climateprediction.net home page
Project announcements discussion

Project announcements discussion

Message boards : climateprediction.net Science : Project announcements discussion
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60377 - Posted: 20 Jun 2019, 10:21:40 UTC

"Front page", right at the bottom on the left.
ID: 60377 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,378,503
RAC: 3,632
Message 60378 - Posted: 20 Jun 2019, 11:42:35 UTC - in response to Message 60377.  

Thanks Les,

I have seen the link before, just never gotten around to clicking on it. I have to place limits somewhere on my digital time!
ID: 60378 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60400 - Posted: 22 Jun 2019, 7:26:32 UTC

In Regard to the OpenIFS models, we can now talk about the batch that's been released for the Launch Event at Reading University.

This new endeavour is based on ECMWF work, rather the Uk Met Office models that we're so used to.
There's a description on the front page of the project, under News.
And a larger one, with photos, here

They can range from the small, as at present, to the huge, as with the Mid May Monsters that we were testing at one stage, all by changes to data files.
I'm not going to say much about the big ones, just that they required over 5 Gigs of ram each, and the other requirements were also big.

They're 64 bit Linux models.
For the current models, which are for Tropical storm Karl, I've seen the ram go from just under 2 Gigs, to 2+, and up to 3+ (under Properties).
One zip, of 64+ Megs.
Run time for 4, which is NO hyper-threading, is around 1 hour 13 minutes.

When I changed to all 8 processors, the time went up to 2 hours 28 minutes or so.
So I went back to 4 at a time.

The project person replied:
We normally run IFS with hyper-threading turned off. The model does a lot of data movement which can lead to cache use inefficiencies. I found I got a faster throughout by running 4 tasks on my 8 virtual core machine.

So, best not to use hyper-threading.

I'm using 2 machines with 3.5 GHZ processors.
Another cruncher with a 3.6 GHz processor is running almost right on 1 hour.

I don't know when these will start appearing on the cpdn site.
ID: 60400 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 60409 - Posted: 22 Jun 2019, 21:00:08 UTC - in response to Message 60400.  
Last modified: 22 Jun 2019, 21:04:34 UTC


Run time for 4, which is NO hyper-threading, is around 1 hour 13 minutes.

When I changed to all 8 processors, the time went up to 2 hours 28 minutes or so.
So I went back to 4 at a time.

The project person replied:
We normally run IFS with hyper-threading turned off. The model does a lot of data movement which can lead to cache use inefficiencies. I found I got a faster throughout by running 4 tasks on my 8 virtual core machine.

So, best not to use hyper-threading.

That is true not just for the IFS, but also for the "HadAM4 at N144 resolution v8.09". When I started on my i7-8700 running only one work unit (with the other cores on Rosetta), I got 11.5 sec/TS; very nice.
https://www.cpdn.org/cpdnboinc/result.php?resultid=21718772

But when I loaded up all the cores (reserving one for a GPU) with the N144 work units, they dropped down to 22 sec/TS
https://www.cpdn.org/cpdnboinc/result.php?resultid=21723417
https://www.cpdn.org/cpdnboinc/result.php?resultid=21723519

(I am assuming that the work units are similar in run times.)
So might as well go with full cores (or 50% of the processors in BOINC Manager).
ID: 60409 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60410 - Posted: 22 Jun 2019, 22:02:09 UTC

This makes the Intel 9700K (Coffee Lake), a good choice for a new build - 8 cores, no hyper-threading, 3.6 GHz.
ID: 60410 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 60411 - Posted: 22 Jun 2019, 22:31:24 UTC - in response to Message 60410.  

This makes the Intel 9700K (Coffee Lake), a good choice for a new build - 8 cores, no hyper-threading, 3.6 GHz.

Curious you should mention that. I am building one now (the non-K for energy efficiency) for the IFS. I am putting 64 GB memory into it. I hope that is enough.
ID: 60411 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60413 - Posted: 23 Jun 2019, 0:31:21 UTC

That's the amount of ram that I'm planning for.
Also, allow 50 Gigs of HD space for BOINC, with the possibility of going to 100 Gigs. Hopefully we won't get to that again, but the future stretches a long way ahead.
ID: 60413 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 60415 - Posted: 23 Jun 2019, 7:21:49 UTC
Last modified: 23 Jun 2019, 7:32:54 UTC

A quick question on the HadAM4 N144 and N216 programs. Are they 64 bit or was it just OpenIFS that’s 64 bit? On the Applications page they’re listed as Linux/x86.

Thanks
BOINC blog
ID: 60415 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60416 - Posted: 23 Jun 2019, 7:23:42 UTC

Just OpenIFS.
ID: 60416 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 60417 - Posted: 23 Jun 2019, 7:34:18 UTC - in response to Message 60416.  

Just OpenIFS.

Thanks, better update my preferences then as I don’t have any 32 bit libs installed.
BOINC blog
ID: 60417 · Report as offensive     Reply Quote
mngn

Send message
Joined: 13 Jul 18
Posts: 38
Credit: 56,768,885
RAC: 5,312
Message 60446 - Posted: 25 Jun 2019, 16:11:30 UTC
Last modified: 25 Jun 2019, 16:11:53 UTC

There's a new reply when requesting OpenIFS tasks. A sign of better days to come, I hope: "No tasks are available for OpenIFS".
ID: 60446 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1045
Credit: 16,506,818
RAC: 5,001
Message 60448 - Posted: 25 Jun 2019, 18:03:50 UTC - in response to Message 60446.  

There's a new reply when requesting OpenIFS tasks. A sign of better days to come, I hope: "No tasks are available for OpenIFS".


That has been true for a while now:

16-Jun-2019 19:16:51 Sending scheduler request: To fetch work.
16-Jun-2019 19:16:51 Requesting new tasks for CPU
16-Jun-2019 19:16:53 Scheduler request completed: got 0 new tasks
16-Jun-2019 19:16:53 No tasks sent
16-Jun-2019 19:16:53 No tasks are available for UK Met Office Coupled Model Full Resolution Ocean
16-Jun-2019 19:16:53 No tasks are available for UK Met Office HadCM3 short
16-Jun-2019 19:16:53 No tasks are available for UK Met Office HadAM4 at N144 resolution
16-Jun-2019 19:16:53 No tasks are available for OpenIFS
16-Jun-2019 19:16:53 No tasks are available for UK Met Office HadAM4 at N216 resolution
ID: 60448 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1045
Credit: 16,506,818
RAC: 5,001
Message 60462 - Posted: 26 Jun 2019, 22:07:33 UTC - in response to Message 60413.  

Also, allow 50 Gigs of HD space for BOINC, with the possibility of going to 100 Gigs.


OK: I just ordered a 2 Terabyte hard drive and will move my boinc stuff to a 100 megabyte partition on it. 7200 rpm, 256 Megabyte cache. But it is a good thing I am not dial-up. These current Linux things upload pretty fast over my FiOS connection (75 Megabit/second up and down).

Wed 26 Jun 2019 11:28:24 AM EDT | climateprediction.net | Started upload of hadam4_a045_200710_12_825_011882504_0_r1567844472_6.zip
Wed 26 Jun 2019 11:28:37 AM EDT | climateprediction.net | Finished upload of hadam4_a045_200710_12_825_011882504_0_r1567844472_6.zip

Wed 26 Jun 2019 11:40:20 AM EDT | climateprediction.net | Started upload of hadam4_a03c_200710_12_825_011882475_0_r29367483_6.zip
Wed 26 Jun 2019 11:40:35 AM EDT | climateprediction.net | Finished upload of hadam4_a03c_200710_12_825_011882475_0_r29367483_6.zip

Wed 26 Jun 2019 01:08:31 PM EDT | climateprediction.net | Started upload of hadam4_a027_200610_12_825_011882434_0_r411654165_6.zip
Wed 26 Jun 2019 01:08:37 PM EDT | climateprediction.net | Finished upload of hadam4_a027_200610_12_825_011882434_0_r411654165_6.zip

I assume the downloads and uploads will be bigger when the grids get finer.
ID: 60462 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60468 - Posted: 26 Jun 2019, 22:51:07 UTC - in response to Message 60462.  

I assume the downloads and uploads will be bigger when the grids get finer.


They can get like that.
All will be revealed "soon".
ID: 60468 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 60492 - Posted: 29 Jun 2019, 14:32:01 UTC - in response to Message 60409.  

So might as well go with full cores (or 50% of the processors in BOINC Manager).

FWIW, my more recent number for the benefits of hyper-threading (virtual cores, symmetric multiprocessing, or whatever) are more encouraging. I will turn it back on.
But it is a bit hard to get consistent numbers; you need to let it run for a while. YMMV, depending in CPU, cache size, etc.
ID: 60492 · Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : climateprediction.net Science : Project announcements discussion

©2024 climateprediction.net