climateprediction.net home page
Posts by Dave Jackson

Posts by Dave Jackson

1) Message boards : Number crunching : New work discussion - 2 (Message 66624)
Posted 1 hour ago by Profile Dave Jackson
Post:
The first of the Perturbed Surface variant of OpenIFS are going out now. App name: oifs_43r3_ps

3000 in this batch. Many more to follow.
Only seem to be 1,000 in this batch. I presume the other 2,000 will arrive soon seeing as the first lot are all gone. 7 showing as completed on batch statistics page which is a bit out of date.
2) Message boards : Number crunching : New work discussion - 2 (Message 66623)
Posted 3 hours ago by Profile Dave Jackson
Post:
Looks like about 13 hours running two at a time on an i7-4790K and about 9 hours running two at a time on my Rzyen 5 5600X.
10hrs47 on my Ryzen7.
3) Message boards : Number crunching : OpenIFS Discussion (Message 66612)
Posted 12 hours ago by Profile Dave Jackson
Post:
123 output files! Waiting till I see the sizes, but I hope Oxford know what they've unleashed on their creaking infrastructure.
Averaging about 14.2MB looking at mine. ( Haven't actually done the arithmetic to calculate the mean.
4) Message boards : Number crunching : New work discussion - 2 (Message 66610)
Posted 12 hours ago by Profile Dave Jackson
Post:
Typical runtime is ~7-10hrs depending on your CPU. Memory should be ~6Gb.
The two I have should finish in a fraction under 10 hours.
5) Message boards : Number crunching : New Work Announcements (Message 66605)
Posted 14 hours ago by Profile Dave Jackson
Post:
#945 Haven't checked the exact number but over 100 OpenIFS tasks. I have two running.

Edit 1,000 tasks.
6) Message boards : Number crunching : OpenIFS Discussion (Message 66604)
Posted 16 hours ago by Profile Dave Jackson
Post:
two running from #945

Edit: Running two at once, no problem with my bored band keeping up. Peak memory usage at the moment seems to be about 13% of 32GB
7) Message boards : Number crunching : OpenIFS Discussion (Message 66601)
Posted 16 hours ago by Profile Dave Jackson
Post:
I got four from testing all failed in just over one and a half minutes with
 ABORT!    1 RRTM_KGB16:ERROR READING FILE RADRRTM
At least that is the only line that leaps out at me. batch D523.

Don't know if it worth setting up trello cards for these or not?
8) Message boards : Number crunching : Excuse me? Zero Credit? (Message 66594)
Posted 1 day ago by Profile Dave Jackson
Post:
On testing site, Andy has a script that uses a lot less resources and so runs daily rather than weekly. The idea had been that this would be brought to the main site but not sure what has happened to it. May be that the work for OpenIFS or something else has taken over or that the new script still used too many resources for running daily with the vastly bigger numbers here. If I find out anything I will report back.
9) Message boards : Number crunching : OpenIFS Discussion (Message 66592)
Posted 2 days ago by Profile Dave Jackson
Post:
I expect the answer is negative but would the app_config file accept wildcards? If it did, that would be a way to limit the total number of OpenIFS tasks.
I think it's highly unlikely, but I can check the code. That's not the way David Anderson's mind works.

Edit - no, no sign of a wildcard handler in the code.
Thanks for checking Richard. I thought it highly unlikely but also that it would be silly not to check.
10) Message boards : Number crunching : New work discussion - 2 (Message 66587)
Posted 3 days ago by Profile Dave Jackson
Post:
That doesn't mean much. It just means it's on a volunteer machine, either 'ready to start', 'running' or 'suspended'. None of the sent jobs have produced any trickles which tells me they are not actually running yet. Probably sitting behind a bunch of HadSM4 jobs. I've asked the workunit deadline to be set to 30 days for these OpenIFS tasks to get around the problem of volunteers who set a large job cache.
Two of the batch of five now showing as completed.
11) Message boards : Number crunching : New work discussion - 2 (Message 66583)
Posted 3 days ago by Profile Dave Jackson
Post:
I see two batches, one of 2 tasks and one of 5 showing on the page moderators can check details of batches on. Nothing showing as ready to send or in progress on server status page at the moment but that is not unusual as it is only updated about every 2 hours or so.

Edit: Unless these have been restricted to go to particular machines, the vagaries of who gets them might mean a long wait till they come back.

Edit 2 Now showing on server status page as ready to go which means they have probably gone.
12) Message boards : Number crunching : no credit awarded? (Message 66582)
Posted 3 days ago by Profile Dave Jackson
Post:
I believe credit is ran once a week on Sundays around 0000 UTC. You should see all of the credit for the week show up shortly afterwards.
It is certainly at the weekends. I haven't noticed what time it happens though. Over the years, it has changed on a number of occasions, usually when the credit script or the server it runs on has had to be restarted for one reason or another.

If no credit after the weekend let us know. Though I might spot if there is an issue anyway as I have had some resends complete this week on Monday and Tuesday.
13) Message boards : Number crunching : OpenIFS Discussion (Message 66575)
Posted 3 days ago by Profile Dave Jackson
Post:
I expect the answer is negative but would the app_config file accept wildcards? If it did, that would be a way to limit the total number of OpenIFS tasks.
14) Message boards : Number crunching : OpenIFS Discussion (Message 66563)
Posted 4 days ago by Profile Dave Jackson
Post:
Last test batch were oifs_43r3_ps so perturbed surface ones.
15) Message boards : Number crunching : OpenIFS Discussion (Message 66560)
Posted 4 days ago by Profile Dave Jackson
Post:
You would to add a separate <app>...</app> section for each IFS variant, once we know the exact application names in use. You could then use <max_concurrent> to limit each IFS type, but I don't see a way to limit the total IFS number of all types, once multiple versions are in play at the same time.
Depends how likely it is that batches of multiple variants will be out there at once I guess.
16) Message boards : Number crunching : New work discussion - 2 (Message 66557)
Posted 5 days ago by Profile Dave Jackson
Post:
Between 11hrs4 minutes and 11:20 here. The fastest one was running on its own for 3/4 of the time but a larger sample will be needed to know whether this is random or due to running on its own. I will be sticking to running just two of these at a time when they hit the main site as any more than that produces data faster than it can upload.
17) Message boards : Number crunching : New work discussion - 2 (Message 66553)
Posted 5 days ago by Profile Dave Jackson
Post:
Surely credit is based on flops, not Boinc estimates?
It is for well behaved projects but projects are free do do their own thing and some do. However, I am guessing that the data BOINC uses to work out an estimate, is somehow related to the amount of computation i.e. flops.

Depending on the project sometimes different tasks even from the same batch might generate wildly differing amounts of computation, especially ones where the result of one computation dictates the data that goes into the next but my experience is that there is very little variation in computation time between tasks from a particular batch for CPDN.
18) Message boards : Number crunching : New work discussion - 2 (Message 66551)
Posted 5 days ago by Profile Dave Jackson
Post:
In fact, I seem to have four running. Total memory use on my system seems to peak just below 21GB out of 32 with four so that is fine. Running 8 at once should probably wait till I have my upgrade to 64GB though and even then, restricting my box to 7 out of 8 real cores may well prove to give a higher throughput.

Edit. 10% complete in one hour ten minutes. but BOINC is estimating over four days to completion. If the credits are based the same data as the estimated time, then that will need to be changed though estimated time on new model types is often wildly out.
19) Message boards : Number crunching : New work discussion - 2 (Message 66546)
Posted 6 days ago by Profile Dave Jackson
Post:
But not automatically within Boinc yet presumably.
Correct.
In projects such as asteroids, there are volunteers helping code OpenCL for GPUs for example. Have you obtained the services of us users for here too? (Not myself, but I'm sure there are many).
For getting Open Box to run automagically in a shorter time frame, if volunteers were to come forward with the requisite skills, I suspect they would be welcomed. Volunteers don't get around the license conditions of the code used, again something that has been said many times.
20) Message boards : Number crunching : New work discussion - 2 (Message 66540)
Posted 6 days ago by Profile Dave Jackson
Post:
Just make it work in Windows please! You'd get so many more hosts....
With all the Linux hosts that are missing the libraries for the Met office models, there will be a lot more Linux hosts to run the tasks hopefully successfully than with previous tasks.


Next 20

©2022 climateprediction.net