climateprediction.net home page
Posts by Dave Jackson

Posts by Dave Jackson

1) Questions and Answers : Unix/Linux : fedora 30 64 bit (Message 62791)
Posted 21 hours ago by Profile Dave Jackson
Post:
Well they did say once: Don't change horses in mid stream.
This may be why. :)

I was in a hurry so upgraded to Groovy without waiting for running tasks to finish and all six survived, (two now completed successfully. :)
2) Message boards : Number crunching : New work Discussion (Message 62790)
Posted 1 day ago by Profile Dave Jackson
Post:
If you get more would you please let us know how many you are running at once and post the output of "free -mw && cat /proc/meminfo"? I would like to know a little bit more what the kernel is actually doing. I may have to reconfigure things to add more swap space in a worst case scenario. Thanks!


This was while running two at once. I suspended my N216's from the main site.
Watching top, each task peaks at about 30% of Ram for a few seconds and then drops back down to about 9%. At one point I did let it run four at once but started at different times so hoped the peaks wouldn't all come at once.

Another thing I noticed was that %work done against time isn't constant but the task slows down a lot after about 75% complete.

dave@swarm:~$ free
              total        used        free      shared  buff/cache   available
Mem:       32853368    23005584     2512156      151960     7335628     9238792
Swap:      41972728           0    41972728
3) Message boards : Number crunching : New work Discussion (Message 62785)
Posted 1 day ago by Profile Dave Jackson
Post:
Still no news as to when they might make the main site but more testing is happening with the OpenIFS tasks.

If the latest batch is anything to go by, they will not be for everyone, the ones I ran recently peaked at 11GB of RAM and about 13GB of swap. (I never saw my swap file usage go up to anything like that so I wonder if that means data swapped out to disk in the BOINC data directory though usage there never got up that high when I looked.

OpenIFS v2.19
x86_64-pc-linux-gnu
Peak working set size 11,797.17 MB
Peak swap size 13,131.91 MB
Peak disk usage 3,516.51 MB
4) Message boards : Number crunching : New work Discussion (Message 62779)
Posted 14 days ago by Profile Dave Jackson
Post:
And now all gone bar the inevitable retreads.
5) Message boards : Number crunching : Welcome back/checking if everything is working? (Message 62777)
Posted 16 days ago by Profile Dave Jackson
Post:
The answer is, that the old data isn't going to be removed.
So, just don't stare at those numbers for long periods. :)


Shucks :)
6) Message boards : Number crunching : Welcome back/checking if everything is working? (Message 62775)
Posted 16 days ago by Profile Dave Jackson
Post:
Can they do the wiping as they need to have all the old tasks available when counting the credits? So nothing can be purged?


If everything was wiped that was past the deadline anything being wiped would I guess miss out on credits making CPDN policy on this the same as most other projects where credit is not granted after the deadline. My personal opinion so not with my moderator hat on or expressing any views of the project is that while we have the very long deadlines currently in use this would be no bad thing.
7) Questions and Answers : Unix/Linux : *** Running 32bit CPDN from 64bit Linux - Discussion *** (Message 62768)
Posted 17 days ago by Profile Dave Jackson
Post:
Instructions for Red Hat Enterprise Linux 7 do not work for Red Hat Enterprise Linux 8. I am not surprised. I have not found out where to find compatibility libraries.


I don't suppose the latest Fedora libraries work? - Just an idea. (It is now over ten years since I switched to Debian based distributions so not based on anything other than guesswork.)
8) Message boards : Number crunching : New Work Announcements (Message 62767)
Posted 17 days ago by Profile Dave Jackson
Post:
Batch 877 HADAM4H 1680 Tasks for Linux 1536 currently showing as available on server but that was close to an hour ago so will be quite a few less by now!
9) Message boards : Number crunching : UK Met Office HadAM4 at N216 resolution (Message 62765)
Posted 17 days ago by Profile Dave Jackson
Post:
I have now got 8 retreads from 843 and 842. Have set to no new tasks now as running more than the number of real cores slows things down too much.
10) Message boards : climateprediction.net Science : No new Task Since Start of Corona Pandemic? (Message 62763)
Posted 18 days ago by Profile Dave Jackson
Post:
Hi Joe,
There has been very little work lately. Most work comes from universities around the world. (Some does come from Oxford's Msc programme and other work there but most does not. The other day there was a small batch (only 40 spin up tasks) but with several thousand computers looking for work they went almost before they were sent out. We (the moderators) have been told that there should be some work for both Linux and Windows platforms in the next few weeks as part of the masters' programme in Oxford. The spin up tasks should lead to some more when they are completed. I am not sure if they are related to the masters tasks or not.

However on Windows at least it is likely that there will be more computers looking for work than there are tasks or certainly more cores available for work than tasks so some may miss out at least on the first couple of batches.

Linux users like myself are a bit better off because lots have not installed some 32bit libraries that are essential for running those tasks which means that every so often there are resends which I picked up eight of over the past couple of days.

Pretty sure one of us will post in the number crunching section when work does arrive.

Sorry I can't be the bearer of better news.
11) Message boards : Number crunching : WAH2 sam50 spinups (Message 62758)
Posted 19 days ago by Profile Dave Jackson
Post:
Hi, if you look at the name of the task e.g.
wah2_sam50_s000_198512_60_876_012028138 for one of this batch,
sam50 tells you it is South America and 50Km squares for the resolution.
198512 gives you the start year and month.
60 is the number of model months the task will run for.
12) Message boards : Number crunching : WAH2 sam50 spinups (Message 62756)
Posted 19 days ago by Profile Dave Jackson
Post:
Quickly grabbed but should keep the 40 cores that have them busy for a while as they are 5 year runs.
13) Message boards : Number crunching : Welcome back/checking if everything is working? (Message 62755)
Posted 19 days ago by Profile Dave Jackson
Post:
I presume these will still require some 32bit libs and not the full blown 64bit jobbies. For linux w/u. I had better make sure the fedora 30 hard disk is plugged in.


For Linux the tasks will be N216 Hadam4 tasks and hadcm3 so yes do make sure the relevant 32bit libraries for your distribution are installed.

That said, the last testing Hadam4 tasks I ran on my new box with a clean install of xubuntu20.04 ran without my installing them explicitly. I did go on and install the ones that were not there. It may have been that I installed what was needed for them while installing everything including the kitchen sink to enable me to compile BOINC from source.
14) Message boards : Number crunching : Welcome back/checking if everything is working? (Message 62751)
Posted 20 days ago by Profile Dave Jackson
Post:
Hi Guys,
Re.
UK Met Office Coupled Model Full Resolution Ocean
Weather At Home 2 (wah2) (region independent)
UK Met Office HadAM4 at N144 resolution

Just a thought.
Now that there's not been any new projects for some months. the 'In progress' numbers for these applications are obviously defunct. They were actually at this level a long time before the pandemic hit the world.
They must be hoarded WU's that are past their useful date. None of the users have registered for months.
Couldn't they be zeroed?

Les has contacted the project, some cleaning up will be done but probably not before some more work appears which will be part of the new season Msc programme which should in the next few weeks have work for both Windows and Linux machines. (Not sure about Mac.
15) Message boards : Number crunching : Welcome back/checking if everything is working? (Message 62747)
Posted 20 days ago by Profile Dave Jackson
Post:
I'll see what the project thinks about wiping everything.


Wiping of the full resolution ocean stuff could certainly happen (in my opinion) and also the WAH2 region independent tasks or I suppose, everything over two years old? I don't know how messy the different options are.
16) Message boards : Number crunching : UK Met Office HadAM4 at N216 resolution (Message 62744)
Posted 21 days ago by Profile Dave Jackson
Post:
And I have just picked up one from #843 as well. (On its fifth and final attempt.
17) Message boards : Number crunching : Welcome back/checking if everything is working? (Message 62739)
Posted 27 days ago by Profile Dave Jackson
Post:
I've been getting upload retry for days.


I think uploads from batch 860 go to a server in Tasmania. I have let Andy know.
18) Message boards : Number crunching : Stopped sending trickles 5 days ago, but still running. Should I let it run or abort? (Message 62734)
Posted 21 Sep 2020 by Profile Dave Jackson
Post:
Hi!

Looks like new trickles aren't showing again.

BR
Rayburner
I will send another email. Based on last time I think it is a script Andy can start remotely. Not sure why it keeps stopping though.

Edit: Done.
19) Message boards : Number crunching : Updated BOINC Clients 7.16.11 - Windows 64-bit and Mac OS X (64-bit Intel) (Message 62732)
Posted 19 Sep 2020 by Profile Dave Jackson
Post:
Just an FYI, the ARP tasks hate SMT and eat L3 cache as much as CPDN N216 tasks. So do one per core or 2MB of L3 (which ever is lower) for best turn around times.

Yep, noticed that. Combining the two with some 216's from testing certainly slows things down.
20) Message boards : Number crunching : Updated BOINC Clients 7.16.11 - Windows 64-bit and Mac OS X (64-bit Intel) (Message 62729)
Posted 17 Sep 2020 by Profile Dave Jackson
Post:
And while weather rather than climate there is the Africa Rainfall Project with World Community Grid.


And almost as difficult to get a w/u for. There are a few now and then........ and there does not seem to be a status page showing if there are w/u's to be had. Or have I missed it?.


I seem to be able to consistently run 6 of these. On a multicore computer the secret is to go to the profile - default, home, work or school the computer is using, click on custom profile and then up the number of concurrent work units for the project from the default of 1.

(Took me a while to work this out!)


Next 20

©2020 climateprediction.net