climateprediction.net home page
New work Discussion

New work Discussion

Message boards : Number crunching : New work Discussion
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 44 · 45 · 46 · 47 · 48 · 49 · 50 . . . 65 · Next

AuthorMessage
Profile Alan K

Send message
Joined: 22 Feb 06
Posts: 405
Credit: 18,110,105
RAC: 2,624
Message 62786 - Posted: 23 Oct 2020, 22:58:45 UTC - in response to Message 62785.  

I would have to resize my virtual memory for those then. Currently only 8GB.
ID: 62786 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7503
Credit: 23,674,684
RAC: 3,244
Message 62787 - Posted: 23 Oct 2020, 23:56:42 UTC - in response to Message 62786.  

Yes, these are classed as high resolution.
ID: 62787 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 590
Credit: 25,540,983
RAC: 4,073
Message 62788 - Posted: 24 Oct 2020, 1:34:19 UTC - in response to Message 62785.  

Still no news as to when they might make the main site but more testing is happening with the OpenIFS tasks.

Works for me. I built an i7-9700 (eight full cores) with 32 GB of memory for it, and never thought I would live to see it.
(I can do 64 GB easily enough too with a little swapping.)

It may beat the next Ice Age yet.
ID: 62788 · Report as offensive     Reply Quote
lazlo_vii

Send message
Joined: 11 Dec 19
Posts: 108
Credit: 3,012,142
RAC: 0
Message 62789 - Posted: 24 Oct 2020, 5:52:21 UTC - in response to Message 62785.  
Last modified: 24 Oct 2020, 5:54:21 UTC

OpenIFS v2.19
x86_64-pc-linux-gnu
Peak working set size 11,797.17 MB
Peak swap size 13,131.91 MB
Peak disk usage 3,516.51 MB



If you get more would you please let us know how many you are running at once and post the output of "free -mw && cat /proc/meminfo"? I would like to know a little bit more what the kernel is actually doing. I may have to reconfigure things to add more swap space in a worst case scenario. Thanks!
ID: 62789 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3176
Credit: 8,795,408
RAC: 7,129
Message 62790 - Posted: 24 Oct 2020, 6:11:31 UTC

If you get more would you please let us know how many you are running at once and post the output of "free -mw && cat /proc/meminfo"? I would like to know a little bit more what the kernel is actually doing. I may have to reconfigure things to add more swap space in a worst case scenario. Thanks!


This was while running two at once. I suspended my N216's from the main site.
Watching top, each task peaks at about 30% of Ram for a few seconds and then drops back down to about 9%. At one point I did let it run four at once but started at different times so hoped the peaks wouldn't all come at once.

Another thing I noticed was that %work done against time isn't constant but the task slows down a lot after about 75% complete.

dave@swarm:~$ free
              total        used        free      shared  buff/cache   available
Mem:       32853368    23005584     2512156      151960     7335628     9238792
Swap:      41972728           0    41972728
ID: 62790 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 396
Credit: 18,843,957
RAC: 11,667
Message 62792 - Posted: 25 Oct 2020, 19:06:16 UTC - in response to Message 62790.  

It would be great if someone could post system requirements for the OpenIFS once more WUs are tested. It seems my i7-4790 with 16Gb and 21 Gb var space and 5.6 Gb swap may not be able to handle more than two WUs at once
ID: 62792 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7503
Credit: 23,674,684
RAC: 3,244
Message 62793 - Posted: 25 Oct 2020, 19:18:55 UTC - in response to Message 62792.  

Testing has only just started, so it'll be a while before there's anything firm to talk about.

But it does seem that these require more than just "rolling up your sleeves". More like "strip to the waist and put your backs into it". :)

But you're right; lots of requirements are going to need to be posted.
ID: 62793 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2026
Credit: 54,668,537
RAC: 8,817
Message 62794 - Posted: 25 Oct 2020, 19:52:42 UTC

If they get to the main site, there should be something in preferences like WCG where the default number that can be run at a time by a host is 1, and that can be changed by the user on a webpage if the system has 24 GB or more of RAM. Otherwise it's going to be a mess.
ID: 62794 · Report as offensive     Reply Quote
Profile Alan K

Send message
Joined: 22 Feb 06
Posts: 405
Credit: 18,110,105
RAC: 2,624
Message 62795 - Posted: 25 Oct 2020, 23:43:18 UTC - in response to Message 62794.  

Could it be done through the cc-config file with a max download flag?
ID: 62795 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7503
Credit: 23,674,684
RAC: 3,244
Message 62796 - Posted: 26 Oct 2020, 0:27:25 UTC - in response to Message 62795.  

Best if it was a server setting, so fiddlers can't put things back to the way they are already.
ID: 62796 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3176
Credit: 8,795,408
RAC: 7,129
Message 62797 - Posted: 26 Oct 2020, 7:45:06 UTC - in response to Message 62792.  

It would be great if someone could post system requirements for the OpenIFS once more WUs are tested. It seems my i7-4790 with 16Gb and 21 Gb var space and 5.6 Gb swap may not be able to handle more than two WUs at once


Based on what happened on my laptop which had 8GB RAM with the earlier tests where the tasks took up about 5GB RAM more would probably run but the use of swap memory would slow them down a lot.
ID: 62797 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3176
Credit: 8,795,408
RAC: 7,129
Message 62798 - Posted: 26 Oct 2020, 13:02:42 UTC

There will be some more resends on batch 877 as they had gone out with the default of three strikes and out rather than the five to allow for the number that fail due to missing 32bit libs. Don't know when Sarah will do this. About 120 so far have failed three times and these will be sent out again.
ID: 62798 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 590
Credit: 25,540,983
RAC: 4,073
Message 62799 - Posted: 26 Oct 2020, 14:03:23 UTC - in response to Message 62798.  

About 120 so far have failed three times and these will be sent out again.

I suppose we have mentioned this before, but if you had a "reliable machine" program, they would be sent to only machines that can do them.
It is implemented on some BOINC projects, so I think it is a standard BOINC thing.
ID: 62799 · Report as offensive     Reply Quote
Profile Alan K

Send message
Joined: 22 Feb 06
Posts: 405
Credit: 18,110,105
RAC: 2,624
Message 62801 - Posted: 30 Oct 2020, 23:40:39 UTC - in response to Message 62798.  
Last modified: 30 Oct 2020, 23:41:06 UTC

Have these been sent out as batch 878? Snaflled 2 of them!
ID: 62801 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7503
Credit: 23,674,684
RAC: 3,244
Message 62802 - Posted: 31 Oct 2020, 0:37:06 UTC - in response to Message 62801.  

Batch 878 are similar to batch 877.
ID: 62802 · Report as offensive     Reply Quote
lazlo_vii

Send message
Joined: 11 Dec 19
Posts: 108
Credit: 3,012,142
RAC: 0
Message 62803 - Posted: 31 Oct 2020, 3:09:36 UTC - in response to Message 62802.  

I see over 1000 N216's in the queue. Are those bounced tasks or new?
ID: 62803 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7503
Credit: 23,674,684
RAC: 3,244
Message 62804 - Posted: 31 Oct 2020, 3:48:00 UTC - in response to Message 62803.  

New. Different experiment.
ID: 62804 · Report as offensive     Reply Quote
lazlo_vii

Send message
Joined: 11 Dec 19
Posts: 108
Credit: 3,012,142
RAC: 0
Message 62805 - Posted: 31 Oct 2020, 4:03:40 UTC - in response to Message 62804.  

With any luck I'll be able to crunch a few of them. I set WGC to "No New Tasks" on half of a Ryzen 7.
ID: 62805 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3176
Credit: 8,795,408
RAC: 7,129
Message 62806 - Posted: 31 Oct 2020, 7:45:20 UTC

I haven't checked all of them yet but at least one of those I have downloaded is already on its third attempt.
ID: 62806 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3176
Credit: 8,795,408
RAC: 7,129
Message 62824 - Posted: 2 Nov 2020, 14:21:27 UTC
Last modified: 2 Nov 2020, 15:18:50 UTC

#877 and #878 have had all tasks waiting to go out withdrawn as many produce uploads greater than the limit allowed causing some to fail at 100% completion. They will be re-issued shortly.

Edit: Those already sent out will be left to run. Credit will be granted.
ID: 62824 · Report as offensive     Reply Quote
Previous · 1 . . . 44 · 45 · 46 · 47 · 48 · 49 · 50 . . . 65 · Next

Message boards : Number crunching : New work Discussion

©2021 climateprediction.net