climateprediction.net home page
New study going out to volunteer's machines

New study going out to volunteer's machines

Message boards : News : New study going out to volunteer's machines
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Andy Bowery
Volunteer moderator
Project administrator

Send message
Joined: 11 Feb 11
Posts: 13
Credit: 53,909
RAC: 0
Message 68287 - Posted: 13 Feb 2023, 11:32:41 UTC

STORMS: Investigating how low-pressure systems may change in the future

(actual project name - “Quantifying controls on the intensity, variability and impacts of extreme European STORMS”)

Victoria Sinclair, Clément Bouvier
Institute for Atmospheric and Earth System Research,
University of Helsinki, Finland

Throughout the year, low-pressure systems regularly move across Europe, usually from west to east, bringing cloud, rain and windy weather. Sometimes these weather systems can become very intense, and the winds and rain associated with them can cause damage to buildings and infrastructure, flooding, and can disrupt electricity supply and travel. Although the short-term weather forecasts of these storms are now quite accurate, it still remains uncertain how these storms, and their impacts, are likely to change in the future as our climate changes. Some of this uncertainty is because our understanding of what controls the strength and impacts of these storms is incomplete.

The aim of this project is to understand what controls the strength and structure of these low-pressure systems. We will quantify how the atmospheric state that the low-pressure systems develop in affects the strength and structure of these low-pressure systems. This atmospheric state can be described by various parameters, for example, the mean temperature, moisture content, and upper-level wind speeds (i.e. the strength and width of the jet stream). Since there are lots of different parameters we want to study (not just the ones described above), we want to do lots of experiments in a high controlled manner. Therefore, we will run a large ensemble of simulations of idealised low-pressure systems using the numerical weather prediction model OpenIFS. Although the simulations are idealised, the weather systems that develop look very like real weather systems that we observed in reality. Each ensemble member differs in its initial atmospheric state, and we choose these initial states to cover everything from the current climate to past pre-industrial climates to the most extreme future climate projections. This is exciting because although idealised simulations of low-pressure systems have been performed before, this is the first time that such an extensive exploration of the parameter space will be conducted.

Once we have the results from the large ensemble, we will calculate different measures of the strength of the storms and then use machine learning techniques to see how these relate to the initial states. Our results will hopefully increase in confidence in how these storms and their impacts will change in the future.

Technical information:

Please ensure 'Leave non-GPU tasks in memory while suspended' is set in Disk/Memory options of boincmgr

CPDN app-name: oifs_43r3_bl
Run time: ~6 hrs/task on a modern CPU
Max memory: ~7Gb
Total number of files: 354 files
Model output: 16Mb per output step (uncompressed)
Total size of uploaded files: 1.5GB
Checkpoint filesize: ~800Mb (these are periodically created & deleted in the slot dir and not uploaded)[/b]
ID: 68287 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1045
Credit: 16,506,818
RAC: 5,001
Message 68289 - Posted: 13 Feb 2023, 13:12:21 UTC - in response to Message 68287.  

Technical information:

Please ensure 'Leave non-GPU tasks in memory while suspended' is set in Disk/Memory options of boincmgr

CPDN app-name: oifs_43r3_bl
Run time: ~6 hrs/task on a modern CPU
Max memory: ~7Gb
Total number of files: 354 files
Model output: 16Mb per output step (uncompressed)
Total size of uploaded files: 1.5GB
Checkpoint filesize: ~800Mb (these are periodically created & deleted in the slot dir and not uploaded)[/b]


Thank you for this very informative post, especially the part I quoted above.

My machine (Computer 1511241) just got six of these tasks and I diddled my app_config.xml file to allow 5 to run at a time and five are currently running. Since this is a new task type, OpenIFS 43r3 Baroclinic Lifecycle 1.11 x86_64-pc-linux-gnu , my boinc manager expects them to take 08:47:45 to complete. Perhaps so, but we will see as it learns the truth, whatever that is. If the latest _ps tasks are anything to go by, the estimated time at the beginning was about double what my machine actually took (based on only two tasks of the _ps tasks) so far.

Since these five all have less than an hour on them, the following RAM usage is probably not typical yet, but there seems to be plenty of RAM for these to run. The biggest one is taking about 4 GBytes with about 1/2 hour of execution time.

$ free -hw
              total        used        free      shared     buffers       cache   available
Mem:           62Gi        24Gi       1.9Gi       115Mi       509Mi        35Gi        37Gi
Swap:          15Gi       127Mi        15Gi

ID: 68289 · Report as offensive     Reply Quote
[SG]Felix

Send message
Joined: 4 Oct 15
Posts: 34
Credit: 9,069,332
RAC: 14,637
Message 68294 - Posted: 13 Feb 2023, 18:29:56 UTC

Quick Note from my side, could it be possible, that you mixed some of the infos up?

I have two WUs running, both have one zip after an hour of runtime.
https://www.cpdn.org/workunit.php?wuid=12209606 has created an _0 Zip with 62,21MB of size
https://www.cpdn.org/workunit.php?wuid=12209550 has created an _0 Zip with 57,37MB of size

I will block the uploads an collect more data until completion.

Greets
Felix
ID: 68294 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 774
Credit: 13,433,329
RAC: 7,110
Message 68296 - Posted: 13 Feb 2023, 18:56:30 UTC - in response to Message 68294.  

Quick Note from my side, could it be possible, that you mixed some of the infos up?
I have two WUs running, both have one zip after an hour of runtime.
https://www.cpdn.org/workunit.php?wuid=12209606 has created an _0 Zip with 62,21MB of size
https://www.cpdn.org/workunit.php?wuid=12209550 has created an _0 Zip with 57,37MB of size

I will block the uploads an collect more data until completion.
Greets
Felix
Felix, zipfiles are not guaranteed to produce the same size with different data. The compression achieved by zip depends on the patterns in the input files. The filesizes are estimates only. I suggest just let them finish, no need to block uploads.
ID: 68296 · Report as offensive     Reply Quote
Profile Friedrich S.
Avatar

Send message
Joined: 22 Jan 05
Posts: 35
Credit: 3,825,199
RAC: 2,237
Message 68301 - Posted: 13 Feb 2023, 20:39:10 UTC - in response to Message 68287.  
Last modified: 13 Feb 2023, 20:39:58 UTC

Good evening,

just one more technical information I would like to see: What OS is required? Linux? Or would Windows work as well?

Thanks!

Friedrich
ID: 68301 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1045
Credit: 16,506,818
RAC: 5,001
Message 68304 - Posted: 13 Feb 2023, 21:10:04 UTC - in response to Message 68289.  

Since these five all have less than an hour on them, the following RAM usage is probably not typical yet, but there seems to be plenty of RAM for these to run. The biggest one is taking about 4 GBytes with about 1/2 hour of execution time.


The first five have finished now. Near the end, it was like this:

localhost:jeandavid8[~]$ date; free -hw
Mon Feb 13 14:46:01 EST 2023
              total        used        free      shared     buffers       cache   available
Mem:           62Gi        20Gi        10Gi       151Mi       509Mi        31Gi        41Gi
Swap:          15Gi       141Mi        15Gi
localhost:jeandavid8[~]$ date; free -hw
Mon Feb 13 15:27:08 EST 2023
              total        used        free      shared     buffers       cache   available
Mem:           62Gi        18Gi        12Gi       147Mi       509Mi        30Gi        42Gi
Swap:          15Gi       141Mi        15Gi
localhost:jeandavid8[~]$ date; free -hw
Mon Feb 13 16:06:48 EST 2023
              total        used        free      shared     buffers       cache   available
Mem:           62Gi        24Gi       4.9Gi       164Mi       509Mi        31Gi        36Gi
Swap:          15Gi       141Mi        15Gi

ID: 68304 · Report as offensive     Reply Quote
David Berg

Send message
Joined: 2 Jul 15
Posts: 16
Credit: 3,885,305
RAC: 2,411
Message 68306 - Posted: 13 Feb 2023, 21:24:31 UTC - in response to Message 68287.  

Thank you, Andy.
I have not seen any new tasks in the last several months. Posted information suggests they have all been LINUX tasks and I run only Windows 10.
Are these new Storms tasks executable in Windows?
I don't see the configuration element 'Leave non-GPU tasks in memory while suspended'. Please tell me where to find it.
Thank you!

- David Berg
ID: 68306 · Report as offensive     Reply Quote
Hugo

Send message
Joined: 24 Sep 09
Posts: 4
Credit: 498,714
RAC: 759
Message 68315 - Posted: 14 Feb 2023, 13:42:16 UTC - in response to Message 68306.  

Indeed! I have the same doubt, no new task for months and so far not being able to catch any of this new ones.
ID: 68315 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 774
Credit: 13,433,329
RAC: 7,110
Message 68316 - Posted: 14 Feb 2023, 16:03:23 UTC - in response to Message 68306.  

Thank you, Andy.
I have not seen any new tasks in the last several months. Posted information suggests they have all been LINUX tasks and I run only Windows 10.
Are these new Storms tasks executable in Windows?
I don't see the configuration element 'Leave non-GPU tasks in memory while suspended'. Please tell me where to find it.
Thank you!

- David Berg
The current batches are linux only. For Windows, running them under WSL (which is what I do), or a virtualbox setup works fine, but that's not for everyone. I do know that CPDN are preparing some more Windows batches though I don't have a timeline for when they will be ready.

The 'leave non-GPU in memory' option can be found in the computing preferences in boincmgr.
ID: 68316 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 925
Credit: 34,100,818
RAC: 11,270
Message 68317 - Posted: 14 Feb 2023, 16:17:05 UTC - in response to Message 68316.  

The 'leave non-GPU in memory' option can be found in the computing preferences in boincmgr.
It can also be found on the Computing Preferences page of this website, https://www.cpdn.org/prefs.php?subset=global ('Memory' section).

You can set preferences via the website, or via BOINC Manager, but you can't mix the two. If you already have some personal preferences set, use the same technique for this one as well.
ID: 68317 · Report as offensive     Reply Quote
Hugo

Send message
Joined: 24 Sep 09
Posts: 4
Credit: 498,714
RAC: 759
Message 68328 - Posted: 14 Feb 2023, 23:43:41 UTC - in response to Message 68316.  

Thanks for lettings us know! I will try to setup a virtual machine later down the line... but its a bummer that for example I cannot use my Ryzen 5 right now in the project because the Linux-only WUs.
ID: 68328 · Report as offensive     Reply Quote
TribbleRED

Send message
Joined: 7 Dec 19
Posts: 1
Credit: 3,936,263
RAC: 9,895
Message 68352 - Posted: 16 Feb 2023, 2:23:24 UTC - in response to Message 68316.  

The current batches are linux only. For Windows, running them under WSL (which is what I do), or a virtualbox setup works fine, but that's not for everyone. I do know that CPDN are preparing some more Windows batches though I don't have a timeline for when they will be ready.


Do you have a write-up or a link to describing how to do this?

Cheers
ID: 68352 · Report as offensive     Reply Quote
KAMasud

Send message
Joined: 6 Oct 06
Posts: 204
Credit: 7,608,986
RAC: 0
Message 68366 - Posted: 17 Feb 2023, 16:17:10 UTC - in response to Message 68352.  

The current batches are linux only. For Windows, running them under WSL (which is what I do), or a virtualbox setup works fine, but that's not for everyone. I do know that CPDN are preparing some more Windows batches though I don't have a timeline for when they will be ready.


Do you have a write-up or a link to describing how to do this?

Cheers


https://www.windowscentral.com/how-install-wsl2-windows-10
ID: 68366 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 669
Credit: 4,391,754
RAC: 6,918
Message 68367 - Posted: 17 Feb 2023, 16:28:54 UTC - in response to Message 68316.  

The current batches are linux only. For Windows, running them under WSL (which is what I do), or a virtualbox setup works fine, but that's not for everyone. I do know that CPDN are preparing some more Windows batches though I don't have a timeline for when they will be ready.
How about doing what LHC does, send the VB wrapper out so anyone can run your Linux stuff?
ID: 68367 · Report as offensive     Reply Quote
KAMasud

Send message
Joined: 6 Oct 06
Posts: 204
Credit: 7,608,986
RAC: 0
Message 68368 - Posted: 18 Feb 2023, 5:40:35 UTC - in response to Message 68367.  

The current batches are linux only. For Windows, running them under WSL (which is what I do), or a virtualbox setup works fine, but that's not for everyone. I do know that CPDN are preparing some more Windows batches though I don't have a timeline for when they will be ready.
How about doing what LHC does, send the VB wrapper out so anyone can run your Linux stuff?


If you want an older version, download it from the Boinc site. If you wish the current version as we all do, go to the VB site and download it. The flavour of Linux is your choice, go and download it from their site.
ID: 68368 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 669
Credit: 4,391,754
RAC: 6,918
Message 68370 - Posted: 18 Feb 2023, 13:20:56 UTC - in response to Message 68368.  
Last modified: 18 Feb 2023, 13:24:03 UTC

The current batches are linux only. For Windows, running them under WSL (which is what I do), or a virtualbox setup works fine, but that's not for everyone. I do know that CPDN are preparing some more Windows batches though I don't have a timeline for when they will be ready.
How about doing what LHC does, send the VB wrapper out so anyone can run your Linux stuff?


If you want an older version, download it from the Boinc site. If you wish the current version as we all do, go to the VB site and download it. The flavour of Linux is your choice, go and download it from their site.
I may have a go at that. But it involves setting up another Boinc client within a virtual machine. I hate the linux interface, it took me 3 hours last time to work out how to install a program. The VB wrapper LHC uses means a task is downloaded from them and run in the Windows client along with other projects, easier to manage.

Would I be able to access this Boinc client from Boinctasks in Windows? I currently use Boinctasks to view all my clients on Windows on other machines across the network. Hopefully the VBox environment has a unique IP I can access?
ID: 68370 · Report as offensive     Reply Quote
computezrmle

Send message
Joined: 9 Mar 22
Posts: 30
Credit: 963,113
RAC: 46,932
Message 68371 - Posted: 18 Feb 2023, 13:50:36 UTC - in response to Message 68370.  

Hopefully the VBox environment has a unique IP...

There's no need for hope.
Just read the VirtualBox manual.
https://www.virtualbox.org/manual/ch06.html#networkingmodes

Hint1: Don't mix self created VMs with those created by vboxwrapper
Hint2: Your use case requires a self created VM with Bridged networking
ID: 68371 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 669
Credit: 4,391,754
RAC: 6,918
Message 68372 - Posted: 18 Feb 2023, 14:29:50 UTC - in response to Message 68371.  
Last modified: 18 Feb 2023, 14:31:45 UTC

Thanks, I've got the network set up and it has a unique IP.

But I can't find the thing to install stuff. I don't even have a dock and every Ubuntu manual keeps telling me to use it.

Edit: nevermind, found the software downloader in the start menu. I'd prefer no dock anyway, this isn't a Mac.
ID: 68372 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 669
Credit: 4,391,754
RAC: 6,918
Message 68373 - Posted: 18 Feb 2023, 14:34:37 UTC
Last modified: 18 Feb 2023, 15:34:26 UTC

No tasks? I thought CPDN had loads of Linux tasks.

Gah! Finally found out how to login as root. Much easier to edit xml files when you're allowed to use your own machine! I mean seriously? There seems to be no way to change an xml file to change Boinc's settings if I'm not root! The mousepad program refuses to save, and I can't give myself permissions to edit the file! Gotta love the warning notices everywhere as though I'm about to start a nuclear war. Well I'm root on Windows and nothing has gone wrong in 32 years.
ID: 68373 · Report as offensive     Reply Quote
Paulie

Send message
Joined: 24 Nov 12
Posts: 12
Credit: 1,596,573
RAC: 962
Message 68374 - Posted: 19 Feb 2023, 2:30:21 UTC - in response to Message 68287.  
Last modified: 19 Feb 2023, 2:54:54 UTC

I would love to run some of these, but I haven't received any tasks from CP. in quite a while,
My computer is more than capable of running these,

How do I set up a virtual box, with detailed instructions?

And yet my laptop has 3 CP programs running!
ID: 68374 · Report as offensive     Reply Quote
1 · 2 · 3 · Next

Message boards : News : New study going out to volunteer's machines

©2024 climateprediction.net