climateprediction.net home page
Hardware requirements for upcoming models

Hardware requirements for upcoming models

Message boards : Number crunching : Hardware requirements for upcoming models
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 5 · Next

AuthorMessage
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7628
Credit: 24,240,330
RAC: 0
Message 65768 - Posted: 6 Aug 2022, 0:23:07 UTC

Processor speed
Memory amount
Hard disk space
etc
ID: 65768 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3560
Credit: 10,761,686
RAC: 5,528
Message 65771 - Posted: 6 Aug 2022, 5:23:42 UTC - in response to Message 65768.  

etc should probably include internet connection speed and allowance for any still on a metered connection.
ID: 65771 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 400
Credit: 21,744,587
RAC: 10,207
Message 65773 - Posted: 6 Aug 2022, 7:40:26 UTC - in response to Message 65771.  
Last modified: 6 Aug 2022, 7:41:53 UTC

I would also include power consumption under others. It seems more and more important as electricity prices soar while people are planning their upgrades.
ID: 65773 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 307
Credit: 2,381,687
RAC: 301
Message 65774 - Posted: 6 Aug 2022, 12:56:17 UTC
Last modified: 6 Aug 2022, 12:56:54 UTC

Sign up to gridcoin, it helps pay for the power. https://gridcoin.us/

Unfortunately they only take projects with regular work: https://gridcoin.us/guides/whitelist.htm
ID: 65774 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3560
Credit: 10,761,686
RAC: 5,528
Message 65775 - Posted: 6 Aug 2022, 15:05:27 UTC - in response to Message 65774.  

I went with gridcoin for a short while. I prefer to have full control over my computer. They are not as bad as SU but I wouldn't go there again.
ID: 65775 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 307
Credit: 2,381,687
RAC: 301
Message 65776 - Posted: 6 Aug 2022, 15:39:23 UTC - in response to Message 65775.  
Last modified: 6 Aug 2022, 15:40:18 UTC

I went with gridcoin for a short while. I prefer to have full control over my computer. They are not as bad as SU but I wouldn't go there again.
You do have full control. You use Boinc as usual, but Gridcoin gives you coins when you do work for their whitelisted projects. You can still do the others, you just won't get any coins. You don't need to bother with a pool anymore, since they've added a function to withdraw coins without investing. Seriously, I don't change my Boinc habits one bit for Gridcoin. It just sits there in the background and I don't have to touch it.
ID: 65776 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3560
Credit: 10,761,686
RAC: 5,528
Message 65777 - Posted: 6 Aug 2022, 16:12:27 UTC - in response to Message 65776.  

Even though they do give out coins to CPDN users occasionally, seeing it as a worthwhile project, as I don't run anything else most of the time and CPDN is not whitelisted, I doubt it is worth my while joining.
ID: 65777 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 307
Credit: 2,381,687
RAC: 301
Message 65778 - Posted: 6 Aug 2022, 16:32:56 UTC

It would only get whitelisted if it has regular work. And I'm guessing that means for Windows aswell.
ID: 65778 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 148
Credit: 1,883,629
RAC: 252
Message 65884 - Posted: 19 Aug 2022, 16:06:38 UTC - in response to Message 65768.  

Hi Les,

Just browsing the forums and came across this message. I'm probably not well known on the forums. I'm retired as of last year but I worked at ECMWF and developed OpenIFS@home with Andy & Sarah.

OpenIFS is a very memory intensive model. That's in part because it's very effficient, it's designed to run as fast as possible for weather forecasting applications.

At the moment we've only run OpenIFS as a single-threaded application and at low resolutions that are not (yet) scientifically all that useful. I've been testing a multi-core version as we'd like to increase the model resolution available for OpenIFS but this implies significant increases in peak RAM used by OpenIFS. Since we are bottlenecked on RAM, we might as well make use of additional cores to speed up the model.

Processor speed is not that relevant here, what is though is available core count. IFS (and OpenIFS) is a highly parallel model. If I enable threading, there is a 2x speedup with 2 cores, 3.5x with 4 cores etc. I plan to release some tests on the dev site which will try out multi-core, higher model resolution (and hence memory) after the summer with assistance from the CPDN team.

Since network upload is the bottleneck (and maybe upload server capacity at the remote end), available local disk space is not really a big deal.

Cheers, Glenn
ID: 65884 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3560
Credit: 10,761,686
RAC: 5,528
Message 65891 - Posted: 19 Aug 2022, 17:55:57 UTC

Looking forward to these!
ID: 65891 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,744,749
RAC: 7
Message 65892 - Posted: 19 Aug 2022, 18:32:28 UTC - in response to Message 65884.  

Thank you. We go from knowing almost nothing, to quite a bit.

I can do lots of cores, and have a good cable modem, so send them out.
And 64GB is no problem, and I can do 128GB if you need it. It is just sitting in the drawer anyway.
ID: 65892 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 148
Credit: 1,883,629
RAC: 252
Message 65895 - Posted: 19 Aug 2022, 20:35:18 UTC - in response to Message 65892.  

Andy B looked at available machines signed to CPDN with > 16Gb RAM assigned to BOINC and there were over 1000 which is a good number for higher resolution experiments. I also forgot to add I agreed with one of the other posts that mentioned energy costs too.

We need to do some tests to find a happy place between model resolution, wall-clock time and model output size for running in the CPDN framework.

BTW, does anyone know how to get alerts or emails when a new reply/post appears to a thread I am subscribed to? It doesn't seem to be working for me, I have to go to the forum page to check.

Thx.
ID: 65895 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 307
Credit: 2,381,687
RAC: 301
Message 65896 - Posted: 19 Aug 2022, 20:43:44 UTC - in response to Message 65895.  

BTW, does anyone know how to get alerts or emails when a new reply/post appears to a thread I am subscribed to? It doesn't seem to be working for me, I have to go to the forum page to check.
They're working for me. Have you got "immediately by email" ticked in here? https://www.cpdn.org/edit_forum_preferences_form.php

If you have, whitelist cpdn.org in your spam filter or ISP's spam filter. I had to do that for another project.
ID: 65896 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7628
Credit: 24,240,330
RAC: 0
Message 65897 - Posted: 19 Aug 2022, 23:08:06 UTC - in response to Message 65884.  

Hi Glenn

That's info's good to know. Time to get ready for them.

I remember when the first ones appeared, at the time of the announcement on our front page: Oxford e-Research Centre teams up with ECMWF to enable scientists to run the OpenIFS weather model on thousands of people’s home computers.
I only got one task from that, which ran for about 18 hours.

Les
ID: 65897 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 148
Credit: 1,883,629
RAC: 252
Message 65898 - Posted: 19 Aug 2022, 23:10:55 UTC - in response to Message 65896.  

BTW, does anyone know how to get alerts or emails when a new reply/post appears to a thread I am subscribed to? It doesn't seem to be working for me, I have to go to the forum page to check.
They're working for me. Have you got "immediately by email" ticked in here? https://www.cpdn.org/edit_forum_preferences_form.php


Yes, that's probably it. I had no email set. We'll see how much of a good idea that was... :)
ID: 65898 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 148
Credit: 1,883,629
RAC: 252
Message 65899 - Posted: 19 Aug 2022, 23:13:02 UTC - in response to Message 65897.  


I remember when the first ones appeared, at the time of the announcement on our front page: Oxford e-Research Centre teams up with ECMWF to enable scientists to run the OpenIFS weather model on thousands of people’s home computers.
I only got one task from that, which ran for about 18 hours.


Yes, that was a short forecast (about 10 days I think). There are some seasonal length runs coming which will take much longer, hence the push for multiple cores.
ID: 65899 · Report as offensive     Reply Quote
Mr. P Hucker

Send message
Joined: 9 Oct 20
Posts: 307
Credit: 2,381,687
RAC: 301
Message 65900 - Posted: 19 Aug 2022, 23:56:51 UTC - in response to Message 65899.  

Yes, that was a short forecast (about 10 days I think). There are some seasonal length runs coming which will take much longer, hence the push for multiple cores.
I suppose it's too much to ask for GPU support :-)
ID: 65900 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3560
Credit: 10,761,686
RAC: 5,528
Message 65901 - Posted: 20 Aug 2022, 7:05:23 UTC - in response to Message 65900.  

Yes, that was a short forecast (about 10 days I think). There are some seasonal length runs coming which will take much longer, hence the push for multiple cores.
I suppose it's too much to ask for GPU support :-)
And worth remembering that OpenIFS is Linux and Mac only. I am not aware of any plans to add Windows to the mix.
ID: 65901 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 148
Credit: 1,883,629
RAC: 252
Message 65906 - Posted: 20 Aug 2022, 20:11:57 UTC - in response to Message 65901.  
Last modified: 20 Aug 2022, 20:12:32 UTC

I suppose it's too much to ask for GPU support :-)
And worth remembering that OpenIFS is Linux and Mac only. I am not aware of any plans to add Windows to the mix.
ECMWF and external colleagues have been testing GPUs in IFS for many years, since they first looked potentially useful for speeding up the forecast. The problem with GPUs is the relative cost of moving data on & off them. i.e. they perform best when a lot of computation is needed on large data arrays. Unfortunately because of the algorithms used in IFS this limits their use to select parts of the model.

There is currently no GPU part in the operational IFS model and since OpenIFS is based on the operational model (not the 'in-development' model) it will be sometime yet, assuming IFS does eventually use them for operational forecasts. To be honest though, I'd rather not have to manage GPU use in a CPDN context with the myriad of graphics cards out there. The 'low-hanging fruit' is to make use of the highly parallel nature of IFS and all the spare cores available on home machines.

As for Windows, I've talked with the CPDN team about creating a virtual machine for OpenIFS that will run on Windows, but that's lower down the priority list at the moment. As for macs, I'm afraid I don't have access to a mac with the new chips, so we'd probably go with a VM for that as well.

If you are interested there are technical reports here (apologies, I only had a quick google search and couldn't find anything more of an easy read):

https://www.ecmwf.int/en/elibrary/17320-openacc-gpu-adaptation-ifs-cloud-microphysics-scheme
https://www.ecmwf.int/en/elibrary/18628-accelerating-weather-prediction-nvidia-gpus
https://www.ecmwf.int/sites/default/files/elibrary/2018/18628-accelerating-weather-prediction-nvidia-gpus.pdf
https://events.ecmwf.int/event/169/contributions/2767/attachments/1460/2636/HPC-WS_Lapillonne.pdf

ECMWF's new supercomputer in Bologna includes GPUs on each node for application development, see: https://www.ecmwf.int/en/about/media-centre/news/2022/taking-ecmwfs-new-high-performance-computing-facility-operation

Hope that's useful, happy to answer any questions.
ID: 65906 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 148
Credit: 1,883,629
RAC: 252
Message 65907 - Posted: 20 Aug 2022, 20:17:58 UTC - in response to Message 65906.  

Quick question, out of interest. How many cores on your machines to you make available to CPDN/BOINC?

I ask because we considering what to create for OpenIFS; thinking 2 & 4 core versions but maybe people on this thread have some thoughts?

Thanks.
ID: 65907 · Report as offensive     Reply Quote
1 · 2 · 3 · 4 . . . 5 · Next

Message boards : Number crunching : Hardware requirements for upcoming models

©2022 climateprediction.net