climateprediction.net home page
News and Announcements 3

News and Announcements 3

Message boards : Number crunching : News and Announcements 3
Message board moderation

To post messages, you must log in.

AuthorMessage
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 63276 - Posted: 5 Jan 2021, 20:46:22 UTC

Reply from project about batch 890
----------------------------------------------

Hi all,

Sorry for the delayed response over Christmas. Just to let you know that I have stopped resends going out from this batch as the failures are very high.

The experiment setup and files etc are the same as we have used for other weather@home regions so I wasn’t expecting this degree of error. I know that the South Africa region is not as stable though as other regions and I am wondering if this is what is causing the problems in this case - it would be consistent with just an initial conditions difference causing an error as Ian reported.

I will check that I picked up the best restart file to use and also what the researchers want to do.

Best wishes and Happy New Year,

Sarah
------------------------------------------------------
Dr Sarah Sparrow
CPDN programme coordinator
ID: 63276 · Report as offensive
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,380,160
RAC: 3,563
Message 63739 - Posted: 29 Mar 2021, 20:01:22 UTC

Those who keep an eye on the server status page will have seen we have anew model type listed. These are Linux only tasks.

This experiment investigates the role of the ocean in the extreme events of the very wet 2013-14 winter in Europe. In the past these studies have been done by forcing the atmospheric model with observed sea surface temperatures and sea ice. In this experiment we use a more realistic representation of air-sea interactions in the form of a Slab ocean model, where the mixed layer is forced at the surface, and from heat convergence from the slow ocean circulation. The setup allows us to evaluate the impact of air-sea coupling on extreme events. This is a the first time this configuration is run on CPDN.

Posted here so more see it but it could as easily go in the science section if another moderator thinks that more appropriate.
ID: 63739 · Report as offensive
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,380,160
RAC: 3,563
Message 64255 - Posted: 2 Aug 2021, 16:41:53 UTC

Hi All,

Just to let you all know that there will be a period of downtime for the CPDN services tomorrow, Tuesday (3rd August). The CPDN services are expected to be disrupted for the majority of the working day. This downtime have been requested by the Engineering Department IT Support in order for them to conduct work on network switches. We apologies for any inconvenience.

Best regards,

Andy Bowery
ID: 64255 · Report as offensive
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 64443 - Posted: 9 Sep 2021, 17:51:54 UTC

Hi All,

The offending RAM has now been removed and the main project and dev projects have now been started again and are up and running.

Best regards,

Andy
ID: 64443 · Report as offensive
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,380,160
RAC: 3,563
Message 64679 - Posted: 21 Oct 2021, 18:36:43 UTC

Missed hearing this earlier in the week. The Royal Netherlands Meteorological Institute says that respected Dutch climate scientist Geert Jan van Oldenborgh has died at age 59 I am sure some working at CPDN would have known him.
ID: 64679 · Report as offensive
Profile Thyme Lawn
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1283
Credit: 15,824,334
RAC: 0
Message 65194 - Posted: 23 Feb 2022, 11:41:11 UTC

Notification from the project team:

Hi All,

Just to let you know that the CPDN project is currently offline. Overnight there has been a power outage on the site where two key servers of the project are based. Engineering IT Support are aware of the issue and will be working to rectify it.

Best wishes,

Andy
ID: 65194 · Report as offensive
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2167
Credit: 64,403,322
RAC: 5,085
Message 65202 - Posted: 24 Feb 2022, 16:41:49 UTC - in response to Message 65194.  
Last modified: 24 Feb 2022, 16:42:25 UTC

Update from Andy at 1500 UTC today:

The dev site is back up and running. Engineer IT Support are currently working on restoring the power to the other server affected by the outage.

Best wishes,

Andy
ID: 65202 · Report as offensive
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,380,160
RAC: 3,563
Message 65203 - Posted: 24 Feb 2022, 18:47:38 UTC

And main site back up. I have one task currently uploading. I can't check download is working as I haven't managed to get OSX emulation working on my Linux box yet.
ID: 65203 · Report as offensive
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2167
Credit: 64,403,322
RAC: 5,085
Message 65204 - Posted: 24 Feb 2022, 18:56:47 UTC - in response to Message 65203.  

And main site back up. I have one task currently uploading. I can't check download is working as I haven't managed to get OSX emulation working on my Linux box yet.

Dave, I don't think it's back up. All the server processes are not running. It's still not accepting trickles or task completion updates. I've been able to upload the yearly files in my hadcm3s tasks, because they must be going to a different server.
ID: 65204 · Report as offensive
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,380,160
RAC: 3,563
Message 65206 - Posted: 24 Feb 2022, 20:19:21 UTC - in response to Message 65204.  
Last modified: 24 Feb 2022, 21:48:24 UTC

Dave, I don't think it's back up. All the server processes are not running. It's still not accepting trickles or task completion updates. I've been able to upload the yearly files in my hadcm3s tasks, because they must be going to a different server.


I had two zips were uploading, they had reached about 40% but I am now getting transient upload errors on them. An out.zip has uploaded successfully. But I see all the servers listed in server status are saying not running. So not sure if things were OK for a short while and there has been a glitch or whether the uploads would have got to 100% and then failed anyway.

I guess it gives me more time to play with quemu and kvm

Edit: My zips from 216 models have all uploaded now but model not reporting so some progress.
ID: 65206 · Report as offensive
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2167
Credit: 64,403,322
RAC: 5,085
Message 65210 - Posted: 24 Feb 2022, 21:51:21 UTC - in response to Message 65206.  

All my hadcm3s zips went up fine, it's just the trickles and successful completion requests that are told the server is temporarily down. Perhaps the hadcm3s zips are going up to a different server (non-Oxford) than the hadam4 zips and that's why we are seeing a difference with them.
ID: 65210 · Report as offensive
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2167
Credit: 64,403,322
RAC: 5,085
Message 65212 - Posted: 25 Feb 2022, 14:52:33 UTC

Looks like things are back up. Latest from Andy...

Hi All,

Just to let you know the main CPDN project and dev project are now back online.

Best wishes,

Andy
ID: 65212 · Report as offensive
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,380,160
RAC: 3,563
Message 66874 - Posted: 12 Dec 2022, 14:27:53 UTC

Uploads update Mon 12th

Have just heard from CPDN. There was a major failure in the cloud system they use at the weekend. It will take a day or two to move over to a new system before uploads will work again.
ID: 66874 · Report as offensive
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,380,160
RAC: 3,563
Message 68613 - Posted: 21 Mar 2023, 11:33:24 UTC

Testing site server software updated to 1.4.2 I will post again if not beaten to it when this is due to be rolled out to main site. Currently zips not showing on testing. Getting rid of obsolete task types on server status page has also been raised with Andy.
ID: 68613 · Report as offensive
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,380,160
RAC: 3,563
Message 68631 - Posted: 27 Mar 2023, 14:56:34 UTC

Hi All,

We are going to be moving the location of the main application/scheduler and database servers. For this we will need to take the project offline for 3.5 days. This works starts with a taking a copy of the database, during which we need to make sure no changes are made to the database, so we will be taking the project offline over that period. This work needs to start soon. So I will be starting this work in 1 hours time. Sorry for the short notice.

Best wishes,

Andy
ID: 68631 · Report as offensive

Message boards : Number crunching : News and Announcements 3

©2024 climateprediction.net