climateprediction.net home page
Posts by Glenn Carver

Posts by Glenn Carver

1) Message boards : Number crunching : New Work Announcements 2024 (Message 71077)
Posted 7 hours ago by Glenn Carver
Post:
Weather@Home East Asia 25km : new batches

New batches to be released in next couple of days. Windows only. These are similar batches to previous EAS25 but using updated input data files and some code improvements. Note these will use a new version 8.32.

The workunit names will be prefixed with: CLIM, ALL, NAT, GHG, AER, P15, P20 & P30. For interest, these mean:

CLIM - Climatological baseline batch. Model runs span years 1985 to 2010.
ALL - 'All' forcings. Model runs span 2009 to 2022. 'All' here means human induced changes to greenhouse gases (GHG) and aerosols (AER) are included i.e. 'real world' scenario.
NAT - 'Nature' run. As 'ALL' but with the human induced changes to greenhouse gases & aerosol removed.
GHG - As 'NAT' but add in changes just from greenhouse gas changes.
AER - As 'NAT' but add in changes just from aerosol changes.
The NAT, GHG & AER batches are known as 'attribution' runs, to look at relative impact of these changes over the East Asia region.

There will also be 3 future scenario batches covering years: 2089-2101. These will assume different levels of global warming:
P15 - 1.5 degree warming.
P20 - 2.0 degree warming.
P30 - 3.0 degree warming.

The number of workunits per batch is not equal but there are thousands per batch. So plenty of workunits :)
2) Message boards : Number crunching : New Weather@Home app version 8.32 to be deployed (Message 71076)
Posted 1 day ago by Glenn Carver
Post:
A heads up a new Weather@home app is going to be deployed. It'll be version 8.32 and will replace the version 8.29 currently in place for the region independent app.

This new app will be used for forthcoming East Asia 25km batches. It contains an important scientific fix compared to earlier batches which had a negative impact on the larger domains.

As it's new, it might trip any anti-virus software until it's in widespread use. It's been tested against VirusTotal but can still cause some anti-virus systems to treat it as suspicious.

The NZ25 batches will continue to use v8.24.
3) Message boards : Number crunching : Windows BOINC Development release ver 8.0.3 (Message 71075)
Posted 7 days ago by Glenn Carver
Post:
Yes, it'll be a long wait before we can confidently make a change. There's no mechanism by which I could do this in the task: if ( client_version > 8.0.2 ) do this, else do that. Be nice if there was. Unless someone knows different....
4) Message boards : Number crunching : Windows BOINC Development release ver 8.0.3 (Message 71073)
Posted 7 days ago by Glenn Carver
Post:
It's important to note that this new 'feature' in the boinc client is not enabled by default. It was found to cause problems for other projects that didn't set appropriate values of the rsc_memory_bound in the task XML as CPDN does, as our tasks need significant memory.

For that reason, the new feature has to be to enabled by the project in the XML sent with the task. CPDN have not yet tested this to see if it behaves in the way we hope it will. So don't feel a need to rush and install this new version as I doubt any other project (maybe LHC) are likely to enable this feature. For the time being, the CPDN scheduler is set to restrict the number of tasks in progress as a workaround (as LHC do). We can't change this until the new client is in widespread use.
5) Message boards : Number crunching : New Work Announcements 2024 (Message 71068)
Posted 8 days ago by Glenn Carver
Post:
If it's just for announcements then it should be closed to comments.
Eager to get new Linux work.
It's not just for announcements as sometimes we need to post updates on batches in progress. But it's clearly not the right thread for discussions on installing versions of boinc etc. (and I'm not interested in moderating).

There won't be any Linux work for some time AFAIK. The focus is on Weather@Home (windows) at the moment, expecting some more EAS25 batches soon. We'll get back to Linux once they're done.
6) Message boards : Number crunching : New Work Announcements 2024 (Message 71065)
Posted 10 days ago by Glenn Carver
Post:
Can we please keep this thread to just about New Work for 2024? Discussions on other topics are best made in new threads to make it easier to find the right information.

Thanks.
7) Message boards : Number crunching : New Work Announcements 2024 (Message 71055)
Posted 20 days ago by Glenn Carver
Post:
Just got a resent task from batch 1006 eas25 win-8.29. It is from February, still useful to run? (was thinking of shutting the box down...)
If the batch was of no use, the resends would have been turned off. But it's your machine and if you want it off, feel free to Abort.
8) Message boards : Number crunching : finish file present too long (Message 71049)
Posted 23 days ago by Glenn Carver
Post:
Ok, that is useful. Thanks Richard. At least I now understand what the error message means. The user in question is also running quite an old version of the client, ~7.11, since that error message has been edited in the latest version. But that means the client didn't have the bugfix you pointed to where the wait time was extended from 10secs to 5 mins.

So maybe the user Aborted the task, the controller process kills the model, finish file is created, but because the machine was swapping(?) the processes were slow to die and the 10 sec wait on the finish file triggered the client error. I still don't understand why this caused the task to be reported a success since it triggered the debugger but never mind. Since the code has since changed and it's not anything I can do on the CPDN side, I will leave it here.

Thanks.
9) Message boards : Number crunching : finish file present too long (Message 71046)
Posted 24 days ago by Glenn Carver
Post:
Two tasks claim to have completed successfully for the 1019 NZ25 batch but actually failed with this error:
<message>
finish file present too long</message>
... as shown by this task:
https://www.cpdn.org/cpdnboinc/result.php?resultid=22454885

Does someone who knows boinc better than me understand what triggers this? The task has clearly failed, I suspect because the model has fallen over. But then something appears to go wrong on the boinc side and triggers this error message which I've not seen before.

Both tasks fail on the same machine, which is probably a clue. Anyone familiar with this message and what exactly is causing it?

Why they got logged as successful is also an issue.
10) Message boards : Number crunching : New Work Announcements 2024 (Message 71045)
Posted 24 days ago by Glenn Carver
Post:
I can answer that! I've just had a brief (1 or 2 seconds) power outage, and everything shut down. On power up (and after waiting ages for the router to restart), I can see that the four tasks I got from this batch (v8.24 app, batch 1019 data) have picked up and restarted running from the point they'd reached.

wah v8.24 has the bug in the code that causes the failed restarts; it just happens alot less on the smaller domains, which NZ25 is. Any version after 8.24 has the fixed code.
11) Message boards : Number crunching : receiving no new tasks (Message 71037)
Posted 24 days ago by Glenn Carver
Post:
Down to 449 ready to send now.
For NZ25 yes, but there are alot more East Asia batches coming as well. There's no need to think there won't be enough Windows work in the near future.

Anyway, back to the topic of this thread. I'd say things are working ok since you have already obtained some CPDN tasks.
12) Message boards : Number crunching : receiving no new tasks (Message 71034)
Posted 24 days ago by Glenn Carver
Post:
Yes, that's right. It will keep resetting. Better to not click 'Update' at all as far as CPDN is concerned. 'No new tasks' is sometimes because the scheduler was a bit slow to respond when you made the request and got told there weren't any -- there are various processes in play behind the scenes. I don't pretend to know details but someone else might. There are plenty of workunits left, I've just checked on the machine.

If you're a new user then the system might be limiting you to 1 workunit per day.

Give it a day or two.
13) Message boards : Number crunching : receiving no new tasks (Message 71032)
Posted 25 days ago by Glenn Carver
Post:
The first thing you should do is set 'Use 50%' cpu resource, not 100%. The CPDN tasks slow down if you try to make them use all threads because they are floating-point heavy.

Are you clicking on 'Update' for the project frequently? The CPDN server had a 1hr wait-time. That gets reset back to 1hr if 'Update' is clicked before an hour is up.
14) Message boards : climateprediction.net Science : Weather forecast models show human impact on extreme weather (Message 71029)
Posted 25 days ago by Glenn Carver
Post:
Followers of weather & climate science might be interested in this article recently published on the Oxford University website:

https://www.ox.ac.uk/news/2024-05-30-first-use-weather-forecasts-show-human-impact-extreme-weather-transformational

The Nature paper it's developed from is referred to in the text.
15) Message boards : Number crunching : New Work Announcements 2024 (Message 71028)
Posted 25 days ago by Glenn Carver
Post:
New Weather@Home batch going out today. NZ25 domain, Windows only, app version 8.24.
3150 25 month tasks. Roll up roll up, they won't last long!
Please don't download and sit on a pile of unstarted tasks though...
16) Message boards : Number crunching : New Work Announcements 2024 (Message 71027)
Posted 25 days ago by Glenn Carver
Post:
Some tasks were sent out as batch 995. This was a mistake. The correct batch is 1019. If you have a task from 995 it can be aborted. Don't waste time running it as the results are not needed. It's an previously run batch.
17) Message boards : Number crunching : New Work Announcements 2024 (Message 71025)
Posted 25 days ago by Glenn Carver
Post:
New Weather@Home batch going out today. NZ25 domain, Windows only, app version 8.24.
18) Message boards : Number crunching : One of my oifs_43r3_bl_1018 taskss errored out. (Message 71024)
Posted 26 days ago by Glenn Carver
Post:
To calculate the percentage done, the controlling program (not boinc) reads one of the model log files to see what model step it's on. I think that sometimes goes wrong. When boinc and therefore the task are restarted, the file is read correctly.
19) Message boards : Number crunching : One of my oifs_43r3_bl_1018 taskss errored out. (Message 71021)
Posted 27 days ago by Glenn Carver
Post:
Which task was it? I can look at the logs and see what happened.
20) Message boards : Number crunching : Windows Task Manager - Automatically set higher priority für CPDN (Message 71018)
Posted 28 days ago by Glenn Carver
Post:
Richard's suggestion of 'Process Lasso' was rather heavyweight for me, so after a bit of online searching it's possible to reset the priorities with a powershell two-liner:

get-process -processname  "wah2rm3m2t_um_8.31_windows_intelx86"   | foreach { $_.PriorityClass = "Normal" }
get-process -processname  "wah2am3m2_um_8.31_windows_intelx86"   | foreach { $_.PriorityClass = "Normal" }
Execute this in a powershell with Admin privilege.

Note 'Normal' is case-sensitive and first letter must be capital. Choice is 'Normal', 'Above normal', 'High' (or 'Below normal' which is higher than 'Low' the boinc default). The '.exe' extension is not needed. Change the process name as required. TaskManager can be used to verify the process priority has changed. It's certainly improved my runtimes by a small percentage.

To make it even easier as a powershell script:
.\set-ProcessPriority

where the script is:
function set-ProcessPriority {
    param($processName = $(throw "Enter process name"), $priority = "Normal")

    get-process -processname $processname | foreach { $_.PriorityClass = $priority }
    echo "`"$($processName)`"'s priority is set to `"$($priority)`""
}
set-ProcessPriority "wah2am3m2_um_8.31_windows_intelx86"   "High"
set-ProcessPriority "wah2rm3m2t_um_8.31_windows_intelx86"  "High"
Save this to a file 'set-ProcessPriority.ps1'. Change 'High' to your preference.

If the script does not run, the execution policy may need to be changed:
Set-ExecutionPolicy RemoteSigned -Scope Process
which only changes it for the current session.

Credit to: https://monadblog.blogspot.com/2006/03/msh-how-to-change-processs-priority.html


Next 20

©2024 climateprediction.net