climateprediction.net home page
Posts by Dave Roberts

Posts by Dave Roberts

1) Message boards : climateprediction.net Science : Climate change in the News (Message 62134)
Posted 19 Feb 2020 by Dave Roberts
Post:
Re Isambard 2.
On top of the news by rbpeake , Message 62128
----------------------------------------------------------------------------------------------------------------------------------------------------
"Met to get a new supercomputer. https://news.sky.com/story/met-office-to-get-16312bn-supercomputer-to-improve-weather-and-climate-forecasting-11936162[/code]" -

1.2 billion investment has been confirmed for a state-of-the-art supercomputer to improve severe weather and climate forecasting,

A supercomputer which can forecast weather and climate 18 times faster than current technology is set to be developed. Data from the supercomputer will be used to inform Government policy as part of leading the global fight against climate change and meeting net zero emission targets.

Predicting severe weather and the impacts of climate change will be faster and more accurate than ever before, thanks to confirmation of £1.2 billion government funding to develop a state-of-the-art supercomputer, Business and Energy Secretary and COP26 President Alok Sharma announced today (17 February 2020).
-----------------------------------------------------------------------------------------------------------------------------------------------------
That's quite something.

Having said that, I have some sympathy with critics :-

But Clara Goldsmith from the Climate Coalition, the UK's largest group dedicated to action against global warming, says that while investing in such technology is important, "what's just as important is that we invest in solutions to climate change that will stop this problem getting worse".
and
But Eric Brotherton, from Ribchester in Lancashire, believes the money would be better spent now helping people like him who are repeatedly left to protect their own homes from flood water.
"£1.2bn on a computer?" he said. "No... It's got to be people first then the computer in my opinion."
2) Questions and Answers : Unix/Linux : Choice of Unix distro? (Message 62068)
Posted 30 Jan 2020 by Dave Roberts
Post:
Hi All,
With CPDN seeming to favour Unix based models in the short/medium term future, I thought I look at using my VMWare Fusion software to advantage.
So, I looking to run a Unix distro in VMWare Fusion on my Mac. Possibly with the intention of moving away from OSX as my main OS (Lengthy story) altogether in the future.
My first thoughts were to look at those distro’s were the most Mac like. (Strong personal preference over Windows.)
Initially, the one that drew my attention the most was ‘elementary OS’ especially since it was a Unbuntu variant.
Then I found that it appeared to be resource intensive in a VM environment, certainly in Virtual Box, although I couldn’t find any reference to VMWare Fusion, which is what I use.
Second variant was ‘Deepin’, although I hadn’t heard of it before.
I’d be grateful for any thoughts/advice/recommendations in this area.

PS My attachment to Fusion is because I run Windows 7 in Fusion with incredibly low overhead. ( On both iMac & Macbook Pro)
3) Message boards : Number crunching : No new tasks for over a month (Message 60883)
Posted 29 Aug 2019 by Dave Roberts
Post:
"I can see that others are still working on tasks, but I haven't been assigned any new tasks sine June. Any suggestions on what is going on?"

CPDN batches of work come mainly from projects of research groups around the world and none have been submitted for a while.

Incidentally, work for us volunteers, from CPN, is not dependent on Oxford academic terms, neither is routine maintenance and/or development of the system. Academic terms are simply there for the undergraduate student teaching activities at the university.
The other work of the university, research, administration etc., carries on in exactly the same way as work in other parts of the economy with holiday times in no way much different.

As the CPND website says, “CPDN is a team of 13 climate scientists, computing experts and post graduate students working on this project, as well as our partners and collaborators working at other universities, research and non-profit organisations around the world.”
The CPDN project takes place within the the Oxford e-Research Centre, which is a multidisciplinary applied research department.

These people work ‘normal’ hours and are not simply there only during university ‘student’ terms. A small caveat being that, as is normal in academic activities, some of these people may be away, at different times, on conferences or other activities related to their work.

We should also remember that these people have work associated with other projects. (Funding issues?)

A very large proportion of the work that is put out by CPDN are projects from researchers around the world which take time to be designed and set up as well as the results analysed and prepared for publication.

There’s not much way in which CPDN (staff or volunteer administrators) can get to know when or from where such work is likely to come.

We just have to be patient.
4) Message boards : Cafe CPDN : This climate change has results beyond the physical .... (Message 60648)
Posted 12 Jul 2019 by Dave Roberts
Post:
There's more than a slight chance that anthropogenic global warming could trigger a runaway scenario ending in global conditions that would last millions of years.
eg Release of all the methane in the Tundra.

Look at the Eocene conditions for comparison, lasting 23 million years.

Once planetary climatic conditions become extreme, there are no quick solutions even in geological timescales. Recovery from the KT impact was in the order of 4 to 9 million years.

At the opposite end of the spectrum, the two Snowball Earth scenarios both lasted 10 million years.
5) Message boards : Number crunching : Certificate authentication (Message 60627)
Posted 10 Jul 2019 by Dave Roberts
Post:
Thanks Les.

Well, The problem vanished this morning and the trickle uploaded.

The problem has been there for a few weeks but I hadn't noticed it as all zips were going up OK so hadn't bothered to look at the log file.

A strange one because I initially thought it was my internet connection but obviously not since zips were OK at the same time that I was getting the message.

I'm on BOINC 7.14.2 so that wasn't the problem.

I'll reply on the BOINC board as well.
6) Message boards : Number crunching : Certificate authentication (Message 60620)
Posted 8 Jul 2019 by Dave Roberts
Post:
Thanks Les
7) Message boards : Number crunching : Certificate authentication (Message 60618)
Posted 8 Jul 2019 by Dave Roberts
Post:
Windows 7 in VirtualBox - latest version of BOINC
One of my tasks :-

wah2_sam50_n301_199312_24_822_01187763_0

is having a problem uploading a trickle with the message :-

Climateprediction.net | scheduler request failed: Peer Certificate cannot be authenticated with given CA certificates

No other associated messages such as "Transient error" message

Zips uploading OK.

Anyone seen this before - can't find anything on the boards relating to trickles.
8) Message boards : Number crunching : Upload failures (Message 60581)
Posted 4 Jul 2019 by Dave Roberts
Post:
I do believe that Les understands the figures. It seems to me that he was simply replying to those who do come onto the boards but ignore his advice and continue to complain.
Actually there have been 5410 visit to this post, at this point, which is a lot more than the no. of replies so it would appear that many people probably have seen the post and may very well have acted accordingly.
9) Message boards : Number crunching : Upload failures (Message 60579)
Posted 4 Jul 2019 by Dave Roberts
Post:
CPDN has a track record of computation errors when suspending tasks so I sure ignored that part. I don't blame any one else for it either.


I have two dual booted Mac systems and a Dell with Windows 7 all of which are run

ning CPDN. the two Macs have Fusion virtual machines running CPDN under Windows 7. (When there are no native MAC jobs available)

I have to swap between the OSX versions on the Macs relatively frequently and have never had any problems with errors of any sort except once at the very beginning when I started to do this. I've now been doing this for years whilst testing variations of RCA software.

The one problem I had at the beginning, occurred when shutting down Fusion before suspending CPDN which was in the middle of a Zip upload.

Since than I've always made sure there are no uploads running before suspending CPDN and only then suspending Fusion. (I always suspend the tasks before suspending the project although I don't have any obvious reason to think that this is strictly necessary.)

Also, I've never had a problem on the Dell with suspending, after taking similar suitable precautions.
10) Message boards : Number crunching : Upload failures (Message 60560)
Posted 2 Jul 2019 by Dave Roberts
Post:
Thanks Jim. So WCG is an IBM CLOUD based operation using BOINC in the same way as CPDN on behalf of a group of research projects which come & go. As you say, a different level of resources.
11) Message boards : Number crunching : Upload failures (Message 60549)
Posted 2 Jul 2019 by Dave Roberts
Post:
Hi Jim, What sort of advantage does WCG offer volunteer computing over connecting via BOINC directly, since one gets connected to BOINC anyway when joining WCG.
12) Message boards : Number crunching : Upload failures (Message 60547)
Posted 2 Jul 2019 by Dave Roberts
Post:
This is nothing special! World Community Grid get hammered with several TB every day with more computers than CPDN... even more during Pentathlon


I'm not sure that this is comparable.
WCG isn't a single project and uses BOINC to link to projects just as one can do via BOINC directly. So the total upload to WCG is spread over all those projects that are currently in operation. I would assume uploads to a WCG project go directly to the computers run by the project, just as in the case of CPDN, not to some mega computing site.
The 128 Mb files that Dave mentions, appear to have been for a one off completed project, not at all similar to the continuous (more or less) streams of CPDN research projects.
13) Message boards : Number crunching : Upload failures (Message 60031)
Posted 27 Apr 2019 by Dave Roberts
Post:
No problems with uploading SAM50's but not had any EU25s :- W1ndows7.
14) Message boards : Number crunching : Project Losing it's Way? (Message 60012)
Posted 25 Apr 2019 by Dave Roberts
Post:
My main concern about CPDN is that it is just spinning numbers for the case of academic studies of past weather events. If they want to keep me, they will make the studies more relevant to the real world.


Just noticed this comment by Jim1348 and hope the following might be useful.

Running simulations over past times is absolutely critical in insuring that the models are as theoretically sound as possible when using them to simulate the future. They really are relevant to the real world

From the empirical data that we have acquired about past climate events and weather, we can propose a theoretical model of the worlds (area) climate which we can then attempt to verify by running it from a known starting point in the past to a known finishing point sometime later.

We can then check to see how closely the results from a run of the model match the known data.

The closer the model results are to the actual data, the more confident we can be that the model is a fair representation of that reality and will simulate the future reasonably accurately.

We often have to have a series of simulations of the past, refining the model both in terms of the theory and its parameters until its results are within reasonable and predefined probabilistic limits of (un)certainty.

It's from this point that we can play around with the model, both theory and parameters, to test the effects of human activity on the future climate.

It's a never ending challenge and generally speaking, every run is useful, even if it 'fails'.
15) Message boards : Number crunching : Android/ARM processors (Message 59671)
Posted 23 Feb 2019 by Dave Roberts
Post:
As Les points out, some people apparently don't bother to read the information available. The list of projects shows quite clearly which project supports which platforms and I did find that quite a few support Android on this URL.
https://boinc.berkeley.edu/projects.php
16) Message boards : Number crunching : New Model Type HadAM4 (Message 59615)
Posted 12 Feb 2019 by Dave Roberts
Post:
RE - Leaving in memory - Jean-David Beyer

"I always leave Leave non-GPU tasks in memory while suspended checked, but "in memory" just means it can be paged in very quickly, not that it is physically in RAM all the time."

Are you sure about that? I might be wrong but all the evidence I can find seems to point to a task state physically staying in system memory when suspended with a 'Leave'. Obviously, in unchecked, suspended tasks are removed from memory, and resume from their last checkpoint. Hence the problems.

Since paging is virtual memory methodology, it would only occur if the system needed more RAM than was available for some process. I wouldn’t have thought that leaving a task state in memory would trigger paging of that state if another task needed more memory. Rather that elements of the the second task would be paged.

Obviously, when the machine is switched off everything in RAM is lost but the system copies the RAM state to disk to be read back on on startup.

Having said all that, I’m always willing to be corrected.

Out of interest, I have used 'Leave non-GPU tasks in memory while suspended' from shortly after I joined CPDN, having had a task crash after a suspend and restart of my Mac. Since then, I have never had a problem, even when running CPDN in a Virtual Machine.

I sometimes have to switch between dual booted systems so the 'Leave' option is invaluable. I still take care to ensure I don't do this if trickles are pending or uploads are occuring, although pending zips don't appear problematic.

I have Mac systems.
17) Message boards : Number crunching : Second project added when climateprediction.net added to BOINC (Message 59532)
Posted 29 Jan 2019 by Dave Roberts
Post:
Bit confused by your original question. Are you referring to the old account "old_user693724" as a project?
18) Message boards : Number crunching : Error while computing??? (Message 59484)
Posted 21 Jan 2019 by Dave Roberts
Post:
Some crunchers are unknowingly crashing their WUs by doing too many Suspends.


In my experience, suspending is not a problem if the option "Leave non-GPU tasks in memory while suspended" is set in the account preferences. I've never had a problem with that setting and I suspend one computer quite regularly and with this option set, the tasks also survive shutdown.

I guess it's a case of how to get the message across.
19) Questions and Answers : Unix/Linux : No work for Linux either now. (Message 59016)
Posted 17 Nov 2018 by Dave Roberts
Post:
I've always understood Dave's interpretation to be correct.
In logs of my recent tasks, there are trickles without zip uploads and also zip uploads without a prior trickle

EG
Sun 11 Nov 00:01:47 2018 | climateprediction.net | Sending scheduler request: To send trickle-up message.
Sun 11 Nov 00:01:47 2018 | climateprediction.net | Not requesting tasks: "no new tasks" requested via Manager
Sun 11 Nov 00:01:48 2018 | climateprediction.net | Scheduler request completed

Sun 11 Nov 04:51:33 2018 | climateprediction.net | Started upload of hadcm3s_n847_190012_120_771_011670374_0_r606276455_out.zip
Sun 11 Nov 04:51:34 2018 | climateprediction.net | Computation for task hadcm3s_n847_190012_120_771_011670374_0 finished

However, many zip uploads hasve been preceded by a trickle, albeit with a slight time delay.

Fri 16 Nov 12:10:49 2018 | climateprediction.net | Sending scheduler request: To send trickle-up message.
Fri 16 Nov 12:10:49 2018 | climateprediction.net | Not requesting tasks: "no new tasks" requested via Manager
Fri 16 Nov 12:10:50 2018 | climateprediction.net | Scheduler request completed

Fri 16 Nov 17:31:01 2018 | climateprediction.net | Sending scheduler request: To send trickle-up message.
Fri 16 Nov 17:31:01 2018 | climateprediction.net | Not requesting tasks: "no new tasks" requested via Manager
Fri 16 Nov 17:31:02 2018 | climateprediction.net | Scheduler request completed

Fri 16 Nov 17:32:09 2018 | climateprediction.net | Started upload of hadcm3s_su139_190012_120_771_011667641_0_r717114851_5.zip
Fri 16 Nov 17:33:49 2018 | climateprediction.net | Finished upload of hadcm3s_su139_190012_120_771_011667641_0_r717114851_5.zip

It's certainly trickles that start the credit calcs, perhaps they have a final task to see if any zips need uploading?
20) Message boards : Number crunching : Work Units for MacIntosh (Message 58912)
Posted 24 Oct 2018 by Dave Roberts
Post:
Just a thought, for what it's worth. Have you considered running a virtual machine?

I started doing this a few years ago when tasks for Macs were running a bit scarce.

I was running Snow Leopard at the time and after some research, bought VMWare Fusion on a special offer. It was exceptionally good and completed tasks in pretty much the same time as using native OSX. Also, it didn't interfere with my normal usage.

Thinking of moving to High Sierra meant I needed the new version of Fusion but since I had a very old version I had to pay the full price, so I retried the free Oracle VirtualBox.

I didn't find this very good since there appeared to a very high background overhead but it did work and was easy to install and setup.

To cut a long story short, I kept with Snow Leopard for some time but having finally decided to move to High Sierra, I bit the bullet and bought the latest VMWare Fusion.

I know it looks a bit expensive but given what we pay for decent hardware, for me at least, it's worthwhile.


PS One can always suspend/stop Fusion, if necessary, after suspending BOINC, and I've never had any problems with restarts or task failures.


Next 20

©2020 climateprediction.net