Richard, what you say must apply to all OSs. We will need to wait a day or two and check again.

Here's a suitable mine-canary: host 788878

Since we had this conversation, that mine-canary has downloaded new work, completed it, reported it, and got credit for it. 1,982.64 credits per task, exactly as before. I think we can reassure Billy that there is no reduction in credits, provided he allows the servers enough time to finish working through all their processes.

Finally I have cleared all tasks from my account.

It is strange that the last 3 hadam3p tasks have all completed to time step 72,000, but should have finished on 72,096 (as they had done previously).
Has any vital information at the end of the task failed to be sent in?

Please put my mind at rest, and I will crunch no more of these tasks until I get an answer in case it is wasted computing time.

Keith

It is strange that the last 3 hadam3p tasks have all completed to time step 72,000, but should have finished on 72,096 (as they had done previously).
Has any vital information at the end of the task failed to be sent in?

If it were just the two recent tasks, then the advice would be to wait for the straggler to arrive. But one of the tasks shows a report date of 26 Jun 2009 3:02:46 UTC, which is far too long. Also, the stderr out dump shows:

Unable to load library hadam3p_se_6.07_i686-apple-darwin.dylib
dlopen error: 3153509

The hadam3p_se_ tasks is the post-processing task, so that doesn't look good.

Thanks for the info, which will be passed onto the programmers. It looks like you're still beta-testing ...

Iain

Zombie's test result on the beta site had the last trickle, and also no missing library file.
If this was part of the Sherlock Holmes series, it would probably be called:
"The curious case of the missing library application"

Zombie's test result on the beta site had the last trickle, and also no missing library file.

Actually, the ones I can find are missing the final mini-trickle and are missing the library as well: here.

Looks like we jumped the gun on this one. :-(

My mistake. He has his computers hidden, so I used the link that he provided here, which actually goes to a 6.06 model on the main site. I didn't notice that.

He did provide a link to a beta model near the start of that thread, which is one that's on your list.
But now the beta server isn't responding.

We need more Mac users to beta test, who WILL post results and comments, and leave their computers visible.

Have to wait for Monday. :)

I don't think the missing last trickle is related to the missing library file. The occurrence of these two things in one model is probably pure coincidence.

I ran a beta Windows HadAM3P that missed its last trickle: it's here. It had no report of a missing library file. The stderr out messages about file conversion errors are something different that Tolu corrected during beta testing.

However, all the beta models that uploaded the last trickle and got 1982 credits have model graphs whereas the model with the missing last trickle that only got 1980 credits has no graphs. This is also the case with Windows main project HadAM3Ps and can be seen in one of my main project tasks here. There are no graphs for the model that only got 1980 credits.

The matter of missing last trickles was discussed on the independent forum here. I said in my post there that we might need to ask Tolu about the missing last trickle but I can't remember whether I or anyone else ever did. If mustn't have contacted him because either he or I would have added a post in that thread.

---

Cpdn news

I ran a beta Windows HadAM3P that missed its last trickle: it's here. It had no report of a missing library file. The stderr out messages about file conversion errors are something different that Tolu corrected during beta testing.

I had thought that the problem was just a 'packaging' problem on the Mac: add the required library back into the Mac distribution and everything would be hunky dory. Perhaps it's more subtle than that.

However, all the beta models that uploaded the last trickle and got 1982 credits have model graphs whereas the model with the missing last trickle that only got 1980 credits has no graphs. This is also the case with Windows main project HadAM3Ps and can be seen in one of my main project tasks here. There are no graphs for the model that only got 1980 credits.

How peculiar. (Model here - the 'all results' view seems not to work for other people.)

The matter of missing last trickles was discussed on the independent forum here. I said in my post there that we might need to ask Tolu about the missing last trickle but I can't remember whether I or anyone else ever did. If mustn't have contacted him because either he or I would have added a post in that thread.

Zydor's trickles mostly turned up, but at least one didn't.

The file http://climateapps2.oucs.ox.ac.uk/cpdnboinc/download/hadam3p_se_6.07_i686-apple-darwin.zip is on the server. That should have been downloaded and unzipped to projects/climateprediction.net by BOINC.

It contains the files hadam3p_se_6.07_i686-apple-darwin.dylib, globe.jpg, libifcoremt.dylib, libimf.dylib, libintlc.dylib, libirc.dylib and libsvml.dylib plus the txf directory (containing the required font files). They're all dated 11th June 2009.

The only difference I can see with zombie67's beta tests and the ones with the missing library errors is the BOINC version number - zombie67 tested with 6.6.20, Keith and the anonymous user on Beta are using 6.6.36.

---

"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer

I've trawled through a batch of HadAM3P workunits looking for what's happening on other Macs.

http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=9091274
BOINC 6.6.36, missing last trickle, no graphs, missing library file.
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=9102559
BOINC 6.6.20, missing last trickle, no graphs, missing file.
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=9102709
idem, on same computer.
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=8898056
BOINC 6.6.36, missing last trickle, no graphs, missing file.
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=9104883
BOINC 6.2.18, missing last trickle, no graphs, missing file.
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=9102998
BOINC 6.6.36, missing last trickle, no graphs, missing file.
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?resultid=8898081
BOINC 5.10.45, missing last trickle, no graphs, missing file.


I'm not going to hunt for any more. That should be enough to show that the same thing is probably happening to all the Macs running HadAM3P v.6.07 and that the BOINC version is not the cause.

---

Cpdn news

Assuming that all of the hosts with the problem do have hadam3p_se_6.07_i686-apple-darwin.dylib in the projects/climateprediction.net directory (only the owners of those hosts can confirm that) that only leaves one possibility. Something in the BOINC API (possibly related to the way BOINC has been installed?) is causing post-processing to be started in the wrong directory.

Historical note: when applications were first ported over to BOINC v6 a lot of tasks were failing with file access problems (particularly during post-processing) because something in the API was leaving the application in the wrong working directory (slots/<n> instead of projects/climateprediction.net).

---

"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer

A fix has been released to beta today. Some Mac beta testers will be running those for a few days and the tested version will then be made available here if everything works.

(Mac beta testers are in short supply: if anyone is feeling brave, the beta site is here.)

I thought new account creation on the CPDN Beta project had been suspended because of an influx of spammers and also because of so many members with either no intention of attaching a computer or no intention of reporting problems in the models, or no experience of CPDN models (some people were attaching to CPDN and CPDN Beta on the same day), or no intention of sorting out problems within their own computers. None of what I've just said is likely to apply to anyone who's taken the time and trouble to read this thread.

If new Beta account creation is as I hope disabled we need to get a system running very quickly whereby, for example, prospective Beta members could send a PM to a moderator who'd have a quick look at their CPDN crunching experience and send them an invitation code.

---

Cpdn news

If new Beta account creation is as I hope disabled we need to get a system running very quickly whereby, for example, prospective Beta members could send a PM to a moderator who'd have a quick look at their CPDN crunching experience and send them an invitation code.

Tolu set Beta registration to require an invitation code on 27th April.

---

"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer

I am crunching 2 HADAM3P tasks starting today, after completing all previous tasks in progress.
I detached and restarted BOINC before doing so.
I would have liked to become a BETA tester if that would help, but do not seem to be able to atach to that project.

My account shows I had 7 successful completed tasks in April with 1982.64 credit using v6.06.
All recently completed tasks using v6.07 completed at 72,000 with the lower credit of 1,980.00.
(I previously stopped crunching these types of tasks because of the fortran compiler conflict with v6.06.)

As soon as completed, I will report the results. (Expected finish at 72,000.)

Keith

I would have liked to become a BETA tester if that would help ...

Thanks, Keith. We're trying to figure out a way of opening the Beta project to interested participants without letting in the usual crowd of spammers.

MOV, Iain, Thyme & Les

At present the 2 HADAM3P tasks are 75% completed.
The problem library "se" file seems to be present as is shown by the following list of files:-

1985_12_init.gz
1995_12_init.gz
globe.jpg
hadam3p_6.07_i686-apple-darwin
hadam3p_data_6.07_i686-apple-darwin.zip
hadam3p_graphics_6.07_i686-apple-darwin
hadam3p_nc3t_1985_2_006094539
hadam3p_nc3t_1985_2_006094539.xml
hadam3p_nc3t_1985_2_006094539.zip
hadam3p_nc3v_1995_2_006094541
hadam3p_nc3v_1995_2_006094541.xml
hadam3p_nc3v_1995_2_006094541.zip
hadam3p_se_6.07_i686-apple-darwin.dylib
hadam3p_se_6.07_i686-apple-darwin.zip
hadam3p_um_6.07_i686-apple-darwin
hadam3p_um_6.07_i686-apple-darwin.zip
hadcm3_40.png
hadcm3_banner_290.png
hadcm3_ss_290_1.png
hadcm3_ss_290_2.png
hadcm3_ss_290_3.png
HadISST_SI_N96_1985_12_1988_01.gz
HadISST_SI_N96_1995_12_1998_01.gz
HadISST_SST_N96_1985_12_1988_01.gz
HadISST_SST_N96_1995_12_1998_01.gz
ic19610824_12_N96.gz
ic19611019_16_N96.gz
libifcoremt.dylib
libimf.dylib
libintlc.dylib
libirc.dylib
libsvml.dylib
slideshow_hadcm3_00
slideshow_hadcm3_01
slideshow_hadcm3_02
slideshow_hadcm3_03
slideshow_hadcm3i_00
slideshow_hadcm3i_01
slideshow_hadcm3i_02
slideshow_hadcm3i_03
so2dms_N96_1985_12_1988_02.gz
so2dms_N96_1995_12_1998_02.gz
stat_icon
trickle_up_hadam3p_nc3t_1985_2_006094539_0_1247963099.xml
trickle_up_hadam3p_nc3t_1985_2_006094539_0_1247982766.xml
trickle_up_hadam3p_nc3v_1995_2_006094541_0_1247962674.xml
trickle_up_hadam3p_nc3v_1995_2_006094541_0_1247982587.xml
txf

Should complete tasks in another 2 or possibly 3 days.
Let me know if any of you want me to do anything further in next 2 days before completion.

It seems the "se" files are not missing at the 75% mark of the crunching cycle.

Keith

On making an examination of the top computers it can be seen that the problem of finishing at time step 72,000 instead of 72,096 on HADAM3P tasks has been occurring on version 6.06 as well as version 6.07, not only on Mac OSX but on other PC OSs too.

So, the problem is not confined to computer type, nor to the version of HADAM3P processing.

Would it not be wise to stop sending these tasks for crunching until this problem is fixed as I understand the last time step is a vital part of the result.

Keith

Tolu's Beta run of the updated Mac science application has finished successfully, here. I imagine he'll move the application over to this site early next week, so that'll be the Mac sorted.

[Edit: Although I've found the odd Windows HADAM3P that's crashed during post-processing and therefore stopped at 1980.0 credits, I can't find any that are doing that repeatedly, nor Linux either. Have you found one?]

Tolu's Beta run of the updated Mac science application has finished successfully, here. I imagine he'll move the application over to this site early next week, so that'll be the Mac sorted.

[Edit: Although I've found the odd Windows HADAM3P that's crashed during post-processing and therefore stopped at 1980.0 credits, I can't find any that are doing that repeatedly, nor Linux either. Have you found one?]

Yes, Iain

As I said in my last post, a quick glance at the top computers will show you that the majority of them are having HADAM3P tasks stopping short of 72,096 (at 72,000) excluding the last time step, which includes the vital information. This is not confined to version 6.07, nor is it confined to Mac OSX.

It is easy to identify the problem tasks that are "successfully" completing their results too early at 72,000.
They do not crash, but do not do the final vital report.

Keith