climateprediction.net home page
Posts by Speedy

Posts by Speedy

1) Message boards : Number crunching : Upload failures (Message 61286)
Posted 20 Oct 2019 by Speedy
Post:
Climate models have lots of files open, which all need saving at checkpoints.
With your computer having so many processors, it will need a VERY fast HD to keep up with all that saving when it occurs at the same time.

Thank you for pointing this out I have cut it down to working on three tasks at a time. I'm not sure but maybe when I turned my machine last night it was trying to upload a trickle message
@Speedy The error message in stderr on the task page says "The system cannot find the drive specified.". This is an error that crops up occasionally. No one knows the cause. It's not typically reproduced in the other tasks in the work unit. It may be some kind of timing issue when the model tries to write to, or read from the disk.

The error listing you pasted into your post are just because the model crashed before those monthly upload files are created. It was expecting to upload them and they were never generated. It's unfortunately not useful for finding the cause of the crash.

Thank you for explaining the error message it makes complete sense. Hopefully I will be able to complete other tasks without them crashing
2) Message boards : Number crunching : Upload failures (Message 61280)
Posted 19 Oct 2019 by Speedy
Post:
I noticed this morning after turning my computer on their is a Weather At Home 2 that failed saying the following files were absent I gather these are from batch #845
    20/10/2019 8:38:16 AM | climateprediction.net | Computation for task wah2_global_c0ey_198812_13_845_011911553_0 finished
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_2.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_3.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_4.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_5.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_6.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_7.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_8.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_9.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_10.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_11.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_12.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_13.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_restart.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent
    20/10/2019 8:38:16 AM | climateprediction.net | Output file wah2_global_c0ey_198812_13_845_011911553_0_r559669980_out.zip for task wah2_global_c0ey_198812_13_845_011911553_0 absent


Another task has been created and sent out so it will be interesting to see whether or not this one fails two

3) Message boards : Number crunching : New work Discussion (Message 61275)
Posted 18 Oct 2019 by Speedy
Post:
Some Windows work: 7160 x WAH2 global, batch #845.

Yes it was a nice surprise I managed to pick up 21
4) Message boards : Number crunching : Upload failures (Message 61189)
Posted 5 Oct 2019 by Speedy
Post:
I am sorry to hear that people are getting stuck uploads. I thought the upload situation was going to be fixed before the release of new work?
Unfortunately I am not able to comment in regards to state uploads as I have not been able to get any work. I have .read in another thread there has been Windows work but I was not one of the lucky recipients
5) Message boards : Number crunching : Credits (Message 60830)
Posted 16 Aug 2019 by Speedy
Post:
And the credit script has run again. Clearly someone has moved the weekend!

I agree it looks like it's moved to Friday New Zealand time or perhaps Thursday because when I checked on Friday morning I had credit granted for half a work unit I have completed other half is still running.
6) Message boards : Number crunching : Upload failures (Message 60755)
Posted 30 Jul 2019 by Speedy
Post:
For everyones info..

sam50 units are limited to 1 MiB/s in upload speed
anz50 units are limitet to 100 KiB/s in upload speed..

Observed sitting waiting for 98 files to upload :D

To see if you can increase your upload speed you could try changing Max files per project to 1. It may not have any effect but it's worth a try
<max_file_xfers_per_project>1</max_file_xfers_per_project>
7) Message boards : Number crunching : Upload failures (Message 60538)
Posted 2 Jul 2019 by Speedy
Post:
With the backlog been 100 TB for argument sake. It will take 16.6667 days to clear the backlog this will be done by 20th July at 6 TB a day. I just did the maths from the numbers in the previous post
8) Message boards : Number crunching : New work Discussion (Message 60402)
Posted 22 Jun 2019 by Speedy
Post:
Thank you for your explanation.
9) Message boards : Number crunching : New work Discussion (Message 60399)
Posted 22 Jun 2019 by Speedy
Post:
I have a question that I am sure are on other people's minds. How come new work gets released when certain people are having trouble uploading work unit parts/tackle to servers because they are full?
10) Message boards : climateprediction.net Science : Project announcements discussion (Message 60355)
Posted 19 Jun 2019 by Speedy
Post:

As I've hinted elsewhere, these are BIG.
But they're still being developed and tested, so I don't want to say much yet.

Thanks Les for this information. I just hope they are able to upgrade storage accordingly
11) Message boards : Number crunching : Upload failures (Message 60344)
Posted 18 Jun 2019 by Speedy
Post:
I already denied new cpdn work.

They should stop delivering new workunits until they have enough space for it!

I completely agree. But maybe when completed results get uploaded from certain models space automatically gets freed?
12) Message boards : Number crunching : Upload failures (Message 60332)
Posted 17 Jun 2019 by Speedy
Post:
Yeah, 260GB of upload is waiting …..

I would recommend suspending crunching/processing work units until you can get some of the uploads to clear. Reason being you may end up losing all of the work that you have processed
13) Message boards : Number crunching : New work Discussion (Message 60115)
Posted 8 May 2019 by Speedy
Post:
Batch 813 has now been expanded to 2200 tasks.

Out of complete curiosity were the 2200 tasks from batch 813 added to the Weather At Home 2 (wah2) application? As I write wah2 has 9202 tasks available
14) Message boards : Number crunching : Upload server is out of disk space (Message 59426)
Posted 13 Jan 2019 by Speedy
Post:
Something is moving in the right direction. I just received some credit for the task in progress. However none of my trickles have uploaded. Has anybody else noticed receiving credit without anything uploading?


Please read my post here about the difference between zips and Trickles.

Thank you for the interesting reading.
Hopefully my 12 zip files will start to upload in the next day or so so I can continue processing the work unit
15) Message boards : Number crunching : Upload server is out of disk space (Message 59413)
Posted 12 Jan 2019 by Speedy
Post:
Something is moving in the right direction. I just received some credit for the task in progress. However none of my trickles have uploaded. Has anybody else noticed receiving credit without anything uploading?
16) Message boards : Number crunching : transient HTTP error (Message 59401)
Posted 12 Jan 2019 by Speedy
Post:
Only manually.

Set it to off.
Then, when other projects need to upload files, set it to ON.

And then back to off again after they finish uploading, even if the cpdn files have started uploading.
They're not going to get anywhere anyway, so you may as well save your bandwidth.

None of this is particularly wonderful; it's just trying to make the best of a bad situation.

Thanks Les, I am on an unlimited plan. Hope the data shifting goes well. It would be useful if we had a storage indicator on the server status page. I am not sure if this is possible. Would help us know what was going on moving forward.
17) Message boards : Number crunching : transient HTTP error (Message 59399)
Posted 11 Jan 2019 by Speedy
Post:
1) Suspend each of the climate models. (This will stop yet more files from being created.)

2) Set the Network access in the BOINC Manager to OFF. (This will stop BOINC's constant attempts to upload the files.)

3) Wait patiently until lots of space has been freed up on the Upload server.
This may take until Monday.

Is it possible to set number 2) if you are wanting to work on other projects? If yes could somebody please explain how to do this
18) Message boards : Number crunching : Upload server is out of disk space (Message 59348)
Posted 8 Jan 2019 by Speedy
Post:
The project people are working at moving data off the relevant Upload server and onto various NASs.
It all backed up because every one there has been on holidays.

Thanks for the information Les. I would find it interesting watching the data being transferred.
19) Message boards : Number crunching : Upload server is out of disk space (Message 59345)
Posted 8 Jan 2019 by Speedy
Post:
We need a bigger cloud.

Do we have any idea how big the current storage capacity is? My uploads are uploading but are not being accepted. I gather there is a rather large backlog to get through with limited storage capacity
20) Message boards : Number crunching : Upload server is out of disk space (Message 59334)
Posted 7 Jan 2019 by Speedy
Post:
[Speedy wrote:]
... Unless someone has a safe recommendation on numbers. ...


Many models have a 2 GB limit for backlogged files, so pausing models before reaching that limit is a good idea.

Thanks, in that case I think I can afford to visit one for quite a while before I need to pause


Next 20

©2021 climateprediction.net