climateprediction.net home page
Disk Quota exeded.

Disk Quota exeded.

Message boards : Number crunching : Disk Quota exeded.
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,846
RAC: 3,590
Message 56346 - Posted: 9 Jun 2017, 16:02:20 UTC

Getting this on all uploads at the moment, project informed but it is 5.00pm on a Friday so space may not be freed up till Monday. - Project has been informed.

Someone must be sending too much work out!
ID: 56346 · Report as offensive     Reply Quote
BRG

Send message
Joined: 24 Dec 16
Posts: 15
Credit: 1,564,952
RAC: 0
Message 56347 - Posted: 9 Jun 2017, 16:11:04 UTC - in response to Message 56346.  

Getting this on all uploads at the moment, project informed but it is 5.00pm on a Friday so space may not be freed up till Monday. - Project has been informed.

Someone must be sending too much work out!


Having the same problem... have 17 WUs trying to upload and other adding every 15 mins!
ID: 56347 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,846
RAC: 3,590
Message 56348 - Posted: 9 Jun 2017, 17:14:00 UTC - in response to Message 56347.  

Sarah has responded to my email. So they will be moving some data off the disks involved to longer term storage. Still not sure it will happen before Monday though which means that there will be a lot of machines trying to upload seeing as virtually everyone will have all the work they can cope with at the moment.
ID: 56348 · Report as offensive     Reply Quote
Profile Iain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1079
Credit: 6,902,393
RAC: 6,787
Message 56349 - Posted: 9 Jun 2017, 18:17:09 UTC

... this appears to be just the final restart upload for my machines.
ID: 56349 · Report as offensive     Reply Quote
Art Masson
Avatar

Send message
Joined: 16 Oct 11
Posts: 254
Credit: 15,954,577
RAC: 20
Message 56350 - Posted: 9 Jun 2017, 19:42:47 UTC - in response to Message 56349.  

Yes...the normal zip trickles are uploading OK but not the final restart upload for finished WUs. Getting the following error:

6/9/2017 1:30:55 PM | climateprediction.net | [error] Error reported by file upload server: can't write file wah2_global_b2kx_199812_25_577_011039931_1_r275968347_restart.zip: Disk quota exceeded

Art
ID: 56350 · Report as offensive     Reply Quote
Art Masson
Avatar

Send message
Joined: 16 Oct 11
Posts: 254
Credit: 15,954,577
RAC: 20
Message 56351 - Posted: 9 Jun 2017, 20:35:05 UTC

I take it back. Perhaps this is function of the WU type, but on another machine all my zip trickles are now failing to upload as well with the following error:

6/9/2017 3:32:21 PM | climateprediction.net | Temporarily failed upload of wah2_global_b0py_199812_25_578_011047520_0_r952609853_23.zip: transient upload error
ID: 56351 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 24,487,746
RAC: 3,014
Message 56352 - Posted: 9 Jun 2017, 21:27:12 UTC - in response to Message 56351.  
Last modified: 9 Jun 2017, 21:28:22 UTC

6/9/2017 1:30:55 PM | climateprediction.net | [error] Error reported by file upload server: can't write file wah2_global_b2kx_199812_25_577_011039931_1_r275968347_restart.zip: Disk quota exceeded

6/9/2017 3:32:21 PM | climateprediction.net | Temporarily failed upload of wah2_global_b0py_199812_25_578_011047520_0_r952609853_23.zip: transient upload error


I got the same for a WU from 578 batch, but it passed through after the second attempt and it was successfully reported as completed. Another WU from 577 batch did not have any upload errors 2h earlier, but after completion it is still "in progress" and I may list in the dedicated thread.
ID: 56352 · Report as offensive     Reply Quote
Profile Alan K

Send message
Joined: 22 Feb 06
Posts: 484
Credit: 29,579,234
RAC: 4,572
Message 56354 - Posted: 9 Jun 2017, 22:25:32 UTC - in response to Message 56352.  

I've got one from a different batch:

09/06/2017 23:01:01 | | [http_xfer] [ID#1630] HTTP: wrote 184 bytes
09/06/2017 23:01:01 | climateprediction.net | [http] [ID#1630] Info: Connection #4319 to host upload2.cpdn.org left intact
09/06/2017 23:01:02 | climateprediction.net | [error] Error reported by file upload server: can't write file hadcm3s_825r_201412_120_564_011005106_1_r869911686_10.zip: Disk quota exceeded
09/06/2017 23:01:02 | climateprediction.net | Temporarily failed upload of hadcm3s_825r_201412_120_564_011005106_1_r869911686_10.zip: transient upload error
09/06/2017 23:01:02 | climateprediction.net | Backing off 01:37:27 on upload of hadcm3s_825r_201412_120_564_011005106_1_r869911686_10.zip
ID: 56354 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,846
RAC: 3,590
Message 56355 - Posted: 10 Jun 2017, 6:47:44 UTC

Unless things have changed, some of the zips go directly to wherever in the world the scientists who give Oxford the work are - various universities either always or on the whole. This would explain zips for some tasks going through and not others. My guess is First thing (start of working day) Monday they will start moving the data.

Meanwhile, a back of envelope calculation making a few assumptions about number of cpus per machine and speed etc. would suggest at least 500GB/day of data is building up between all the machines out there crunching so when Oxford does start accepting the data again the servers are going to get hammered. Many will get transient http errors as a result of this till things calm down.
ID: 56355 · Report as offensive     Reply Quote
Profile Alan K

Send message
Joined: 22 Feb 06
Posts: 484
Credit: 29,579,234
RAC: 4,572
Message 56357 - Posted: 10 Jun 2017, 9:35:53 UTC - in response to Message 56355.  

I've suspended network activity for the moment until things get sorted.
ID: 56357 · Report as offensive     Reply Quote
Desti

Send message
Joined: 6 Aug 04
Posts: 124
Credit: 9,195,838
RAC: 0
Message 56364 - Posted: 11 Jun 2017, 13:42:43 UTC

The wah2 583 batch gets the trickles uploaded with no problems.

11-Jun-2017 15:01:45 [climateprediction.net] Started upload of wah2_wus25_thkh_199809_25_583_011070077_1_r95718452_1.zip
11-Jun-2017 15:01:52 [climateprediction.net] Finished upload of wah2_wus25_thkh_199809_25_583_011070077_1_r95718452_1.zip
Linux Users Everywhere @ BOINC
ID: 56364 · Report as offensive     Reply Quote
DJStarfox

Send message
Joined: 27 Jan 07
Posts: 300
Credit: 3,288,263
RAC: 26,370
Message 56367 - Posted: 11 Jun 2017, 21:54:34 UTC

Got a dozen or so stuck file transfers. This hurts my other projects, but I've suspended all network activity until this is resolved.

Even if they get the file move started on Monday, I expect the transfer to take many hours. So, don't expect much free space until mid-day Tuesday.

BTW, why don't the programmers know how much data each model needs to upload? They should tell the sysadmin this information #models x upload size x #uploads. That way, the sysadmin can check if they'll run out of space before the tasks are finished.
ID: 56367 · Report as offensive     Reply Quote
Profile Alan K

Send message
Joined: 22 Feb 06
Posts: 484
Credit: 29,579,234
RAC: 4,572
Message 56373 - Posted: 12 Jun 2017, 10:21:39 UTC - in response to Message 56364.  

wus25 goes to upload5. It is upload2 that has the problems.
ID: 56373 · Report as offensive     Reply Quote
Profile Iain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1079
Credit: 6,902,393
RAC: 6,787
Message 56374 - Posted: 12 Jun 2017, 10:46:29 UTC

My blocked uploads have started clearing now ...
ID: 56374 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,846
RAC: 3,590
Message 56375 - Posted: 12 Jun 2017, 11:42:56 UTC - in response to Message 56374.  

Mine too.
Disk quota has been increased to give some headroom.
ID: 56375 · Report as offensive     Reply Quote
DJStarfox

Send message
Joined: 27 Jan 07
Posts: 300
Credit: 3,288,263
RAC: 26,370
Message 56376 - Posted: 12 Jun 2017, 13:57:49 UTC

I had about 1 GB upload this morning. :)
ID: 56376 · Report as offensive     Reply Quote

Message boards : Number crunching : Disk Quota exeded.

©2024 climateprediction.net