climateprediction.net home page
Problems with uploads of zip files :- Backing off ... on upload of...

Problems with uploads of zip files :- Backing off ... on upload of...

Message boards : Number crunching : Problems with uploads of zip files :- Backing off ... on upload of...
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Dave Roberts

Send message
Joined: 15 Jan 11
Posts: 175
Credit: 6,242,691
RAC: 699
Message 56801 - Posted: 8 Sep 2017, 11:22:26 UTC

Task 20652406 Windows 7

I have 2 tasks that have been trying to upload zip files for the last 5 days -
getting mesages :- eg

Started upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_11.zip 07/09/2017 21:07:13 | climateprediction.net |
Backing off 00:02:00 on upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_11.zip
08/09/2017 02:17:41 | climateprediction.net |
Started upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_6.zip
08/09/2017 02:17:41 | climateprediction.net |
Started upload of wah2_afr50_k736_201512_13_641_011202321_0_r1780376226_10.zip
08/09/2017 02:18:06 | climateprediction.net |
Backing off 04:34:31 on upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_6.zip
etc etc

Can't see any problems with the servers.
Any ideas anyone?
ID: 56801 · Report as offensive     Reply Quote
Dave Roberts

Send message
Joined: 15 Jan 11
Posts: 175
Credit: 6,242,691
RAC: 699
Message 56802 - Posted: 8 Sep 2017, 11:24:30 UTC - in response to Message 56801.  

PS Internet connection is OK, no time restrictions.
ID: 56802 · Report as offensive     Reply Quote
WB8ILI

Send message
Joined: 1 Sep 04
Posts: 161
Credit: 81,421,805
RAC: 1,225
Message 56804 - Posted: 8 Sep 2017, 12:32:08 UTC

Not an expert, but I think I remember reading that all uploads don't go to Oxford. As I remember, sometime back there was an issue with some upload server in Mexico that prevented uploads from working. If your uploads issues are with just one batch number, this might be a clue.
ID: 56804 · Report as offensive     Reply Quote
Profile JIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,053,321
RAC: 4,417
Message 56805 - Posted: 8 Sep 2017, 13:16:08 UTC - in response to Message 56801.  

All the regional models, the zips go to the Universities that contracted for them except of the last zip. If they are having trouble receiving these zips files it would not show on the Server Status page. The last zip, the restart dump used to create the next segment of the model, still goes to Oxford.
ID: 56805 · Report as offensive     Reply Quote
Dave Roberts

Send message
Joined: 15 Jan 11
Posts: 175
Credit: 6,242,691
RAC: 699
Message 56809 - Posted: 9 Sep 2017, 16:13:46 UTC - in response to Message 56805.  

Thanks for your replies. FYI, I'm getting the same problem with restart zips??

08/09/2017 19:29:11 | climateprediction.net Started upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_9.zip
08/09/2017 19:29:11 | climateprediction.net | Started upload of wah2_afr50_k736_201512_13_641_011202321_0_r1780376226_restart.zip
08/09/2017 19:29:34 | climateprediction.net | Backing off 00:06:43 on upload of wah2_afr50_k3vw_201512_13_644_011209698_0_r873231541_9.zip
08/09/2017 19:29:34 | climateprediction.net | Backing off 00:07:42 on upload of wah2_afr50_k736_201512_13_641_011202321_0_r1780376226_restart.zip
ID: 56809 · Report as offensive     Reply Quote
Dave Roberts

Send message
Joined: 15 Jan 11
Posts: 175
Credit: 6,242,691
RAC: 699
Message 56819 - Posted: 11 Sep 2017, 10:02:21 UTC - in response to Message 56809.  

Well, everything got uploaded late last night, after 8 days.

I'd previously checked all the forums to see if anyone had seen anything like this in the past. Various combinations of 'project' 'backup', 'pending', 'upload' etc.. with no luck. (But I think the searches only look at titles)

It was the restsrt zips that threw me, since, if they all went to Oxford, and no server problems had been reported there, was there another problem?

Anyway, looks like patience is the key.
ID: 56819 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2167
Credit: 64,403,322
RAC: 5,085
Message 56821 - Posted: 11 Sep 2017, 15:24:56 UTC

I e-mailed them last night about the problem and someone must have taken care of it. I didn't hear back what the issue was.

When you were searching, did you do an advanced search? The default search limit is 30 days, but you can change that to longer when searching with advanced options.
ID: 56821 · Report as offensive     Reply Quote
Dave Roberts

Send message
Joined: 15 Jan 11
Posts: 175
Credit: 6,242,691
RAC: 699
Message 56822 - Posted: 11 Sep 2017, 16:41:41 UTC

Thanks for the info geophi, I'd used an advanced search but didn't spot anything like my tasks - upload problems after successful completion - that had such a long delay before uploading properly.

There was a similar problem back in 2005, but that had a "Temporarily failed upload" message whilst my error was "Backing off... "'.

It would be interesting to know the reason for the problem.
Thanks again.
Dave
ID: 56822 · Report as offensive     Reply Quote
KANE47

Send message
Joined: 11 Sep 12
Posts: 2
Credit: 2,188,559
RAC: 0
Message 57440 - Posted: 6 Dec 2017, 3:43:48 UTC - in response to Message 56822.  

Is it safe to say this one (see log below) will get resolved with some patience?
Seems like I can download work with no probs ever.

But uploading finished work? forget about it.

Other projects via BOINC transfer up and down with no problem; just this one seems to be odd.


12/5/2017 21:25:00 | | Project communication failed: attempting access to reference site
12/5/2017 21:25:01 | | Internet access OK - project servers may be temporarily down.
12/5/2017 21:30:14 | climateprediction.net | Started upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_restart.zip
12/5/2017 21:30:14 | climateprediction.net | Started upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_out.zip
12/5/2017 21:30:17 | climateprediction.net | Temporarily failed upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_restart.zip: connect() failed
12/5/2017 21:30:17 | climateprediction.net | Backing off 05:58:54 on upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_restart.zip
12/5/2017 21:30:17 | climateprediction.net | Temporarily failed upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_out.zip: connect() failed
12/5/2017 21:30:17 | climateprediction.net | Backing off 05:47:08 on upload of wah2_eas50_a2de_201212_12_686_011362222_0_r2042822419_out.zip
12/5/2017 21:30:18 | | Project communication failed: attempting access to reference site
12/5/2017 21:30:19 | | Internet access OK - project servers may be temporarily down.
ID: 57440 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2167
Credit: 64,403,322
RAC: 5,085
Message 57441 - Posted: 6 Dec 2017, 5:52:27 UTC - in response to Message 57440.  

The server that accepts the EAS model files has been having problems for the last week or two. The computer staff is looking into it/working on it. No ETA for a fix at this time.
ID: 57441 · Report as offensive     Reply Quote
Henk Haneveld

Send message
Joined: 9 Dec 17
Posts: 2
Credit: 145,579
RAC: 0
Message 57465 - Posted: 12 Dec 2017, 8:08:12 UTC - in response to Message 57441.  

After a long time away I returned to this project only to find my old acount had been deleted (not very happy about that) and to run straight in this upload problem.

Any progress on fixing this?
ID: 57465 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,378,503
RAC: 3,632
Message 57468 - Posted: 12 Dec 2017, 8:43:50 UTC - in response to Message 57465.  
Last modified: 12 Dec 2017, 8:51:35 UTC

Hi Henk, still no news on this, I know one of the key people at Oxford is away for a few weeks so those covering may not be as familiar with what needs to be done making it take longer.
ID: 57468 · Report as offensive     Reply Quote
KANE47

Send message
Joined: 11 Sep 12
Posts: 2
Credit: 2,188,559
RAC: 0
Message 57484 - Posted: 13 Dec 2017, 23:37:43 UTC

geo and Dave -


Thanks much!

-K
ID: 57484 · Report as offensive     Reply Quote
Alex Plantema

Send message
Joined: 3 Sep 04
Posts: 126
Credit: 26,363,193
RAC: 0
Message 57488 - Posted: 15 Dec 2017, 22:59:06 UTC - in response to Message 57440.  

I also have a task from batch 686 that won't upload at all. Trickles 1 to 12, restart and out are still in the upload queue.
ID: 57488 · Report as offensive     Reply Quote
Eirik Redd

Send message
Joined: 31 Aug 04
Posts: 391
Credit: 219,888,554
RAC: 1,481,373
Message 57489 - Posted: 16 Dec 2017, 6:50:49 UTC
Last modified: 16 Dec 2017, 7:16:40 UTC

batches 686 and 690 eas50
Yeah, these batches do, and have for at least a month, possibly once accepted uploads.
But mostly never.
Trickles, yes.

What I do, is , if the sponsor of the batch doesn't respond in a few weeks to an upload problem -- either they don't know or don't care, and if the growing backlog of unwanted uploads starts to delay subprojects that do care, or fill my disk,
it's "cancel another few "eas50 wu's that were misconfigured" no biggie.
It is annoying that I have to also cancel the worthless uploads of "eas50 - 1-12 singly or one at a time even after the tasks have completed (according to BOINC)

Short form -- eas50 workunits -- whatever batch -- let them finish, but if you have a lot of unaccepted uploads for a week or two -- CANCEL the tasks and the uploads - we aren't paying for upstream misconfigs. NOT cpdn project's fault.
ID: 57489 · Report as offensive     Reply Quote
Sergey Lovtsov

Send message
Joined: 13 Nov 06
Posts: 3
Credit: 706,723
RAC: 0
Message 57502 - Posted: 20 Dec 2017, 11:58:03 UTC

Hello!

So, I can't upload 14 files. For example:

20.12.2017 14:56:25 | climateprediction.net | Started upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_1.zip
20.12.2017 14:56:25 | climateprediction.net | Started upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_2.zip
20.12.2017 14:56:28 | climateprediction.net | Backing off 05:16:10 on upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_1.zip
20.12.2017 14:56:28 | climateprediction.net | Backing off 03:14:22 on upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_2.zip
20.12.2017 14:56:29 | climateprediction.net | Started upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_3.zip
20.12.2017 14:56:29 | climateprediction.net | Started upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_4.zip
20.12.2017 14:56:32 | climateprediction.net | Backing off 03:03:50 on upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_3.zip
20.12.2017 14:56:32 | climateprediction.net | Backing off 04:07:17 on upload of wah2_eas50_a0ff_200412_12_690_011369324_0_r1426637179_4.zip
ID: 57502 · Report as offensive     Reply Quote
Alex Plantema

Send message
Joined: 3 Sep 04
Posts: 126
Credit: 26,363,193
RAC: 0
Message 57508 - Posted: 20 Dec 2017, 21:42:44 UTC

It's strange that the 12 trickles are both on the result page and in the upload queue.
ID: 57508 · Report as offensive     Reply Quote
Sergey Lovtsov

Send message
Joined: 13 Nov 06
Posts: 3
Credit: 706,723
RAC: 0
Message 57511 - Posted: 21 Dec 2017, 8:08:10 UTC

I have 6 projects, but I have problems only with this one.
ID: 57511 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,378,503
RAC: 3,632
Message 57512 - Posted: 21 Dec 2017, 8:39:14 UTC

I have 6 projects, but I have problems only with this one.


In many ways it is not surprising that this project has more problems than many others. The program files are much longer - something like a million lines of Fortran altogether. The tasks are put together by a lot of different people so more chance of getting something wrong and it isn't just the computers at Oxford that can mess things up as the zips often go to computers at the institution where the scientists who get Oxford to send the work out on their behalf are based.

In an ideal world the code which is used on license from the met office here in UK would I guess be rewritten from scratch. Certainly some bits of the Linux code would obviating the need to install 32bit libraries. However, I suspect the project will never have the resources to make that possible.
ID: 57512 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,378,503
RAC: 3,632
Message 57513 - Posted: 21 Dec 2017, 8:45:20 UTC - in response to Message 57508.  

It's strange that the 12 trickles are both on the result page and in the upload queue.


The trickles are what appear on the result page. They are different from the zip files. It is just that with recent model types they are produced at the same time. On older model types I think that there were only zips produced at the end but the trickles were still produced and allowed people to gain credit for a task they crunched for 9 months or more which still crashed before finishing.

That is the route cause of most of the problems there have been with the credit system over the years.
ID: 57513 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : Problems with uploads of zip files :- Backing off ... on upload of...

©2024 climateprediction.net