climateprediction.net home page
ANOTHER UPLOAD PROBLEM

ANOTHER UPLOAD PROBLEM

Message boards : Number crunching : ANOTHER UPLOAD PROBLEM
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 33 · Next

AuthorMessage
Profile JIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,053,321
RAC: 4,417
Message 49397 - Posted: 21 Jun 2014, 5:51:51 UTC

The zip file uploading problem is back. I presently have 2 hadam3p zip files stuck in my transfer tab. These 2 files are # 10 and 11.

6/20/2014 10:31:32 PM | climateprediction.net | Started upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip
6/20/2014 10:31:55 PM | climateprediction.net | Temporarily failed upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip: transient HTTP error
6/20/2014 10:31:55 PM | climateprediction.net | Backing off 00:02:36 on upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip
6/20/2014 10:31:58 PM | | Project communication failed: attempting access to reference site
6/20/2014 10:32:00 PM | | Internet access OK - project servers may be temporarily down.
6/20/2014 10:34:32 PM | climateprediction.net | Started upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip
6/20/2014 10:39:42 PM | climateprediction.net | Temporarily failed upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip: transient HTTP error
6/20/2014 10:39:42 PM | climateprediction.net | Backing off 00:04:53 on upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip
6/20/2014 10:39:45 PM | | Project communication failed: attempting access to reference site
6/20/2014 10:39:47 PM | | Internet access OK - project servers may be temporarily down.
6/20/2014 10:52:46 PM | climateprediction.net | Started upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip
6/20/2014 10:53:08 PM | climateprediction.net | Temporarily failed upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip: transient HTTP error
6/20/2014 10:53:08 PM | climateprediction.net | Backing off 00:13:16 on upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip
6/20/2014 10:53:12 PM | | Project communication failed: attempting access to reference site
6/20/2014 10:53:14 PM | | Internet access OK - project servers may be temporarily down.
6/20/2014 11:23:22 PM | climateprediction.net | Started upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip
6/20/2014 11:23:46 PM | climateprediction.net | Temporarily failed upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip: transient HTTP error
6/20/2014 11:23:46 PM | climateprediction.net | Backing off 00:18:38 on upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip
6/20/2014 11:23:49 PM | | Project communication failed: attempting access to reference site
6/20/2014 11:23:51 PM | | Internet access OK - project servers may be temporarily down.
6/21/2014 12:05:49 AM | climateprediction.net | Started upload of hadam3p_eu_f5n0_2013_1_008759104_0_10.zip
6/21/2014 12:05:49 AM | climateprediction.net | Started upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip
6/21/2014 12:06:12 AM | climateprediction.net | Temporarily failed upload of hadam3p_eu_f5n0_2013_1_008759104_0_10.zip: transient HTTP error
6/21/2014 12:06:12 AM | climateprediction.net | Backing off 03:27:42 on upload of hadam3p_eu_f5n0_2013_1_008759104_0_10.zip
6/21/2014 12:06:12 AM | climateprediction.net | Temporarily failed upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip: transient HTTP error
6/21/2014 12:06:12 AM | climateprediction.net | Backing off 00:56:41 on upload of hadam3p_eu_f5n0_2013_1_008759104_0_11.zip
6/21/2014 12:06:15 AM | | Project communication failed: attempting access to reference site



ID: 49397 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 49398 - Posted: 21 Jun 2014, 6:02:23 UTC

This is known about, they've been emailed, and it's the weekend.
I was expecting something like this to happen because it IS the weekend. :(

It's going to be a loooooong wait until next week gets here. sigh

And there's another week (at least) of the upgrading/re-arranging to go. SIGH



ID: 49398 · Report as offensive     Reply Quote
Profile Iain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1079
Credit: 6,904,049
RAC: 6,657
Message 49399 - Posted: 21 Jun 2014, 13:42:27 UTC

Well, at least the World Cup won't be a distraction. :-((
ID: 49399 · Report as offensive     Reply Quote
Lars Vindal

Send message
Joined: 8 Nov 04
Posts: 3
Credit: 994,108
RAC: 0
Message 49400 - Posted: 21 Jun 2014, 17:59:06 UTC - in response to Message 49329.  

Got same problem here, but with a different upload server than quoted below.


<file>
<name>hadam3p_eu_r858_2013_1_008755442_0_13.zip</name>
<nbytes>36821677.000000</nbytes>
<max_nbytes>150000000.000000</max_nbytes>
<md5_cksum>b4d07615d3c72c2219551c58bb075fba</md5_cksum>
<status>1</status>
<upload_url>http://cpdn-restarts.oerc.ox.ac.uk/cgi-bin/file_upload_handler</upload_url>
<persistent_file_xfer>
<num_retries>3</num_retries>
<first_request_time>1402397041.847410</first_request_time>
<next_request_time>1402402255.478571</next_request_time>
<time_so_far>1403.103088</time_so_far>
<last_bytes_xferred>36821891.000000</last_bytes_xferred>
<is_upload>1</is_upload>
</persistent_file_xfer>
</file>




My problem files:


<file>
<name>hadam3p_eu_f4wd_2013_1_008758145_0_5.zip</name>
<nbytes>37337842.000000</nbytes>
<max_nbytes>150000000.000000</max_nbytes>
<md5_cksum>c8299b7170a9c3a9af4c77e1b45dcc70</md5_cksum>
<status>1</status>
<upload_url>http://cpdn-upload2.oerc.ox.ac.uk/cgi-bin/file_upload_handler</upload_url>
<persistent_file_xfer>
<num_retries>10</num_retries>
<first_request_time>1403290251.734221</first_request_time>
<next_request_time>1403376890.309406</next_request_time>
<time_so_far>1185.701311</time_so_far>
<last_bytes_xferred>0.000000</last_bytes_xferred>
<is_upload>1</is_upload>
</persistent_file_xfer>
</file>

<file>
<name>hadam3p_eu_f4wd_2013_1_008758145_0_6.zip</name>
<nbytes>36961859.000000</nbytes>
<max_nbytes>150000000.000000</max_nbytes>
<md5_cksum>9a76cafabf4ba7992a2b303f20ae703e</md5_cksum>
<status>1</status>
<upload_url>http://cpdn-upload2.oerc.ox.ac.uk/cgi-bin/file_upload_handler</upload_url>
<persistent_file_xfer>
<num_retries>7</num_retries>
<first_request_time>1403343215.686512</first_request_time>
<next_request_time>1403379096.991080</next_request_time>
<time_so_far>664.749817</time_so_far>
<last_bytes_xferred>0.000000</last_bytes_xferred>
<is_upload>1</is_upload>
</persistent_file_xfer>
</file>



Seems like it's not just one upload server with problems.

Currently, the server status page reports uploader.oerc and cpdnupload2.oerc as being down. Would that be the same ones causing the problems reported here?
ID: 49400 · Report as offensive     Reply Quote
Mike.Gibson

Send message
Joined: 2 May 07
Posts: 20
Credit: 657,542
RAC: 0
Message 49401 - Posted: 23 Jun 2014, 15:12:52 UTC

23/06/2014 12:14:15 | climateprediction.net | [error] Error reported by file upload server: can't open file /storage/cpdn-restarts/incoming/uploader/hadam3p_eu_r5iz_2013_1_008752049_1_13.zip: No such file or directory
23/06/2014 12:14:15 | climateprediction.net | Temporarily failed upload of hadam3p_eu_r5iz_2013_1_008752049_1_13.zip: transient upload error
23/06/2014 12:14:15 | climateprediction.net | Backing off 03:47:41 on upload of hadam3p_eu_r5iz_2013_1_008752049_1_13.zip
23/06/2014 12:22:53 | climateprediction.net | Sending scheduler request: To send trickle-up message.
23/06/2014 12:22:53 | climateprediction.net | Not requesting tasks: "no new tasks" requested via Manager
23/06/2014 12:22:56 | climateprediction.net | Scheduler request failed: Error 403

This is occurring on the final upload and all 35.11 MB are sent but then aren't accepted. the same thing then happens another 3 hours later.

Any ideas, please?

Mike
ID: 49401 · Report as offensive     Reply Quote
Mike.Gibson

Send message
Joined: 2 May 07
Posts: 20
Credit: 657,542
RAC: 0
Message 49402 - Posted: 23 Jun 2014, 15:17:51 UTC

Would you believe it. It had been refusing to accept it for the best part of a day and then the minute that I posted about it, it went through!

Mike
ID: 49402 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 49403 - Posted: 23 Jun 2014, 15:22:42 UTC - in response to Message 49402.  

Jonathan has been working on the server problems that occurred over the weekend.


ID: 49403 · Report as offensive     Reply Quote
iancantwell

Send message
Joined: 19 Mar 14
Posts: 2
Credit: 36,321
RAC: 0
Message 49423 - Posted: 26 Jun 2014, 3:08:03 UTC

I finished this work unit in 128 hrs and successfully uploaded all sections (though have not received any credit updates) with the following exception

25/06/2014 22:36:17 | climateprediction.net | Started upload of hadam3p_eu_fce0_2013_1_008767852_0_1.zip
25/06/2014 22:36:25 | climateprediction.net | [error] Error reported by file upload server: can't open file
25/06/2014 22:36:25 | climateprediction.net | Temporarily failed upload of hadam3p_eu_fce0_2013_1_008767852_0_1.zip: transient upload error
25/06/2014 22:36:25 | climateprediction.net | Backing off 04:21:08 on upload of hadam3p_eu_fce0_2013_1_008767852_0_1.zip

This message is repeated every time their is an attempted upload. What do I do now? let it be or abort? If I abort do I get any credit for the work successfully uploaded?

In the meantime I have received another similar wu but have temporarily suspended until I get advice

Many thanks

ID: 49423 · Report as offensive     Reply Quote
Profile JIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,053,321
RAC: 4,417
Message 49424 - Posted: 26 Jun 2014, 4:22:52 UTC

Me too. I have the same problem. File does not upload at all.

6/26/2014 12:17:38 AM | climateprediction.net | Started upload of hadam3p_eu_f5mz_2013_1_008759103_0_1.zip
6/26/2014 12:18:01 AM | climateprediction.net | Temporarily failed upload of hadam3p_eu_f5mz_2013_1_008759103_0_1.zip: transient HTTP error
6/26/2014 12:18:01 AM | climateprediction.net | Backing off 03:08:50 on upload of hadam3p_eu_f5mz_2013_1_008759103_0_1.zip
6/26/2014 12:18:04 AM | | Project communication failed: attempting access to reference site
6/26/2014 12:18:06 AM | | Internet access OK - project servers may be temporarily down.

ID: 49424 · Report as offensive     Reply Quote
Niall

Send message
Joined: 18 Dec 13
Posts: 62
Credit: 1,078,935
RAC: 0
Message 49425 - Posted: 26 Jun 2014, 8:28:28 UTC

I'm having exactly the same problem. It's been an intermittent issue for a while, and we know there may be temporary server issues while the tech crew work on them. http://www.climateprediction.net/possible-server-issues-in-june/ The files have always cleared eventually, and see no reason why the same will not apply. The clue is in the words "transient" and "temporarily". The crunch continues.
ID: 49425 · Report as offensive     Reply Quote
Profile Alan K

Send message
Joined: 22 Feb 06
Posts: 484
Credit: 29,579,234
RAC: 4,572
Message 49426 - Posted: 26 Jun 2014, 9:24:37 UTC - in response to Message 49425.  

I am not sure - and someone will correct me if I am wrong - but doesn't BOINC keep trying to upload the files for up to 14days after the initial failure?
ID: 49426 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 925
Credit: 34,100,818
RAC: 11,270
Message 49428 - Posted: 26 Jun 2014, 10:24:40 UTC - in response to Message 49426.  

I am not sure - and someone will correct me if I am wrong - but doesn't BOINC keep trying to upload the files for up to 14days after the initial failure?

I think the time limit was extended to 90 days in newer clients, specifically because of our experiences here.

There was one occasion when to cope with the sheer volume of data, the staff had to specify, fund, order, build, deliver, install, and configure a new server to hold uploaded data. That's hard to do in 14 days.
ID: 49428 · Report as offensive     Reply Quote
Profile Iain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1079
Credit: 6,904,049
RAC: 6,657
Message 49430 - Posted: 26 Jun 2014, 14:23:21 UTC - in response to Message 49428.  

I am not sure - and someone will correct me if I am wrong - but doesn't BOINC keep trying to upload the files for up to 14days after the initial failure?

I think the time limit was extended to 90 days in newer clients, specifically because of our experiences here.

There was one occasion when to cope with the sheer volume of data, the staff had to specify, fund, order, build, deliver, install, and configure a new server to hold uploaded data. That's hard to do in 14 days.


... unless, of course, you do some capacity planning beforehand. ;-)
ID: 49430 · Report as offensive     Reply Quote
Profile Iain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1079
Credit: 6,904,049
RAC: 6,657
Message 49438 - Posted: 27 Jun 2014, 15:19:10 UTC

My recently stalled EU uploads have all now cleared. Is anyone still having problems?
ID: 49438 · Report as offensive     Reply Quote
Profile astroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 49439 - Posted: 27 Jun 2014, 16:45:39 UTC - in response to Message 49438.  

Some of my tasks uploaded okay. Then, weekend gremlins returned. Most failures are the familiar HTTP kind. A few had the 'unable to resolve...' error.

At least, when the servers were down for maintenance, there was no bandwidth overload on my DSL connection. Now, we're back to Upload/Hang at 100%/HTTP error/Backoff/Start over...
"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 49439 · Report as offensive     Reply Quote
Profile KWSN THE Holy Hand Grenade!

Send message
Joined: 9 Apr 07
Posts: 7
Credit: 1,630,807
RAC: 0
Message 49457 - Posted: 29 Jun 2014, 20:34:30 UTC - in response to Message 49438.  
Last modified: 29 Jun 2014, 20:34:58 UTC

My recently stalled EU uploads have all now cleared. Is anyone still having problems?


Yes, my final upload on several WU's has gone through, but the prior (partial results) are still stuck in "Transient HTTP error" hell. With each of these at 35 Mb, and me having (currently) 11 to upload, that's 385 Mb to upload - and my connection only gives me a 100Kb/sec upload speed! (around 5 minutes per file...)
ID: 49457 · Report as offensive     Reply Quote
Rick

Send message
Joined: 25 Mar 14
Posts: 3
Credit: 280,087
RAC: 0
Message 49460 - Posted: 30 Jun 2014, 8:10:13 UTC

Hi,
I have an upload issue. There are 16 uploads waiting and they have been there all day.
All hadam3p_eu. It appears that each time the countdown ends they go into upload retry in 04:00:00. I don't know enough about this to do anything about it. Help for a novice would be appreciated.
ID: 49460 · Report as offensive     Reply Quote
Profile JIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,053,321
RAC: 4,417
Message 49461 - Posted: 30 Jun 2014, 8:19:06 UTC

I have the same problem. I have about 15 hadam3p_eu zip files stuck in the transfer tabs of 2 machines. It is the usual transient HTTP error again.

6/29/2014 11:24:30 PM | climateprediction.net | Started upload of hadam3p_eu_f5my_2013_1_008759102_0_2.zip
6/29/2014 11:24:53 PM | climateprediction.net | Temporarily failed upload of hadam3p_eu_f5my_2013_1_008759102_0_2.zip: transient HTTP error
6/29/2014 11:24:53 PM | climateprediction.net | Backing off 00:02:21 on upload of hadam3p_eu_f5my_2013_1_008759102_0_2.zip

Please Fix.
ID: 49461 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,378,503
RAC: 3,632
Message 49462 - Posted: 30 Jun 2014, 8:19:31 UTC - in response to Message 49460.  

I don't know enough about this to do anything about it. Help for a novice would be appreciated.


When I get this problem, I suspend network activity through BOINC and re-enable it once a day to see if things have resolved. It is a server issue and with lots of work happening on the system at Oxford at the moment, I don't know how sorting the problem interacts with the other work which may mean sorting it needs to wait till something else is finished.

Also worth noting that when problem is resolved, there are still often difficulties initially as the server gets flooded with upload attempts. Only trying once a day helps to reduce this. If everyone did it, there probably wouldn't be an overload problem when things start working again.
ID: 49462 · Report as offensive     Reply Quote
Profile JS

Send message
Joined: 4 Mar 14
Posts: 7
Credit: 183,494
RAC: 0
Message 49463 - Posted: 30 Jun 2014, 9:23:42 UTC - in response to Message 49457.  

Yes, having problems uploading.
ID: 49463 · Report as offensive     Reply Quote
Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 33 · Next

Message boards : Number crunching : ANOTHER UPLOAD PROBLEM

©2024 climateprediction.net