climateprediction.net home page
Upload failures

Upload failures

Message boards : Number crunching : Upload failures
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 19 · Next

AuthorMessage
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60112 - Posted: 7 May 2019, 20:00:33 UTC

It's going to be looked at once the new day rolls around.
ID: 60112 · Report as offensive     Reply Quote
wolfman1360

Send message
Joined: 18 Feb 17
Posts: 81
Credit: 12,866,674
RAC: 7,509
Message 60114 - Posted: 7 May 2019, 21:10:11 UTC - in response to Message 60105.  

Wolfman

If you want to see the file sizes, you'll have to set the network to Suspend network activity, and then wait for one to be created.
Once the data is zipped up, it gets returned asap. Then the zip file is deleted.

The files should be in:
ProgramData/BOINC/projects/climateprediction.net

Zip sizes:
Batch 719 sam25 were 92.77 Megs
Batch 785 hadam4 were 17.3 Megs
The current batch 813 sam50 are 74.6 Megs

It all depends on the size of the area of interest, and the total of the amount of data the researcher wants to get.

Thanks, this is great to know!
I'll keep this in mind and give it a shot.
ID: 60114 · Report as offensive     Reply Quote
mngn

Send message
Joined: 13 Jul 18
Posts: 38
Credit: 56,768,885
RAC: 5,312
Message 60124 - Posted: 9 May 2019, 14:20:59 UTC
Last modified: 9 May 2019, 14:28:22 UTC

Upload trouble again.

2019-05-09 16:26:28 | climateprediction.net | [fxd] starting upload, upload_offset -1
2019-05-09 16:26:28 | climateprediction.net | Started upload of wah2_sam50_a03a_201412_24_813_011837270_1_r562800830_7.zip
2019-05-09 16:26:28 | climateprediction.net | [file_xfer] URL: http://jasmin-upload.cpdn.org/cgi-bin/file_upload_handler
2019-05-09 16:26:29 | climateprediction.net | [file_xfer] http op done; retval 0 (Success)
2019-05-09 16:26:29 | climateprediction.net | [error] Error reported by file upload server: can't open log file '../log_jasmin-upload/file_upload_handler.log' (errno: 9)
2019-05-09 16:26:29 | climateprediction.net | [file_xfer] parsing upload response: <data_server_reply> <status>1</status> <message>can't open log file '../log_jasmin-upload/file_upload_handler.log' (errno: 9)</message></data_server_reply>
2019-05-09 16:26:29 | climateprediction.net | [file_xfer] parsing status: -127
2019-05-09 16:26:29 | climateprediction.net | [file_xfer] file transfer status -127 (transient upload error)
2019-05-09 16:26:29 | climateprediction.net | Temporarily failed upload of wah2_sam50_a03a_201412_24_813_011837270_1_r562800830_7.zip: transient upload error
2019-05-09 16:26:29 | climateprediction.net | Backing off 00:31:52 on upload of wah2_sam50_a03a_201412_24_813_011837270_1_r562800830_7.zip
ID: 60124 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 24,488,575
RAC: 2,962
Message 60125 - Posted: 9 May 2019, 17:28:03 UTC - in response to Message 60124.  

Upload trouble again.

2019-05-09 16:26:28 | climateprediction.net | [fxd] starting upload, upload_offset -1
2019-05-09 16:26:28 | climateprediction.net | Started upload of wah2_sam50_a03a_201412_24_813_011837270_1_r562800830_7.zip
2019-05-09 16:26:28 | climateprediction.net | [file_xfer] URL: http://jasmin-upload.cpdn.org/cgi-bin/file_upload_handler
2019-05-09 16:26:29 | climateprediction.net | [file_xfer] http op done; retval 0 (Success)
2019-05-09 16:26:29 | climateprediction.net | [error] Error reported by file upload server: can't open log file '../log_jasmin-upload/file_upload_handler.log' (errno: 9)
2019-05-09 16:26:29 | climateprediction.net | [file_xfer] parsing upload response: <data_server_reply> <status>1</status> <message>can't open log file '../log_jasmin-upload/file_upload_handler.log' (errno: 9)</message></data_server_reply>
2019-05-09 16:26:29 | climateprediction.net | [file_xfer] parsing status: -127
2019-05-09 16:26:29 | climateprediction.net | [file_xfer] file transfer status -127 (transient upload error)
2019-05-09 16:26:29 | climateprediction.net | Temporarily failed upload of wah2_sam50_a03a_201412_24_813_011837270_1_r562800830_7.zip: transient upload error
2019-05-09 16:26:29 | climateprediction.net | Backing off 00:31:52 on upload of wah2_sam50_a03a_201412_24_813_011837270_1_r562800830_7.zip


I have the same upload problem as well. Log info is the same.
ID: 60125 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60126 - Posted: 9 May 2019, 18:02:24 UTC

OK, I've just sent an email.
ID: 60126 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60127 - Posted: 9 May 2019, 19:26:28 UTC

How is it now?
ID: 60127 · Report as offensive     Reply Quote
mngn

Send message
Joined: 13 Jul 18
Posts: 38
Credit: 56,768,885
RAC: 5,312
Message 60128 - Posted: 9 May 2019, 19:59:51 UTC

It is uploading fine now.
ID: 60128 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 24,488,575
RAC: 2,962
Message 60130 - Posted: 10 May 2019, 5:27:51 UTC - in response to Message 60127.  

All cleared, thanks.
ID: 60130 · Report as offensive     Reply Quote
mngn

Send message
Joined: 13 Jul 18
Posts: 38
Credit: 56,768,885
RAC: 5,312
Message 60174 - Posted: 20 May 2019, 20:11:50 UTC

Here we go again:

20/05/2019 22:01:36 | climateprediction.net | Started upload of wah2_sam50_n3u6_201412_24_814_011843666_0_r1742785198_14.zip
20/05/2019 22:01:36 | climateprediction.net | [file_xfer] URL: http://jasmin-upload.cpdn.org/cgi-bin/file_upload_handler
20/05/2019 22:01:37 | climateprediction.net | [file_xfer] http op done; retval 0 (Success)
20/05/2019 22:01:37 | climateprediction.net | [error] Error reported by file upload server: [wah2_sam50_n3u6_201412_24_814_011843666_0_r1742785198_14.zip] locked by file_upload_handler PID=15383
20/05/2019 22:01:37 | climateprediction.net | [file_xfer] parsing upload response: <data_server_reply> <status>1</status> <message>[wah2_sam50_n3u6_201412_24_814_011843666_0_r1742785198_14.zip] locked by file_upload_handler PID=15383</message></data_server_reply>
20/05/2019 22:01:37 | climateprediction.net | [file_xfer] parsing status: -127
20/05/2019 22:01:37 | climateprediction.net | [file_xfer] file transfer status -127 (transient upload error)
20/05/2019 22:01:37 | climateprediction.net | Temporarily failed upload of wah2_sam50_n3u6_201412_24_814_011843666_0_r1742785198_14.zip: transient upload error
20/05/2019 22:01:37 | climateprediction.net | [file_xfer] project-wide xfer delay for 3129.587695 sec
20/05/2019 22:01:37 | climateprediction.net | Backing off 00:44:39 on upload of wah2_sam50_n3u6_201412_24_814_011843666_0_r1742785198_14.zip
ID: 60174 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60175 - Posted: 20 May 2019, 21:25:33 UTC

"Jasmin" is used for several things, and gets a bit overwhelmed at times.
The message "locked by file_upload_handler" usually happens when you do a Retry now too soon.

I'll leave it for a few hours and ask here again.
If it's still not working, I'll send an email "this evening".
ID: 60175 · Report as offensive     Reply Quote
mngn

Send message
Joined: 13 Jul 18
Posts: 38
Credit: 56,768,885
RAC: 5,312
Message 60177 - Posted: 21 May 2019, 6:09:20 UTC

It uploaded just one hour later.
ID: 60177 · Report as offensive     Reply Quote
Mephist0

Send message
Joined: 21 Feb 08
Posts: 47
Credit: 7,929,915
RAC: 0
Message 60178 - Posted: 21 May 2019, 7:10:47 UTC - in response to Message 60177.  

I have problems too.. Started trying to upload results yesterday and got this error.. Today still same problem. The whole file seem to upload but does not disappear and retries over and over..

2019-05-21 09:06:48 | | Project communication failed: attempting access to reference site
2019-05-21 09:06:48 | climateprediction.net | Temporarily failed upload of wah2_sam50_n2j1_201412_24_814_011842629_0_r254395716_17.zip: transient HTTP error
2019-05-21 09:06:48 | climateprediction.net | Backing off 00:02:58 on upload of wah2_sam50_n2j1_201412_24_814_011842629_0_r254395716_17.zip
ID: 60178 · Report as offensive     Reply Quote
Mephist0

Send message
Joined: 21 Feb 08
Posts: 47
Credit: 7,929,915
RAC: 0
Message 60179 - Posted: 21 May 2019, 7:14:39 UTC - in response to Message 60178.  
Last modified: 21 May 2019, 7:36:18 UTC

My queue consists of 34 files total file size about 2.5 GB.

I have to add too that this computer does not have internet access, it has to be connected manually. Thats why it has such a long queue. I usually connect the computer once a week or when the tasks has finished if that is earlier than 7 days :)
ID: 60179 · Report as offensive     Reply Quote
_Ryle_

Send message
Joined: 17 Aug 05
Posts: 21
Credit: 15,091,561
RAC: 22,127
Message 60209 - Posted: 25 May 2019, 17:49:46 UTC - in response to Message 60038.  

I just remembered something from last year.

I got some cams (Central America), which seemed to run OK, but there were problems with the zip uploads. (cam and sam models go to a server in Mexico.)

With the first one, zip 3 refused to upload past about 30%. Then zip 6 did the same. I was wondering if zip 9 would be the same, but all of the others were OK.

Every time a new zip was created and I tried to upload it, first the stuck zip uploaded to the fail point, waited until it timed out, and then the new zip got to upload.
In the end I Aborted the model, on the grounds that it had been given enough chances, and if the researchers wanted that particular data set, they could just reissue it as part of a new batch.

The same thing happened with a second cam. There was only one zip failed, forget which. Same process - let anything that wanted to, upload, then Aborted.

A 3rd cam, on the other hand, was no trouble at all.

So, I'm guessing that it's Tullio's sam50s that are not uploading properly.
If so, then there's 2 answers:
1) Do what I did, and give them as much data as will upload, then Abort, or
2) Just Abort the problem sam50 now.

And, as with my cams, it may not be all of them that won't upload.


I've got a cam25 running currently around 78 percent done. It's got 3 zips that are not keen on uploading it seems. Zip 1, 6 and now 14. I think I will let it finish if it can, and then abort the uploads if they don't clear after some time.
ID: 60209 · Report as offensive     Reply Quote
Mario

Send message
Joined: 30 Apr 18
Posts: 3
Credit: 242,814
RAC: 0
Message 60210 - Posted: 25 May 2019, 19:10:10 UTC

Hello,
I have a problem uploading task 21646610, wah2_eu25_a0f2_201712_13_800_011781166_1

The task seems to be uploaded to the server but it is still showing as in progress. The server has even posted points for the task.

Any ideas?

Mario
ID: 60210 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60211 - Posted: 25 May 2019, 21:10:58 UTC - in response to Message 60210.  

If there's no mention of that task in the Tasks tab of your Manager, then BOINC has already "Reported" it to the server.

If the scheduler was having problems at that moment, the message has become lost, so that task will always be shown as "In Progress".

Credits are related to the trickle_up files, not the zips.
ID: 60211 · Report as offensive     Reply Quote
Mario

Send message
Joined: 30 Apr 18
Posts: 3
Credit: 242,814
RAC: 0
Message 60212 - Posted: 25 May 2019, 21:18:19 UTC - in response to Message 60211.  
Last modified: 25 May 2019, 21:18:54 UTC

Les,

Thank you.

I checked my data files and the task zip files are gone from there as well. I will not worry about this task anymore.

Mario
ID: 60212 · Report as offensive     Reply Quote
Mephist0

Send message
Joined: 21 Feb 08
Posts: 47
Credit: 7,929,915
RAC: 0
Message 60216 - Posted: 27 May 2019, 8:25:46 UTC

I'm still having issues with the uploads. Anyone know when this will be fixed? I now have 50 uploads queued 3700 MB of data. I have paused the network traffic due to the network traffic consumed trying to upload OVER AND OVER again ..

2019-05-27 10:23:52 | climateprediction.net | Temporarily failed upload of wah2_sam50_n088_201412_24_814_011846908_0_r734526431_16.zip: transient HTTP error
ID: 60216 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 60217 - Posted: 27 May 2019, 9:16:33 UTC

I'm not having any problems with a sam50, batch 815.
Everything is uploading OK.

I'll send an email, but it's a long weekend in the UK.

What server is yours going to?
ID: 60217 · Report as offensive     Reply Quote
Mephist0

Send message
Joined: 21 Feb 08
Posts: 47
Credit: 7,929,915
RAC: 0
Message 60218 - Posted: 27 May 2019, 10:29:24 UTC - in response to Message 60217.  

Here is the log with file_xfer switched on. Looks like my project has not been switched to https:// yet for this client.

2019-05-27 12:26:24 | | Project communication failed: attempting access to reference site
2019-05-27 12:26:24 | climateprediction.net | [file_xfer] http op done; retval -184 (transient HTTP error)
2019-05-27 12:26:24 | climateprediction.net | [file_xfer] file transfer status -184 (transient HTTP error)
2019-05-27 12:26:24 | climateprediction.net | Temporarily failed upload of wah2_sam50_n2j1_201412_24_814_011842629_0_r254395716_18.zip: transient HTTP error
2019-05-27 12:26:24 | climateprediction.net | [file_xfer] project-wide xfer delay for 19851.142918 sec
2019-05-27 12:26:24 | climateprediction.net | Backing off 00:02:59 on upload of wah2_sam50_n2j1_201412_24_814_011842629_0_r254395716_18.zip
2019-05-27 12:26:24 | climateprediction.net | [fxd] starting upload, upload_offset -1
2019-05-27 12:26:24 | climateprediction.net | Started upload of wah2_sam50_n2j1_201412_24_814_011842629_0_r254395716_19.zip
2019-05-27 12:26:24 | climateprediction.net | [file_xfer] URL: http://jasmin-upload.cpdn.org/cgi-bin/file_upload_handler
2019-05-27 12:26:25 | climateprediction.net | [file_xfer] http op done; retval 0 (Success)
2019-05-27 12:26:25 | climateprediction.net | [file_xfer] parsing upload response: <data_server_reply> <status>0</status> <file_size>0</file_size></data_server_reply>
2019-05-27 12:26:25 | climateprediction.net | [file_xfer] parsing status: 0
2019-05-27 12:26:25 | climateprediction.net | [fxd] starting upload, upload_offset 0
2019-05-27 12:26:26 | | Internet access OK - project servers may be temporarily down.
2019-05-27 12:26:29 | | Project communication failed: attempting access to reference site
2019-05-27 12:26:29 | climateprediction.net | [file_xfer] http op done; retval -184 (transient HTTP error)
2019-05-27 12:26:29 | climateprediction.net | [file_xfer] file transfer status -184 (transient HTTP error)
2019-05-27 12:26:29 | climateprediction.net | Temporarily failed upload of wah2_sam50_n088_201412_24_814_011846908_0_r734526431_18.zip: transient HTTP error
2019-05-27 12:26:29 | climateprediction.net | [file_xfer] project-wide xfer delay for 15371.880245 sec
2019-05-27 12:26:29 | climateprediction.net | Backing off 00:02:09 on upload of wah2_sam50_n088_201412_24_814_011846908_0_r734526431_18.zip
2019-05-27 12:26:29 | climateprediction.net | [fxd] starting upload, upload_offset -1
2019-05-27 12:26:29 | climateprediction.net | Started upload of wah2_sam50_n088_201412_24_814_011846908_0_r734526431_19.zip
2019-05-27 12:26:29 | climateprediction.net | [file_xfer] URL: http://jasmin-upload.cpdn.org/cgi-bin/file_upload_handler
2019-05-27 12:26:30 | | Internet access OK - project servers may be temporarily down.
2019-05-27 12:26:30 | climateprediction.net | [file_xfer] http op done; retval 0 (Success)
2019-05-27 12:26:30 | climateprediction.net | [file_xfer] parsing upload response: <data_server_reply> <status>0</status> <file_size>0</file_size></data_server_reply>
2019-05-27 12:26:30 | climateprediction.net | [file_xfer] parsing status: 0
2019-05-27 12:26:30 | climateprediction.net | [fxd] starting upload, upload_offset 0
2019-05-27 12:26:48 | | Suspending network activity - user request
ID: 60218 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 19 · Next

Message boards : Number crunching : Upload failures

©2024 climateprediction.net