climateprediction.net home page
Persistent upload problems

Persistent upload problems

Message boards : Number crunching : Persistent upload problems
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47452 - Posted: 2 Nov 2013, 3:14:29 UTC - in response to Message 47451.  
Last modified: 2 Nov 2013, 3:18:21 UTC

Can you get to the internal connection diagnostics for your router? To find out what speed you connection is really running at, rather than what the ISP's salesmen are charging you for?

Mine says:
Connection Information
Downstream: 5.461 Mbps
Upstream: 1005 Kbps

and other useful stuff

I have tried with just one machine communicating at a time. Didn't make any difference.

Link rate downstream is 20946 Kbps and upstream is 1020 Kbps so plenty of speed available. Noise margins are 6.4dB down and 13.8dB up. Line attenuation is 13.0dB down and 5.5dB up. Router doesn't give the other figures.
BOINC blog
ID: 47452 · Report as offensive     Reply Quote
alvin

Send message
Joined: 12 Mar 12
Posts: 29
Credit: 666,199
RAC: 0
Message 47453 - Posted: 2 Nov 2013, 4:11:45 UTC - in response to Message 47452.  
Last modified: 2 Nov 2013, 4:12:04 UTC

definetely some internet/network settings affected
then you switched to dial-up via USB or cam1 - probably USB, you've created brand new port with default settings.
as proposed before to reset NIC adapter settings will help in future, but not via netsh.
simplest is remove NIC, restart in safe mode
go to registry and delete everything in connection with old NIC

enable hidden devices in device manager and remove them all if something remains
ID: 47453 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47456 - Posted: 2 Nov 2013, 11:29:30 UTC - in response to Message 47453.  

definetely some internet/network settings affected
then you switched to dial-up via USB or cam1 - probably USB, you've created brand new port with default settings.
as proposed before to reset NIC adapter settings will help in future, but not via netsh.
simplest is remove NIC, restart in safe mode
go to registry and delete everything in connection with old NIC

enable hidden devices in device manager and remove them all if something remains

I setup an old XP box with a 56k modem in one of the PCI slots. I use it as a proxy server, that way I can point the number crunchers at it one at a time without having to install modem drivers etc. The only problem is the line gets disconnected when people make calls (wife or kid). Its painfully slow of course.

The NIC for the number crunchers are all on the motherboards so can't remove them, as I said before 5 different machines so not likely to be a NIC.
BOINC blog
ID: 47456 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 943
Credit: 34,360,365
RAC: 9,337
Message 47457 - Posted: 2 Nov 2013, 12:26:48 UTC - in response to Message 47456.  

Agreed, highly unlikely to be a NIC or driver if you can consistently get to 91% before something times out.

I'd like to see timings and <http_debug> log messages for a single HADcm3n file upload attempt over broadband, with minimal other activity on the line.
ID: 47457 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47458 - Posted: 2 Nov 2013, 14:32:56 UTC - in response to Message 47457.  

Agreed, highly unlikely to be a NIC or driver if you can consistently get to 91% before something times out.

I'd like to see timings and <http_debug> log messages for a single HADcm3n file upload attempt over broadband, with minimal other activity on the line.

HTTP debug is in the other message thread here

To clarify they don't all get to 91% before failing on the transfer, just some were. others fail at different spots, so no consistent pattern.

BTW David had a look at the sched_request file and while its full of trickle stuff didn't see anything unexpected.
BOINC blog
ID: 47458 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 943
Credit: 34,360,365
RAC: 9,337
Message 47459 - Posted: 3 Nov 2013, 20:01:06 UTC - in response to Message 47458.  

Well, I've had a look at my own most recent uploads:

31-Oct-2013 12:26:28 [climateprediction.net] Started upload of hadcm3n_n088_1880_40_008399121_2_3.zip
31-Oct-2013 12:36:29 [climateprediction.net] Finished upload of hadcm3n_n088_1880_40_008399121_2_3.zip

01-Nov-2013 03:11:52 [climateprediction.net] Started upload of hadcm3n_n11u_1960_40_008378631_2_3.zip
01-Nov-2013 03:21:33 [climateprediction.net] Finished upload of hadcm3n_n11u_1960_40_008378631_2_3.zip

- about ten minutes a throw and no errors, whether I'm awake or asleep (and we agreed your line was running at much the same speed as mine)

That was done under BOINC v7.2.26, but I've noticed no change (intentional or unintentional) in client upload behaviour over the last many iterations. I think you can cross client bugs off your list.

Your log says - or said, over a month ago -

28/09/2013 9:34:05 AM | climateprediction.net | [http] [ID#519] Sent header to server:
28/09/2013 9:41:49 AM | climateprediction.net | [http] [ID#519] Info: Recv failure: Connection was reset
28/09/2013 9:41:49 AM | climateprediction.net | [http] [ID#519] Info: Closing connection #0
28/09/2013 9:41:49 AM | climateprediction.net | [http] HTTP error: Failure when receiving data from the peer
28/09/2013 9:41:50 AM | | Project communication failed: attempting access to reference site
28/09/2013 9:41:50 AM | | [http] HTTP_OP::init_get(): http://www.google.com/

- so the failure occurred after about 07:44 - not long enough. That's not a client timeout - it times out on inactivity, not total duration.

BTW - google.com - you do know you can set

<dont_contact_ref_site>0|1</dont_contact_ref_site>
To determine if a physical network connection exists, the client occasionally contacts a highly-available web site (google.com). If this flag is set, this behavior is suppressed.

Not needed, and google just clutters up the logs. K.I.S.S.

I think your time has come for a FRESH log, under conditions as I suggested last time, but maybe with even more logging. By coincidence, I was just considering what flags to suggest, when this popped up:

03/11/2013 19:53:41 | climateprediction.net | [sched_op] Starting scheduler request
03/11/2013 19:53:41 | climateprediction.net | Sending scheduler request: To send trickle-up message.
03/11/2013 19:53:41 | climateprediction.net | Not requesting tasks: "no new tasks" requested via Manager
03/11/2013 19:53:41 | climateprediction.net | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
03/11/2013 19:53:41 | climateprediction.net | [sched_op] NVIDIA work request: 0.00 seconds; 0.00 devices
03/11/2013 19:53:43 | | [http_xfer] [ID#1] HTTP: wrote 1276 bytes
03/11/2013 19:53:43 | | [http_xfer] [ID#1] HTTP: wrote 1424 bytes
03/11/2013 19:53:43 | | [http_xfer] [ID#1] HTTP: wrote 1372 bytes
03/11/2013 19:53:43 | | [http_xfer] [ID#1] HTTP: wrote 1068 bytes
03/11/2013 19:53:44 | climateprediction.net | Scheduler request completed
03/11/2013 19:53:44 | climateprediction.net | [sched_op] Server version 613
03/11/2013 19:53:44 | climateprediction.net | Project requested delay of 3636 seconds
03/11/2013 19:53:44 | climateprediction.net | [sched_op] Deferring communication for 01:00:36
03/11/2013 19:53:44 | climateprediction.net | [sched_op] Reason: requested by project

I don't think <http_xfer_debug> is going to be very helpful - try <file_xfer_debug> (with <http_debug> as before), but be prepared for a *very* long log.
ID: 47459 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 47460 - Posted: 3 Nov 2013, 21:38:38 UTC

Mark

I don't want to add to your woes, but the target date for finishing conversion of the servers to BOINC version 7 has, for about the past year, been said to be November.

I don't know how, if at all, this will impact on your upload attempts.

ID: 47460 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47462 - Posted: 4 Nov 2013, 11:12:35 UTC
Last modified: 4 Nov 2013, 11:19:40 UTC

Okay here is the scheduler request, still nothing of interest. I'll post a file transfer in the next message. I've asterisked out the auth id's.

4/11/2013 10:07:32 PM | | Using proxy info from GUI
4/11/2013 10:07:32 PM | | Not using a proxy
4/11/2013 10:07:36 PM | | Suspending network activity - user request
4/11/2013 10:07:44 PM | | Resuming network activity
4/11/2013 10:07:44 PM | climateprediction.net | [fxd] starting upload, upload_offset -1
4/11/2013 10:07:44 PM | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
4/11/2013 10:07:44 PM | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
4/11/2013 10:07:44 PM | climateprediction.net | Started upload of hadcm3n_84lg_1980_40_008463912_0_2.zip
4/11/2013 10:07:44 PM | climateprediction.net | [file_xfer] URL: http://rapid-watch.badc.rl.ac.uk/cpdn_cgi/file_upload_handler
4/11/2013 10:07:44 PM | | [http] HTTP_OP::init_get(): http://asteroidsathome.net/boinc/notices.php?userid=1778&auth=***
4/11/2013 10:07:44 PM | | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
4/11/2013 10:07:44 PM | | [http] HTTP_OP::libcurl_exec(): ca-bundle set
4/11/2013 10:07:44 PM | climateprediction.net | Sending scheduler request: To send trickle-up message.
4/11/2013 10:07:44 PM | climateprediction.net | Reporting 1 completed tasks
4/11/2013 10:07:44 PM | climateprediction.net | Not requesting tasks: "no new tasks" requested via Manager
4/11/2013 10:07:44 PM | climateprediction.net | [http] HTTP_OP::init_post(): http://climateapps2.oerc.ox.ac.uk/cpdnboinc_cgi/cgi
4/11/2013 10:07:44 PM | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
4/11/2013 10:07:44 PM | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Info: About to connect() to climateapps2.oerc.ox.ac.uk port 80 (#2)
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Info: Trying 129.67.195.185...
4/11/2013 10:07:44 PM | | [http] [ID#0] Info: About to connect() to asteroidsathome.net port 80 (#1)
4/11/2013 10:07:44 PM | | [http] [ID#0] Info: Trying 62.129.48.181...
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Info: Connected to climateapps2.oerc.ox.ac.uk (129.67.195.185) port 80 (#2)
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Info: Connected to climateapps2.oerc.ox.ac.uk (129.67.195.185) port 80 (#2)
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Sent header to server: POST /cpdnboinc_cgi/cgi HTTP/1.0
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.2.26)
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Sent header to server: Host: climateapps2.oerc.ox.ac.uk
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Sent header to server: Accept: */*
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Sent header to server: Accept-Encoding: deflate, gzip
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Sent header to server: Content-Type: application/x-www-form-urlencoded
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Sent header to server: Content-Length: 6043059
4/11/2013 10:07:44 PM | climateprediction.net | [http] [ID#1] Sent header to server:
4/11/2013 10:07:45 PM | | [http] [ID#0] Info: Connected to asteroidsathome.net (62.129.48.181) port 80 (#1)
4/11/2013 10:07:45 PM | | [http] [ID#0] Info: Connected to asteroidsathome.net (62.129.48.181) port 80 (#1)
4/11/2013 10:07:45 PM | | [http] [ID#0] Sent header to server: GET /boinc/notices.php?userid=***HTTP/1.0
4/11/2013 10:07:45 PM | | [http] [ID#0] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.2.26)
4/11/2013 10:07:45 PM | | [http] [ID#0] Sent header to server: Host: asteroidsathome.net
4/11/2013 10:07:45 PM | | [http] [ID#0] Sent header to server: Accept: */*
4/11/2013 10:07:45 PM | | [http] [ID#0] Sent header to server: Accept-Encoding: deflate, gzip
4/11/2013 10:07:45 PM | | [http] [ID#0] Sent header to server: Content-Type: application/x-www-form-urlencoded
4/11/2013 10:07:45 PM | | [http] [ID#0] Sent header to server:
4/11/2013 10:07:46 PM | climateprediction.net | [http] [ID#5] Info: About to connect() to rapid-watch.badc.rl.ac.uk port 80 (#0)
4/11/2013 10:07:46 PM | climateprediction.net | [http] [ID#5] Info: Trying 130.246.191.84...
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Info: Connected to rapid-watch.badc.rl.ac.uk (130.246.191.84) port 80 (#0)
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Info: Connected to rapid-watch.badc.rl.ac.uk (130.246.191.84) port 80 (#0)
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Sent header to server: POST /cpdn_cgi/file_upload_handler HTTP/1.0
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.2.26)
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Sent header to server: Host: rapid-watch.badc.rl.ac.uk
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Sent header to server: Accept: */*
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Sent header to server: Accept-Encoding: deflate, gzip
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Sent header to server: Content-Type: application/x-www-form-urlencoded
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Sent header to server: Content-Length: 291
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Sent header to server:
4/11/2013 10:07:47 PM | | [http] [ID#0] Received header from server: HTTP/1.1 200 OK
4/11/2013 10:07:47 PM | | [http] [ID#0] Received header from server: Date: Mon, 04 Nov 2013 11:07:46 GMT
4/11/2013 10:07:47 PM | | [http] [ID#0] Received header from server: Server: Apache/2.2.22 (Debian)
4/11/2013 10:07:47 PM | | [http] [ID#0] Received header from server: X-Powered-By: PHP/5.4.4-14+deb7u5
4/11/2013 10:07:47 PM | | [http] [ID#0] Received header from server: Expires: Mon, 04 Nov 2013 11:07:47 GMT
4/11/2013 10:07:47 PM | | [http] [ID#0] Received header from server: Last-Modified: Mon, 04 Nov 2013 11:07:47 GMT
4/11/2013 10:07:47 PM | | [http] [ID#0] Received header from server: Content-Length: 1354
4/11/2013 10:07:47 PM | | [http] [ID#0] Received header from server: Connection: close
4/11/2013 10:07:47 PM | | [http] [ID#0] Received header from server: Content-Type: application/xml
4/11/2013 10:07:47 PM | | [http] [ID#0] Received header from server:
4/11/2013 10:07:47 PM | | [http] [ID#0] Info: Closing connection #1
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Received header from server: HTTP/1.1 200 OK
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Received header from server: Date: Mon, 04 Nov 2013 11:07:47 GMT
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Received header from server: Server: Apache/2.2.12 (Linux/SUSE)
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Received header from server: Connection: close
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Received header from server: Content-Type: text/plain
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Received header from server:
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Info: Closing connection #0
4/11/2013 10:07:47 PM | climateprediction.net | [file_xfer] http op done; retval 0 (Success)
4/11/2013 10:07:47 PM | climateprediction.net | [file_xfer] parsing upload response: <data_server_reply> <status>0</status> <file_size>0</file_size></data_server_reply>
4/11/2013 10:07:47 PM | climateprediction.net | [file_xfer] parsing status: 0
4/11/2013 10:07:47 PM | climateprediction.net | [fxd] starting upload, upload_offset 0
4/11/2013 10:07:47 PM | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
4/11/2013 10:07:47 PM | | [http] HTTP_OP::init_get(): http://einstein.phys.uwm.edu/rss_main.php
4/11/2013 10:07:47 PM | | [http] HTTP_OP::libcurl_exec(): ca-bundle set
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Info: About to connect() to rapid-watch.badc.rl.ac.uk port 80 (#0)
4/11/2013 10:07:47 PM | climateprediction.net | [http] [ID#5] Info: Trying 130.246.191.84...
4/11/2013 10:07:48 PM | | [http] [ID#0] Info: About to connect() to einstein.phys.uwm.edu port 80 (#1)
4/11/2013 10:07:48 PM | | [http] [ID#0] Info: Trying 129.89.61.70...
4/11/2013 10:07:48 PM | climateprediction.net | [http] [ID#5] Info: Connected to rapid-watch.badc.rl.ac.uk (130.246.191.84) port 80 (#0)
4/11/2013 10:07:48 PM | climateprediction.net | [http] [ID#5] Info: Connected to rapid-watch.badc.rl.ac.uk (130.246.191.84) port 80 (#0)
4/11/2013 10:07:48 PM | climateprediction.net | [http] [ID#5] Sent header to server: POST /cpdn_cgi/file_upload_handler HTTP/1.0
4/11/2013 10:07:48 PM | climateprediction.net | [http] [ID#5] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.2.26)
4/11/2013 10:07:48 PM | climateprediction.net | [http] [ID#5] Sent header to server: Host: rapid-watch.badc.rl.ac.uk
4/11/2013 10:07:48 PM | climateprediction.net | [http] [ID#5] Sent header to server: Accept: */*
4/11/2013 10:07:48 PM | climateprediction.net | [http] [ID#5] Sent header to server: Accept-Encoding: deflate, gzip
4/11/2013 10:07:48 PM | climateprediction.net | [http] [ID#5] Sent header to server: Content-Type: application/x-www-form-urlencoded
4/11/2013 10:07:48 PM | climateprediction.net | [http] [ID#5] Sent header to server: Content-Length: 54567068
4/11/2013 10:07:48 PM | climateprediction.net | [http] [ID#5] Sent header to server:
4/11/2013 10:07:48 PM | | [http] [ID#0] Info: Connected to einstein.phys.uwm.edu (129.89.61.70) port 80 (#1)
4/11/2013 10:07:48 PM | | [http] [ID#0] Info: Connected to einstein.phys.uwm.edu (129.89.61.70) port 80 (#1)
4/11/2013 10:07:48 PM | | [http] [ID#0] Sent header to server: GET /rss_main.php HTTP/1.0
4/11/2013 10:07:48 PM | | [http] [ID#0] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.2.26)
4/11/2013 10:07:48 PM | | [http] [ID#0] Sent header to server: Host: einstein.phys.uwm.edu
4/11/2013 10:07:48 PM | | [http] [ID#0] Sent header to server: Accept: */*
4/11/2013 10:07:48 PM | | [http] [ID#0] Sent header to server: Accept-Encoding: deflate, gzip
4/11/2013 10:07:48 PM | | [http] [ID#0] Sent header to server: Content-Type: application/x-www-form-urlencoded
4/11/2013 10:07:48 PM | | [http] [ID#0] Sent header to server:
4/11/2013 10:07:49 PM | | [http] [ID#0] Received header from server: HTTP/1.1 200 OK
4/11/2013 10:07:49 PM | | [http] [ID#0] Received header from server: Date: Mon, 04 Nov 2013 11:07:48 GMT
4/11/2013 10:07:49 PM | | [http] [ID#0] Received header from server: Server: Apache/2.2.3 (CentOS)
4/11/2013 10:07:49 PM | | [http] [ID#0] Received header from server: X-Powered-By: PHP/5.1.6
4/11/2013 10:07:49 PM | | [http] [ID#0] Received header from server: Expires: Tue, 05 Nov 2013 11:07:49 GMT
4/11/2013 10:07:49 PM | | [http] [ID#0] Received header from server: Last-Modified: Wed, 23 Oct 2013 12:44:14 GMT
4/11/2013 10:07:49 PM | | [http] [ID#0] Received header from server: Content-Length: 1691
4/11/2013 10:07:49 PM | | [http] [ID#0] Received header from server: Connection: close
4/11/2013 10:07:49 PM | | [http] [ID#0] Received header from server: Content-Type: application/xml
4/11/2013 10:07:49 PM | | [http] [ID#0] Received header from server:
4/11/2013 10:07:49 PM | | [http] [ID#0] Info: Closing connection #1
4/11/2013 10:07:50 PM | | [http] HTTP_OP::init_get(): http://setiathome.berkeley.edu/notices.php?userid=8823491&auth=***
4/11/2013 10:07:50 PM | | [http] HTTP_OP::libcurl_exec(): ca-bundle set
4/11/2013 10:07:50 PM | | [http] [ID#0] Info: About to connect() to setiathome.berkeley.edu port 80 (#1)
4/11/2013 10:07:50 PM | | [http] [ID#0] Info: Trying 169.229.217.150...
4/11/2013 10:07:50 PM | | [http] [ID#0] Info: Connected to setiathome.berkeley.edu (169.229.217.150) port 80 (#1)
4/11/2013 10:07:50 PM | | [http] [ID#0] Info: Connected to setiathome.berkeley.edu (169.229.217.150) port 80 (#1)
4/11/2013 10:07:50 PM | | [http] [ID#0] Sent header to server: GET /notices.php?userid=*** HTTP/1.0
4/11/2013 10:07:50 PM | | [http] [ID#0] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.2.26)
4/11/2013 10:07:50 PM | | [http] [ID#0] Sent header to server: Host: setiathome.berkeley.edu
4/11/2013 10:07:50 PM | | [http] [ID#0] Sent header to server: Accept: */*
4/11/2013 10:07:50 PM | | [http] [ID#0] Sent header to server: Accept-Encoding: deflate, gzip
4/11/2013 10:07:50 PM | | [http] [ID#0] Sent header to server: Content-Type: application/x-www-form-urlencoded
4/11/2013 10:07:50 PM | | [http] [ID#0] Sent header to server:
4/11/2013 10:07:50 PM | | [http] [ID#0] Received header from server: HTTP/1.1 200 OK
4/11/2013 10:07:50 PM | | [http] [ID#0] Received header from server: Date: Mon, 04 Nov 2013 11:07:50 GMT
4/11/2013 10:07:50 PM | | [http] [ID#0] Received header from server: Server: Apache/2.2.15 (Scientific Linux)
4/11/2013 10:07:50 PM | | [http] [ID#0] Received header from server: X-Powered-By: PHP/5.3.3
4/11/2013 10:07:50 PM | | [http] [ID#0] Received header from server: Expires: Mon, 04 Nov 2013 11:07:50 GMT
4/11/2013 10:07:50 PM | | [http] [ID#0] Received header from server: Last-Modified: Mon, 04 Nov 2013 11:07:50 GMT
4/11/2013 10:07:50 PM | | [http] [ID#0] Received header from server: Content-Length: 1529
4/11/2013 10:07:50 PM | | [http] [ID#0] Received header from server: Connection: close
4/11/2013 10:07:50 PM | | [http] [ID#0] Received header from server: Content-Type: application/xml
4/11/2013 10:07:50 PM | | [http] [ID#0] Received header from server:
4/11/2013 10:07:50 PM | | [http] [ID#0] Info: Closing connection #1
4/11/2013 10:09:27 PM | climateprediction.net | [http] [ID#1] Received header from server: HTTP/1.1 500 Internal Server Error
4/11/2013 10:09:27 PM | climateprediction.net | [http] [ID#1] Received header from server: Date: Mon, 04 Nov 2013 11:07:45 GMT
4/11/2013 10:09:27 PM | climateprediction.net | [http] [ID#1] Received header from server: Server: Apache
4/11/2013 10:09:27 PM | climateprediction.net | [http] [ID#1] Received header from server: Content-Length: 623
4/11/2013 10:09:27 PM | climateprediction.net | [http] [ID#1] Received header from server: Connection: close
4/11/2013 10:09:27 PM | climateprediction.net | [http] [ID#1] Received header from server: Content-Type: text/html; charset=iso-8859-1
4/11/2013 10:09:27 PM | climateprediction.net | [http] [ID#1] Received header from server:
4/11/2013 10:09:27 PM | climateprediction.net | [http] [ID#1] Info: we are done reading and this is set to close, stop send
4/11/2013 10:09:27 PM | climateprediction.net | [http] [ID#1] Info: Closing connection #2
4/11/2013 10:09:28 PM | climateprediction.net | Scheduler request failed: HTTP internal server error
4/11/2013 10:11:08 PM | climateprediction.net | [http] [ID#5] Info: Recv failure: Connection was reset
4/11/2013 10:11:08 PM | climateprediction.net | [http] [ID#5] Info: Closing connection #0
4/11/2013 10:11:08 PM | climateprediction.net | [http] HTTP error: Failure when receiving data from the peer
4/11/2013 10:11:09 PM | | Project communication failed: attempting access to reference site
BOINC blog
ID: 47462 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47463 - Posted: 4 Nov 2013, 11:32:20 UTC

And a file xfer

4/11/2013 10:11:09 PM | climateprediction.net | Started upload of hadcm3n_84lg_1980_40_008463912_0_4.zip
4/11/2013 10:11:09 PM | climateprediction.net | [file_xfer] URL: http://rapid-watch.badc.rl.ac.uk/cpdn_cgi/file_upload_handler
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Info: About to connect() to rapid-watch.badc.rl.ac.uk port 80 (#1)
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Info: Trying 130.246.191.84...
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Info: Connected to rapid-watch.badc.rl.ac.uk (130.246.191.84) port 80 (#1)
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Info: Connected to rapid-watch.badc.rl.ac.uk (130.246.191.84) port 80 (#1)
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Sent header to server: POST /cpdn_cgi/file_upload_handler HTTP/1.0
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.2.26)
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Sent header to server: Host: rapid-watch.badc.rl.ac.uk
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Sent header to server: Accept: */*
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Sent header to server: Accept-Encoding: deflate, gzip
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Sent header to server: Content-Type: application/x-www-form-urlencoded
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Sent header to server: Content-Length: 291
4/11/2013 10:11:10 PM | climateprediction.net | [http] [ID#6] Sent header to server:
4/11/2013 10:11:11 PM | climateprediction.net | [http] [ID#6] Received header from server: HTTP/1.1 200 OK
4/11/2013 10:11:11 PM | climateprediction.net | [http] [ID#6] Received header from server: Date: Mon, 04 Nov 2013 11:11:10 GMT
4/11/2013 10:11:11 PM | climateprediction.net | [http] [ID#6] Received header from server: Server: Apache/2.2.12 (Linux/SUSE)
4/11/2013 10:11:11 PM | climateprediction.net | [http] [ID#6] Received header from server: Connection: close
4/11/2013 10:11:11 PM | climateprediction.net | [http] [ID#6] Received header from server: Content-Type: text/plain
4/11/2013 10:11:11 PM | climateprediction.net | [http] [ID#6] Received header from server:
4/11/2013 10:11:11 PM | climateprediction.net | [http] [ID#6] Info: Closing connection #1
4/11/2013 10:11:11 PM | climateprediction.net | [file_xfer] http op done; retval 0 (Success)
4/11/2013 10:11:11 PM | climateprediction.net | [file_xfer] parsing upload response: <data_server_reply> <status>0</status> <file_size>0</file_size></data_server_reply>
4/11/2013 10:11:11 PM | climateprediction.net | [file_xfer] parsing status: 0
4/11/2013 10:11:11 PM | climateprediction.net | [fxd] starting upload, upload_offset 0
4/11/2013 10:11:11 PM | climateprediction.net | [http] HTTP_OP::libcurl_exec(): ca-bundle set
4/11/2013 10:11:11 PM | climateprediction.net | [http] [ID#6] Info: About to connect() to rapid-watch.badc.rl.ac.uk port 80 (#0)
4/11/2013 10:11:11 PM | climateprediction.net | [http] [ID#6] Info: Trying 130.246.191.84...
4/11/2013 10:11:12 PM | climateprediction.net | [http] [ID#6] Info: Connected to rapid-watch.badc.rl.ac.uk (130.246.191.84) port 80 (#0)
4/11/2013 10:11:12 PM | climateprediction.net | [http] [ID#6] Info: Connected to rapid-watch.badc.rl.ac.uk (130.246.191.84) port 80 (#0)
4/11/2013 10:11:12 PM | climateprediction.net | [http] [ID#6] Sent header to server: POST /cpdn_cgi/file_upload_handler HTTP/1.0
4/11/2013 10:11:12 PM | climateprediction.net | [http] [ID#6] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.2.26)
4/11/2013 10:11:12 PM | climateprediction.net | [http] [ID#6] Sent header to server: Host: rapid-watch.badc.rl.ac.uk
4/11/2013 10:11:12 PM | climateprediction.net | [http] [ID#6] Sent header to server: Accept: */*
4/11/2013 10:11:12 PM | climateprediction.net | [http] [ID#6] Sent header to server: Accept-Encoding: deflate, gzip
4/11/2013 10:11:12 PM | climateprediction.net | [http] [ID#6] Sent header to server: Content-Type: application/x-www-form-urlencoded
4/11/2013 10:11:12 PM | climateprediction.net | [http] [ID#6] Sent header to server: Content-Length: 54540834
4/11/2013 10:11:12 PM | climateprediction.net | [http] [ID#6] Sent header to server:
4/11/2013 10:26:18 PM | climateprediction.net | [http] [ID#6] Info: Recv failure: Connection was reset
4/11/2013 10:26:18 PM | climateprediction.net | [http] [ID#6] Info: Closing connection #0
4/11/2013 10:26:18 PM | climateprediction.net | [http] HTTP error: Failure when receiving data from the peer
4/11/2013 10:26:19 PM | climateprediction.net | [file_xfer] http op done; retval -184 (transient HTTP error)
4/11/2013 10:26:19 PM | climateprediction.net | [file_xfer] file transfer status -184 (transient HTTP error)
4/11/2013 10:26:19 PM | climateprediction.net | Temporarily failed upload of hadcm3n_84lg_1980_40_008463912_0_4.zip: transient HTTP error
4/11/2013 10:26:19 PM | climateprediction.net | Backing off 03:24:24 on upload of hadcm3n_84lg_1980_40_008463912_0_4.zip

BOINC blog
ID: 47463 · Report as offensive     Reply Quote
alvin

Send message
Joined: 12 Mar 12
Posts: 29
Credit: 666,199
RAC: 0
Message 47464 - Posted: 4 Nov 2013, 11:47:50 UTC
Last modified: 4 Nov 2013, 12:08:29 UTC

could you disable any proxies for these uploads as discussed?
you may enable this later
also do you have antivirus on run?
would you exclude BOINC both folders from it or fully disable it?
ID: 47464 · Report as offensive     Reply Quote
Profile Thyme Lawn
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1283
Credit: 15,824,334
RAC: 0
Message 47465 - Posted: 4 Nov 2013, 14:03:43 UTC
Last modified: 4 Nov 2013, 14:18:18 UTC

Mark,

If your scheduler requests are failing due to their size (6,043,059 bytes for the one you posted debug for) there's a relatively easy way to work around that.

In the projects/climateprediction.net directory you should find a load trickle_up_*.xml files. BOINC will include all of them in CPDN scheduler requests and they will account for most of a request's size. Restricting the number of trickles in a request should work:

  1. create a temporary directory
  2. move some of the trickles to the temporary directory
  3. perform a manual project update
  4. if it succeeds move some of the trickles back from the temporary directory, if it doesn't move more of the trickles to the temporary directory
  5. repeat the last 2 steps until all of the trickles have been sent


The debug for your upload shows it failing after 15 minutes. Have you tried increasing the <http_transfer_timeout> value as I suggested in this post? I suspect the way data is being transferred between BOINC's upload cache and your proxy server is causing the cache to be refreshed too slowly.


"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer
ID: 47465 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 47466 - Posted: 4 Nov 2013, 20:22:20 UTC

Ah, I'd forgotten about that trick. It works great, once you work out the max number of trickles to have at a time. It's been 6, 7 years or so since the last time this was necessary, but I think that the number was less than something; 20?, 50?

And if some of the trickles get sent in the wrong order, don't worry. They get listed in a strange order on the model's page, but are stored correctly behind the scenes.

ID: 47466 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47469 - Posted: 5 Nov 2013, 8:46:32 UTC

I've managed to clear the trickles via the 56k dial up. Still working on getting the zip files cleared, at 10 hours each and about 6 or 8 each on the remaining machines it takes a while

The http debug stuff above was done without a proxy server. The transfers time out at some point with my proxy, a uk-based proxy or no proxy. I did try setting a 10 minute time-out at one point but it didn't seem to help.
BOINC blog
ID: 47469 · Report as offensive     Reply Quote
alvin

Send message
Joined: 12 Mar 12
Posts: 29
Credit: 666,199
RAC: 0
Message 47470 - Posted: 5 Nov 2013, 9:22:24 UTC - in response to Message 47469.  

Is any chance for project admins to establish file storage where you could upload files or you upload it to dropbox or whatever and they manually import it back to system to stop this saga to happen and give your wife a chance to use home phone?)
ID: 47470 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47483 - Posted: 6 Nov 2013, 11:00:35 UTC
Last modified: 6 Nov 2013, 11:02:48 UTC

3rd machine now cleared. 2 more to go. I managed to get a blazing fast 3.5k upload speed.

I let the wife use the phone all day Monday, so I'm good to use the phone line until Friday :)
BOINC blog
ID: 47483 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47517 - Posted: 10 Nov 2013, 3:36:52 UTC

Started on the 4th machine but now the server is out of space, so the 56k is having a rest :)
BOINC blog
ID: 47517 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47590 - Posted: 17 Nov 2013, 10:03:58 UTC

Went away for a few days and left the 4th machine to clear itself off. Its done and the 5th (last) machine is down to two files left to upload.
BOINC blog
ID: 47590 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47593 - Posted: 17 Nov 2013, 20:18:05 UTC

Final machine cleared off. All tasks reported.
BOINC blog
ID: 47593 · Report as offensive     Reply Quote
MarkJ
Avatar

Send message
Joined: 28 Mar 09
Posts: 126
Credit: 9,825,980
RAC: 0
Message 47619 - Posted: 21 Nov 2013, 10:03:11 UTC

My ISP has started asking questions after I complained. One of the things they asked for was a tracert. I did one. It was very slow after the first 2 hops (which are the ISP) upto about hop 13 and then all the remaining hops just timed out. It was (for some reason) going via Japan which is where it seems to die.
BOINC blog
ID: 47619 · Report as offensive     Reply Quote
alvin

Send message
Joined: 12 Mar 12
Posts: 29
Credit: 666,199
RAC: 0
Message 47620 - Posted: 21 Nov 2013, 10:13:06 UTC - in response to Message 47619.  

try same via dialup and show them
ID: 47620 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : Persistent upload problems

©2024 climateprediction.net