climateprediction.net home page
PNW upload issues

PNW upload issues

Message boards : Number crunching : PNW upload issues
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Profile Byron Leigh Hatch @ team Carl ...
Avatar

Send message
Joined: 17 Aug 04
Posts: 289
Credit: 44,103,664
RAC: 0
Message 52742 - Posted: 27 Oct 2015, 17:47:38 UTC
Last modified: 27 Oct 2015, 18:01:22 UTC

I think the servers must be getting hammered with all the uploads,
and so I think I will suspend my network activity on BOINC for a day or so to give the servers a break.
patience is required :)

10/27/2015 9:33:42 AM | climateprediction.net | Temporarily failed upload of hadam3p_afr_e1uy_2011_1_010355437_0_3.zip: transient HTTP error
10/27/2015 9:33:42 AM | climateprediction.net | Backing off 03:43:10 on upload of hadam3p_afr_e1uy_2011_1_010355437_0_3.zip
10/27/2015 9:33:43 AM | | Internet access OK - project servers may be temporarily down.
10/27/2015 9:38:13 AM | climateprediction.net | Temporarily failed upload of hadam3p_eu_f4b0_1996_1_010197472_1_9.zip: transient HTTP error
10/27/2015 9:38:13 AM | climateprediction.net | Backing off 04:21:34 on upload of hadam3p_eu_f4b0_1996_1_010197472_1_9.zip
10/27/2015 9:38:14 AM | | Project communication failed: attempting access to reference site
10/27/2015 9:38:18 AM | | Internet access OK - project servers may be temporarily down.
ID: 52742 · Report as offensive     Reply Quote
Profile astroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 52744 - Posted: 27 Oct 2015, 21:07:43 UTC - in response to Message 52721.  

Some of these comments are what Prof.Wallom meant when he asked us to be patient. After all, bandwidth on the receiving end is not infinite.

Since my referenced post, half of my machines have transfers suspended, which takes more than half my queue out of play -- for a day or more. (My files are no more important than those of other participants.)

Patience, please.
"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 52744 · Report as offensive     Reply Quote
Eirik Redd

Send message
Joined: 31 Aug 04
Posts: 391
Credit: 219,888,554
RAC: 1,481,373
Message 52745 - Posted: 27 Oct 2015, 22:06:15 UTC

Patience is the virtue now. I'm allowing only 2 out of 7 machines to upload, at a combined total of 90 KB/s (about 1 megabit/s). Those two machines won't be clear of uploads until early next day. Then I'll let the others upload, one at a time.
Patience, patience.
Give the upload servers a day or three at least, to get back to normal. After an 11-day outage, that's a good upload rate.


ID: 52745 · Report as offensive     Reply Quote
Profile JIM

Send message
Joined: 31 Dec 07
Posts: 1152
Credit: 22,107,227
RAC: 2,727
Message 52747 - Posted: 28 Oct 2015, 3:56:39 UTC

One of my machines has now uploaded all of its 70 or so zip files. The other machine is starting to upload a roughly equal number of zips. So it is coming along nicely.

ID: 52747 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4349
Credit: 16,552,689
RAC: 4,403
Message 52749 - Posted: 28 Oct 2015, 13:43:16 UTC - in response to Message 52747.  

Three zip files left out of about 40. However all three are giving "can't resolve host name" in the log now. One of them got about three quarters of the way through before it fell over.
ID: 52749 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4349
Credit: 16,552,689
RAC: 4,403
Message 52750 - Posted: 28 Oct 2015, 19:49:12 UTC - in response to Message 52749.  

And now down to the last zip which has started :)
ID: 52750 · Report as offensive     Reply Quote
Roland Hughes

Send message
Joined: 7 Mar 12
Posts: 11
Credit: 3,921,344
RAC: 0
Message 52751 - Posted: 28 Oct 2015, 21:31:31 UTC

I turned off new work units on my AMD 6-core machine with SSD and 24Gig of RAM about a week ago. Was trying to be "polite" before wiping machine for new Ubuntu 15.10 release. Still waiting on about a dozen uploads. Had to abort one climate prediction job which estimated it had another 150+ hours to complete as I didn't want to wait yet another week for it to finish. Machine has a video card with 384 CUDA core so kind of odd package would take that long being only thing running on the box.

Odd that the upload server is so resource constrained when the downloads seem to happen at full throttle pace.

I can give it one more day then I _need_ to change the Linux flavor on that machine for another project.
ID: 52751 · Report as offensive     Reply Quote
noderaser
Avatar

Send message
Joined: 17 Nov 04
Posts: 14
Credit: 1,678,427
RAC: 289
Message 52752 - Posted: 28 Oct 2015, 21:49:30 UTC - in response to Message 52751.  

Machine has a video card with 384 CUDA core so kind of odd package would take that long being only thing running on the box.

CPDN doesn't have any GPU work, everything runs on a single core of your CPU. IMHO, multi-threaded apps would be nice but I doubt they'll be making any major changes mid-experiment.
Click Here to see My Detailed BOINC Stats
ID: 52752 · Report as offensive     Reply Quote
Eirik Redd

Send message
Joined: 31 Aug 04
Posts: 391
Credit: 219,888,554
RAC: 1,481,373
Message 52755 - Posted: 29 Oct 2015, 3:16:48 UTC - in response to Message 52751.  

I turned off new work units on my AMD 6-core machine with SSD and 24Gig of RAM about a week ago. Was trying to be "polite" before wiping machine for new Ubuntu 15.10 release. Still waiting on about a dozen uploads. Had to abort one climate prediction job which estimated it had another 150+ hours to complete as I didn't want to wait yet another week for it to finish. Machine has a video card with 384 CUDA core so kind of odd package would take that long being only thing running on the box.

Odd that the upload server is so resource constrained when the downloads seem to happen at full throttle pace.

I can give it one more day then I _need_ to change the Linux flavor on that machine for another project.


A dozen uploads should take less than 6-8 hours, based on the upload speeds I'm seeing here now, and typical 70-90MB zip files, even accounting for server congestion.

Please don't sweat it, if you need to reconfigure or update, or change your OS, or whatever, and ditch a few CPDN models -- cest la vie. Whatever has been uploaded already -- is uploaded -- and thanks.

On the other hand, it's entirely possible -- I've done it more than once -- to save the boinc folder with its status files and the /etc/ and /etc/default/ files, and restart successfully after a total disk wipe and version or even distro change, but that takes an hour or four hours or -- of your human time -- checking libc versions, getting all the 32-bit libs right -- if you like doing such detail work - good -- if not -- a few uploads won't be missed that much.

About the relative slowness of uploads vs downloads at CPDN - some recent models download about 100MB, and upload (70*12)+90 MB. There may be other factors.

Noli perspirare, and thanks for contributing

ID: 52755 · Report as offensive     Reply Quote
Roland Hughes

Send message
Joined: 7 Mar 12
Posts: 11
Credit: 3,921,344
RAC: 0
Message 52765 - Posted: 29 Oct 2015, 21:03:47 UTC - in response to Message 52755.  

That will happen tomorrow. I have to get up and do that in the early morning hours. Have satellite Internet here so "bonus time" is 2am-8am local time. You only get 10Gig/month when ordinary people are awake, but I'm allowed 50Gig/month during that narrow window. When I was young I would just stay up, but now I must set alarm for something like 4am then go to office to kick things off. It has been my experience that one can pull down the shiny new Ubuntu ISO only seconds after "official" release and it __still__ needs to pull down 500Meg of updates/language packs...so...have to sacrifice the sleep.
ID: 52765 · Report as offensive     Reply Quote
Profile astroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 52766 - Posted: 29 Oct 2015, 22:25:42 UTC - in response to Message 52765.  

Roland,

Please note the option in boinc manager:

Preferences/network usage (option) Network usage allowed . . .
You can use it to limit hours of connection and allow boinc to stay awake late to get the job done. Hope it works for you.

Please note that settings made in boinc manager work only on that machine.
"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 52766 · Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Number crunching : PNW upload issues

©2024 climateprediction.net