|
Message boards :
Number crunching :
The uploads are stuck
Message board moderation
Previous · 1 . . . 7 · 8 · 9 · 10 · 11 · 12 · 13 . . . 25 · Next
Author | Message |
---|---|
Send message Joined: 29 Oct 17 Posts: 938 Credit: 15,039,630 RAC: 24,176 |
Stony666 wrote: I have waited nearly two weeks until I post here because I had the hope that it could be fixed in a few days.The upload server is still up and functioning ok. There is an enormous backlog, so please be patient for a couple of days. I'm sure your uploads will happen soon. wujj123456 wrote: Best case: Server is saturated and we just need to be patient and wait for our turn.It was the storage. The cloud provider over-provisioned the disks and we suspect also put the OS on the same bare-metal as the data devices (slaps palm to forehead). But I didn't say that... leloft wrote: Doubly unlucky: I've just had the same refusal from both the first machine and now a second one, both refer to the same value 7168.00 MB. The good news is that one of the hosts has managed to upload 8 tasks.The 7168Mb tasks are fine. Perhaps the disk is just too full of results files at the moment and it will sort itself out in good time. |
![]() Send message Joined: 17 Aug 07 Posts: 8 Credit: 36,041,450 RAC: 17,297 |
I am currently uploading with 32MBit/s. It's all my internet connection can provide. Location is germany. |
Send message Joined: 5 Aug 04 Posts: 1106 Credit: 16,986,528 RAC: 5,566 |
I am currently uploading with 32MBit/s. It's all my internet connection can provide. Location is germany. I am currently uploading with 6.3 MegaBytes/second. 50 Megabits/second. My internet connection can provide 75 megabits/second. Location near New York City. Actual speeds are: Timestamp Download Upload Latency Jitter Quality Score Test Server 1/11/2023 14:35:27 76.28 Mbps 79.91 Mbps 5 ms 1 ms Excellent newyork02.speedtest.windstream.net ![]() |
Send message Joined: 7 Sep 16 Posts: 262 Credit: 33,662,515 RAC: 23,561 |
... both of my connections are pegged at 3Mbit up. No, that's not MB/s. That's Mbit. I'm hoping Starlink will clear up overnight, but it's gonna be a while. |
Send message Joined: 27 Mar 21 Posts: 79 Credit: 78,302,757 RAC: 1,077 |
FWIW, when I came home from work 3 hours ago it worked somewhat. It quickly worsened, and now _all_ transfer attempts fail. I've got the exact same situation now as the one @Stony666 described. Glenn Carver wrote: The upload server is still up and functioning ok. There is an enormous backlog, so please be patient for a couple of days. I'm sure your uploads will happen soon.There is some good news and some bad news: The good: Relative to the upload server outage (now almost three weeks), the backlog can't actually be very big. That's because many of us had to stop computing rather soon after the upload server outage started. (Everybody whose production is either constrained by Internet connection throughput, or by disk space.) The bad: The fact that _nothing_ is moving anymore for myself and evidently for some others doesn't make me optimistic that the backlog (however modest or enormous it might be) would clear anytime soon. And as long as nothing is moving, folks as myself can't resume computation. |
Send message Joined: 28 Jul 19 Posts: 148 Credit: 12,830,559 RAC: 228 |
I’ve now cleared my backlog and files are still trickling up slowly - only trouble is that I’m generating new files just slightly more often than the link is clearing them :-) Definitely progress as all of the outstanding WUs have gone and I can see my task list again. |
![]() Send message Joined: 29 Nov 17 Posts: 55 Credit: 7,750,140 RAC: 29,287 |
All mine have uploaded this afternoon and new ZIP files are disappearing and not being held up. |
Send message Joined: 7 Jun 17 Posts: 23 Credit: 44,434,789 RAC: 2,600,991 |
Fix for - Need more disk space. You currently have 0.00 MB available. Good advice, thank you. I set 'use no more than' to 1000G, 'leave at least' to 1G free, 'use at most' to 99% of disk, updated project and now the two hosts in question processing new units. 2/4 hosts uploading as well. Onwards and forwards. fraser |
Send message Joined: 29 Oct 17 Posts: 938 Credit: 15,039,630 RAC: 24,176 |
FWIW, when I came home from work 3 hours ago it worked somewhat. It quickly worsened, and now _all_ transfer attempts fail. I've got the exact same situation now as the one @Stony666 described.There may be some boinc-ness things going on. I vaguely remember Richard saying something about uploads will stop processing if it tries & fails 3 times? Or something like that? Uploads are still ok for me, I've got another 1000 to do. Richard? |
![]() Send message Joined: 15 May 09 Posts: 4439 Credit: 17,707,089 RAC: 27,575 |
There may be some boinc-ness things going on. I vaguely remember Richard saying something about uploads will stop processing if it tries & fails 3 times? Or something like that? Uploads are still ok for me, I've got another 1000 to do. I have vague memories of a limit on the number of tries and fails but it is a lot more than 3! Edit: Cleared all from the VM. The others after a nudge have just started again. Edit2: Something like 300 tasks an hour are being uploaded. (based on the reduction in tasks in progress. I didn't check tasks available to send which would increase it but not sure by how much. Will perhaps check that tomorrow. |
Send message Joined: 14 Sep 08 Posts: 103 Credit: 36,916,155 RAC: 53,863 |
Curious for folks with no problem connecting, are you able to ping upload11.cpdn.org? I still couldn't connect, but given the latency, I more or less expect to be the last wave. I just don't want to keep retrying in BOINC since it has the tendency to upload partial files. If ping can give me the signal, that would be much easier for me to know when I might be able to upload. |
Send message Joined: 29 Oct 17 Posts: 938 Credit: 15,039,630 RAC: 24,176 |
Curious for folks with no problem connecting, are you able to ping upload11.cpdn.org?I've got 10 concurrent uploads going (all day in fact), and if I try ping upload11.cpdn.org now it just hangs. So either it's too busy or they have disabled ping on the system for security. traceroute does 30 hops and ends in (several tries, same result): .... 27 * * * 28 * * * 29 * * * 30 * * *But both ping & traceroute were working fine for me yesterday evening when the server was up but httpd was not enabled on the machine. Probably load. Still, as you say, might be prudent just to wait a few more hours for the load to ease a bit. p.s. I'm in Cambridge so Oxford's not too far... |
Send message Joined: 4 Dec 15 Posts: 52 Credit: 2,235,763 RAC: 2,409 |
Curious for folks with no problem connecting, are you able to ping upload11.cpdn.org? I was able to upload my files a while ago, but atm a ping isn't getting through. Trying from Hanover, Germany. - - - - - - - - - - Greetings, Jens |
Send message Joined: 14 Sep 08 Posts: 103 Credit: 36,916,155 RAC: 53,863 |
Thanks. Looks like ICMP packets are blocked then. Guess will just let boinc client handle the retry on its own. Given others are making progress, hopefully it won't take too long to be my turn. |
Send message Joined: 16 Aug 16 Posts: 73 Credit: 52,932,477 RAC: 8,823 |
I haven't seen any uploads in the last week either wujj123456. |
Send message Joined: 29 Oct 17 Posts: 938 Credit: 15,039,630 RAC: 24,176 |
I haven't seen any uploads in the last week either wujj123456.FYI. The upload server only came back about 10:00GMT today (weds 11th). |
Send message Joined: 21 Dec 22 Posts: 5 Credit: 7,062,239 RAC: 17,035 |
I was able to upload all files, 100GB gone. |
Send message Joined: 1 Jan 07 Posts: 1012 Credit: 35,614,613 RAC: 25,152 |
There may be some boinc-ness things going on. I vaguely remember Richard saying something about uploads will stop processing if it tries & fails 3 times? Or something like that? Uploads are still ok for me, I've got another 1000 to do.If three uploads in succession fail to get through, BOINC will go into 'Project backoff' for a stated period - starting at a few minutes, increasing to several hours. During this time, existing uploads will not be retried. But any new upload will be tried just once. If that succeeds, the project backoff will be cleared, and existing uploads will be retried - until the next batch of failures. This is purely a temporary thing - the user can over-ride it at any time, either by retrying a single transfer, or choosing 'retry pending transfers' from the tools menu in BOINC Manager. |
Send message Joined: 5 Aug 04 Posts: 173 Credit: 14,648,469 RAC: 62,372 |
climateprediction.net 12-01-2023 01:22 [error] Error reported by file upload server: Server is out of disk space Supporting BOINC, a great concept ! |
Send message Joined: 29 Oct 17 Posts: 938 Credit: 15,039,630 RAC: 24,176 |
climateprediction.net 12-01-2023 01:22 [error] Error reported by file upload server: Server is out of disk spaceYes, I've seen it and reported it back. I believe that's 27Tb filled then. The uploaded tasks should be moving off to a transfer server, maybe that's not working. |
©2024 climateprediction.net