Anybody had one of these?

If I want to do VBox Boinc WUs, i have to start the Client with Admin rights. Also, if I want to start a regular Virtual Machine, I have to do it with Admin rights. If i don't, i get the following VBox error:

Critical error

COM-Object for VirtualBox couldn't be created

Errorcode: 
E_ACCESSDENIED (0x80070005)
Komponente: 
VirtualBoxClientWrap

sudo usermod -a -G vboxusers $USER
sudo usermod -a -G vboxusers boinc

Please think about having one section on your website that lists all the major issues that one could have, with clear solutions.

These are 3 issues I have hit in the last week or so, all of which stopped us crunching

1) 100GB UI Limit
2) 2x core_count, cant download too many uploads Limit
3) Invalidated tasks by switching out HDD

[...] I posted that I had just bought a new HD and was about to swap it out due to so many uploads. Not one mod or team member mentioned the problems that would happen if I did this.

[...] In terms of this problem, from what I gather this "option" has been turned on by CPDN and GPUGrid, but has been left off by most other projects.

[...] Either way it is a bitter pill to swallow which is why there is no point us running this project now.

-rw-r--r-- 1 boinc boinc        19 Jan 17 20:31 boinc_ufs_upload_file_0.zip
-rw-r--r-- 1 boinc boinc         0 Jan 17 20:39 boinc_ufs_upload_file_1.zip
-rw-r--r-- 1 boinc boinc        19 Jan 17 20:49 boinc_ufs_upload_file_2.zip
-rw-r--r-- 1 boinc boinc        19 Jan 17 20:55 boinc_ufs_upload_file_3.zip
-rw-r--r-- 1 boinc boinc        19 Jan 17 21:02 boinc_ufs_upload_file_4.zip
-rw-r--r-- 1 boinc boinc        19 Jan 17 21:12 boinc_ufs_upload_file_5.zip
-rw-r--r-- 1 boinc boinc        19 Jan 17 21:14 boinc_ufs_upload_file_6.zip

handle_upload_file_status: can't parse boinc_ufs_upload_file_1.zip

It seems to me that BOINC upload is for the most part a background process that does its job relatively well. Pretty much the only times uploading generates user complaints are when upload servers aren't working right. The length of this upload outage is rather unique but even so the progress has been very good so far.

Richard Haselgrove wrote:

I've found a technique which seems to help. Go through this sequence:

• Suspend network activity (BOINC Manager, Advanced view, Activity menu)
  
• Retry all transfers (Tools Menu)
  
• Allow network activity
  

That just cleared the last six tasks due by 25 January, on one machine.

And it's just worked on my second machine as well - tied off the loose ends from 14 tasks in a single hour.

14/01/2023 17:12:27 | climateprediction.net | Reporting 14 completed tasks

I usually wait until both connections have stalled, and the queue has gone into 'project backoff'. Not sure if that's a significant part of the procedure, but it can't spoil it.

I am getting pretty good response from the upload server, though not as good as it was about 10 (?) days ago. I have high speed (75 megabit/second) fiber optic Internet connection, but I am in USA and the server is in England. So right now, traceroute does not make it all the way to the server. It did recently.

But notice the big delay from New York to London. Step 8 to step 9. This is usual and unchanged. IIRC, the server is at about step 22.
[...]

16  ral-r26.ja.net (146.97.41.34)  80.929 ms  83.553 ms  79.803 ms

Thanks for the traceroute output. The server doesn't respond to ICMP packets, probably blocked for security. ral-r26.ja.net seems to be the last hop everyone sees from traceroute. Your latency is pretty low, compared to my 140-150ms to reach ral-r26.ja.net.

14  ral-r26.ja.net (146.97.41.34)  34.969 ms  34.954 ms  34.178 ms

I have 1400 tasks to upload. This means 2.5 TB. if there is no wonder the backlog is forever.

I have about 2.5 TB result files and can upload about 10 GB. This means to resolve the backlog takes 250 days

I am now down to 16 tasks uploading. I think I will be clear by the end of play tomorrow. Keeping to just one task running till backlog is cleared.

I think I've got a workaround to the 'too many uploads' issue. Thanks to all who contributed bits towards this. It appeared that actively crunching clients had more success at securing upload slots, so I changed <ncpus> from 24 to 40 in cc_config and reread it. The client downloaded 8 units and started to process them. The host has been uploading solidly since 21.00 last night and has no trouble regaining an upload slot within seconds of dropping it. I have no real idea why this should have worked, except to guess that the ability to secure an upload slot is somehow enhanced by having an actively crunching client.
Best,
fraser

A staggered start for OpeniFS would not help. OpenIFS hits its peak memory every timestep, the tasks will not stay the same 'distance' apart and would still crash if memory is over-provisioned. This is a current feature of boinc client that needs fixing. It's not a criticism of the code, PC hardware & projects have moved on from early boinc apps and the code needs to adapt.

The only sane way to do this is for the client to sum up what the task has told it are its memory requirements and not launch any more than the machine has space for. OpenIFS needs the client to be smarter in its use of volunteer machine's memory.

And I don't agree this is for the user to manage. I don't want to have to manage the ruddy thing myself, it should be the client looking after it for me.

I think all we can do at present is provide a 'Project preferences' set of options on the CPDN volunteer page and set suitable defaults for no. of workunits per host, default them to low. With clear warnings about running too many openifs tasks at once.

xii5ku wrote:

It's specific to CPDN. IIRC I had basically flawless transfers early after the start of the current OpenIFS campaign, but soon this changed to a state in which a certain amount of transfers got stuck at random transfer percentage. I worked around this by increasing max_file_xfers_per_project a little. This period ended at the server's X-Mas outage. — When I came home from work yesterday (evening in the UTC+0100 zone), I found one of the two active computers transferring flawlessly, the other one having frequent but not too many transfers getting stuck at random percentage.

Maybe the first is occupying available bandwidth/slot. I see similar with my faster broadband.

Getting stuck is perhaps to be expected giving your remote location??

As for other projects: I don't have projects with large uploads active at the moment. TN-Grid and Asteroids with infrequent small transfers and Universe with frequent smallish transfers are working fine right now on the same two computers, in the same boinc client instances as CPDN. Earlier in December, I took part in the PrimeGrid challenge which had one large result file for each task, and they transferred flawlessly too.

Depends what you mean by 'large' & small in relation to transfer size.

We opted for more 'smaller' files to upload rather than less 'larger' files to upload, precisely for people further afield. Each upload file is ~15Mb for this project. I still think that's the right approach even though I recognise people got a bit alarmed by the number of upload files that was building up. Despite the problem with 'too many uploads' blocking new tasks, I am reluctant to up the maximum upload size limit, esp if you are having problems now.

So to summarize: It's a problem specifically with CPDN's upload server, and this problem existed with lower severity before the Holidays outage.

I think that's part of your answer there. When there isn't a backlog of transfer clogging the upload server, it can tick along fine.

If it really troubles people to see so many uploads building up in number we can modify it for these longer model runs (3 month forecasts).

It's kinda funny I was not able to upload anything due to transient HTTP error, but can see these messages like everyone else. ¯\_(ツ)_/¯

xii5ku wrote:

It's not for a lack of the client's trying. First, I have still 1 running task per each of my two active computers, which causes a new untried file to be produced every ~7.5 minutes. Second, even if I intervene when the client applies an increasing "project backoff", i.e. make it retry regardless, nothing changes. Earlier yesterday, while I still was able to transfer some (with decreasing success), the unsuccessful transfers gut stuck at random transfer percentages. Later on, still long before the new 'server is out of disk space' issue came up, eventually all of the transfer attempts got stuck at 0 bytes transferred.

Unlike some other contributors here, I have a somewhat limited upload bandwidth of about 8 Mbit/s. Maybe this plays a role why I have been less successful than others, maybe not.

Do have these issues with other projects? And, when CPDN isn't playing catch up with their uploads, do you have the problem then? Just trying to understand if this is a general problem you have or whether it's just related to what's going on with CPDN at the moment.

xii5ku wrote:

FWIW, when I came home from work 3 hours ago it worked somewhat. It quickly worsened, and now _all_ transfer attempts fail. I've got the exact same situation now as the one @Stony666 described.

The bad: The fact that _nothing_ is moving anymore for myself and evidently for some others doesn't make me optimistic that the backlog (however modest or enormous it might be) would clear anytime soon. And as long as nothing is moving, folks as myself can't resume computation.

There may be some boinc-ness things going on. I vaguely remember Richard saying something about uploads will stop processing if it tries & fails 3 times? Or something like that? Uploads are still ok for me, I've got another 1000 to do.

Hardware requirements for current "OpenIFS 43r3 Perturbed Surface" work:

The following items need to be taken into account, in descending order of concern:
1. Upload bandwidth of your internet link.
2. Disk space.
3. RAM capacity.
99. CPU. This one doesn't really matter, except of course that CPU core count has got an influence on how many tasks you may want to run in parallel, and that core count × core speed influences how many tasks you can complete per day at most. One or both of these factors (concurrent tasks, average rate of task completions) influence the sizing of items 1…3 in this list.

The upload server is still up and functioning ok. There is an enormous backlog, so please be patient for a couple of days. I'm sure your uploads will happen soon.

There are completed WUs approaching deadline soon. I have WUs due in 11 days and 11 days no longer look that long given the upload server has been down for two weeks. If somehow the new storage array again has problems, or some new issues showing up which is not that uncommon for new systems, we probably need server to extend deadlines to not waste work.