Over the last few weeks, a lot of my tasks have uploaded. And if I look at the CPDN web site for my tasks, almost all of them have received credit. Yet if I look at my projects list, my average work done has steadily decreased to about 50.54. If I look at the statistics graph for the last month, it has dropped from 1600 to about 50.

I am confused that the CPDN web site seems to know I have received lots of credit but wherever the projects list and the statistics graph get their data from, they have not been getting the credits for a month or so.

Notice: my signature box (below) seems to know I am getting credits (though I do not know if it is up-to-date).

Server: 128c/256t

Currently running 512GB memory with a default OS Swap file of 2.1GB (which is 75% - 100% in usage).

What amount I should set it for the best results for CPDN?

Hi
One of mine just ended but not accepted:
https://www.cpdn.org/result.php?resultid=22272660

<core_client_version>7.17.0</core_client_version>
<![CDATA[
<message>
Process still present 5 min after writing finish file; aborting</message>
<stderr_txt>

Looks silly

Is there any way that this could be made a user selectable option to set the default value before it is downloaded? I would want this on every WU I process and I can imagine so would all the other volunteers who process 24/7.

We've had this discussion about adjusting the checkpointing already in this (or another) thread - if I wasn't supposed to be wrapping Christmas presents I'd find it.

This is never going to be a user selectable option because it requires an understanding of how the model works and if you get it wrong, it could both seriously thrash your filesystem and delay your task. The model is capable of generating very large amounts of output, which have to be tuned carefully to run on home PCs. We might tweak it after these batches if it proves to be causing problems, which is why feedback is always welcome.

Adjusting write I/O from OpenIFS tasks

Further to Conan's point about the amount of write I/O. It can be adjusted but only once a task has started running. The adjustment made will reduce the checkpoint frequency, meaning if the model does have to restart from a shutdown, it will have to repeat more steps. This change does NOT affect the model's scientific output as that's controlled differently.

ONLY make this change if you leave the model running near-continuously with minimal possibility of a restart. Do NOT do it if you often shutdown the PC or boinc client, otherwise it will hamper the progress of the task. If in doubt, just leave it.

To make the change:
1/ shutdown the boinc client & make sure all processes with 'oifs' in their name have gone.
2/ change to the slot directory.
3/ make a backup copy of the fort.4 file (just in case): cp fort.4 fort.4.old
4/ edit the text file fort.4, locate the line:
NFRRES=-24,
and change it to:
NFRRES=-72,
Preserve the minus sign and the comma. This will reduce the checkpoint frequency from 1 day (24 model hrs) to 3 days (72 model hrs). But, it will mean the model might have to repeat as many as 3 model days if it has to restart.
5/ restart the boinc client.

The changes can only be made once the model has started in a slot directory, not before.

Add a project max concurrent to that and job's a good'un

Do you mean like this?

$ cat app_config.xml 
<app_config>
    <project_max_concurrent>5</project_max_concurrent>
<app>
    <name>OpenIFSname1</name>
    <max_concurrent>2</max_concurrent>
    </app>
<app>
    <name>OpenIFSname2</name>
    <max_concurrent>2</max_concurrent>
   </app>
<app>
    <name>hadam3_8.09</name>
    <max_concurrent>3</max_concurrent>
    </app>
<app>
    <name>hadam3_8.52</name>
    <max_concurrent>3</max_concurrent>
    </app>
<app>
    <name>hadcm3s_8.36</name>
    <max_concurrent>3</max_concurrent>
    </app>
<app>
    <name>hadsm3_8.02</name>
    <max_concurrent>3</max_concurrent>
    </app>
</app_config>

If so, why have the the itemized list of the traditional work units in there at all?

You would to add a separate <app>...</app> section for each IFS variant, once we know the exact application names in use. You could then use <max_concurrent> to limit each IFS type, but I don't see a way to limit the total IFS number of all types, once multiple versions are in play at the same time.

Depends how likely it is that batches of multiple variants will be out there at once I guess.

I have an upper limit of 12 (in the winter, and 8 in the summer) of total BOINC jobs. (No air conditioning.)
I am currently allowing an upper limit of 5 CPDN jobs. So now I have a prototype app_config.xml file that allows a max of 2 for each of the OpenIFS types and a max of 3 for each of the "traditional" ones. If there are only one variant of each distributed at a time, I should be OK. And the max limit of BOINC jobs will prevent disaster. I hope.

It looks, in part, like this:

$ cat app_config.xml 
<app_config>
<app>
    <name>OpenIFSname1</name>
    <max_concurrent>2</max_concurrent>
    </app>
<app>
    <name>OpenIFSname2</name>
    <max_concurrent>2</max_concurrent>
   </app>
<app>
    <name>hadam3_8.09</name>  <---<<<
    <max_concurrent>3</max_concurrent>
    </app>

I guessed at the names of the traditional tasks. One is marked with <---<<< . Is that the correct way the traditional ones are named?


This prototype is not in effect yet

I will not restrict my 24 core box to running 4 cores with the other 20 waiting for memory - I’ll block the OpenIFS jobs if they won’t play happily.

24 cores, or 24 threads? There's a difference, and especially for CPDN tasks, "more threads" is not always better.

I've got a pair of 3900X boxes (12C/24T), and I've written some scripts that track "instructions retired per second." I rarely see a difference between 12 and 18 tasks running for most BOINC workloads (and if I do, the 18 task box is usually accomplishing less actual work per second), and the CPDN tasks typically seem to peak around 8 threads, though I don't recall seeing much of a difference dropping to 6. It's not just the cores that matter - it's the cache. I've absolutely seen "more threads mean lower aggregate system throughput," and CPDN is particularly bad for that.

I would expect 4C on a 12T/24C processor to be below optimum, but... depending on the tasks, not by much. Though we'll have to see once the actual OpenIFS tasks show up.

There are some WCG tasks that use very little cache and I get linear speedups with number of threads assigned, but CPDN is definitely not that.

I will not restrict my 24 core box to running 4 cores with the other 20 waiting for memory - I’ll block the OpenIFS jobs if they won’t play happily.

I might be wrong but I think in this situation you would not get the OpenIFS tasks anyway, because the server would see there's not enough free memory available. Remember it's boinc making the decisions, not the model.

"BOINC starts multiple OpenIFS tasks because there are free CPU slots, even though the total memory for the tasks exceeds what's available. "

Can this be overcome by limiting the number of cores available to BOINC before downloading any of the IFS models? Allthough I have a four core CPU the box only has 24Gb of RAM.

I'd like to put it more bluntly and say that CPDN tasks are definitely very sensitive to interruptions (and I believe it's relatively well documented in the forums). By far the worst of any project I'm aware of. Even a couple of LHC subprojects that must be run to completion without interruption, will just restart from the beginning. CPDN's error rate is at least 10%, Bryn Mawr's (who posted above) is over 11%. Mine is over 22%. Many of those are due to restarts (especially if happens more than once). I'd expect CPDN to have a higher error rate than other projects due to valid reasons (i.e. "Negative Pressure Detected"). But for a project that has workunits that take days to weeks to complete, 10%+ error rate is too high, I think, as that means that days' and weeks' worth of processing time is wasted because the tasks can't handle interruptions well. Glenn, it's encouraging to hear that you'd like to look into this and potentially fix it. I'm not sure which OS is worse but the issue affects Windows, macOS, and Linux tasks.

Dave, that's a very poor survival for the linux tasks. Other projects seem to handle a cold restart just fine. I am surprised because operational models are pretty resilient to hardware & data failures but it could be something in the wrapper code that's not tolerating restarts properly. I'll ask the CPDN team as I'm interested to find out.

May not be quite that bad. I will when work appears again, start keeping some real data on this rather than relying on my impressions.

Milkyway for example has a 'project preferences' page under the user account which allows you to limit the number of cores in workunits sent to you. CPDN doesn't support this at present because until now they have not done any multicore work.

True. For all projects, I believe the <project_max_concurrent>4</project_max_concurrent> will work. it certainy works for all of mine. You can pick a different number for each project. For MilkyWay, the app_version stuff, especially <avg_ncpus>4</avg_ncpus> limits the number of processors per work unit. I would be very surprised if current ClimatePrediction tasks look at this. If you put something like this in for CPDN, I do not know if it would cause errors, but it would almost certainly be ignored.
[/b]

[/var/lib/boinc/projects/milkyway.cs.rpi.edu_milkyway]$ cat app_config.xml 
<app_config>
    <project_max_concurrent>4</project_max_concurrent>
<app_version>
      <app_name>milkyway_nbody</app_name>
      <plan_class>mt</plan_class>
      <avg_ncpus>4</avg_ncpus>
   </app_version>
</app_config>

[/var/lib/boinc/projects/climateprediction.net]$ cat app_config.xml 
<app_config>
    <project_max_concurrent>4</project_max_concurrent>
</app_config>

Its a Linux host.

Any way to limit the number of jobs it does at once? I run other projects on the VM as well and don't want to lower the thread count.

Sorry, I've been away from this project for a while, and hadn't kept up to date with recent changes. It would normally be on https://www.cpdn.org/prefs.php?subset=project, but I see it's been taken away.

It worked fine before, why are they messing about?

Because they need back end systems to create WUs in the first place and validate and post process the WUs on return, all of which is project related and not part of Boinc.

But they already had this and will be using the same scientific programs as before, they're not going to change all that. And why on earth didn't they get this one up and running before they stopped using the other one?! Imagine if Google shut down for 3 months while they moved house.

If you look at the updates they’ve provided the development they’re doing is WebSphere / Message Broker and whilst MB has been around for a long time WS is quite new and the combination is very much current technology for real time transaction processing and is complex and difficult to get right - especially if you get into scenarios like dual centre working for security fallback.

This I know having worked on just such a system and seen the problems first hand.

It worked fine before, why are they messing about?

What's concerning to me is that BOINC is not using fancy, bleeding edge, failure prone technology. It's an absolutely ancient technology stack - straight up LAMP, as far as I can tell from the server install guides (Linux, Apache, MySQL, PHP). It's not the sort of thing that should be hard to port, and while WCG is more complex than some others, unless they've gone absolutely nuts or have zero "legacy Linux sysadmins," it shouldn't take more than a couple weeks to move. I assume the bulk of the time was moving data around, but... even then, just drive a server around and hook up 10G server to server. I don't get how this is nearly so long a transition as it is, and I'm not at all optimistic that they're "Nearly, almost, just a tippy tappy bit more... almost... any day now, soon... *crickets*" about the move.

Is this project generating any work for windows machines? I've had a Windows 10 machine attached for several months and nada . . . I used to get work up until a year or so -- stopped getting work so removed the project from my four machines and put them to work on other things. Then reconnected one machine awhile back and still nada . . .