climateprediction.net home page
Cleaning up Boinc Projects Data Files

Cleaning up Boinc Projects Data Files

Questions and Answers : Windows : Cleaning up Boinc Projects Data Files
Message board moderation

To post messages, you must log in.

AuthorMessage
MartinNZ

Send message
Joined: 22 Mar 06
Posts: 144
Credit: 24,695,428
RAC: 0
Message 46250 - Posted: 17 May 2013, 5:07:42 UTC

Hi,

I have 66GB of data in the C:\ProgramData\BOINC\projects\climateprediction.net directory and wish to clean this up a bit as it is obvious BOINC is not doing this. Most of the data relates to old projects and it would be great if I could get rid of it.

I assume it is safe to delete data directories that are no longer running e.g. hadam3p_pnw_aqpb_1990_1_007873682, but there are many other files including zipped folders, exe, dll, png, gz, xml, most of which are related by name to the old data directories, but some are not, e.g. hadcm3trans_data_6.04_windows_intelx86.zip. Also some odd ones like zia0692, globe.rgb which I assume would stay.

Are there any guidelines on what can be deleted?

Also, anything I can do to make sure that BOINC deletes the old data?

Regards,
Martin
ID: 46250 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7477
Credit: 23,566,382
RAC: 2,956
Message 46252 - Posted: 17 May 2013, 6:25:03 UTC - in response to Message 46250.  

Seems to be 3 separate questions, so:

1) This isn't a BOINC thing, it's something for each project.
In our case, when a model crashes before finishing, it leaves behind some remnant data, which has to be cleaned up manually. The "clean up" routine is in the code at the end of the program, which doesn't get reached.

2) This has been asked about and answered quite a few times over the years, but I'm not going looking for them.

3) So, starting again, one doesn't get rid of everything, only folders of defunct models.
a) Make a note of the names of models still in your Tasks tab.
b) Go through your \projects\climateprediction.net folder, checking the names of sub-folders. If any of the names DON'T appear in the Tasks tab, then delete that sub-folder.



Backups: Here
ID: 46252 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3135
Credit: 8,577,104
RAC: 14,206
Message 46253 - Posted: 17 May 2013, 6:44:27 UTC - in response to Message 46252.  
Last modified: 17 May 2013, 6:51:12 UTC

You mention other files, not in the task directories. I have deleted the .xml and .zip files without problems.

I appreciate that these take up very little space, (1.5kb) typically for the .xml files but One word of caution would be that I think it would be much easier to delete something one shouldn't if careless doing this rather than just deleting the obsolete folders which can contain a million times or more data each.
ID: 46253 · Report as offensive     Reply Quote
MartinNZ

Send message
Joined: 22 Mar 06
Posts: 144
Credit: 24,695,428
RAC: 0
Message 46254 - Posted: 17 May 2013, 12:25:17 UTC - in response to Message 46253.  

Thanks Les and Dave,

Pretty much as I thought, so those main directories deleted (59GB worth). I've left the rest of the files as in the scheme of things they don't take up much room. I did have a search of the posts for something, but didn't find what I wanted - just depends on the search terms used I guess.

ID: 46254 · Report as offensive     Reply Quote
Profile Iain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1048
Credit: 6,463,915
RAC: 66
Message 46255 - Posted: 17 May 2013, 21:42:29 UTC

If you would prefer BOINC Manager to do the work for you, then a project reset or remove/attach will clear things up without the need to 'pick and choose'. The only thing you have to remember after a remove is to merge the new host that will be created with the old host. Of course, this can only be done when no CPDN models are in progress, otherwise they will be lost.
ID: 46255 · Report as offensive     Reply Quote
MartinNZ

Send message
Joined: 22 Mar 06
Posts: 144
Credit: 24,695,428
RAC: 0
Message 46262 - Posted: 21 May 2013, 2:48:09 UTC - in response to Message 46255.  

Hi Iain, I was thinking of that for other reasons and to do so I normally stop accepting tasks and let the old ones finish - takes a couple of weeks at the moment. I had noticed that I was getting quite a lot of task failures, but when I looked into it decided that my PC (ID: 960478) seemed to be no better or worse that other PCs, in fact my hours run on the failures was often better than other PCs on the same task. Was considering updating BOINC as I'm still running on v6.10.58, although my memory of previous posts is that BOINC updates have little effects on the CPDN number crunching, so decided against it.

Stderr messages were mixed with model crashes and suspend CPDN monitor messages. For reference BOINC is always shut for Windows updates, runs as a service and is shutdown and restarted each night while backups are done (auto), runs on 75% of processors and memory (12GB) is 60% max. I do run heavy graphics packages which occassionally cause the CPDN tasks to idle, but if I need full power I normally suspend the CPDN project for a few hours. Noticed there was some talk on other threads about the susceptibility of some models to this, but couldn't see what else I could to do ensure the smooth running. In the end I decided there wasn't much I could do so left things are they are. Let me know if you think different.
ID: 46262 · Report as offensive     Reply Quote
Profile Iain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1048
Credit: 6,463,915
RAC: 66
Message 46264 - Posted: 21 May 2013, 7:46:12 UTC

That all looks very good to me. There has been a run of work-unit configuration and download errors, which volunteers can do nothing about: we've all got messy-looking model records - and the failures can leave an amount of junk behind.

The 7-series BOINC has worked well on my Windows machines (though, to be fair, I've been running almost exclusively work simulations for about a year). My 7-series BOINC on Mac happily churns out HADCM3N whenever it can get them. So a BOINC upgrade should be no problem.
ID: 46264 · Report as offensive     Reply Quote
Profile MikeMarsUK
Volunteer moderator
Avatar

Send message
Joined: 13 Jan 06
Posts: 1498
Credit: 15,613,038
RAC: 0
Message 46265 - Posted: 21 May 2013, 8:56:59 UTC - in response to Message 46262.  
Last modified: 21 May 2013, 8:59:45 UTC

...

Stderr messages were mixed with model crashes and suspend CPDN monitor messages. For reference BOINC is always shut for Windows updates, runs as a service and is shutdown and restarted each night while backups are done (auto), runs on 75% of processors and memory (12GB) is 60% max. ...


As Iain says, there have been lots of duff workunits recently so seeing a lot of failures in your list is not surprising.


I seem to recall that the 'suspend CPDN monitor' messages are caused when Boinc decides that the PC is busy and drops the models out of memory temporarily. In theory this might cause a problem if it happened at exactly the wrong moment (for example, when building the 25/50/75% upload files). It is probably only a theoretical concern, but I did make the following changes on my own PC:
* Suspend work when computer is in use = no
* Suspend work if CPU usage is above = 0%
* Leave tasks in memory while suspended? = yes
* Memory: when computer is in use, use at most = same as the 'not in use' figure

I have plenty of memory so the above is fine for me, but of course it depends on how you use your PC.
I'm a volunteer and my views are my own.
News and Announcements and FAQ
ID: 46265 · Report as offensive     Reply Quote
MartinNZ

Send message
Joined: 22 Mar 06
Posts: 144
Credit: 24,695,428
RAC: 0
Message 46270 - Posted: 23 May 2013, 8:09:54 UTC - in response to Message 46265.  

Thanks Mike. The discussion has lead me to think about my system and I seem to remember discussion on PCs, but can't seem to find any now. Perhaps they were on the climateprediction.net forum that now seems to be unavailable. I'll post a separate item as it's pretty off topic.
ID: 46270 · Report as offensive     Reply Quote

Questions and Answers : Windows : Cleaning up Boinc Projects Data Files

©2021 climateprediction.net