climateprediction.net home page
Managing Disk Space Usage - a question

Managing Disk Space Usage - a question

Questions and Answers : Windows : Managing Disk Space Usage - a question
Message board moderation

To post messages, you must log in.

AuthorMessage
old_user99076

Send message
Joined: 20 Sep 05
Posts: 5
Credit: 14,499
RAC: 0
Message 22248 - Posted: 21 Apr 2006, 10:07:43 UTC

I am sure this is covered somewhere.

I am running a sulphur cycle 4.22 work unit. It is reaching way over 800MB in disk storage used.

I understand the complexity of climate and atmospjheoical modelling as I used to work for the local Meteorology Office here..

But I find that my disk is slowly being used up by the model I am running.

What files are work files that can be deleted after a trickle up or similar?

What has to remain in the Data Out directory? This directory has nearly 400 files totalling 750MB all generated over the last 5 weeks.

I am assuming that the growing list of files are of some value to someone, but if they are important to a scientist then I wish that scientist would provide the storage for it and let the application send them to his computer :). How many terabytes are being used up on people\'s PC around the world??
ID: 22248 · Report as offensive     Reply Quote
Profile Keck_Komputers
Avatar

Send message
Joined: 5 Aug 04
Posts: 426
Credit: 2,426,069
RAC: 0
Message 22250 - Posted: 21 Apr 2006, 10:53:31 UTC

Any directory under projects->climateprediction_net that does not match an active workunit listed in the BOINC manager can be deleted. While it is active I believe it needs all the files to be left alone.

One good thing about the coupled and any future models is that they will do this cleanup automatically in most cases.
BOINC WIKI

BOINCing since 2002/12/8
ID: 22250 · Report as offensive     Reply Quote
old_user99076

Send message
Joined: 20 Sep 05
Posts: 5
Credit: 14,499
RAC: 0
Message 22254 - Posted: 21 Apr 2006, 13:05:40 UTC - in response to Message 22250.  

I have deleted all directories/files not belonging to the current model being run. I am probably going to have to abandon the work unit and do no more work for Climate as it is just far too disk resource thirsty.

Has anyone that is running BOINC Manager 5.2.13 actually running multiple projects and the sum of all disk space being used by projects is in excess of 1GB? I have projects unable to download more work because of lack of disk space, despite the fact there is actually about 5GB free, BOINC is set to use over 2GB, but the maths tell me it is refusing to go beyond the 1GB mark. I suspect there is a variable in BOINC Manager\'s code that is limiting the calculations of disk space to one GB. If the variable is an unsigned interger it can hold values up to 2^31 (2GB), but if it is signed it would only hold values up to 2^30 or 1GB.
ID: 22254 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 22256 - Posted: 21 Apr 2006, 13:42:21 UTC

David

It\'s probably best if you Abort the sulphur unit. They aren\'t needed now.
Sulphur models reach a couple of Gigs before the end, where their data is zipped and uploaded. The new models only use about 600Megs.

And, as several of us told you on the BOINC site, there are a LOT of us with BOINC sizes greater than 1Gig. The problem is with your preference settings.


ID: 22256 · Report as offensive     Reply Quote
old_user122149

Send message
Joined: 30 Nov 05
Posts: 7
Credit: 42,511
RAC: 0
Message 22320 - Posted: 23 Apr 2006, 11:20:52 UTC - in response to Message 22250.  

not to put too finer point on it but why does the Climate prediction boinc project not clear up after its self?!?!?


I have 618mb in two folders that are for failed/errored climate prediction work units. My disk space is being wasted.

Is this an issue that is being looked into?
ID: 22320 · Report as offensive     Reply Quote
Profile astroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 22340 - Posted: 23 Apr 2006, 20:10:00 UTC - in response to Message 22320.  
Last modified: 23 Apr 2006, 20:20:19 UTC

not to put too finer point on it but why does the Climate prediction boinc project not clear up after its self?!?!?


I have 618mb in two folders that are for failed/errored climate prediction work units. My disk space is being wasted.

Is this an issue that is being looked into?

The \'slab\' and \'Sulphur Cycle\' Model results were zipped for possible future use. (Some of us were asked to upload entire Thermohaline Circulation [THC] Models a while back.)

The Coupled Model is designed to clean-up after itself but some crashes prevent that. (Edit: Given wider access to Broadband these days, more of the Model\'s output is uploaded to the server during the Run and, because of complaints over disk space, no longer kept by participants.)
"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 22340 · Report as offensive     Reply Quote
old_user122149

Send message
Joined: 30 Nov 05
Posts: 7
Credit: 42,511
RAC: 0
Message 22709 - Posted: 10 May 2006, 22:54:48 UTC - in response to Message 22340.  

I am up to 1.51 gig disk usage now after one or two more failed work units.
(It seem a little optimistic to expect no failure in workunits running for 2400+ hours - probability of failure is high - is there a thread about why work units are not broken down (optionally) into smaller sizes?)

Can I/Should I delete all the folders in the CP directory that have a different name from the current workunit being crunched?
ID: 22709 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 22710 - Posted: 10 May 2006, 23:35:07 UTC

is there a thread about why work units are not broken down (optionally) into smaller sizes?)

A thread, no. Lots of threads, yes. Also on the BOINC Message boards (here), the php Message Board (here), and the boards on the BBC site. Also in the FAQ. (To the left of here.)

Basically, small part of a model / BIG uploads/downloads. Also loss of data integrity caused by different maths librarys in different processors.

*******************

Can I/Should I delete all the folders in the CP directory that have a different name from the current workunit being crunched?

Yes.

ID: 22710 · Report as offensive     Reply Quote

Questions and Answers : Windows : Managing Disk Space Usage - a question

©2024 climateprediction.net