climateprediction.net home page
How to make and use backups.

How to make and use backups.

Questions and Answers : Windows : How to make and use backups.
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile adrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,021,020
RAC: 816
Message 16337 - Posted: 30 Sep 2005, 14:37:59 UTC

Because of the long run times of CPDN units. I started out making a backup copy of the CPDN directories once a week or so, when I started.

Sure enough, I had a model 66% done when something glitched it and it crashed. No worries I thought, I have the backup. I found out, however, that the backup is useless without the slots directories and the clientstate xml file. Also, that it is not simply a case of recovering these directories and files, that it is necessary to edit the xml file in some undiclosed manner.

Now with the sulphur cycle, (and indeed as suggested in the wiki), backups would seem to be even more desirable.

Can someone who knows, therefore, either directly here at the project site or in the Wiki, document the prescribed procedure for making backups, and if necessary, recovering a model from the backups.

This should take into consideration that most people have more then 1 BOINC project running, (hence the xml file editing), and will not want to loose all their other projects whilst trying to recover CPDN\'s. Also considering the variation in the core client versions out there.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 16337 · Report as offensive     Reply Quote
crandles
Volunteer moderator

Send message
Joined: 16 Oct 04
Posts: 692
Credit: 277,679
RAC: 0
Message 16339 - Posted: 30 Sep 2005, 15:55:02 UTC

You mean like
<a href=\"http://boinc-doc.net/boinc-wiki/index.php?title=Backup_BOINC\">Wiki backup BOINC</a>?

I regard that page as rather incomplete. I was hoping someone else in the wiki editting team would pick up on this and improve it.
Visit BOINC WIKI for help

And join BOINC Synergy for all the news in one place.
ID: 16339 · Report as offensive     Reply Quote
Profile adrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,021,020
RAC: 816
Message 16340 - Posted: 30 Sep 2005, 16:05:48 UTC

2 comments, first, yes, it is no good as it is because it does not address the issues of multiple projects, second, it may help if when one searches the Wiki for \"backup\" or \"backups\" it didn\'t say \"no pages\" - okay, if I was used to the Wiki format and read on, I might have found the obviously inadequate current entry.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 16340 · Report as offensive     Reply Quote
old_user23880
Volunteer tester

Send message
Joined: 10 Oct 04
Posts: 223
Credit: 4,664
RAC: 0
Message 16348 - Posted: 1 Oct 2005, 3:23:29 UTC

On the other cpdn boinc board there\'s another thread running about backups. Maybe Ray Brown\'s post there will help.
__________________________________________________

ID: 16348 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 16350 - Posted: 1 Oct 2005, 8:00:37 UTC

I don\'t think that you are going to get much, if any, advice on editing client_stat.xml for multiple projects, as it is fairly complex.
You may have to \'write the book\' yourself.
Basically, though, just suspend BOINC to unlock the lock file, and then copy the *entire* BOINC folder to another part of your hd.

I think it\'s possible that the only people who do backups are those who are really serious about cpdn, and that they run cpdn as the only project on a computer, with other projects, if any on other computers.
So they don\'t have the problem of editing the xml.
If anyone posts differently, you can then ask them how they do it.

Perhaps the simple way, (after a crash), is to suspend cpdn, finish the other projects, suspend them, get back the saved BOINC, suspend all the other projects, (which have been finished), and restart cpdn.
This will probably require BOINC 4.45, which has, I think, an option to stop the download of more wus from a project.

ID: 16350 · Report as offensive     Reply Quote
Profile adrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,021,020
RAC: 816
Message 16352 - Posted: 1 Oct 2005, 11:16:51 UTC
Last modified: 1 Oct 2005, 11:18:03 UTC

@Les

The thought of running the other projects to completion before XCOPY\'ing the whole lot back had occurred to me. It struck me, however, that even with the other projects prevented from downloading work, (you can do this with any client if you use the disk space setting on the projects web site to prevent downloading - you don\'t need 4.45), after you have restored the entire BOINC structure and fire it up, it will start crunching the old wu\'s from the other projects again unless you edit the files to prevent this.

I do not know if it will screw the other projects up if they start getting wu\'s returned again.

Surely if people want to run CPDN and no other projects, they would use the stand-alone CPDN and not BOINC?
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 16352 · Report as offensive     Reply Quote
Profile Andrew Hingston
Volunteer moderator

Send message
Joined: 17 Aug 04
Posts: 753
Credit: 9,804,700
RAC: 0
Message 16354 - Posted: 1 Oct 2005, 11:51:17 UTC - in response to Message 16352.  

I do not know if it will screw the other projects up if they start getting wu\'s returned again.

I guess that if you just abort WUs already returned it shouldn\'t cause a problem.

Surely if people want to run CPDN and no other projects, they would use the stand-alone CPDN and not BOINC?

No. CPDN is now designed to use the BOINC front end. There is still the legacy classic option for those who need it, but it is no longer under development and does not offer new experiemnts, such as the sulphur models currently. Users are asked to use BOINC if possible.

ID: 16354 · Report as offensive     Reply Quote
crandles
Volunteer moderator

Send message
Joined: 16 Oct 04
Posts: 692
Credit: 277,679
RAC: 0
Message 16356 - Posted: 1 Oct 2005, 12:29:53 UTC - in response to Message 16354.  
Last modified: 1 Oct 2005, 12:33:20 UTC

I guess that if you just abort WUs already returned it shouldn\'t cause a problem.


I also would hope so, but I wonder if

Start BOINC without internet connection,
Detatch all projects except climateprediction,
Reconnect internet, and
Reattach all projects

would be a safer procedure?
Visit BOINC WIKI for help

And join BOINC Synergy for all the news in one place.
ID: 16356 · Report as offensive     Reply Quote
crandles
Volunteer moderator

Send message
Joined: 16 Oct 04
Posts: 692
Credit: 277,679
RAC: 0
Message 16357 - Posted: 1 Oct 2005, 13:10:47 UTC

I have put some stuff into the <a href=\"http://boinc-doc.net/boinc-wiki/index.php?title=Backup_BOINC#No_other_projects\"> Wiki backup article</a>

I could easily have written something stupid or left something out, so do review whether it makes sense and make any suggested comments you think are appropriate.

When I say copy a backup, I have not said whether it is necessary to delete what is there first. Is it necessary/wise to delete first?
Visit BOINC WIKI for help

And join BOINC Synergy for all the news in one place.
ID: 16357 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 16359 - Posted: 1 Oct 2005, 13:26:08 UTC

Thanks for the additions. I was working things out as I typed it, and wasn\'t sure if it made sense, or was just plain wrong in places.
Disconnecting from the net is good, as is \'aborting\' in place of \'suspending\'.

I think the Wiki makes sense, but it would be best if a few other people commented to make sure.
We also need a volunteer to try it out. adrianxw?

ID: 16359 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 16362 - Posted: 1 Oct 2005, 14:44:30 UTC

Just a bit on editing client_state.xml, for anyone else reading this thread later.

I have done a bit of editing previously, and have found that BOINC seems to constantly rebuild the file.
Even if you disconnect the cable to the internet, and then delete the file, it will soon re-appear, I think after a data save to the hd.

So it\'s probably not possible to use editing to solve the \"backup one project, but not others\" problem.
BOINC is too sneaky for this.

ID: 16362 · Report as offensive     Reply Quote
Profile adrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,021,020
RAC: 816
Message 16363 - Posted: 1 Oct 2005, 14:46:21 UTC
Last modified: 1 Oct 2005, 15:03:03 UTC

-----
There is still the legacy classic option for those who need it, but it is no longer under development and does not offer new experiemnts, such as the sulphur models currently. Users are asked to use BOINC if possible.
-----

Okay, I didn\'t know that. When I joined CPDN, BOINC was a \"new\" way to crunch in addition to the stand-alone, I\'d never run the SA so have not followed it\'s fate.

>>> We also need a volunteer to try it out. adrianxw?

Got it. I will have a play, write down EXACTLY what I did, and what happened, (looks longingly at his 280 hour crunched sulphur cycle and wonders if he will see it again..). May take a day or so to run down the other 5 projects on this machine, but CPDN was set to 27% of the machines time, so was the heaviest user, thus might not be too bad.

Bear in mind I am using the 4.25 core client, so will be fiddling disk quotas to get the downloads stopped.

1 thing that occurred to me. Why do I need to detach and then reattach my other projects? If, after the restore, I just abort all the wu\'s in progress, the client for the project will just ask for new wu\'s won\'t it? If the client has been updated in the intervening period, it will download with the wu.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 16363 · Report as offensive     Reply Quote
Profile adrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,021,020
RAC: 816
Message 16368 - Posted: 1 Oct 2005, 15:56:04 UTC
Last modified: 1 Oct 2005, 16:05:05 UTC

Okay, I have aborted my CPDN and am waiting for the queues for my other projects to run down.

>>> BOINC is too sneaky for this.

Perhaps the facility to backup a project on demand, or at timed intervals/milestones should be built into the BOINC Manager software? Other projects like CPDN may come along later.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 16368 · Report as offensive     Reply Quote
Profile old_user5994

Send message
Joined: 31 Aug 04
Posts: 239
Credit: 2,933,299
RAC: 0
Message 16382 - Posted: 2 Oct 2005, 6:21:19 UTC - in response to Message 16339.  

You mean like
<a href=\"http://boinc-doc.net/boinc-wiki/index.php?title=Backup_BOINC\">Wiki backup BOINC</a>?

I regard that page as rather incomplete. I was hoping someone else in the wiki editting team would pick up on this and improve it.

I thought it was pretty good!

Hmmm, ok, I will look at it next week ...
ID: 16382 · Report as offensive     Reply Quote
Profile adrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,021,020
RAC: 816
Message 16386 - Posted: 2 Oct 2005, 16:45:33 UTC

I have conducted an experiment. A description and blow by blow account can be found <a href=\"http://www.adrianxw.dk/personalsite/boinc/cpdn_bu.html\">here</a>. I would appreciate any comments.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 16386 · Report as offensive     Reply Quote
crandles
Volunteer moderator

Send message
Joined: 16 Oct 04
Posts: 692
Credit: 277,679
RAC: 0
Message 16387 - Posted: 2 Oct 2005, 17:08:13 UTC

Thanks for trying it out Adrianxw.

I suggested xcopy in the wiki so a simple batch file can be created for simpler running of backup procedure muliple times. Yes, I would use windows explorer for copying to a second (one-off) backup. The Wiki doesn\'t go into much detail about this.....


Visit BOINC WIKI for help

And join BOINC Synergy for all the news in one place.
ID: 16387 · Report as offensive     Reply Quote
Profile adrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,021,020
RAC: 816
Message 16388 - Posted: 2 Oct 2005, 17:48:25 UTC

>>> suggested xcopy

Yes, I know, I didn\'t do that so as a divergence from the published procedure, I felt I should document it.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 16388 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 16389 - Posted: 2 Oct 2005, 19:57:27 UTC

I\'m glad that it work, Adrian.
It\'s all a bit messy, but as long as people are methodical, and keep a proceedure list, this may be the answer.
(Makes me glad that I don\'t run mutiple projects, as Murphy would be sure to get me if I tried this.)

The only bad bit, is that now cpdn has the error message on your account page.
This will stay there, even if you successfully complete the model.
But it\'s only words. It spoils the look of your page, but doesn\'t affect credits, the user graphs, or the science data.
This part may need working on.

ID: 16389 · Report as offensive     Reply Quote
Profile adrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,021,020
RAC: 816
Message 16391 - Posted: 2 Oct 2005, 21:06:36 UTC

The bigger question is will it affect the reply when the wu is completed? I don\'t really care how my page looks. As long as the wu is accepted as completed when it is, in the best faith, then we have lost nothing.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 16391 · Report as offensive     Reply Quote
Profile astroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 16406 - Posted: 3 Oct 2005, 17:48:35 UTC

Hi, Adrian,

There is no connection between the Trickle/stats and Science databases. If the summary files are uploaded at end of run, the work is golden. (I have several \"errored\" runs that finished okay after restart from a backup.)

Jim
"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 16406 · Report as offensive     Reply Quote
1 · 2 · Next

Questions and Answers : Windows : How to make and use backups.

©2024 climateprediction.net