climateprediction.net home page
Moving projects from /var to /home

Moving projects from /var to /home

Questions and Answers : Unix/Linux : Moving projects from /var to /home
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,018
RAC: 3,616
Message 43289 - Posted: 26 Oct 2011, 6:16:15 UTC

I recently changed from Mandriva to the latest Ubuntu. As part of this change I installed BOINC from the package manager instead of running it from my home directory as I have always done in the past. Doing this it seems that all the data for projects is contained in /var/lib/boinc-client/projects. This means that to allow BOINC enough space to run more than one cpdn task at once I need to allow it to use 80% of the disk space. Can I move this data to my home directory and then create a link to it? I have /home on it's own much larger partition.

Thanks

Dave
ID: 43289 · Report as offensive     Reply Quote
Profile astroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 43295 - Posted: 26 Oct 2011, 16:52:17 UTC

It's been years since I ran Linux (SuSE and openSuSE) so I'm more than a little rusty. That said, I see no reason you can't simply exit CPDN and boinc, uninstall boinc, move the Data directory to wherever you want it in /home, and reinstall boinc manually so you can control the process.
"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 43295 · Report as offensive     Reply Quote
Profile Greg van Paassen

Send message
Joined: 17 Nov 07
Posts: 142
Credit: 4,271,370
RAC: 0
Message 43296 - Posted: 26 Oct 2011, 19:50:56 UTC - in response to Message 43289.  

Hi Dave,

I did this myself a couple of years back. Should be no problem. I didn't need to uninstall Boinc, just stop it, move the boinc-client directory and then create a symlink in its place. (I'm on Arch linux so I can't say for sure with Ubuntu.)

As usual, take a backup of the folder before restarting Boinc to be on the safe side. ;-)
ID: 43296 · Report as offensive     Reply Quote
old_user170894
Avatar

Send message
Joined: 3 Mar 06
Posts: 96
Credit: 353,185
RAC: 0
Message 43302 - Posted: 27 Oct 2011, 2:27:23 UTC
Last modified: 27 Oct 2011, 2:30:50 UTC

Moving the data and creating the link might work but the best way to do it is move the directory then open /etc/init.d/boinc-client in a text editor and change the line that says "BOINC_DIR=/var/lib/boinc-client" to "BOINC_DIR=/home/{username}/BOINC". That assumes the data will be in a directory named BOINC but you could put it anywhere in your home dir. {username} is of course your login name.

Make sure there are no spaces on either side of the = sign then save the file and exit the editor. That file is called "the boinc init script" and it is the script the system runs at boot time to start the BOINC daemon. It is a Sys V compliant script. Here is a page that has commands for starting/stopping/enabling/disabling the daemon, very handy commands to know so bookmark that page.

Whether you link to the data directory or change it in the init script, the problem you will run into is that installed from package manager the way you now have, BOINC runs on an account named boinc which does not have permission to write to any of your directories. One way to handle that is to include boinc in your group but that's not terribly secure. The better way to do it is to create the BOINC directory yourself then make boinc the owner of that directory.

If you move the existing data files into /home/BOINC/ and you become root to do so then ownership of those files might transfer to root. If so, you have to change the owner of the files back to boinc else boinc won't be able to access those files.

Another problem you'll run into is that BOINC manager doesn't know the RPC password when installed from package manager. If it doesn't know the password it cannot connect to the client. So, after you have the BOINC directory setup, start the BOINC service. The first time the client runs it will write the password to /home/BOINC/gui_rpc_auth.cfg. Then create a link to gui_rpc_auth.cfg in your home directory. When you start the manager it will run on your account (not boinc), it will look in your home directory for gui_rpc_auth.cfg, find it via the link, and read the password from it. Therefore give your account read access to /home/BOINC/gui_rpc_auth.cfg.

That should be enough to get you started. Let us know if any problems occur.
ID: 43302 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,018
RAC: 3,616
Message 43303 - Posted: 27 Oct 2011, 9:35:00 UTC - in response to Message 43302.  

Many thanks, worked after a few blips - for some reason clicking the x in boinc manager just minimises it on my system which I didn't know. So I have messed up some tasks which no doubt will go out again. Don't know why the package manager installs things the way it does, it would make more sense to me to put all the data in a .BOINC folder in home/username as most of the programs I use do.

Dave
ID: 43303 · Report as offensive     Reply Quote
old_user170894
Avatar

Send message
Joined: 3 Mar 06
Posts: 96
Credit: 353,185
RAC: 0
Message 43308 - Posted: 28 Oct 2011, 3:44:04 UTC - in response to Message 43303.  
Last modified: 28 Oct 2011, 3:49:34 UTC

Linux purists would argue that since the data files belong to the user named boinc, they should not be in your home directory. It makes sense from a security standpoint. Boinc downloads and runs a multitude of binaries on your computer. Getting you to run their binaries is every hacker's dream but we crunchers do it every day and don't give it a second thought. Suppose one day you attach to a new project, a bogus project run by hackers who claim to be nice people at some nice research institute studying puppies. Who doesn't like puppies? They sound legit (how could a BOINC project not be legit?) but their science app steals all your pr0n, credit card numbers, bank account numbers, passwords and then deletes all your files. Who needs that?

Now you might ask... then why aren't the data files stored in boinc's home directory? The response to that is... boinc is an unprivileged user who doesn't have a home directory therefore the files should go in /var/lib/boinc-client to limit any damage science apps might do. If you want easy access to that directory and the files in it then put a link to it on your desktop or in your home directory and add yourself to boinc's group. Don't add boinc to your group because then he has access to your files unless you go to the extra bother of restricting your group members from accessing your files which is doable but there is an easier way... put boinc in his own sandbox, make him play there and don't let him know anything about your sandbox. You, of course, are allowed to play in his sandbox.

You have other programs that store their data files in your home dir but they don't download binaries and run them. BOINC is unique in that respect. Use it but never trust it.
ID: 43308 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,018
RAC: 3,616
Message 43309 - Posted: 28 Oct 2011, 7:21:35 UTC - in response to Message 43308.  

Good points. - I don't currently run anything except cpdn on boinc though I did see that it has been hacked recently. Another alternative is I guess to make boinc a user with it's own home directory. This wouldn't put other user's data at risk and it would avoid disk space problems if a system has been set up with too small a / directory.

Dave
ID: 43309 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,018
RAC: 3,616
Message 43312 - Posted: 29 Oct 2011, 5:37:12 UTC - in response to Message 43309.  

Following above comments I decided to mover everything back. - must have somehow screwed up. couldn't work out what was wrong - uninstalled an reinstalled BOINC had to re-attach to CPDN. However on re-starting doesn't seem to attach to local host. - Had downloaded 2 tasks prior to this which seemed to be running fine. I Suspended them and shut down BOINC before turning computer off.

Dave
ID: 43312 · Report as offensive     Reply Quote
old_user170894
Avatar

Send message
Joined: 3 Mar 06
Posts: 96
Credit: 353,185
RAC: 0
Message 43313 - Posted: 29 Oct 2011, 6:51:27 UTC - in response to Message 43312.  

BOINC manager needs to be able to read the file file that contains the password in order to be able to connect to localhost (BOINC client). The file is gui_rpc_auth.cfg which was in /home/dave/BOINC/. If you followed my other suggestions, you put a link to /home/dave/BOINC/gui_rpc_auth.cfg in /home/dave/. Now the gui_rpc_auth.cfg is back in /var/lib/boinc-client so you need to delete the current link in /home/dave and create a new link that points to the new location of gui_rpc_auth.cfg. If you've already done that and manager still can't connect to client then perhaps the client isn't running.


ID: 43313 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,018
RAC: 3,616
Message 43314 - Posted: 29 Oct 2011, 7:02:53 UTC - in response to Message 43313.  

Thanks, all seems to be working now.

Dave
ID: 43314 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,018
RAC: 3,616
Message 43315 - Posted: 29 Oct 2011, 9:42:55 UTC - in response to Message 43314.  

Have gone back to old system of using the installer rather than package manager version. That way I don't have to worry too much about space etc. I will try the package manager system again when I put in a ssd to use for the system.

Dave
ID: 43315 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,018
RAC: 3,616
Message 43324 - Posted: 30 Oct 2011, 7:38:00 UTC - in response to Message 43315.  

Thought I had this cracked by going back to my old method of using the sh install to run from home directory. It runs fine when I start after install. However when I shut down and restart boinc-manager tells me it is not connected. when I try to reconnect to project it tells me taht boinc-client has shut down three times in last two minutes do I want to try again. Somewhere I assume there is a file that I have not deleted which is causing the problem but I haven't been able to work out where it is.

Any help gratefully received.

Dave
ID: 43324 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,018
RAC: 3,616
Message 43325 - Posted: 30 Oct 2011, 7:53:18 UTC - in response to Message 43324.  

Solved I think!
Un-installing the package manager version didn't get it going again. However I noticed in the terminal I started it from a notice about missing libcurl.so4. Installing the dev package for libcurl it has now started and resumed the tasks I was running.
ID: 43325 · Report as offensive     Reply Quote
Profile Greg van Paassen

Send message
Joined: 17 Nov 07
Posts: 142
Credit: 4,271,370
RAC: 0
Message 43335 - Posted: 30 Oct 2011, 16:33:30 UTC - in response to Message 43315.  

I will try the package manager system again when I put in a ssd to use for the system.

Dave
Hi Dave,

I just installed Ubuntu myself (hence my being quiet here, sorry about that) and moved the package's data dir as you wanted to do. Here are some notes if you do try the package manager boinc again.

The symlink method didn't seem to work (not sure why, didn't spend time on it) but changing the directory in both /etc/init.d/boinc-client and /etc/default/boinc-client did the trick. Just the latter should work in theory.

As Dagorath says, be careful of the ownership of the boinc directory, add yourself to the boinc group, and get gui_rpc_auth.cfg links sorted out.

If you suspend network communication beforehand then you can back up the boinc folder before the change, and restore it as many times as needed until boinc is working OK, without losing work units.
ID: 43335 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,018
RAC: 3,616
Message 43339 - Posted: 30 Oct 2011, 20:23:38 UTC - in response to Message 43335.  

Thanks Greg.

It is now working and not stopping when I switch off and on again so I think I will resist the temptation to fiddle for a while and ignore the maxim, "If it ain't broke fix it till it breaks then fix it!"

Dave
ID: 43339 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1045
Credit: 16,506,818
RAC: 5,001
Message 45465 - Posted: 17 Jan 2013, 3:18:49 UTC - in response to Message 43308.  

What I do is have a separate partition for boinc. I do mount it under /home.
I have a user named boinc in /etc/passwd and friends. So it is a real user.
No one else on the machine need be in the boinc group. So basically, boinc applications can try denial of service by using up the entire partition, but it will not hurt anyone else as far as disk space is concerned. I do not have it chroot-ed though, which might be a good idea. I have not thought about how to do that.
ID: 45465 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,376,018
RAC: 3,616
Message 45467 - Posted: 17 Jan 2013, 8:23:59 UTC

Food for thought - I will think about doing it that way next time I have to do a re-install. For the moment, as everything is working on both boxes I am not inclined to change anything!
ID: 45467 · Report as offensive     Reply Quote
Greg Tippitt

Send message
Joined: 22 Nov 05
Posts: 2
Credit: 2,202,322
RAC: 0
Message 51429 - Posted: 18 Feb 2015, 6:53:39 UTC - in response to Message 45467.  

I have a small linux cluster with 7 nodes at home that I use for playing with MPC stuff. The nodes all boot from USB flash drives and are otherwise diskless. To feed data to its 182 AMD cores, I needed more disk space than I had on the USB thumb drives.

The easiest way I found to move the BOINC data folder on linux was to edit the file "/etc/default/boinc-client". I changed the "BOINC_USER=" to the user id that I normally use for logging onto the machines to avoid permission problems. I changed the "BOINC_DIR=" to a "BOINC" folder in my home folder that is a mount for a iSCSI drive from my disk server.

To move an existing installation of BOINC on a machine, is used the following commands to move the files to the new folder and change the ownership of the files.

sudo move /var/lib/boinc-client/* /home/gtippitt/BOINC
sudo chown -R /home/gtippitt/BOINC gtippitt

I reinstalled the BOINC software to be on the safe side. If asked about the config file, remember to keep the edited one rather than replace it with the default.

sudo aptitude reinstall boinc boinc-client boinc-manager


When I add a new node to the cluster, I copy the "/etc/default/boinc-client" from another node before installing BOINC, and everything lands in its new folder.

Each of the cluster's nodes is a SuperMicro H8QME-2 motherboard with 4 hex core CPUs. With all the jobs running, I got lots of errors when using NFS or CIFS/SMB for disk mounts. Even with dedicated hardwired full duplex gigabit ethernet links from each node to the disk server, no amount of network tuning seemed to eliminate the errors. Changing to iSCSI for the disk mounts has been much cleaner and faster.


ID: 51429 · Report as offensive     Reply Quote

Questions and Answers : Unix/Linux : Moving projects from /var to /home

©2024 climateprediction.net