climateprediction.net home page
Will start crunching with linux.. Advice starting up?

Will start crunching with linux.. Advice starting up?

Questions and Answers : Unix/Linux : Will start crunching with linux.. Advice starting up?
Message board moderation

To post messages, you must log in.

AuthorMessage
Mephist0

Send message
Joined: 21 Feb 08
Posts: 47
Credit: 7,929,915
RAC: 0
Message 64313 - Posted: 10 Aug 2021, 7:39:22 UTC

Hello. I have been running CPDN a long time on Windows machines. But since there is no work for windows anymore i have not been running the project for quite some while now.

Planning of starting up a Ubuntu guest machine in my vmware environment to start crunching.
I have read about the 32bit libs that needs adding.. But.. Any more advice to be up and running quick?

Do you install the boinc package via apt? What is the name of the package?

How do you guys monitor the process of the WU? I will be running Ubuntu server.. So command line is what im going to use..

Any tips getting up and running quick is appreciated..

Also.. What can i expect in memory usage per WU? (Trying to plan how many CPU's i should add).
What disk space should i add to my VM?
ID: 64313 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,380,160
RAC: 3,563
Message 64314 - Posted: 10 Aug 2021, 8:13:54 UTC - in response to Message 64313.  

This thread gives the instructions to install via apt. There are currently server problems at Oxford preventing new work from being downloaded but I know Andy is on the IT support staff's back to try and get these sorted.
ID: 64314 · Report as offensive     Reply Quote
Mephist0

Send message
Joined: 21 Feb 08
Posts: 47
Credit: 7,929,915
RAC: 0
Message 64319 - Posted: 10 Aug 2021, 12:46:10 UTC - in response to Message 64314.  

Thanks, but i think you read my thread a little quick. :) I have already found the 32bit libs thread. But were looking for a hint of how to install and monitor it via linux (the boinc client).. and any other advice :) I tried looking through some threads here on the forum but did not find anything special. I guess its that easy to install so no one talks about it? :)
ID: 64319 · Report as offensive     Reply Quote
_Ryle_

Send message
Joined: 17 Aug 05
Posts: 21
Credit: 15,091,561
RAC: 22,127
Message 64322 - Posted: 10 Aug 2021, 16:15:21 UTC
Last modified: 10 Aug 2021, 16:17:33 UTC

"boinctui" is probably the monitor application you'd want on linux commandline.
sudo apt install boinctui

and just
boinctui

to start it in commandline.
Even though it is text based, it is still informative. I use it a lot myself, to monitor my rigs.

And of course
sudo apt install boinc-client

for the boinc application itself.

I dont know what people recommend for hdd size, here. I usually give it at least 100 GB to be safe. Maybe it's a bit much. Depends on amount of cores you run I guess. Maybe someone with more knowledge can answer more precise. I'm not familiar with vmware, so don't know if you can resize later on. It does work in Virtualbox fx, but that is different thing maybe.

I'd love to learn vmware some day, or some other virtualization. Virtualbox is user friendly, but maybe not as advanced.
ID: 64322 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1045
Credit: 16,506,818
RAC: 5,001
Message 64325 - Posted: 10 Aug 2021, 22:27:18 UTC - in response to Message 64322.  


I dont know what people recommend for hdd size, here. I usually give it at least 100 GB to be safe. Maybe it's a bit much. Depends on amount of cores you run I guess. Maybe someone with more knowledge can answer more precise.


Here is what I use with a 16-core processor, but the boinc-client is told to use at most 8 cores.Generally 3 or 4 climateprediction models are running, and the other 4 or 5 are running WCG and Rosetta. (The extra space is for OpenIFS tasks if I ever get any.)

Filesystem            1K-blocks      Used Available Use% Mounted on

/dev/sdb3             122908728  24737500  91904756  22% /var/lib/boinc

ID: 64325 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 64326 - Posted: 10 Aug 2021, 22:47:07 UTC - in response to Message 64313.  

Mephist0

If you're missing any 32 bit libs, then the task will crash at about 8 seconds, and you'll get a message on the task page that says which ones are missing.

The n216 models like lots of L3 cache.
With only 4 megs of this on processors in the recent past, this has been a limiting factor.
Unless you don't mind the tasks taking months to slowly finish, when you cram dozens of them onto the computer.
ID: 64326 · Report as offensive     Reply Quote
wateroakley

Send message
Joined: 6 Aug 04
Posts: 185
Credit: 27,083,655
RAC: 6,161
Message 64337 - Posted: 11 Aug 2021, 15:50:04 UTC - in response to Message 64313.  

Win10, Oracle VirtualBox 6.1 with ubuntu 20.04 VM.
Some tried and tested settings from our i7-3770 with 4 cores (8 threads) and 32GB memory,

Memory. Allow 3GB memory per cpu for CPDN and the VM OS. It may be more than some model types require, but others have suggested 3GB. I have no idea what future needs may be.
32GB physical memory lets you run a VM memory of 20GB and leaves enough room for Windoze to be happy.
We found that 16GB physical memory for four ubuntu VM cpus to run CPDN was too small. CPDN tasks were throttled by about 25%. It became unusable for heavy office applications (Word and Excel).
We tested a Q9650 cpu and 8GB memory with an ubuntu VM running 2 cpus, Yes it ran cpdn, but insufficient memory. Not worth the effort.

CPU. 4 physical cores, 8 threads. We run the ubuntu VM with 4 cpus.
Task manager reports that VirtualBox (running the VM & CPDN) takes about 55% of CPU.
We can run the rare Windows CPDN tasks on 2 or 3 cores in the Win10 host. Otherwise they run the Africa Rainfall tasks.
Trying to put all i7 cores into use (4 VM ubuntu threads with CPDN and 4 Win10 threads for CPDN) is not a good idea. CPDN tasks slowed down by circa 20% The throttle was probably from running out of L3 cache.

Disc. Put the ubuntu VM onto an SSD.
Allow at least 10GB of .vdi virtual disc per VM cpu. Use an expandable .vdi format. Minimum 40GB for 4 cpus.
Beware the generic VM youtubers, they wildly underestimate the disc size in their videos. My new friend is gparted.
If you want to avoid all the extra cpdn processing from continually hitting the data TBW on a large SSD, a 128GB SSD is an inexpensive option.

Good luck.
ID: 64337 · Report as offensive     Reply Quote

Questions and Answers : Unix/Linux : Will start crunching with linux.. Advice starting up?

©2024 climateprediction.net