climateprediction.net home page
Run Linux work units with Windows 10 WSL

Run Linux work units with Windows 10 WSL

Questions and Answers : Unix/Linux : Run Linux work units with Windows 10 WSL
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 63911 - Posted: 25 Apr 2021, 18:32:25 UTC - in response to Message 63906.  

I have installed WSL as indicated in “Guide to using The BOINC Client in Microsoft´s WSL” (Version 0.8): Title: WSL installation, Steps: 1 to 7.
I am wondering how I might check, if WSL version 2 is running? Where and with which command, it might get the version of WSL?
Under Titel: BOINC Client installation in WSL and link to a project, there is a Note: “WSL, by default uses all available cores and memory of you PC”and a command
notepad "$ env: USERPROFILE / .wslconfig"
When I execute this command, I get the following error (Spanish): “Bloc de Notas, El nombre de archive, el nombre de directorio o la sintaxis de la etiqueta del volume no son correctos.”
However I am able to click this messages away and put the lines:
[wsl2]
memory = 3GB # Limit the machine memory to 3GB
processors = 4 # Limit the cores available to the VM to 4
in the archive.
I would like to ask: where, with which name and with which extension the file shall be saved? I guess somebody with VM experience might answer this question as well.
Thanks a lot!
ID: 63911 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 63912 - Posted: 25 Apr 2021, 19:46:31 UTC - in response to Message 63911.  

Just to remind everyone: You have to enable VM in the BIOS... for WSL2. I did this 5 minutes ago, now I am on WSL2. Hope we will soon see, if it works with climateprediction.net
I am still interested how to limit the processor count and RAM for WSL2.
ID: 63912 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2035
Credit: 55,359,893
RAC: 20,056
Message 63913 - Posted: 25 Apr 2021, 22:54:45 UTC - in response to Message 63911.  
Last modified: 25 Apr 2021, 22:55:24 UTC

I would like to ask: where, with which name and with which extension the file shall be saved? I guess somebody with VM experience might answer this question as well.
Thanks a lot!

I haven't done much with WSL so can't be of a lot of help. However, this link

https://docs.microsoft.com/en-us/windows/wsl/wsl-config#configure-global-options-with-wslconfig

tells you to save the file .wslconfig (no extension), in your Windows user's root directory, i.e. C:\users\klepel (or whatever name you use for your Windows user). Then restart WSL 2 and it will read the file upon restart and that should limit CPUs and RAM used in the VM.
ID: 63913 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 63914 - Posted: 26 Apr 2021, 3:10:24 UTC - in response to Message 63913.  

I would like to ask: where, with which name and with which extension the file shall be saved? I guess somebody with VM experience might answer this question as well.
Thanks a lot!

I haven't done much with WSL so can't be of a lot of help. However, this link

https://docs.microsoft.com/en-us/windows/wsl/wsl-config#configure-global-options-with-wslconfig

tells you to save the file .wslconfig (no extension), in your Windows user's root directory, i.e. C:\users\klepel (or whatever name you use for your Windows user). Then restart WSL 2 and it will read the file upon restart and that should limit CPUs and RAM used in the VM.
Worked! Thanks a lot!
ID: 63914 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 63915 - Posted: 26 Apr 2021, 3:25:39 UTC
Last modified: 26 Apr 2021, 3:28:38 UTC

I just would like to give an up-date!
It looks very promising since I got WSL2 installed today!

Do not despair of the two WUs, which crashed at about 13000 seconds mark. It was too many restarts after limiting WSL to 2 processors and 16GB RAM.
Task Manager indicates the use of about 7% CPU and 6.7 GB RAM at the moment.

It was fairly strait forward with the Guide from our Italian friends and once I had figured out, I had to make a change in the BIOS to make VM to work.
Now it seems easy to operate: I made a text file with the boinccmd commands I will use and now it is just copy past to manage Boinc in WSL2.
ID: 63915 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 63921 - Posted: 29 Apr 2021, 18:56:00 UTC - in response to Message 63915.  

Second up-date:
I received the first tickles for the two WUs running on this WSL2 enabled computer: https://www.cpdn.org/results.php?hostid=1517859
So this looks very promising! Hopefully this will motivate some heavy Windows users to install WSL2 on their computers as it is a tool from Microsoft Windows. After the initial hassle all works very well!
ID: 63921 · Report as offensive     Reply Quote
Eirik Redd

Send message
Joined: 31 Aug 04
Posts: 381
Credit: 180,987,000
RAC: 135,173
Message 63959 - Posted: 8 May 2021, 4:11:54 UTC

WSL seems to " get better all the time" like the way this going
ID: 63959 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 63960 - Posted: 9 May 2021, 18:54:55 UTC
Last modified: 9 May 2021, 18:55:20 UTC

Third up-date:
I am able to report back that the first two WUs have finished successfully. So my WSL2 installation works!
I am considering installing it on a second Win10 computer, which runs 24/7.
ID: 63960 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3204
Credit: 8,978,317
RAC: 7,027
Message 63961 - Posted: 9 May 2021, 19:26:53 UTC - in response to Message 63960.  

Great news!
ID: 63961 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 63963 - Posted: 12 May 2021, 17:46:47 UTC

Sad news: Automated Windows May Up-date crashed two running climetaprediction.net WUs in my WSL2 instant.
ID: 63963 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 64015 - Posted: 30 May 2021, 0:44:50 UTC
Last modified: 30 May 2021, 0:45:32 UTC

Forth up-date:
I finished another two WUs on this WSL Computer. However the RAM usage is increasing by the day, now it is: 10.3 GB. I will wait a few days more and restart, if it does not stop to increase.
ID: 64015 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 64137 - Posted: 7 Jul 2021, 21:01:49 UTC

Monthly MS Windows Up-Date crashed two climateprediction.net WUs on WSL computer again!
Sorry for that!
ID: 64137 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2035
Credit: 55,359,893
RAC: 20,056
Message 64139 - Posted: 7 Jul 2021, 23:58:24 UTC - in response to Message 64137.  

@klepel

Have you done a google search for a command line/script that will either cleanly shutdown, or pause/save the vm before rebooting. I've seen people with vms running within VirtualBox be able to do this. It wouldn't do much good for a power failure or Windows blue screen problem, but for a Windows reboot or shutdown, it could prevent the loss of models?
ID: 64139 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 64140 - Posted: 8 Jul 2021, 1:13:31 UTC - in response to Message 64139.  

Have you done a google search for a command line/script that will either cleanly shutdown, or pause/save the vm before rebooting. I've seen people with vms running within VirtualBox be able to do this. It wouldn't do much good for a power failure or Windows blue screen problem, but for a Windows reboot or shutdown, it could prevent the loss of models?

No, I have not searched for this kind of solution, It would not have occurred to me, and it might be well behind my skills…
But to this particular incident: I opened a second Ubuntu stance on my computer and used the “—quit” command and waited until BOINC closed in the first Ubuntu stance (where BOINC was running). Then I closed both WSL Ubuntu Windows (At least I think so) and restarted the computer to install the Windows Up-Dates. This has worked before! But it seems, it does not work always.
I was warned about that, as I lost two WUs during the Win10 May Up-Date.
On the other WSL Computer, I am guilty… I installed a NVIDIA Driver without quitting WSL Ubuntu where BOINC was running - NVIDIA Driver unexpectedly rebooted before I was able to quit BOINC and close WSL Ubuntu.
So everybody shall be warned: Quit BOINC on WSL and wait a few seconds before your close Windows or restart your computer - sometimes it works sometimes it does not!
ID: 64140 · Report as offensive     Reply Quote
wateroakley

Send message
Joined: 6 Aug 04
Posts: 118
Credit: 17,920,192
RAC: 4,236
Message 64146 - Posted: 8 Jul 2021, 16:35:03 UTC - in response to Message 64139.  

Running CPDN tasks with ubuntu VM under Virtualbox appears to be reasonably resilient to the unannounced Windows10 updates. Certainly better with the VM than the loss rate of CPDN tasks on native windoze from automatic updates. Since I lost one task on 14/4, five more updates have passed without loss. ubuntu VM has complained a couple of times when I restart the VM, with 'file error' issues, necessitating a manual reboot and restart.

I recently increased the physical memory to 32GB (from 16GB) and allocated 20GB memory to the ubuntu VM (from 10.5GB). Yesterday was the first unannounced windows update since the memory upgrade. No losses and the ubuntu VM did not complain after restarting. It's also got enough physical memory to stop windoze swapping out office applications/data or blanking the video to the screens when I'm trying to work on documents.
ID: 64146 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3204
Credit: 8,978,317
RAC: 7,027
Message 64148 - Posted: 8 Jul 2021, 19:40:41 UTC - in response to Message 64146.  

I was impressed when an electrician working here caused the main trip switch to flip and on restarting all five running tasks resumed with no issues and have since completed.
ID: 64148 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 61
Credit: 55,914,059
RAC: 15,423
Message 64151 - Posted: 9 Jul 2021, 1:19:41 UTC

Just to add: The WUs crashed with missing ZIP files. Unfortunately, I have not written down the exact Error message, but I think this error is quite usual after a system failure.
Beside this monthly WIN Up-Dates, WSL is pretty stable.
ID: 64151 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7528
Credit: 23,789,883
RAC: 3,624
Message 64154 - Posted: 9 Jul 2021, 2:00:50 UTC - in response to Message 64151.  

The missing files are the ones that haven't yet been created.
ID: 64154 · Report as offensive     Reply Quote
wateroakley

Send message
Joined: 6 Aug 04
Posts: 118
Credit: 17,920,192
RAC: 4,236
Message 64213 - Posted: 25 Jul 2021, 16:56:22 UTC

Virtual machines. If you are considering running a Linux virtual machine for CPDN using Windows 10 with Oracle VirtualBox 6.1 and ubuntu 20.04 VM, after four months ...

Memory. 16GB physical memory is the bare minimum. The usable VM memory will max out at circa 11GB, leaving 5GB for the M$ OS and bloatware. The Win10 i7-3770 16GB memory PC example, with ubuntu VM and four CPDN tasks, was unworkable for heavy office application users (excel/word/pdf) - the screens would regularly go blank. Resource Manager reported all physical memory was in use. With three CPDN tasks and 10.5GB VM memory it was just about workable, slightly better with two tasks. QED: more physical memory needed.

With 32 GB of physical memory this Win10 PC now runs VitrtualBox with the ubuntu VM and four CPDN tasks; plus heavy office applications and three of the very rare Windows CPDN tasks. The ubuntu VM config is 4 cpu (4 of 8 threads) and 20GB virtual memory. Resource manager reports 28GB physical memory in use, 55% cpu load for VirtualBox and 45% cpu for everything else.

I recall 3GB memory per CPDN task is required? If you don't do anything else on a 4 cpu (8 threads) PC, 24GB physical memory may be enough for running Win10 and VirtualBox with an ubuntu VM and 4 CPDN tasks. If you have a newer multi-processer PC then more memory will be needed,

Virtual Disc. The ubuntu VM will need about 10GB of virtual disc per virtual cpu. That's a virtual disc size of 40GB to run the VM and four CPDN tasks. Youtube videos with tutorials for creating an ubuntu VM in VirtualBox wildly understate the virtual disc storage that is required for CPDN. Schoolboy disc size mistakes in Linux can be fixed with Gparted. If you're worried about Total Bytes Written (TBW) on your expensive SSD, then a small (128GB) SSD is very cheap option.

Old machines, are they worth the effort? We created an ubuntu VM with CPDN on an older headless PC with Win10 and VirtualBox (Q9650 quad cpu, 8GB memory, 128GB SSD). Yes, CPDN worked with two tasks, but there's insufficient memory to run more than 2 tasks on a 4-cpu machine. Not much else would run happily either. QED: Don't bother.
ID: 64213 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3204
Credit: 8,978,317
RAC: 7,027
Message 64214 - Posted: 25 Jul 2021, 17:34:20 UTC

I recall 3GB memory per CPDN task is required?


The current Linux tasks on my Ryzen seem to max out at less than 2GB/task but by the time you count overheads, I would agree 3GB/task is about right. Don't forget that while I am not holding my breath, at some point the OpenIFS tasks will reach the main site and some of mine in testing used well over 5GB/task.
ID: 64214 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · Next

Questions and Answers : Unix/Linux : Run Linux work units with Windows 10 WSL

©2021 climateprediction.net