climateprediction.net home page
Run Linux work units with Windows 10 WSL

Run Linux work units with Windows 10 WSL

Questions and Answers : Unix/Linux : Run Linux work units with Windows 10 WSL
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
klepel

Send message
Joined: 9 Oct 04
Posts: 54
Credit: 53,934,394
RAC: 25,426
Message 63767 - Posted: 2 Apr 2021, 19:42:44 UTC - in response to Message 63514.  

I have successfully installed WSL Ubuntu on this computer:https://www.cpdn.org/results.php?hostid=1517859

The WSL sub-directory was created in:
C:\Users\[User]\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\home\[user]

However I am not able to read the copied cc_confi.xml in this folder:
<cc_config>
<options>
<allow_multiple_clients>1</allow_multiple_clients>
<allow_remote_gui_rpc>1</allow_remote_gui_rpc>
</options>
</cc_config>

And after attaching to climateprediction.net:
I am also not able to read the corresponding app_config, which a copied in the project folder:
 <app_config>
   <app>
      <name>openifs</name>
      <max_concurrent>2</max_concurrent>
      <report_results_immediately/>
   </app>
   <app>
      <name>hadcm3n</name>
      <max_concurrent>2</max_concurrent>
      <report_results_immediately/>
   </app>
   <app>
      <name>wah2</name>
      <max_concurrent>2</max_concurrent>
      <report_results_immediately/>
   </app>
   <app>
      <name>wah2_ri</name>
      <max_concurrent>2</max_concurrent>
      <report_results_immediately/>
   </app>
   <app>
      <name>hadam4</name>
      <max_concurrent>2</max_concurrent>
      <report_results_immediately/>
   </app>
   <app>
      <name>hadam4h</name>
      <max_concurrent>2</max_concurrent>
      <report_results_immediately/>
   </app>
   <app>
      <name>hadcsm4</name>
      <max_concurrent>2</max_concurrent>
      <report_results_immediately/>
   </app>
</app_config>

All the WUs have errored inmediatly, any ideas?
I was thinking if I might download leafpad and try to make the cc_config and app_config with Leafpad, but will it even start?
And secodndly and even more important: Any ideas why the WUs do not run at all?
ID: 63767 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 589
Credit: 25,391,770
RAC: 6,891
Message 63768 - Posted: 2 Apr 2021, 20:50:42 UTC - in response to Message 63767.  

All the WUs have errored inmediatly, any ideas?
I was thinking if I might download leafpad and try to make the cc_config and app_config with Leafpad, but will it even start?
And secodndly and even more important: Any ideas why the WUs do not run at all?

It looks to me that you did not install the 32-bit libraries.

Start WSL by opening the command window "ubuntu2004.exe", located in
C:\Users\(Username)\AppData\Local\Microsoft\WindowsApps\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc
(Though you probably have different numbers at the end)
And in the command window run: sudo apt install lib32ncurses6 lib32z1 lib32stdc++-7-dev

As for the other problems, my memory is hazy, but my notes indicate that the cc_config.xml should be located in:
/etc/boinc-client/
which you can get to by opening the WSL command window as above.
I used the "nano" editor in Ubuntu to just create the file, and then copy and paste the contents from my Windows machine.
There may be better ways to do it. I expect that Leafpad will work fine.

And the app_config.xml should probably be located in:
/var/lib/boinc-client/projects/climateprediction.net/
HOWEVER, I did not actually use an app_config.xml file with WSL myself, and don't know if that will really work or not, but it is the best place to start.

That may get you further along I hope.
ID: 63768 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 54
Credit: 53,934,394
RAC: 25,426
Message 63774 - Posted: 3 Apr 2021, 0:10:51 UTC - in response to Message 63768.  

After your post I deinstalled Ubuntu completely and installed it from Microsoft store anew.
It looks to me that you did not install the 32-bit libraries.

I installed all lib 32 bit libraries your mentioned with your command.
Start WSL by opening the command window "ubuntu2004.exe", located in
C:\Users\(Username)\AppData\Local\Microsoft\WindowsApps\CanonicalGroupLimited.Ubuntu20.04onWindows_79rhkp1fndgsc
(Though you probably have different numbers at the end)

I start it right from the menu, it seems indifferent to me.
And in the command window run: sudo apt install lib32ncurses6 lib32z1 lib32stdc++-7-dev

Did this.
As for the other problems, my memory is hazy, but my notes indicate that the cc_config.xml should be located in:
/etc/boinc-client/
which you can get to by opening the WSL command window as above.

Yes, there is an example cc_config.xml, but it is not read starting BOINC. I might change the names and change some lines in it to check again.
I used the "nano" editor in Ubuntu to just create the file, and then copy and paste the contents from my Windows machine.
There may be better ways to do it. I expect that Leafpad will work fine. [quote]
[quote]And the app_config.xml should probably be located in:
/var/lib/boinc-client/projects/climateprediction.net/
HOWEVER, I did not actually use an app_config.xml file with WSL myself, and don't know if that will really work or not, but it is the best place to start.

I will try this further down the road, as WUs still error out at the start see more below. But the project archive is created at:
…\LocalState\rootfs\home\roland\projects
, and might be stored there and not at
/var/lib/boinc-client/projects/climateprediction.net/
as in a linux computer.
That may get you further along I hope.

Thanks for your help!
As I wrote above, the WUs error immediately! I get the following Error in the Ubuntu Shell (WSL):
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_1.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_2.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_3.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_4.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_5.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_6.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_7.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_8.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_9.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_10.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_restart.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent
02-Apr-2021 18:43:35 [climateprediction.net] Output file hadcm3s_a2sw_191012_120_900_012074303_0_r554985389_out.zip for task hadcm3s_a2sw_191012_120_900_012074303_0 absent

Maybe this gives a clue, which lib is missing! Highly appreciate any help!
In the meanwhile, I will try to attach to another project, just to see if those WUs run successfully.
ID: 63774 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 589
Credit: 25,391,770
RAC: 6,891
Message 63775 - Posted: 3 Apr 2021, 0:58:55 UTC - in response to Message 63774.  

On my Ubuntu machines, I also do the following, though I didn't think it was necessary under WSL.
But maybe I did it anyway and didn't write it down?

• Join the root group: sudo adduser (USERNAME) root
• Join the BOINC group: sudo adduser (USERNAME) boinc
• Allow access by all: sudo chmod -R 777 /etc/boinc-client
• Allow access by all: sudo chmod -R 777 /var/lib/boinc-client

Then, reboot.
At that point, try adding the cc_config.xml and app_config.xml files.
You have good hardware. Good luck.
ID: 63775 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 54
Credit: 53,934,394
RAC: 25,426
Message 63776 - Posted: 3 Apr 2021, 1:31:39 UTC

Thank you for your fast answers!
As I wrote I tried another project on the same installation and it works:
https://boinc.multi-pool.info/latinsquares/results.php?hostid=103983
Although the finished work units, are not confirmed yet! But I do not expect, that they will not validad!
So it has to be a problem with climateprediction.net. A missing lib or some problem in the downloaded files, but it seems more like a Linux problem. On my real Linux computers I do not use the standard BOINC installation and it is another flavor, so I do not know, where to look for missing lib files.
Thanks! For commenting on my hardware! I normally do climateprediction.net on Linux computers – and Windows WUs on my Windows computers – but if we are not able to bring more users to volunteer more linux computers the actual back lock will take ages. So, when I read your WSL experience, I thought I might give it a try. But my Linux Know-How is quite limited, so if it does not work with WSL I might give up after Easter Weekend.
ID: 63776 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7474
Credit: 23,547,955
RAC: 3,044
Message 63778 - Posted: 3 Apr 2021, 1:48:40 UTC - in response to Message 63776.  

A "missing lib problem" will give you an error message similar to what is at the bottom of this page. (It's the loooooong line.)
missing lib

************************

There's a thread at the top of the Linux section: *** Running 32bit CPDN from 64bit Linux - Important ***, which is a help file for finding the lib(s).
ID: 63778 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 54
Credit: 53,934,394
RAC: 25,426
Message 63779 - Posted: 3 Apr 2021, 4:09:53 UTC - in response to Message 63778.  

A "missing lib problem" will give you an error message similar to what is at the bottom of this page. (It's the loooooong line.)
missing lib

************************

There's a thread at the top of the Linux section: *** Running 32bit CPDN from 64bit Linux - Important ***, which is a help file for finding the lib(s).


I already executed
sudo  apt-get  install  lib32ncurses6  lib32z1  lib32stdc++-7-dev

and I am not getting a message about a missing lib either.

I have more and more the feeling that climateprediction.net has a problem that the project files are at another place than normal, but then how JIM1348 was able to run climateprediction.net WUs and I am not.
Or is it, that I am not running WSL 2 but WSL, but then how the other project works?
ID: 63779 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2024
Credit: 54,073,462
RAC: 21,544
Message 63780 - Posted: 3 Apr 2021, 4:39:41 UTC - in response to Message 63779.  

The errors all have similar to the following in stderr:

<message>
process exited with code 8 (0x8, -248)</message>
<stderr_txt>
Process creation (../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu) failed: Error -1, errno=8
execv: Exec format error


I'm not sure if that's a permission/ownership problem or a path problem. I haven't used WSL so can't really say.
ID: 63780 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3135
Credit: 8,506,890
RAC: 18,656
Message 63781 - Posted: 3 Apr 2021, 5:54:42 UTC - in response to Message 63780.  

Process creation (../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu) failed: Error -1, errno=8
execv: Exec format error


I have seen a number of tasks with this error when looking at tasks that have failed on all three tries and have mentioned it on one of the forums open to moderators on at least one batch. As far as I know, you are the first with this error to have posted here. It would be good to try and track it down. If no one here can come up with an answer, it may be worth asking over on the BOINC forums.

For what it is worth, that particular error occurs on both Linux and Mac systems and the Linux systems I have seen it has occurred on have been running Linux natively so may not be specifically to do with WSL. MacOS being a Unix based system presumably has a common cause for the problem. Not having seen it on any computer I have owned, I can't really be of much more help. Hopefully one of the users who has had it in the past might see this and give us some more information.
ID: 63781 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7474
Credit: 23,547,955
RAC: 3,044
Message 63783 - Posted: 3 Apr 2021, 8:32:05 UTC

On one of my Linux machines, the files are here:
/var/lib/boinc-client/
In here is: notices, projects, slots, and a whole lot of files.


I suspect that Windows is putting things in the wrong places.
ID: 63783 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3135
Credit: 8,506,890
RAC: 18,656
Message 63784 - Posted: 3 Apr 2021, 11:33:23 UTC - in response to Message 63783.  

On one of my Linux machines, the files are here:
/var/lib/boinc-client/
In here is: notices, projects, slots, and a whole lot of files.


I suspect that Windows is putting things in the wrong places.


Possibly but why does it allow other projects to work and what about the other machines with that error, more common on hadcm3 tasks than hadam4 from memory?
ID: 63784 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 589
Credit: 25,391,770
RAC: 6,891
Message 63785 - Posted: 3 Apr 2021, 13:16:14 UTC - in response to Message 63783.  
Last modified: 3 Apr 2021, 13:38:56 UTC

I suspect that Windows is putting things in the wrong places.

I think so too. At first, I had the impression that BOINC was installed in the Home directory, which is different than for Ubuntu.
I did a search and found the configuration files in unusual places (for me), and stuck cc_config.xml in several places.
It eventually worked, but I am not sure of the right one.

EDIT: To be a little more precise, I found the cc_config.xml file (and others) in the Home directory, which is unusual, and as I recall also in /etc/boinc-client
At first, I did not even bother with a custom cc_confg.xml, and so did not notice the problem, or I would have mentioned it. I am out of WSL for now. Maybe you can find the real thing.
ID: 63785 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3135
Credit: 8,506,890
RAC: 18,656
Message 63787 - Posted: 3 Apr 2021, 15:29:13 UTC

this is one that failed with the same error on native Linux. (Arch Linux) in this case. The computer is crashing everything with this error. Given how easily I found an example, I am really surprised there haven't been users posting to track down the problem before now.
ID: 63787 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 54
Credit: 53,934,394
RAC: 25,426
Message 63788 - Posted: 3 Apr 2021, 17:22:13 UTC - in response to Message 63781.  

Process creation (../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu) failed: Error -1, errno=8
execv: Exec format error


I have seen a number of tasks with this error when looking at tasks that have failed on all three tries and have mentioned it on one of the forums open to moderators on at least one batch. As far as I know, you are the first with this error to have posted here. It would be good to try and track it down. If no one here can come up with an answer, it may be worth asking over on the BOINC forums.`

Seems to be interesting to check out this forum, a user already mentioned that he is running WSL with BOINC Manager (This might be intersting for JIM1348, Mensaje 103645). I might have to study the guide "Guide to using The BOINCClient in Microsoft's WSL" as well. In the same message, it is mentioned that climateprediction.net does not work properly.
For what it is worth, that particular error occurs on both Linux and Mac systems and the Linux systems I have seen it has occurred on have been running Linux natively so may not be specifically to do with WSL. MacOS being a Unix based system presumably has a common cause for the problem. Not having seen it on any computer I have owned, I can't really be of much more help. Hopefully one of the users who has had it in the past might see this and give us some more information.

I looked-up my tasks and most failed on other computers as well. There might be a general problem with these tasks - it seems the WUs expect certain items in particular places, and if they are not, they error out immediately. These gives some hope, other tasks might work! So as my computer is rarely restarted, I might as well as let WSL run for itself and look if any further tasks runs properly.
ID: 63788 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 54
Credit: 53,934,394
RAC: 25,426
Message 63790 - Posted: 3 Apr 2021, 17:57:25 UTC

On one of my Linux machines, the files are here:
/var/lib/boinc-client/
In here is: notices, projects, slots, and a whole lot of files.

I suspect that Windows is putting things in the wrong places.

It seems that WSL does not work like an clean Linux Computer installation or a duel boot installation, as I do have on other computers. So the files are not stored at the expected places.
I did a search and found the configuration files in unusual places (for me), and stuck cc_config.xml in several places.

Yes, the cc_config.xml is all over the place!
C:\Users\(User)\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\etc\boinc-client
C:\Users\(User)\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\etc\boinc-client
C:\Users\(User)\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\var\lib\boinc-client

and I put mine in:
C:\Users\(User)\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\home\(user)

The cc_cofig.xml in the first place and in my place are not read/recognized.I have not tried the other two yet.0
I also placed an app_config.xls in its usual place:
C:\Users\(User)\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\home\(user)\projects\climateprediction.net

No joy either!
What stresses me, ist that JIM1348, got it to work somehow! I think I will try to install WSl in another Windows Computer, following exactly the guide from BOINC Italy and see, if it works with this installation, at least cc_config.xml and app_config.xml. Or I might as well just let it run and see, if a lucky WUs finishes as the other project does.
ID: 63790 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 589
Credit: 25,391,770
RAC: 6,891
Message 63791 - Posted: 3 Apr 2021, 18:04:29 UTC - in response to Message 63790.  

What stresses me, ist that JIM1348, got it to work somehow!

I would tell you how I did it if I could. But there is a certain randomness in file locations, and maybe permissions too.
You are dealing with both Windows and Linux, so it could be the worst of both.
ID: 63791 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 54
Credit: 53,934,394
RAC: 25,426
Message 63792 - Posted: 3 Apr 2021, 18:26:33 UTC
Last modified: 3 Apr 2021, 19:19:49 UTC

Sorry this was not right place for this post.
ID: 63792 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3135
Credit: 8,506,890
RAC: 18,656
Message 63794 - Posted: 4 Apr 2021, 6:25:27 UTC

This one failed with the process creation error but completed on another machine. Took a while to find going through the successfully completed tasks from #837 to find it though! It does demonstrate that those with this error are not all down to the task at least.
ID: 63794 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3135
Credit: 8,506,890
RAC: 18,656
Message 63798 - Posted: 5 Apr 2021, 9:18:57 UTC

Looking through hard fails on #899 all the failures I found with
 Process creation (../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu) failed: Error -1, errno=2
execv: No such file or directory 
showed the computer as having Virtual Box Though the #837 one linked to below using Arch Linux doesn't. I have very limited experience of VB so hoping someone who has used it more than I have might be able to shed light on this?
ID: 63798 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7474
Credit: 23,547,955
RAC: 3,044
Message 63799 - Posted: 5 Apr 2021, 11:26:58 UTC - in response to Message 63798.  

just guessing, but it may mean that the climate program tried to write in a location that either hadn't been created / was in an incorrect location, or that the directory didn't have write permissions.

All of which may well mean that virtual box isn't suited for use with this project.
Perhaps time to stop trying to fit a square peg into a round hole using wedges.
ID: 63799 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · Next

Questions and Answers : Unix/Linux : Run Linux work units with Windows 10 WSL

©2021 climateprediction.net