climateprediction.net home page
*** Running 32bit CPDN from 64bit Linux - Discussion ***

*** Running 32bit CPDN from 64bit Linux - Discussion ***

Questions and Answers : Unix/Linux : *** Running 32bit CPDN from 64bit Linux - Discussion ***
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 18 · Next

AuthorMessage
John Eric Hopkinson

Send message
Joined: 27 Jan 05
Posts: 74
Credit: 1,047,809
RAC: 0
Message 54207 - Posted: 26 May 2016, 22:29:18 UTC - in response to Message 54201.  

Astrowx:
Thanks for the information.
I have four smallish wah2 tasks running on Win10, approx 32% progress without incident thus far. Windows upgrading has not been an issue but that may change when MS unleashes Windows 10 Redstone (whatever that is) which was reported in CDNet "How to get started with Ubuntu and Bash in Windows 10", in April.
MS does not hesitate to muscle in on any venture which provides even a hint of either income or acquisition, and it seems they are alarmed by Linux. In the Anniversary Update to which you referred, MS introduces Ubuntu and Bash for Win10. I am adequately confused by the proliferations of "distros" and cross-platform stuff, and have no intention of investigating anything beyond what is necessary to carry on in Linux to learn Python coding and installing CPDN.
This all started when i was trying to find a replacement for Win XP whose demise was very destructive for my interest in these things.
How are you surviving in the degenerative climatic conditions in the Pacific Northwest?


ID: 54207 · Report as offensive     Reply Quote
Thunder

Send message
Joined: 1 Sep 04
Posts: 42
Credit: 6,475,117
RAC: 0
Message 54250 - Posted: 4 Jun 2016, 16:32:58 UTC

Well, I can't say I didn't try, but I'm finally giving up running CPDN on 64-bit linux (or at least the 3 different versions/flavors of Ubuntu I've tried).

I've spent 3 months trying every suggestion, systematically working down this thread and every page it links to, installing so many additional libraries that I've utterly lost count, but to no avail.

The best I've gotten is that all models make it through until they try to generate the final upload files and then fail. I highly doubt it's a permissions issue because it's a standard package installation and all other projects work fine.

In any case, 15,000 hours of CPU time wasted is enough trying for me. :-P


ID: 54250 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2167
Credit: 64,403,322
RAC: 5,085
Message 54252 - Posted: 4 Jun 2016, 19:05:39 UTC - in response to Message 54250.  

Well, I can't say I didn't try, but I'm finally giving up running CPDN on 64-bit linux (or at least the 3 different versions/flavors of Ubuntu I've tried).

I've spent 3 months trying every suggestion, systematically working down this thread and every page it links to, installing so many additional libraries that I've utterly lost count, but to no avail.

The best I've gotten is that all models make it through until they try to generate the final upload files and then fail. I highly doubt it's a permissions issue because it's a standard package installation and all other projects work fine.

In any case, 15,000 hours of CPU time wasted is enough trying for me. :-P


Looking at the stderr on the tasks that failed on your Ubuntu PC, it appears to be a missing 32bit zlib/libz problem. On an Ubuntu 15.10 PC, this is what I get when I do an

ldd wah2_se_8.12_i686-pc-linux-gnu.so in the projects/climateprediction.net directory

linux-gate.so.1 => (0xf7749000)
libz.so.1 => /lib/i386-linux-gnu/libz.so.1 (0xf7682000)
libnsl.so.1 => /lib/i386-linux-gnu/libnsl.so.1 (0xf7667000)
libstdc++.so.6 => /usr/lib/i386-linux-gnu/libstdc++.so.6 (0xf74ef000)
libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf74a2000)
libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf7484000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf72c9000)
/lib/ld-linux.so.2 (0x565b0000)

What do you get?
ID: 54252 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,377,675
RAC: 3,657
Message 54253 - Posted: 5 Jun 2016, 8:27:50 UTC - in response to Message 54250.  
Last modified: 5 Jun 2016, 8:35:23 UTC

The alternative is to run them in WINE. Thos seems to me to give fewer fails on unexpected re-starts than running them natively. I have one of my three linux boxes running the nix version of BOINC but that one is only running cpdn and not doing anything else. Though it may get a rest if we get a decent summer here in UK :)

Edit: Don't think I have every had a missing lib error at the end of computation though, mine have always crashed out in the first twenty seconds or so. Still the fix of installing the missing lib should be simple enough.

Edit2: Sometimes because of changes in a distribution you may need to do some googling to identify the 32bit package containing the missing lib.I certainly have had to do this in the past.

It would be nice if someone could do rewrite of the relevant sections of the Met Office code so that it worked out of the box with 64 bit Linux distributions however I am well aware of the problems inherent in Oxford not owning the code but only using it under license from the Met Office.
ID: 54253 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2167
Credit: 64,403,322
RAC: 5,085
Message 54254 - Posted: 5 Jun 2016, 15:31:52 UTC - in response to Message 54253.  

Edit: Don't think I have every had a missing lib error at the end of computation though, mine have always crashed out in the first twenty seconds or so. Still the fix of installing the missing lib should be simple enough.


That's because, in this case, he's got everything else, but not the right 32bit zip/unzip capability. So there were multiple libz errors and despite trickling 24 times, none of the 24 expected monthly zips were uploaded.

Edit2: Sometimes because of changes in a distribution you may need to do some googling to identify the 32bit package containing the missing lib.I certainly have had to do this in the past.


Yes, this is the case. They instructions are updated to a certain version of a distribution, then later versions may rename one or more of the packages needed, and at least part of the instructions don't work anymore. Very frustrating.

That's why a 64bit version would solve most of these problems. No more trying to follow outdated instructions for getting the needed 32bit packages, that are different among different linux distributions, and then outdated even within the same distribution as new versions are created.
ID: 54254 · Report as offensive     Reply Quote
Profile astroWX
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 1496
Credit: 95,522,203
RAC: 0
Message 54255 - Posted: 5 Jun 2016, 21:09:37 UTC - in response to Message 54207.  

Astrowx:
...
How are you surviving in the degenerative climatic conditions in the Pacific Northwest?

Off topic but I don't want to ignore your question. At my age, these periods of too-frequent record heat are difficult. I moved to coastal Washington state a quarter century ago to escape extreme weather. (It works, sometimes.)

If you have further questions, we can work with them offline. (Unfortunately, we can't, at the moment, open new threads.)

"We have met the enemy and he is us." -- Pogo
Greetings from coastal Washington state, the scenic US Pacific Northwest.
ID: 54255 · Report as offensive     Reply Quote
Thunder

Send message
Joined: 1 Sep 04
Posts: 42
Credit: 6,475,117
RAC: 0
Message 54270 - Posted: 8 Jun 2016, 1:27:00 UTC - in response to Message 54252.  

Sorry Geophi, but I already detached the project from that machine, so the executables are gone. I may check back once in a while to see if they're going to ever embrace these new fangled 64-bit computers. *sigh*

I do know that libz.so.1 was one of the libraries that I specifically located and installed the package that contained it, so the solution is likely more complicated than that.

I'd like to help further, but even though CPDN is my first (and longest) BOINC project, I'm just too frustrated with it to continue. I'll leave it going as a low-priority project on a handful of windows boxes that seem to be stable enough that they won't trash 3/4 of their models, but beyond that, I'm done.
ID: 54270 · Report as offensive     Reply Quote
Jim1348

Send message
Joined: 15 Jan 06
Posts: 637
Credit: 26,751,529
RAC: 653
Message 54375 - Posted: 24 Jun 2016, 15:52:22 UTC

My Win7 64-bit machine crashed, and I reloaded it with Ubuntu 16.04, naively thinking it would work on the WAH2s. I am not that much of a Linux expert, and don't intend to become one. Thanks for all the info though.
ID: 54375 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 76
Credit: 67,812,914
RAC: 5,809
Message 54378 - Posted: 24 Jun 2016, 23:17:50 UTC
Last modified: 24 Jun 2016, 23:18:43 UTC

Plop!! I run 5 Weather At Home 2 (wah2) v8.12 – units on Lubuntu 64-bit computer I have set-up as a BOINC cruncher, and all errored out with the same problem as Thunder has experienced.

I am just trying as it won’t work with GPUGRID and waiting for Linux Mint 18, to solve some problems with Nvidia Drivers.

However, I have not touched the computer for 12 days so Climteprediction.net would be able to finish its tasks and now there is not one zip file. Just frustrating…

Although I guess the Project-Administration won´t care: I am on the same boat as Thunder, it was my first and main project since 2004, I will run it on my MS Windows Computers, but will not insist on any Linux 64-bit cruncher I plan to set up especially for BOINC as many other BOINC projects oblige to use a Linux 64-bit to work.
ID: 54378 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 24,487,746
RAC: 3,014
Message 54385 - Posted: 26 Jun 2016, 21:43:58 UTC

Hi folks,
I've just successfully finished crunching my first two wah2_8.12 under Linux. It is a machine on which until recently I was running successfully BOINC under WINE and before MOSES II TRIFIDS under the Linux client.

It runs Ubuntu 14.04.4 LTS with kernel: Linux 3.13.0-88-generic (x86_64)
Here is what "ldd" gives about the libraries used, I hope it is of help to other Linux users. Cheers

ldd wah2_8.12_i686-pc-linux-gnu
linux-gate.so.1 => (0xf77bb000)
libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf777b000)
libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf7776000)
libstdc++.so.6 => /usr/lib/i386-linux-gnu/libstdc++.so.6 (0xf768d000)
libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf7647000)
libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf762a000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf747b000)
/lib/ld-linux.so.2 (0xf77bc000)

sudo ldd wah2_se_8.12_i686-pc-linux-gnu.so
linux-gate.so.1 => (0xf7723000)
libz.so.1 => /lib/i386-linux-gnu/libz.so.1 (0xf765b000)
libnsl.so.1 => /lib/i386-linux-gnu/libnsl.so.1 (0xf7642000)
libstdc++.so.6 => /usr/lib/i386-linux-gnu/libstdc++.so.6 (0xf7559000)
libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf7513000)
libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf74f6000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf7347000)
/lib/ld-linux.so.2 (0xf7724000)
ID: 54385 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54386 - Posted: 26 Jun 2016, 22:38:04 UTC

Hi klepel

The bitness isn't up to cpdn.
The code for the programs belongs to the UK Met Office. I's what the researchers around the globe use. We're just being allowed to use it via a special arrangement. And the project people don't have the source code for the main modelling programs, just that for the parts that access it.

We've also said more than once over the years that cpdn isn't for everyone.
So, sorry that it's not working out for you, but that's how things go.

ID: 54386 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 76
Credit: 67,812,914
RAC: 5,809
Message 54404 - Posted: 29 Jun 2016, 2:26:17 UTC - in response to Message 54386.  

The code for the programs belongs to the UK Met Office. I's what the researchers around the globe use. We're just being allowed to use it via a special arrangement. And the project people don't have the source code for the main modelling programs, just that for the parts that access it.

I am well aware of this. I was not insisting that climateprediction.net shall implement a 64-bit version of UK Met Office model as I am quite aware that it is not up to climateprediction.net to do so.

What I am looking for and I am sure many user do is that climateprediction.net creates libraries for the main 64-bit LINUX distros and maintains these libraries, so that the 32-bit climateprediction.net model will work out of the box or at least we do have a starting point to implement it, so I will not run 5 models for 12 days only to find out the ZIP files have not been created. This is a waste of resources which might have been used for other important projects.

We've also said more than once over the years that cpdn isn't for everyone.
So, sorry that it's not working out for you, but that's how things go.

I have supported this stance for long and I do understand it, but somehow it sounds hallow to me now. It is a reality, that the soft- and hardware is developed for 64-bit, other BOINC projects do require it, and I think I am not quite far away, when I stipulate that “POWER-USERS” as Thunder and myself are trying to migrate to LINUX to safe at least the expenses for the OS, so we might invest it in additional Hardware.



ID: 54404 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54406 - Posted: 29 Jun 2016, 2:58:28 UTC - in response to Message 54404.  

OK, sorry, perhaps I got a bit lectury.

The most frequent missing lib is libjpeg.so.62
This is/was needed, I think, by earlier versions of Centos, the recent versions of which have dropped it.
It's the "version 6" part that's the problem, as other distros are up to 8 something now.
I solved this by getting the file from a 32 bit install, and transferring it.

And if it's missing, models fail at about 6 seconds, not at the finish.

The only thing special that happens at the finish, is that data is gathered to create the re-start zip file, and this is done by a program which is only used at this time in the entire modelling process.

This applies to all 3 OS's, and I don't recall hearing about this being a problem before on Linux.

ID: 54406 · Report as offensive     Reply Quote
Profile tullio

Send message
Joined: 6 Aug 04
Posts: 264
Credit: 965,476
RAC: 0
Message 54408 - Posted: 29 Jun 2016, 11:35:57 UTC

1 PNW task crashed on my Linux box after 433.35052 hours when at 99% but gave me 13k credits. I had to reboot the SUN WS with Opteron 1210, 8 GB RAM, SuSE Leap 42.1 64 bit when it was running also SETI Beta GPU tasks on a AMD/ATI HD 7770 GPU board. Maybe one of them crashed and the CPDN task did not survive the reboot.
Tullio
ID: 54408 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 76
Credit: 67,812,914
RAC: 5,809
Message 54424 - Posted: 5 Jul 2016, 15:09:59 UTC

I am keep trying to run a climateprediction.net WU to the end. So fare it runs quite well, but it does not create the intermediate zip files along with the processing of the WU.

So I am looking for a library which equals the libz.so.1 from Ubuntu in “lubuntu” as this is the missing library the tasks which failed previously. Is it libzip4? Thanks for your help!

ID: 54424 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4314
Credit: 16,377,675
RAC: 3,657
Message 54427 - Posted: 5 Jul 2016, 18:27:54 UTC

As detailed about a mile back in this thread,

sudo apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0 libstdc++6 libstdc++5 freeglut3


Worked for me with xubuntu15.10 also in case some models with graphics appear

sudo apt-get install libjpeg62:i386


Was needed on my system. I haven't checked whether this works on a fresh install of xubuntu16.04

The problem as noted earlier in thread is that the instructions often change between releases of the same distribution. The instructions also vary between distributions. That is why there are such problems with it. As suggested, perhaps a download of the relevent libs could be provided by cpdn along with instructions on where to place them so theyh will be found?
ID: 54427 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2167
Credit: 64,403,322
RAC: 5,085
Message 54434 - Posted: 6 Jul 2016, 4:22:15 UTC - in response to Message 54424.  

I am keep trying to run a climateprediction.net WU to the end. So fare it runs quite well, but it does not create the intermediate zip files along with the processing of the WU.

So I am looking for a library which equals the libz.so.1 from Ubuntu in “lubuntu” as this is the missing library the tasks which failed previously. Is it libzip4? Thanks for your help!


As Dave said, from the terminal window command line, run

sudo apt-get install lib32z1

Afterward in the ..../projects/climateprediction.net directory do a

sudo ldd wah2_se_8.12_i686-pc-linux-gnu.so

which should give you satisfied dependencies for libz, which is required by that program for zipping.
ID: 54434 · Report as offensive     Reply Quote
klepel

Send message
Joined: 9 Oct 04
Posts: 76
Credit: 67,812,914
RAC: 5,809
Message 54619 - Posted: 8 Aug 2016, 16:54:18 UTC

I just wanted to report back: I somehow managed to get all the necessary libs installed and the Lubuntu 64-bit computer has already crunched 6 WUs satisfactory and reported the results back.
ID: 54619 · Report as offensive     Reply Quote
nanoprobe

Send message
Joined: 18 May 12
Posts: 1
Credit: 12,442
RAC: 0
Message 54672 - Posted: 19 Aug 2016, 17:23:10 UTC - in response to Message 54250.  

Well, I can't say I didn't try, but I'm finally giving up running CPDN on 64-bit linux.


I'm in the same boat on 16.04 LTS 64 bit. I've installed every lib file, package whatever I could find to no avail. Every task fails in the first 10 seconds. I'll keep an eye on this thread is case someone finds a solution.
ID: 54672 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54673 - Posted: 19 Aug 2016, 21:18:16 UTC - in response to Message 54672.  

My solution is now buried way down this thread, but basically:

1)Check the error messages. The usual culprit is a version 6 jpeg lib.
2)Get an old computer, and install a 32 version of Unix.
3)Locate the 2 files with the appropriate names, copy them to a memory stick, and transfer them to the computer(s) in question.

The difficulty occurs, not during installing the libs, but in the clean up afterwards, when the installer finds a version 8 of the files, and gets rid of the version 6 that was just installed.

ID: 54673 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 18 · Next

Questions and Answers : Unix/Linux : *** Running 32bit CPDN from 64bit Linux - Discussion ***

©2024 climateprediction.net