climateprediction.net home page
New work Discussion

New work Discussion

Message boards : Number crunching : New work Discussion
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 77 · 78 · 79 · 80 · 81 · 82 · 83 · Next

AuthorMessage
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3372
Credit: 10,104,924
RAC: 8,765
Message 65464 - Posted: 20 May 2022, 12:07:52 UTC

Don't bother with nested virt, it has a substantial impact on performance. I got it working on cloud machines with nested virt, but performance wasn't any better than an older Intel box.
I guessed there would be a substantial performance hit. If I do it, it will be more about proof of concept than anything. Certainly not going to be doing it regularly.
ID: 65464 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 104
Credit: 18,019,200
RAC: 39,899
Message 65465 - Posted: 20 May 2022, 15:23:51 UTC - in response to Message 65463.  


Well, this does matter if the interfaces are connected to the same network segment.
Like IPs, MACs must be unique.


The reason it doesn't matter (in terms of network function) is that my VMs are each on separate physical hosts, and the default network configuration for KVM (which I'm not changing) is to have NAT happening on each machine. So the VMs get an IP in the 192.168.122.0/24 range, while the actual host machines have their unique MAC addresses and are on the physical network segment out here.

LAN -> VM Hosts (unique MACs) -> VMs (identical MACs) works, just leads to BOINC confusion. Apparently it can deal with some conditions, but not all of them, and if it can't tell the difference between two VMs, you end up with tasks getting abandoned.

So if you're using the scripts to run the VMs, just twiddle a few bits in the MAC address line.

I guessed there would be a substantial performance hit. If I do it, it will be more about proof of concept than anything. Certainly not going to be doing it regularly.


It's certainly a fun little trick to get working! I ran a bunch of tasks on some cloud boxes (so nested virt with MacOS on Linux on presumably Linux), but they were sub-1%/hour progress, and not particularly cheap to run.
ID: 65465 · Report as offensive     Reply Quote
Peter Hucker of the Scottish Boinc Team

Send message
Joined: 9 Oct 20
Posts: 197
Credit: 1,761,668
RAC: 0
Message 65475 - Posted: 2 Jun 2022, 23:15:43 UTC

Any news on the Windows tasks that were promised a couple of months back? Seems even Linux has run out now.
ID: 65475 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7585
Credit: 24,102,463
RAC: 2,603
Message 65476 - Posted: 3 Jun 2022, 0:37:56 UTC - in response to Message 65475.  

That was about Mac tasks running in a VM on Windows.
ID: 65476 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 104
Credit: 18,019,200
RAC: 39,899
Message 65478 - Posted: 3 Jun 2022, 2:15:15 UTC

I was so excited to see this thread yellow. :(

But nothing to find.
ID: 65478 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3372
Credit: 10,104,924
RAC: 8,765
Message 65479 - Posted: 3 Jun 2022, 5:15:40 UTC

The series of Mac tests has still been going ton. They have all been successful in that they finished without any errors or obvious problems in file sizes uploaded etc. However, the scientists will be looking at the science in the uploads to confirm they are actually working as intended and to guide exactly how they do the next main site batch, ie. do they just send out a lot more tasks than in the test ones or do they tweak things and run another test.
ID: 65479 · Report as offensive     Reply Quote
Peter Hucker of the Scottish Boinc Team

Send message
Joined: 9 Oct 20
Posts: 197
Credit: 1,761,668
RAC: 0
Message 65480 - Posted: 3 Jun 2022, 6:22:42 UTC
Last modified: 3 Jun 2022, 6:26:28 UTC

I'm sure someone told me a few months back they were testing a Windows batch. I only run Windows, I've had enough of using different OSs, it's too tedious to make all the unintuitive interfaces work (and yes that includes Windows now they've gone all Mac-like with Windows 11). They've in fact taken the "simplicity" (can't find any settings) of Mac and the "security" (can't change anything without permission from myself which I don't know how to give) of Linux and put them together in a terrible mess. Please make Windows good again.....
ID: 65480 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2100
Credit: 58,039,568
RAC: 1,756
Message 65481 - Posted: 3 Jun 2022, 7:08:18 UTC - in response to Message 65480.  

We tested some WAH2 tasks for ANZ back around April 1st. They were successful, and the main investigator from New Zealand was going to look at the output and get back to Sarah if things looked okay to send out main site batches. No communications since then. I will query if this is still the plan.
ID: 65481 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 32
Credit: 3,212,803
RAC: 24,092
Message 65482 - Posted: 3 Jun 2022, 9:15:06 UTC

The server status page shows over 22000 Windows tasks in progress. Wasn't there a bunch of Windows tasks released last July? If so, shouldn't we expect a bunch of re-sends soon due to time outs?
ID: 65482 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3372
Credit: 10,104,924
RAC: 8,765
Message 65484 - Posted: 3 Jun 2022, 9:44:37 UTC - in response to Message 65482.  

If so, shouldn't we expect a bunch of re-sends soon due to time outs?
If the researchers have got enough data from those that have been returned, then they won't bother resending those that have timed out. This is quite likely to be the case as batches are often part of PhD projects and if the person responsible has finished their degree then they won't be wanting them. Post doc research, it will depend on whether the extra information is useful or not. (And in both cases whether the project has been told if the tasks are no longer needed!)
ID: 65484 · Report as offensive     Reply Quote
Peter Hucker of the Scottish Boinc Team

Send message
Joined: 9 Oct 20
Posts: 197
Credit: 1,761,668
RAC: 0
Message 65486 - Posted: 3 Jun 2022, 10:26:39 UTC - in response to Message 65482.  

The server status page shows over 22000 Windows tasks in progress. Wasn't there a bunch of Windows tasks released last July? If so, shouldn't we expect a bunch of re-sends soon due to time outs?
The server status page on this project is infamous for being completely inaccurate. There are jobs sat in there from a decade ago.
ID: 65486 · Report as offensive     Reply Quote
Peter Hucker of the Scottish Boinc Team

Send message
Joined: 9 Oct 20
Posts: 197
Credit: 1,761,668
RAC: 0
Message 65487 - Posted: 3 Jun 2022, 10:27:21 UTC - in response to Message 65484.  

If so, shouldn't we expect a bunch of re-sends soon due to time outs?
If the researchers have got enough data from those that have been returned, then they won't bother resending those that have timed out. This is quite likely to be the case as batches are often part of PhD projects and if the person responsible has finished their degree then they won't be wanting them. Post doc research, it will depend on whether the extra information is useful or not. (And in both cases whether the project has been told if the tasks are no longer needed!)
Which is a why a 1 year timeout is absurd. If they got resent at an earlier date, the guy might still be there.
ID: 65487 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 104
Credit: 18,019,200
RAC: 39,899
Message 65501 - Posted: 8 Jun 2022, 0:34:06 UTC

The "Thump" you just heard was the sound of the outbound workunit server's bandwidth limiter slamming into the limit and remaining there. ;)

As a bunch of Linux boxes play Hungry Hungry Hippos with the available workunits...
ID: 65501 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7585
Credit: 24,102,463
RAC: 2,603
Message 65502 - Posted: 8 Jun 2022, 1:45:44 UTC

Yes, it took several hours for my machine to get all the files for two of them.
ID: 65502 · Report as offensive     Reply Quote
Peter Hucker of the Scottish Boinc Team

Send message
Joined: 9 Oct 20
Posts: 197
Credit: 1,761,668
RAC: 0
Message 65506 - Posted: 8 Jun 2022, 17:07:56 UTC

PAH! Still no Windows work.

Hours?! Dial-up? Outer Mongolia? Mars?
ID: 65506 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3372
Credit: 10,104,924
RAC: 8,765
Message 65507 - Posted: 8 Jun 2022, 17:37:26 UTC - in response to Message 65506.  

PAH! Still no Windows work.


Three testing tasks currently crunching on my machine in VM suing Windows BOINC7.20.0 under WINE proving that the latest version (in alpha) has solved the problem of 64bit BOINC crashing under WINE. Hoping these might lead to some Windows tasks. It seems there is funding for the work somewhere in the antipodes.
ID: 65507 · Report as offensive     Reply Quote
Peter Hucker of the Scottish Boinc Team

Send message
Joined: 9 Oct 20
Posts: 197
Credit: 1,761,668
RAC: 0
Message 65508 - Posted: 8 Jun 2022, 17:40:44 UTC - in response to Message 65507.  

Surely most Windows people are actually using Windows machines, so it needs to work in Windows, not in WINE.
ID: 65508 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3372
Credit: 10,104,924
RAC: 8,765
Message 65509 - Posted: 8 Jun 2022, 19:17:32 UTC - in response to Message 65508.  

Surely most Windows people are actually using Windows machines, so it needs to work in Windows, not in WINE.


I challenge you to find a single program written for Windows that works under WINE on Linux and doesn't work with Windows.

I agree most using the new BOINC will be using a native Windows environment, however the number of people who do not have such an environment to run Windows work is not insignificant and prior to this version the 64bit versions of BOINC all crashed on trying to change to advanced view as well as on trying to attach a project. Given that 32bit versions of BOINC are not being produced any more, that condemned us to older and older versions which at some point would stop working with up to date systems.
ID: 65509 · Report as offensive     Reply Quote
Peter Hucker of the Scottish Boinc Team

Send message
Joined: 9 Oct 20
Posts: 197
Credit: 1,761,668
RAC: 0
Message 65510 - Posted: 8 Jun 2022, 19:43:31 UTC - in response to Message 65509.  

What I meant was WINE could be more fussy than Windows, so you might not have to fix the bug. But as you say there are a lot of WINE users so I guess that needs sorted.

When did Boinc stop being 32bit? I ran CPDN on Windows ok a couple of years back, I had whatever the latest Boinc was then. I thought people over at Boinc were putting off going to 64bit because of their own problems.
ID: 65510 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2100
Credit: 58,039,568
RAC: 1,756
Message 65511 - Posted: 8 Jun 2022, 21:07:05 UTC - in response to Message 65510.  

When did Boinc stop being 32bit? I ran CPDN on Windows ok a couple of years back, I had whatever the latest Boinc was then. I thought people over at Boinc were putting off going to 64bit because of their own problems.

They haven't updated the 32bit windows Boinc for 3.5+ years and the 32bit linux version for 7.5+ years. I guess they think if you are running older operating systems, you can download older versions of boinc and make do. The problem came about last fall due to the certificates being installed with older version of Windows boinc expiring. They came out with a new 64 bit version with a newer certificates list, but no new 32 bit version. They do have a certificates file to download at the top of this page: https://boinc.berkeley.edu/download_all.php if one is still running the older Windows versions.
ID: 65511 · Report as offensive     Reply Quote
Previous · 1 . . . 77 · 78 · 79 · 80 · 81 · 82 · 83 · Next

Message boards : Number crunching : New work Discussion

©2022 climateprediction.net