climateprediction.net home page
Memory speed and hadam4h (N216) performance

Memory speed and hadam4h (N216) performance

Message boards : Number crunching : Memory speed and hadam4h (N216) performance
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2096
Credit: 57,971,859
RAC: 5,479
Message 64839 - Posted: 1 Dec 2021, 2:55:11 UTC

On my Ryzen 5 5600X with a pair of DDR4 3600 RAM modules, I had been running 5 hadam4h models at a time. In that configuration the sec/TS for the models was around 13.9. I ran the 5600X over at World Community Grid on the ARP models for awhile, and after a month or so, I started getting sporadic "Invalids" as the output was checked against two other Ryzens. I took it offline, ran Memtest86 from PassMark, and it found occasional errors in one of the 13 tests. I replaced the modules with low latency DDR4 3200 RAM which resulted in no Memtest86 errors and no invalids at WCG for the last month. Well running this fast DDR4 3200 memory on the 5 hadam4h models here at cpdn, the average sec/TS was now around 15.9, about 15% slower than before. I just received replacement 3600 modules from GSkill and running them with the 5 hadam4h models, the speed is back to about 13.8 sec/TS. Hard to believe that a 12.5% increase in memory speed made a 2 sec/TS difference in model speed.

This may or may not translate in a similar fashion to other platforms and other workloads, but I know with my i7-4970K, replacing the DDR3 2400 memory with DDR3 1866, resulted in a big slowdown for both hadam4h and hadam4 models. Over at WCG on the ARP models, the slower memory did result in slower model speed, but it was less than 10%.
ID: 64839 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 3340
Credit: 9,948,339
RAC: 791
Message 64840 - Posted: 1 Dec 2021, 7:00:05 UTC - in response to Message 64839.  

Thanks for that George, I am looking at getting a memory upgrade for my Ryzen7. I had been thinking of another 32GB of PC3200. but reading what you have said, maybe I should replace it with 32GB of the faster speed or possibly even get two sticks of 32GB at the faster speed?
ID: 64840 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 601
Credit: 9,422,989
RAC: 376
Message 64841 - Posted: 1 Dec 2021, 14:21:14 UTC - in response to Message 64839.  


This may or may not translate in a similar fashion to other platforms and other workloads, but I know with my i7-4970K, replacing the DDR3 2400 memory with DDR3 1866, resulted in a big slowdown for both hadam4h and hadam4 models. Over at WCG on the ARP models, the slower memory did result in slower model speed, but it was less than 10%.


Here is typicalresults for my machine, 22159799. It typically runs four CPDM processes and three to four WCG processes and a Rosetta process, depending on the whims of the boinc-client.
CPU type 	GenuineIntel
Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz [Family 6 Model 85 Stepping 7]
Number of processors 	16

Operating System 	Linux Red Hat Enterprise Linux
Red Hat Enterprise Linux 8.5 (Ootpa) [4.18.0-348.2.1.el8_5.x86_64|libc 2.28 (GNU libc)]
BOINC version 	7.16.11
Memory 	62.4 GB  (64GB 4x16GB DDR4 2933MHz RDIM M ECC Memory)
Cache 	16896 KB


Time Sent (UTC) 	Host ID 	Result ID 	Result Name 	Phase 	Timestep 	CPU Time (sec) 	Average (sec/TS)
01 Dec 2021 08:39:47 	1511241 	22159799 	hadam4h_21m0_210002_4_922_012121464_0 	1 	34,763 	551,665 	15.8693
29 Nov 2021 08:49:12 	1511241 	22159799 	hadam4h_21m0_210002_4_922_012121464_0 	1 	26,123 	414,976 	15.8855
27 Nov 2021 16:22:47 	1511241 	22159799 	hadam4h_21m0_210002_4_922_012121464_0 	1 	17,483 	277,539 	15.8748
26 Nov 2021 00:18:49 	1511241 	22159799 	hadam4h_21m0_210002_4_922_012121464_0 	1 	8,843 	140,722 	15.9134

ID: 64841 · Report as offensive     Reply Quote
Profile Alan K

Send message
Joined: 22 Feb 06
Posts: 419
Credit: 18,919,008
RAC: 1,263
Message 64851 - Posted: 6 Dec 2021, 23:31:45 UTC

Interestingly I recently increased my machines RAM from 16Gb to 24Gb (10 GB to 16Gb for the VM) and have noticed an approx 10% reduction in the sec/ts values for the batch 922 tasks.
ID: 64851 · Report as offensive     Reply Quote
SolarSyonyk

Send message
Joined: 7 Sep 16
Posts: 88
Credit: 17,214,585
RAC: 17,971
Message 64855 - Posted: 7 Dec 2021, 19:39:35 UTC

That's not at all surprising. If you run something that shows memory bandwidth utilization (pcm.x utils on Intel/Linux work, not sure what's out there for AMD), the CPDN tasks are absolutely thrashing the memory system, even if they're limited in count so they can use as much cache as possible. They're mostly memory bandwidth limited, far as I can tell, so faster RAM having an impact makes a ton of sense.

I wonder where the benefits start tapering off?
ID: 64855 · Report as offensive     Reply Quote
Profile Alan K

Send message
Joined: 22 Feb 06
Posts: 419
Credit: 18,919,008
RAC: 1,263
Message 64856 - Posted: 7 Dec 2021, 23:18:44 UTC - in response to Message 64855.  

There was no increase in memory speed in this case. Mistakenly I ordered 1333MHz for the extra rather than the 1600MHz already fitted. Hum ho.
ID: 64856 · Report as offensive     Reply Quote
wateroakley

Send message
Joined: 6 Aug 04
Posts: 138
Credit: 18,598,590
RAC: 4,448
Message 64863 - Posted: 9 Dec 2021, 17:01:38 UTC - in response to Message 64856.  

There was no increase in memory speed in this case. Mistakenly I ordered 1333MHz for the extra rather than Assuming the 1600MHz already fitted. Hum ho.
The formula* that calculates the precise RAM speed, or absolute latency,:may well result in finding the 1333MHz memory is theoretically quicker. The 32GB 1600Mhz (CAS 11) memory (in our old i7) runs 2% slower than the 16GB 1333MHz (CAS 9) memory it replaced. From our experience in 2021 we found that 16GB physical memory, with a 10GB VM, would run out of physical memory and result in memory swaps. The biggest win will be from increasing the physical memory to 24GB, which is the minimum we found to allow the VM (with four CPDN tasks) and Windoze enough physical memory to play nicely together.

* Absolute latency = (CAS latency*2000) / Frequency
ID: 64863 · Report as offensive     Reply Quote

Message boards : Number crunching : Memory speed and hadam4h (N216) performance

©2022 climateprediction.net