climateprediction.net home page
Long runs and low credits
Long runs and low credits
log in

Advanced search

Message boards : Number crunching : Long runs and low credits

Author Message
Kim
Send message
Joined: 17 May 14
Posts: 5
Credit: 1,702,892
RAC: 2,265
Message 59983 - Posted: 17 Apr 2019, 21:57:42 UTC

Hi
Since my upgrade to a more powerful system, I have noticed that I'm receiving long run time files. That's not an issue as I have lots of idle time. But I am noticing that the last few weeks 10-20 day runs have been giving small credits and this seems very odd considering the amount of computing time. What going on or is there an error somewhere.

Kim

Les Bayliss
Volunteer moderator
Send message
Joined: 5 Sep 04
Posts: 6909
Credit: 20,843,205
RAC: 108
Message 59984 - Posted: 17 Apr 2019, 23:41:32 UTC

The models these days ARE longer, and some cover a VERY large (regional) area, so they are slower and take longer to process.

And there are new models on the horizon which are much bigger, and will take the project into a similar resolution as the Met Office super computer models.

You've crashed a lot of tasks, so you won't get much, if any, credit for them.

(And repeating an old story, the credit system only runs once per week, on Saturdays.)

Aurum
Avatar
Send message
Joined: 15 Jul 17
Posts: 15
Credit: 5,064,521
RAC: 1,612
Message 60043 - Posted: 1 May 2019, 18:34:34 UTC - in response to Message 59984.

Strange, I seem to get more credit for "crashed" WUs:

21146213 11527147 1453910 26 Mar 2018, 5:16:23 UTC 18 Apr 2018, 13:08:52 UTC Error while computing 539,241.69 533,577.10 14,458.52 Weather At Home 2 (wah2) v8.24
windows_intelx86
____________

bernard_ivo
Send message
Joined: 18 Jul 13
Posts: 342
Credit: 9,747,196
RAC: 32,487
Message 60046 - Posted: 1 May 2019, 19:34:26 UTC
Last modified: 1 May 2019, 19:39:34 UTC

There is some discrepancy between credits and long runs. Most notable with EU25 WUs, where my reliable win7 machine gets them a lot but the least credit per CPU time

Run time (sec) --- CPU time (sec) -- Credit
1,017,256.94 --- 1,002,254.00 --- 9,898.9 --- EU25 13 months
523,449.11 ---516,650.60--- 9,898.90 ---SAM50 13 months
973,122.62 ---960,097.10--- 18,258.20---SAM50 24 months
2,512,569.76 ---2,486,619.00--- 18,258.20---SAM25 24 months

Whether these credits are enough is different issue.

Profile Alan K
Send message
Joined: 22 Feb 06
Posts: 284
Credit: 14,477,549
RAC: 9,631
Message 60048 - Posted: 1 May 2019, 22:07:05 UTC - in response to Message 60046.

As fsr as I am aware the credits are based on the number of trickles not the CPU time. Since the SAM25 is a 24 month model it will produce 24 trickles - the same number as the 24 month SAM50 even though the CPU time is almost 3 times longer. I agree that this does not fully reflect the CPU time used. Time for a "fiddle factor" perhaps?

Profile Dave Jackson
Volunteer moderator
Send message
Joined: 15 May 09
Posts: 2415
Credit: 3,076,286
RAC: 342
Message 60049 - Posted: 2 May 2019, 4:46:29 UTC - in response to Message 60048.

Different types of work units do have different amounts of credit/trickle or certainly used to. If no one else does it before I get a chance to I will try and find enough detail to pass on to the project. It is some years since I last looked at the credit I get from specific model types so I may be out of date. There are also so many now of differing complexities that there are variations between what appears to be the same model type in time taken to complete because of different number of variables being included.

I don't know if the system can cope with different credit scores for models of the same nominal type?

Les Bayliss
Volunteer moderator
Send message
Joined: 5 Sep 04
Posts: 6909
Credit: 20,843,205
RAC: 108
Message 60050 - Posted: 2 May 2019, 5:46:52 UTC

Credits are based on the amount of work a computer has to do to create a model.
This is determined during the testing.

Then a value for each type of model is stored in a table, and used by the credit calc program to determine a suitable amount of credit for a model, based on the number of trickle_up files received up to that point.

And the project doesn't intend to play catch-up with other projects that use gpus or optimised apps.

Profile Iain Inglis
Volunteer moderator
Send message
Joined: 16 Jan 10
Posts: 976
Credit: 2,899,749
RAC: 10,458
Message 60054 - Posted: 2 May 2019, 18:50:44 UTC
Last modified: 2 May 2019, 21:25:49 UTC

Here's an estimate of relative credits/hour (and GFLOPS) for 530 models run across seven machines of mine (most now retired).

ID App GFLOPs Normalised GFLOPS Credits Normalised Credits/Hour AFR25/12 7.08 9,138.96 AFR50/9 8.24 6,859.15 AFR50/12 8.24 1,776,974 1.25 9,138.96 1.13 AFR50/13 8.12 1,924,645 0.97 9,898.90 0.87 8.24 1.27 1.15 AFR50/120 8.24 91,212.09 ANZ50/8 8.24 1,346,181 1.32 6,099.22 1.05 ANZ50/13 8.24 2,184,053 1.34 9,898.90 1.06 ANZ50/20 8.24 3,357,074 1.34 15,218.45 1.07 ANZ50/120 8.24 91,212.09 ATLA50/11 8.24 3,532,559 8,379.03 ATLA50/120 8.24 38,431,251 91,212.09 CAFR25/13 8.24 5,222,246 1.50 9,898.90 0.50 CAFR25/120 8.24 91,212.09 CAM25/5 8.24 3,819.41 CAM25/10 8.24 3,212,387 1.32 7,619.09 0.55 CAM25/18 8.24 5,773,759 1.30 CAM25/120 8.24 91,212.09 CAM50/12 8.24 784,376 0.71 9,138.96 1.45 CAM50/13 8.12 849,559 0.51 9,898.90 1.05 CAM50/120 8.24 91,212.09 CARI25/11 8.24 3,532,559 1.36 8,379.03 0.57 CARI25/120 8.24 91,212.09 DUB25/12 8.12 9,138.96 DUB25/13 8.12 3,327,828 0.67 9,898.90 0.35 EAS50/10 8.24 1,482,156 1.05 7,619.09 0.94 EAS50/12 8.24 1,777,602 1.02 9,138.96 0.92 EAS50/13 8.24 1,925,325 1.05 9,898.90 0.95 EAS50/18 8.24 2,663,941 1.01 13,698.58 0.92 EAS50/120 8.24 91,212.09 EU25/12 8.12 1,903,330 0.54 9,138.96 0.46 EU25/13 8.12 2,061,502 0.56 9,898.90 0.47 8.24 0.69 0.58 EU25/120 8.24 91,212.09 EU50r/3 8.24 209,165 0.87 2,299.53 1.68 EU50r/4 8.24 278,121 0.87 3,059.47 1.68 EU50r/13 8.12 898,721 0.86 9,898.90 1.66 EU50r/120 8.24 91,212.09 GLOBAL25/1 8.24 194.40 GLOBAL25/2 8.24 384.39 GLOBAL25/3 8.24 2,299.53 GLOBAL25/16 8.24 319,399 0.53 3,044.17 0.89 GLOBAL25/25 8.24 498,688 0.53 4,754.02 0.88 GLOBAL25/133 8.24 25,272.31 GLOBAL25/145 8.24 2,889,203 0.52 27,552.12 0.86 MEX25/5 8.12 3,819.41 MEX25/13 8.12 4,172,902 9,898.90 MEX50/12 8.12 784,376 0.50 9,139.96 1.03 MEX50/13 8.12 849,559 0.53 9,898.90 1.09 NAM50/13 8.24 2,014,634 1.43 9,898.90 1.23 NAM50/120 8.24 91,212.09 NAWA25/1 8.12 194.40 NAWA25/12 8.24 9,138.96 NAWA25/13 8.12 3,327,828 9,898.90 NAWA25/120 8.24 91,212.09 PNW25/2 8.24 1,539.60 PNW25/16 8.24 1,277,597 0.92 12,178.71 1.55 PNW25/21 8.24 1,616,017 0.89 15,978.39 1.55 PNW25/24 8.12 1,915,068 0.71 18,258.20 1.18 8.24 0.93 1.55 PNW25/25 8.24 19,018.14 PNW25/28 8.24 2,233,803 0.93 21,297.94 1.55 PNW25/48 8.24 33,456.93 PNW25/49 8.19 3,907,164 0.93 37,256.61 1.56 8.24 0.93 1.56 PNW25/60 8.24 4,783,686 0.94 45,615.91 1.58 PNW25/61 8.24 46,375.85 PNW25/80 8.24 60,814.64 PNW25/120 8.24 91,212.09 PNW25/121 8.24 9,644,400 0.92 91,972.03 1.54 SAFR50/0 8.24 0.00 SAFR50/2 8.24 1,539.60 SAFR50/13 8.24 4,172,902 2.47 9,898.90 1.03 SAFR50/14 8.24 4,493,073 SAFR50/16 8.24 5,133,416 2.27 12,178.71 0.95 SAFR50/24 8.24 7,694,788 18,258.20 SAFR50/120 8.24 91,212.09 SAM25/0 8.24 0.0 SAM25/2 8.24 1,539.60 SAM25/13 8.24 4,172,902 0.98 9,898.90 0.41 SAM25/24 8.24 18,258.20 SAM25/60 8.24 19,220,962 0.88 45,615.91 0.37 SAM25/61 8.24 46,375.85 SAM25/72 8.24 53,975.21 SAM25/73 8.24 23,383,191 1.01 55,495.08 0.42 SAM25/85 8.24 27,225,249 0.84 64,614.32 0.35 SAM25/120 8.24 91,212.09 SAM50/12 8.24 9,138.96 SAM50/13 8.12 2,147,398 0.93 9,898.90 0.75 8.24 1.35 1.09 SAM50/16 8.24 2,641,684 1.34 12,178.71 1.08 SAM50/24 8.24 3,959,780 1.36 18,258.20 1.10 SAM50/25 8.24 4,124,542 1.35 19,018.14 1.09 SAM50/120 8.24 91,212.09 SAS50/1 8.24 779.66 SAS50/3 8.24 348,111 0.89 2,299.53 1.03 SAS50/4 8.24 462,873 0.89 3,059.47 1.03 SAS50/8 8.12 921,921 0.67 6,099.22 0.77 8.24 0.89 1.03 SAS50/10 8.24 1,151,445 0.88 7,619.09 1.02 SAS50/12 8.24 1,380,969 0.87 9,138.96 1.01 SAS50/13 8.12 1,495,731 0.69 9,898.90 0.80 8.24 0.89 1.03 SAS50/22 8.24 2,528,589 0.88 13,698.58 0.83 SAS50/120 8.24 91,212.09 WUS25/9 8.19 6,859.15 WUS25/16 8.24 12,178.71 WUS25/21 8.24 15,978.39 WUS25/24 8.24 18,258.20 WUS25/25 8.24 3,291,506 1.13 19,018.14 1.15 WUS25/120 8.24 91,212.09


The best were EU50r, with 1.68 times the average credit rate. The worst were SAFR50/0 and SAM25/0, which had no trickles and no credits.

Note that this sample only includes model types that I have managed to download - there may be other good and bad payers.

bernard_ivo
Send message
Joined: 18 Jul 13
Posts: 342
Credit: 9,747,196
RAC: 32,487
Message 60501 - Posted: 30 Jun 2019, 6:10:09 UTC

Hi
I've noticed quite some credit discrepancy between the HadAM4 at N144 and pnw25

The first is very resource heavy and took more than 12 days for 17,599.91 credit (i7-4790 @3.6GHz)
The second took a bit over 3 days and the credit for it is 12,178.71 (E5-2630 v2 @ 2.60GHz)

It would be nice if the heavy ones are granted fairer level of credit

Les Bayliss
Volunteer moderator
Send message
Joined: 5 Sep 04
Posts: 6909
Credit: 20,843,205
RAC: 108
Message 60502 - Posted: 30 Jun 2019, 7:31:01 UTC

We did try to get it right during testing, but I'll raise the matter after all of the server problems have been solved.

Profile geophi
Volunteer moderator
Send message
Joined: 7 Aug 04
Posts: 1811
Credit: 36,354,717
RAC: 10,605
Message 60508 - Posted: 30 Jun 2019, 13:24:34 UTC - in response to Message 60501.

The PNW ones have gotten inflated credits relative to the other models types for a long time (when looking at credit for time crunched) so they are not the ones to compare other models to. The HADAM4 are in line with the other models such as SAM50, SAFR, SAS, ANZ and GLOBAL.

It's the SAM25, CAM25 and EU25 that are getting too low credits relative to the other models and those are being looked at for possible retroactive adjustment upward.

bernard_ivo
Send message
Joined: 18 Jul 13
Posts: 342
Credit: 9,747,196
RAC: 32,487
Message 60509 - Posted: 30 Jun 2019, 16:02:49 UTC - in response to Message 60508.

Thanks,

As long as it is dealt with in fair mode we should all be fine :)

Message boards : Number crunching : Long runs and low credits


Main page · Your account · Message boards


Copyright © 2019 climateprediction.net