climateprediction.net home page
Long runs and low credits

Long runs and low credits

Message boards : Number crunching : Long runs and low credits
Message board moderation

To post messages, you must log in.

AuthorMessage
Kim

Send message
Joined: 17 May 14
Posts: 5
Credit: 1,794,104
RAC: 25
Message 59983 - Posted: 17 Apr 2019, 21:57:42 UTC

Hi
Since my upgrade to a more powerful system, I have noticed that I'm receiving long run time files. That's not an issue as I have lots of idle time. But I am noticing that the last few weeks 10-20 day runs have been giving small credits and this seems very odd considering the amount of computing time. What going on or is there an error somewhere.

Kim
ID: 59983 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 6960
Credit: 20,843,205
RAC: 0
Message 59984 - Posted: 17 Apr 2019, 23:41:32 UTC

The models these days ARE longer, and some cover a VERY large (regional) area, so they are slower and take longer to process.

And there are new models on the horizon which are much bigger, and will take the project into a similar resolution as the Met Office super computer models.

You've crashed a lot of tasks, so you won't get much, if any, credit for them.

(And repeating an old story, the credit system only runs once per week, on Saturdays.)
ID: 59984 · Report as offensive     Reply Quote
Aurum
Avatar

Send message
Joined: 15 Jul 17
Posts: 15
Credit: 5,064,521
RAC: 1
Message 60043 - Posted: 1 May 2019, 18:34:34 UTC - in response to Message 59984.  

Strange, I seem to get more credit for "crashed" WUs:

21146213 11527147 1453910 26 Mar 2018, 5:16:23 UTC 18 Apr 2018, 13:08:52 UTC Error while computing 539,241.69 533,577.10 14,458.52 Weather At Home 2 (wah2) v8.24
windows_intelx86
ID: 60043 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 344
Credit: 9,943,922
RAC: 25
Message 60046 - Posted: 1 May 2019, 19:34:26 UTC
Last modified: 1 May 2019, 19:39:34 UTC

There is some discrepancy between credits and long runs. Most notable with EU25 WUs, where my reliable win7 machine gets them a lot but the least credit per CPU time

Run time (sec) --- CPU time (sec) -- Credit
1,017,256.94 --- 1,002,254.00 --- 9,898.9 --- EU25 13 months
523,449.11 ---516,650.60--- 9,898.90 ---SAM50 13 months
973,122.62 ---960,097.10--- 18,258.20---SAM50 24 months
2,512,569.76 ---2,486,619.00--- 18,258.20---SAM25 24 months

Whether these credits are enough is different issue.
ID: 60046 · Report as offensive     Reply Quote
Profile Alan K

Send message
Joined: 22 Feb 06
Posts: 293
Credit: 14,738,424
RAC: 92
Message 60048 - Posted: 1 May 2019, 22:07:05 UTC - in response to Message 60046.  

As fsr as I am aware the credits are based on the number of trickles not the CPU time. Since the SAM25 is a 24 month model it will produce 24 trickles - the same number as the 24 month SAM50 even though the CPU time is almost 3 times longer. I agree that this does not fully reflect the CPU time used. Time for a "fiddle factor" perhaps?
ID: 60048 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 2472
Credit: 3,125,721
RAC: 304
Message 60049 - Posted: 2 May 2019, 4:46:29 UTC - in response to Message 60048.  

Different types of work units do have different amounts of credit/trickle or certainly used to. If no one else does it before I get a chance to I will try and find enough detail to pass on to the project. It is some years since I last looked at the credit I get from specific model types so I may be out of date. There are also so many now of differing complexities that there are variations between what appears to be the same model type in time taken to complete because of different number of variables being included.

I don't know if the system can cope with different credit scores for models of the same nominal type?
ID: 60049 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 6960
Credit: 20,843,205
RAC: 0
Message 60050 - Posted: 2 May 2019, 5:46:52 UTC

Credits are based on the amount of work a computer has to do to create a model.
This is determined during the testing.

Then a value for each type of model is stored in a table, and used by the credit calc program to determine a suitable amount of credit for a model, based on the number of trickle_up files received up to that point.

And the project doesn't intend to play catch-up with other projects that use gpus or optimised apps.
ID: 60050 · Report as offensive     Reply Quote
Profile Iain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 979
Credit: 3,103,609
RAC: 47
Message 60054 - Posted: 2 May 2019, 18:50:44 UTC
Last modified: 2 May 2019, 21:25:49 UTC

Here's an estimate of relative credits/hour (and GFLOPS) for 530 models run across seven machines of mine (most now retired).

ID             App    GFLOPs       Normalised GFLOPS   Credits      Normalised Credits/Hour
AFR25/12       7.08                                    9,138.96     
AFR50/9        8.24                                    6,859.15     
AFR50/12       8.24   1,776,974    1.25                9,138.96     1.13
AFR50/13       8.12   1,924,645    0.97                9,898.90     0.87
               8.24                1.27                             1.15
AFR50/120      8.24                                    91,212.09    
ANZ50/8        8.24   1,346,181    1.32                6,099.22     1.05
ANZ50/13       8.24   2,184,053    1.34                9,898.90     1.06
ANZ50/20       8.24   3,357,074    1.34                15,218.45    1.07
ANZ50/120      8.24                                    91,212.09    
ATLA50/11      8.24   3,532,559                        8,379.03     
ATLA50/120     8.24   38,431,251                       91,212.09    
CAFR25/13      8.24   5,222,246    1.50                9,898.90     0.50
CAFR25/120     8.24                                    91,212.09    
CAM25/5        8.24                                    3,819.41     
CAM25/10       8.24   3,212,387    1.32                7,619.09     0.55
CAM25/18       8.24   5,773,759    1.30                             
CAM25/120      8.24                                    91,212.09    
CAM50/12       8.24   784,376      0.71                9,138.96     1.45
CAM50/13       8.12   849,559      0.51                9,898.90     1.05
CAM50/120      8.24                                    91,212.09    
CARI25/11      8.24   3,532,559    1.36                8,379.03     0.57
CARI25/120     8.24                                    91,212.09    
DUB25/12       8.12                                    9,138.96     
DUB25/13       8.12   3,327,828    0.67                9,898.90     0.35
EAS50/10       8.24   1,482,156    1.05                7,619.09     0.94
EAS50/12       8.24   1,777,602    1.02                9,138.96     0.92
EAS50/13       8.24   1,925,325    1.05                9,898.90     0.95
EAS50/18       8.24   2,663,941    1.01                13,698.58    0.92
EAS50/120      8.24                                    91,212.09    
EU25/12        8.12   1,903,330    0.54                9,138.96     0.46
EU25/13        8.12   2,061,502    0.56                9,898.90     0.47
               8.24                0.69                             0.58
EU25/120       8.24                                    91,212.09    
EU50r/3        8.24   209,165      0.87                2,299.53     1.68
EU50r/4        8.24   278,121      0.87                3,059.47     1.68
EU50r/13       8.12   898,721      0.86                9,898.90     1.66
EU50r/120      8.24                                    91,212.09    
GLOBAL25/1     8.24                                    194.40       
GLOBAL25/2     8.24                                    384.39       
GLOBAL25/3     8.24                                    2,299.53     
GLOBAL25/16    8.24   319,399      0.53                3,044.17     0.89
GLOBAL25/25    8.24   498,688      0.53                4,754.02     0.88
GLOBAL25/133   8.24                                    25,272.31    
GLOBAL25/145   8.24   2,889,203    0.52                27,552.12    0.86
MEX25/5        8.12                                    3,819.41     
MEX25/13       8.12   4,172,902                        9,898.90     
MEX50/12       8.12   784,376      0.50                9,139.96     1.03
MEX50/13       8.12   849,559      0.53                9,898.90     1.09
NAM50/13       8.24   2,014,634    1.43                9,898.90     1.23
NAM50/120      8.24                                    91,212.09    
NAWA25/1       8.12                                    194.40       
NAWA25/12      8.24                                    9,138.96     
NAWA25/13      8.12   3,327,828                        9,898.90     
NAWA25/120     8.24                                    91,212.09    
PNW25/2        8.24                                    1,539.60     
PNW25/16       8.24   1,277,597    0.92                12,178.71    1.55
PNW25/21       8.24   1,616,017    0.89                15,978.39    1.55
PNW25/24       8.12   1,915,068    0.71                18,258.20    1.18
               8.24                0.93                             1.55
PNW25/25       8.24                                    19,018.14    
PNW25/28       8.24   2,233,803    0.93                21,297.94    1.55
PNW25/48       8.24                                    33,456.93    
PNW25/49       8.19   3,907,164    0.93                37,256.61    1.56
               8.24                0.93                             1.56
PNW25/60       8.24   4,783,686    0.94                45,615.91    1.58
PNW25/61       8.24                                    46,375.85    
PNW25/80       8.24                                    60,814.64    
PNW25/120      8.24                                    91,212.09    
PNW25/121      8.24   9,644,400    0.92                91,972.03    1.54
SAFR50/0       8.24                                    0.00         
SAFR50/2       8.24                                    1,539.60     
SAFR50/13      8.24   4,172,902    2.47                9,898.90     1.03
SAFR50/14      8.24   4,493,073                                     
SAFR50/16      8.24   5,133,416    2.27                12,178.71    0.95
SAFR50/24      8.24   7,694,788                        18,258.20    
SAFR50/120     8.24                                    91,212.09    
SAM25/0        8.24                                    0.0          
SAM25/2        8.24                                    1,539.60     
SAM25/13       8.24   4,172,902    0.98                9,898.90     0.41
SAM25/24       8.24                                    18,258.20    
SAM25/60       8.24   19,220,962   0.88                45,615.91    0.37
SAM25/61       8.24                                    46,375.85    
SAM25/72       8.24                                    53,975.21    
SAM25/73       8.24   23,383,191   1.01                55,495.08    0.42
SAM25/85       8.24   27,225,249   0.84                64,614.32    0.35
SAM25/120      8.24                                    91,212.09    
SAM50/12       8.24                                    9,138.96     
SAM50/13       8.12   2,147,398    0.93                9,898.90     0.75
               8.24                1.35                             1.09
SAM50/16       8.24   2,641,684    1.34                12,178.71    1.08
SAM50/24       8.24   3,959,780    1.36                18,258.20    1.10
SAM50/25       8.24   4,124,542    1.35                19,018.14    1.09
SAM50/120      8.24                                    91,212.09    
SAS50/1        8.24                                    779.66       
SAS50/3        8.24   348,111      0.89                2,299.53     1.03
SAS50/4        8.24   462,873      0.89                3,059.47     1.03
SAS50/8        8.12   921,921      0.67                6,099.22     0.77
               8.24                0.89                             1.03
SAS50/10       8.24   1,151,445    0.88                7,619.09     1.02
SAS50/12       8.24   1,380,969    0.87                9,138.96     1.01
SAS50/13       8.12   1,495,731    0.69                9,898.90     0.80
               8.24                0.89                             1.03
SAS50/22       8.24   2,528,589    0.88                13,698.58    0.83
SAS50/120      8.24                                    91,212.09    
WUS25/9        8.19                                    6,859.15     
WUS25/16       8.24                                    12,178.71    
WUS25/21       8.24                                    15,978.39    
WUS25/24       8.24                                    18,258.20    
WUS25/25       8.24   3,291,506    1.13                19,018.14    1.15
WUS25/120      8.24                                    91,212.09    


The best were EU50r, with 1.68 times the average credit rate. The worst were SAFR50/0 and SAM25/0, which had no trickles and no credits.

Note that this sample only includes model types that I have managed to download - there may be other good and bad payers.
ID: 60054 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 344
Credit: 9,943,922
RAC: 25
Message 60501 - Posted: 30 Jun 2019, 6:10:09 UTC

Hi
I've noticed quite some credit discrepancy between the HadAM4 at N144 and pnw25

The first is very resource heavy and took more than 12 days for 17,599.91 credit (i7-4790 @3.6GHz)
The second took a bit over 3 days and the credit for it is 12,178.71 (E5-2630 v2 @ 2.60GHz)

It would be nice if the heavy ones are granted fairer level of credit
ID: 60501 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 6960
Credit: 20,843,205
RAC: 0
Message 60502 - Posted: 30 Jun 2019, 7:31:01 UTC

We did try to get it right during testing, but I'll raise the matter after all of the server problems have been solved.
ID: 60502 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 1822
Credit: 36,393,036
RAC: 5
Message 60508 - Posted: 30 Jun 2019, 13:24:34 UTC - in response to Message 60501.  

The PNW ones have gotten inflated credits relative to the other models types for a long time (when looking at credit for time crunched) so they are not the ones to compare other models to. The HADAM4 are in line with the other models such as SAM50, SAFR, SAS, ANZ and GLOBAL.

It's the SAM25, CAM25 and EU25 that are getting too low credits relative to the other models and those are being looked at for possible retroactive adjustment upward.
ID: 60508 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 344
Credit: 9,943,922
RAC: 25
Message 60509 - Posted: 30 Jun 2019, 16:02:49 UTC - in response to Message 60508.  

Thanks,

As long as it is dealt with in fair mode we should all be fine :)
ID: 60509 · Report as offensive     Reply Quote

Message boards : Number crunching : Long runs and low credits

©2019 climateprediction.net