climateprediction.net home page
Task 13541576

Task 13541576

Name hadcm3n_yekc_1900_40_007517949_0
Workunit 7715424
Created 28 Oct 2011, 12:54:53 UTC
Sent 21 Nov 2011, 6:27:37 UTC
Report deadline 20 Feb 2012, 13:54:48 UTC
Received 18 Jan 2012, 22:43:59 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 882595
Run time 27 days 19 hours 27 min 54 sec
CPU time 27 days 19 hours 27 min 54 sec
Validate state Invalid
Credit 8,398.08
Device peak FLOPS 1.47 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr
<core_client_version>5.10.45</core_client_version>
<![CDATA[
<message>
The device does not recognize the command. (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:00:36 (81076): No heartbeat from core client for 30 sec - exiting
18:00:37 (81076): No heartbeat from core client for 30 sec - exiting
18:00:38 (81076): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
18:00:38 (89584): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:00:37 (4012): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...
18:00:39 (4180): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:00:38 (4824): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:00:37 (5472): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
18:00:37 (2132): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
18:00:39 (4152): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
18:17:31 (5504): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:17:33 (5504): No heartbeat from core client for 30 sec - exiting
18:17:34 (5504): No heartbeat from core client for 30 sec - exiting
18:18:45 (5432): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:28:50 (4792): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:28:51 (4792): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
04:13:15 (4548): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:13:16 (4548): No heartbeat from core client for 30 sec - exiting
06:27:52 (3528): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:27:54 (3528): No heartbeat from core client for 30 sec - exiting

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
07:23:44 (4916): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:23:46 (4916): No heartbeat from core client for 30 sec - exiting
07:23:47 (4916): No heartbeat from core client for 30 sec - exiting
07:25:06 (5176): No heartbeat from core client for 30 sec - exiting
07:25:07 (5176): No heartbeat from core client for 30 sec - exiting
07:25:08 (5176): No heartbeat from core client for 30 sec - exiting
07:25:09 (5176): No heartbeat from core client for 30 sec - exiting
07:25:10 (5176): No heartbeat from core client for 30 sec - exiting
07:25:11 (5176): No heartbeat from core client for 30 sec - exiting
07:25:12 (5176): No heartbeat from core client for 30 sec - exiting
07:25:13 (5176): No heartbeat from core client for 30 sec - exiting
07:25:15 (5176): No heartbeat from core client for 30 sec - exiting
07:25:16 (5176): No heartbeat from core client for 30 sec - exiting
07:25:17 (5176): No heartbeat from core client for 30 sec - exiting
07:25:18 (5176): No heartbeat from core client for 30 sec - exiting
07:25:19 (5176): No heartbeat from core client for 30 sec - exiting
07:25:20 (5176): No heartbeat from core client for 30 sec - exiting
07:25:21 (5176): No heartbeat from core client for 30 sec - exiting
07:25:22 (5176): No heartbeat from core client for 30 sec - exiting
07:25:23 (5176): No heartbeat from core client for 30 sec - exiting
07:25:24 (5176): No heartbeat from core client for 30 sec - exiting
07:25:26 (5176): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
03:45:44 (5792): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:45:46 (5792): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
05:04:15 (4540): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:04:17 (4540): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
18:00:40 (5024): No heartbeat from core client for 30 sec - exiting
18:00:41 (5024): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:07:09 (4796): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:07:11 (4796): No heartbeat from core client for 30 sec - exiting
18:08:06 (5752): No heartbeat from core client for 30 sec - exiting
18:08:07 (5752): No heartbeat from core client for 30 sec - exiting
18:08:08 (5752): No heartbeat from core client for 30 sec - exiting
18:08:09 (5752): No heartbeat from core client for 30 sec - exiting
18:08:11 (5752): No heartbeat from core client for 30 sec - exiting
18:08:12 (5752): No heartbeat from core client for 30 sec - exiting
18:08:13 (5752): No heartbeat from core client for 30 sec - exiting
18:08:14 (5752): No heartbeat from core client for 30 sec - exiting
18:08:15 (5752): No heartbeat from core client for 30 sec - exiting
18:08:16 (5752): No heartbeat from core client for 30 sec - exiting
18:08:17 (5752): No heartbeat from core client for 30 sec - exiting
18:08:18 (5752): No heartbeat from core client for 30 sec - exiting
18:08:19 (5752): No heartbeat from core client for 30 sec - exiting
18:08:20 (5752): No heartbeat from core client for 30 sec - exiting
18:08:21 (5752): No heartbeat from core client for 30 sec - exiting
18:08:23 (5752): No heartbeat from core client for 30 sec - exiting
18:08:24 (5752): No heartbeat from core client for 30 sec - exiting
18:08:25 (5752): No heartbeat from core client for 30 sec - exiting
18:08:26 (5752): No heartbeat from core client for 30 sec - exiting
18:08:27 (5752): No heartbeat from core client for 30 sec - exiting
18:08:28 (5752): No heartbeat from core client for 30 sec - exiting
18:08:29 (5752): No heartbeat from core client for 30 sec - exiting
18:08:30 (5752): No heartbeat from core client for 30 sec - exiting
18:08:31 (5752): No heartbeat from core client for 30 sec - exiting
18:08:32 (5752): No heartbeat from core client for 30 sec - exiting
18:08:33 (5752): No heartbeat from core client for 30 sec - exiting
18:08:35 (5752): No heartbeat from core client for 30 sec - exiting
18:08:36 (5752): No heartbeat from core client for 30 sec - exiting
18:08:37 (5752): No heartbeat from core client for 30 sec - exiting
18:08:38 (5752): No heartbeat from core client for 30 sec - exiting
18:08:39 (5752): No heartbeat from core client for 30 sec - exiting
18:08:40 (5752): No heartbeat from core client for 30 sec - exiting
18:08:41 (5752): No heartbeat from core client for 30 sec - exiting
18:08:42 (5752): No heartbeat from core client for 30 sec - exiting
18:08:43 (5752): No heartbeat from core client for 30 sec - exiting
18:08:44 (5752): No heartbeat from core client for 30 sec - exiting
18:08:45 (5752): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:14:18 (3972): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:14:21 (3972): No heartbeat from core client for 30 sec - exiting
18:15:44 (3476): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:18:20 (5352): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:18:58 (4452): No heartbeat from core client for 30 sec - exiting
18:18:59 (4452): No heartbeat from core client for 30 sec - exiting
18:19:00 (4452): No heartbeat from core client for 30 sec - exiting
18:19:01 (4452): No heartbeat from core client for 30 sec - exiting
18:19:02 (4452): No heartbeat from core client for 30 sec - exiting
18:19:03 (4452): No heartbeat from core client for 30 sec - exiting
18:19:04 (4452): No heartbeat from core client for 30 sec - exiting
18:19:05 (4452): No heartbeat from core client for 30 sec - exiting
18:19:06 (4452): No heartbeat from core client for 30 sec - exiting
18:19:08 (4452): No heartbeat from core client for 30 sec - exiting
18:19:09 (4452): No heartbeat from core client for 30 sec - exiting
18:19:10 (4452): No heartbeat from core client for 30 sec - exiting
18:19:11 (4452): No heartbeat from core client for 30 sec - exiting
18:19:12 (4452): No heartbeat from core client for 30 sec - exiting
18:19:13 (4452): No heartbeat from core client for 30 sec - exiting
18:19:14 (4452): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:19:15 (4452): No heartbeat from core client for 30 sec - exiting
18:19:16 (4452): No heartbeat from core client for 30 sec - exiting
18:22:32 (3464): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:22:33 (3464): No heartbeat from core client for 30 sec - exiting
18:23:32 (5140): No heartbeat from core client for 30 sec - exiting
18:23:33 (5140): No heartbeat from core client for 30 sec - exiting
18:23:34 (5140): No heartbeat from core client for 30 sec - exiting
18:23:36 (5140): No heartbeat from core client for 30 sec - exiting
18:23:37 (5140): No heartbeat from core client for 30 sec - exiting
18:23:38 (5140): No heartbeat from core client for 30 sec - exiting
18:23:39 (5140): No heartbeat from core client for 30 sec - exiting
18:23:40 (5140): No heartbeat from core client for 30 sec - exiting
18:23:41 (5140): No heartbeat from core client for 30 sec - exiting
18:23:42 (5140): No heartbeat from core client for 30 sec - exiting
18:23:43 (5140): No heartbeat from core client for 30 sec - exiting
18:23:44 (5140): No heartbeat from core client for 30 sec - exiting
18:23:45 (5140): No heartbeat from core client for 30 sec - exiting
18:23:46 (5140): No heartbeat from core client for 30 sec - exiting
18:23:48 (5140): No heartbeat from core client for 30 sec - exiting
18:23:49 (5140): No heartbeat from core client for 30 sec - exiting
18:23:50 (5140): No heartbeat from core client for 30 sec - exiting
18:23:51 (5140): No heartbeat from core client for 30 sec - exiting
18:23:52 (5140): No heartbeat from core client for 30 sec - exiting
18:23:53 (5140): No heartbeat from core client for 30 sec - exiting
18:23:54 (5140): No heartbeat from core client for 30 sec - exiting
18:23:55 (5140): No heartbeat from core client for 30 sec - exiting
18:23:56 (5140): No heartbeat from core client for 30 sec - exiting
18:23:57 (5140): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:23:58 (5140): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
18:00:40 (5944): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:00:37 (6060): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
CPDN Monitor - Quit request from BOINC...
18:54:38 (4368): No heartbeat from core client for 30 sec - exiting
18:54:39 (4368): No heartbeat from core client for 30 sec - exiting
18:54:40 (4368): No heartbeat from core client for 30 sec - exiting
18:54:41 (4368): No heartbeat from core client for 30 sec - exiting
18:54:42 (4368): No heartbeat from core client for 30 sec - exiting
18:54:43 (4368): No heartbeat from core client for 30 sec - exiting
18:54:44 (4368): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
Error converting file to netcdf: dataout/yekcko.pjc6c10
Error converting file to netcdf: dataout/yekcko.pic6c10
Error converting file to netcdf: dataout/yekcko.pfc6c10
Error converting file to netcdf: dataout/yekcka.phc6c10
Error converting file to netcdf: dataout/yekcka.pgc6c10
Error converting file to netcdf: dataout/yekcka.pec6c10
Error converting file to netcdf: dataout/yekcka.pdc6c10

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    
22:23:42 (2768): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:23:44 (2768): No heartbeat from core client for 30 sec - exiting
22:23:45 (2768): No heartbeat from core client for 30 sec - exiting
22:23:46 (2768): No heartbeat from core client for 30 sec - exiting
22:25:04 (5432): No heartbeat from core client for 30 sec - exiting
22:25:05 (5432): No heartbeat from core client for 30 sec - exiting
22:25:06 (5432): No heartbeat from core client for 30 sec - exiting
22:25:07 (5432): No heartbeat from core client for 30 sec - exiting
22:25:08 (5432): No heartbeat from core client for 30 sec - exiting
22:25:09 (5432): No heartbeat from core client for 30 sec - exiting
22:25:10 (5432): No heartbeat from core client for 30 sec - exiting
22:25:11 (5432): No heartbeat from core client for 30 sec - exiting
22:25:12 (5432): No heartbeat from core client for 30 sec - exiting
22:25:13 (5432): No heartbeat from core client for 30 sec - exiting
22:25:15 (5432): No heartbeat from core client for 30 sec - exiting
22:25:16 (5432): No heartbeat from core client for 30 sec - exiting
22:25:17 (5432): No heartbeat from core client for 30 sec - exiting
22:25:18 (5432): No heartbeat from core client for 30 sec - exiting
22:25:19 (5432): No heartbeat from core client for 30 sec - exiting
22:25:20 (5432): No heartbeat from core client for 30 sec - exiting
22:25:21 (5432): No heartbeat from core client for 30 sec - exiting
22:25:22 (5432): No heartbeat from core client for 30 sec - exiting
22:25:23 (5432): No heartbeat from core client for 30 sec - exiting
22:25:24 (5432): No heartbeat from core client for 30 sec - exiting
22:25:26 (5432): No heartbeat from core client for 30 sec - exiting
22:25:27 (5432): No heartbeat from core client for 30 sec - exiting
22:25:28 (5432): No heartbeat from core client for 30 sec - exiting
22:25:29 (5432): No heartbeat from core client for 30 sec - exiting
22:25:30 (5432): No heartbeat from core client for 30 sec - exiting
22:25:31 (5432): No heartbeat from core client for 30 sec - exiting
22:25:32 (5432): No heartbeat from core client for 30 sec - exiting
22:25:33 (5432): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:25:34 (5432): No heartbeat from core client for 30 sec - exiting
22:25:36 (5432): No heartbeat from core client for 30 sec - exiting
22:32:45 (3600): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:34:52 (4248): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:00:39 (5012): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: TEMPHIST: Failed in OPEN of history file                                                                                                                                                                                                                        tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
17 Jan 2012 07:52:58 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 699,840 2,398,887 3.4278
15 Jan 2012 19:04:31 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 673,920 2,304,436 3.4195
13 Jan 2012 22:29:41 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 648,000 2,216,265 3.4202
12 Jan 2012 00:15:48 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 622,080 2,117,526 3.4039
10 Jan 2012 02:16:54 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 596,160 2,022,020 3.3917
08 Jan 2012 03:03:44 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 570,240 1,921,343 3.3694
06 Jan 2012 05:27:20 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 544,320 1,824,778 3.3524
04 Jan 2012 07:51:01 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 518,400 1,728,716 3.3347
02 Jan 2012 08:14:42 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 492,480 1,668,388 3.3877
31 Dec 2011 08:31:23 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 466,560 1,603,466 3.4368
29 Dec 2011 08:43:53 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 440,640 1,525,753 3.4626
27 Dec 2011 18:45:32 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 414,720 1,448,303 3.4922
25 Dec 2011 19:10:48 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 388,800 1,346,115 3.4622
23 Dec 2011 19:25:31 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 362,880 1,243,273 3.4261
15 Dec 2011 02:37:42 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 336,960 1,167,507 3.4648
13 Dec 2011 02:18:44 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 311,040 1,063,402 3.4189
11 Dec 2011 03:45:37 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 285,120 964,231 3.3818
09 Dec 2011 06:52:09 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 259,200 870,633 3.3589
07 Dec 2011 18:41:03 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 233,280 775,266 3.3233
05 Dec 2011 20:50:23 882595 13541576 hadcm3n_yekc_1900_40_007517949_0 207,360 679,127 3.2751


©2024 climateprediction.net