 |
|
| Name | hadcm3n_4l4z_1940_40_008304136_1 |
| Workunit | 8455271 |
| Created | 21 Mar 2013, 2:53:46 UTC |
| Sent | 21 Mar 2013, 2:53:57 UTC |
| Report deadline | 20 Jun 2013, 10:21:08 UTC |
| Received | 18 May 2013, 19:02:42 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1275223 |
| Run time | 12 days 3 hours 38 min 24 sec |
| CPU time | 11 days 11 hours 29 min 55 sec |
| Validate state | Invalid |
| Credit | 6,220.80 |
| Device peak FLOPS | 2.60 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>7.0.27</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0971ac50 *** ======= Backtrace: ========= /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70f01)[0xf74d1f01] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x72768)[0xf74d3768] /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d)[0xf74d681d] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xf76544bf] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xf7477e46] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 09:00 3850308 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 09:00 3850308 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 096c5000-0972a000 rw-p 00000000 00:00 0 [heap] f6e00000-f6e21000 rw-p 00000000 00:00 0 f6e21000-f6f00000 ---p 00000000 00:00 0 f6fe3000-f745e000 rw-s 00000000 09:00 3899597 /var/lib/boinc-client/slots/2/133955 f745e000-f7461000 rw-p 00000000 00:00 0 f7461000-f75bd000 r-xp 00000000 09:00 3211393 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so f75bd000-f75be000 ---p 0015c000 09:00 3211393 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so f75be000-f75c0000 r--p 0015c000 09:00 3211393 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so f75c0000-f75c1000 rw-p 0015e000 09:00 3211393 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so f75c1000-f75c4000 rw-p 00000000 00:00 0 f75c4000-f75e0000 r-xp 00000000 09:00 3211603 /lib/i386-linux-gnu/libgcc_s.so.1 f75e0000-f75e1000 rw-p 0001b000 09:00 3211603 /lib/i386-linux-gnu/libgcc_s.so.1 f75e1000-f7605000 r-xp 00000000 09:00 3211380 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so f7605000-f7606000 r--p 00023000 09:00 3211380 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so f7606000-f7607000 rw-p 00024000 09:00 3211380 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so f7607000-f76e7000 r-xp 00000000 09:00 5614195 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f76e7000-f76eb000 r--p 000e0000 09:00 5614195 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f76eb000-f76ec000 rw-p 000e4000 09:00 5614195 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f76ec000-f76f4000 rw-p 00000000 00:00 0 f76f4000-f76f6000 r-xp 00000000 09:00 3211372 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so f76f6000-f76f7000 r--p 00001000 09:00 3211372 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so f76f7000-f76f8000 rw-p 00002000 09:00 3211372 /lib/i386-linux-gnu/i686/cmov/libdl-2.13.so f76f8000-f770d000 r-xp 00000000 09:00 3211366 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so f770d000-f770e000 r--p 00014000 09:00 3211366 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so f770e000-f770f000 rw-p 00015000 09:00 3211366 /lib/i386-linux-gnu/i686/cmov/libpthread-2.13.so f770f000-f7711000 rw-p 00000000 00:00 0 f7730000-f7731000 rw-p 00000000 00:00 0 f7731000-f7732000 ---p 00000000 00:00 0 f7732000-f7735000 rw-p 00000000 00:00 0 [stack:1960] f7735000-f7737000 rw-s 00000000 09:00 3899595 /var/lib/boinc-client/slots/2/boinc_mmap_file f7737000-f7739000 rw-p 00000000 00:00 0 f7739000-f773a000 r-xp 00000000 00:00 0 [vdso] f773a000-f7756000 r-xp 00000000 09:00 3211347 /lib/i386-linux-gnu/ld-2.13.so f7756000-f7757000 r--p 0001b000 09:00 3211347 /lib/i386-linux-gnu/ld-2.13.so f7757000-f7758000 rw-p 0001c000 09:00 3211347 /lib/i386-linux-gnu/ld-2.13.so ffd12000-ffd83000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (17 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf7739400] [0xf7739430] /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x51)[0xf748b941] /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x182)[0xf748ed72] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x66e15)[0xf74c7e15] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70f01)[0xf74d1f01] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x72768)[0xf74d3768] /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d)[0xf74d681d] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xf76544bf] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xf7477e46] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 18 May 2013 19:05:21 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 518,400 | 991,791 | 1.9132 |
| 17 May 2013 00:42:12 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 492,480 | 942,799 | 1.9144 |
| 15 May 2013 20:31:22 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 466,560 | 893,835 | 1.9158 |
| 06 May 2013 02:41:51 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 440,640 | 844,977 | 1.9176 |
| 02 May 2013 18:40:54 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 414,720 | 796,463 | 1.9205 |
| 30 Apr 2013 23:11:12 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 388,800 | 748,270 | 1.9246 |
| 28 Apr 2013 20:32:58 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 362,880 | 697,279 | 1.9215 |
| 26 Apr 2013 19:15:53 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 336,960 | 647,527 | 1.9217 |
| 23 Apr 2013 19:23:11 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 311,040 | 598,888 | 1.9254 |
| 19 Apr 2013 18:56:11 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 285,120 | 550,931 | 1.9323 |
| 17 Apr 2013 03:09:56 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 259,200 | 499,016 | 1.9252 |
| 14 Apr 2013 22:37:06 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 233,280 | 449,840 | 1.9283 |
| 12 Apr 2013 18:40:44 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 207,360 | 398,473 | 1.9216 |
| 10 Apr 2013 21:14:53 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 181,440 | 348,908 | 1.9230 |
| 07 Apr 2013 01:30:31 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 155,520 | 299,038 | 1.9228 |
| 04 Apr 2013 20:13:46 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 129,600 | 250,560 | 1.9333 |
| 31 Mar 2013 01:52:18 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 103,680 | 199,041 | 1.9198 |
| 28 Mar 2013 05:22:29 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 77,760 | 147,616 | 1.8984 |
| 25 Mar 2013 21:25:56 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 51,840 | 97,834 | 1.8872 |
| 22 Mar 2013 21:11:59 | 1275223 | 15674354 | hadcm3n_4l4z_1940_40_008304136_1 | 25,920 | 49,011 | 1.8909 |
©2024 climateprediction.net