Task 11871697

Name	hadam3p_saf_v8hp_1973_1_006722651_0
Workunit	6925901
Created	6 Sep 2010, 11:58:09 UTC
Sent	7 Sep 2010, 8:24:47 UTC
Report deadline	20 Aug 2011, 13:44:47 UTC
Received	19 Sep 2010, 22:07:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1066791
Run time	3 days 14 hours 19 min 11 sec
CPU time	2 days 15 hours 8 min 46 sec
Validate state	Invalid
Credit	1,122.84
Device peak FLOPS	2.27 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.06 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.44</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (5261): called boinc_finish CPDN Monitor - Quit request from BOINC... (6173): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... (708): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (7735): called boinc_finish Signal 15 received, exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... (32352): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (27448): called boinc_finish Signal 15 received, exiting... (27428): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (4588): called boinc_finish Signal 15 received, exiting... Signal 15 received, exiting... (4258): called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (9847): called boinc_finish CPDN Monitor - Quit request from BOINC... (14470): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (14470): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (28851): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (28851): No heartbeat from core client for 30 sec - exiting (1592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (2842): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (3440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (4120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting (4120): No heartbeat from core client for 30 sec - exiting SIGSEGV: segmentation violation Stack trace (14 frames): /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x836249c] [0xf0fb9400] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x81351ee] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x813b38c] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x8130d28] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x813cb2b] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x8133041] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x8078164] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x82d7263] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x82f7b3b] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x82f1a7f] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x82f9572] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf0e0dbd6] /home/juerschen/BOINC/projects/climateprediction.net/hadam3p_saf_um_6.06_i686-pc-linux-gnu[0x804c631] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6003, selfPID=5987, iMonCtr=1 Model crash detected, will try to restart... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6003, selfPID=6003, iMonCtr=2 Leaving CPDN_Main::Monitor... (5987): called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_saf_v8hp_1973_1_006722651_0_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_v8hp_1973_1_006722651_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_v8hp_1973_1_006722651_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_v8hp_1973_1_006722651_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_v8hp_1973_1_006722651_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_v8hp_1973_1_006722651_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Sep 2010 12:40:01	1066791	11871697	hadam3p_saf_v8hp_1973_1_006722651_0	69,217	207,397	2.9963
14 Sep 2010 12:18:40	1066791	11871697	hadam3p_saf_v8hp_1973_1_006722651_0	69,216	206,894	2.9891
12 Sep 2010 01:09:20	1066791	11871697	hadam3p_saf_v8hp_1973_1_006722651_0	57,707	172,826	2.9949
11 Sep 2010 23:58:26	1066791	11871697	hadam3p_saf_v8hp_1973_1_006722651_0	57,700	172,291	2.9860
11 Sep 2010 23:17:45	1066791	11871697	hadam3p_saf_v8hp_1973_1_006722651_0	57,697	171,783	2.9773
11 Sep 2010 22:26:40	1066791	11871697	hadam3p_saf_v8hp_1973_1_006722651_0	57,696	171,283	2.9687
11 Sep 2010 11:15:42	1066791	11871697	hadam3p_saf_v8hp_1973_1_006722651_0	46,176	137,371	2.9749
11 Sep 2010 01:24:55	1066791	11871697	hadam3p_saf_v8hp_1973_1_006722651_0	34,656	103,200	2.9778
10 Sep 2010 15:36:37	1066791	11871697	hadam3p_saf_v8hp_1973_1_006722651_0	23,136	68,792	2.9734
09 Sep 2010 14:28:25	1066791	11871697	hadam3p_saf_v8hp_1973_1_006722651_0	11,616	34,660	2.9838