Task 13492631

Name	hadam3p_eu_6pao_2000_1_007500053_0
Workunit	7697528
Created	14 Oct 2011, 20:05:44 UTC
Sent	16 Oct 2011, 20:42:11 UTC
Report deadline	28 Sep 2012, 2:02:11 UTC
Received	26 Oct 2011, 4:48:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1170513
Run time	1 days 9 hours 7 min 11 sec
CPU time	6 hours 41 min 43 sec
Validate state	Invalid
Credit	399.11
Device peak FLOPS	2.87 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>6.13.6</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... 04:03:39 (5716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:04:56 (6160): No heartbeat from core client for 30 sec - exiting 04:04:57 (6160): No heartbeat from core client for 30 sec - exiting 04:04:58 (6160): No heartbeat from core client for 30 sec - exiting 04:04:59 (6160): No heartbeat from core client for 30 sec - exiting 04:05:00 (6160): No heartbeat from core client for 30 sec - exiting 04:05:01 (6160): No heartbeat from core client for 30 sec - exiting 04:05:02 (6160): No heartbeat from core client for 30 sec - exiting 04:05:03 (6160): No heartbeat from core client for 30 sec - exiting 04:05:04 (6160): No heartbeat from core client for 30 sec - exiting 04:05:05 (6160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:05:06 (6160): No heartbeat from core client for 30 sec - exiting 06:50:59 (5696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:51:01 (5696): No heartbeat from core client for 30 sec - exiting 06:51:02 (5696): No heartbeat from core client for 30 sec - exiting 06:51:03 (5696): No heartbeat from core client for 30 sec - exiting 06:51:04 (5696): No heartbeat from core client for 30 sec - exiting 06:51:05 (5696): No heartbeat from core client for 30 sec - exiting 06:51:06 (5696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:38:44 (11048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:38:46 (11048): No heartbeat from core client for 30 sec - exiting 13:38:47 (11048): No heartbeat from core client for 30 sec - exiting 13:38:48 (11048): No heartbeat from core client for 30 sec - exiting 13:38:49 (11048): No heartbeat from core client for 30 sec - exiting 13:38:50 (11048): No heartbeat from core client for 30 sec - exiting 13:43:38 (10836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:53:16 (10280): No heartbeat from core client for 30 sec - exiting 13:53:17 (10280): No heartbeat from core client for 30 sec - exiting 13:53:18 (10280): No heartbeat from core client for 30 sec - exiting 13:53:19 (10280): No heartbeat from core client for 30 sec - exiting 13:53:20 (10280): No heartbeat from core client for 30 sec - exiting 13:53:21 (10280): No heartbeat from core client for 30 sec - exiting 13:53:22 (10280): No heartbeat from core client for 30 sec - exiting 13:53:23 (10280): No heartbeat from core client for 30 sec - exiting 13:53:24 (10280): No heartbeat from core client for 30 sec - exiting 13:53:25 (10280): No heartbeat from core client for 30 sec - exiting 13:53:26 (10280): No heartbeat from core client for 30 sec - exiting 13:53:27 (10280): No heartbeat from core client for 30 sec - exiting 13:53:28 (10280): No heartbeat from core client for 30 sec - exiting 13:53:29 (10280): No heartbeat from core client for 30 sec - exiting 13:53:30 (10280): No heartbeat from core client for 30 sec - exiting 13:53:31 (10280): No heartbeat from core client for 30 sec - exiting 13:53:32 (10280): No heartbeat from core client for 30 sec - exiting 13:53:33 (10280): No heartbeat from core client for 30 sec - exiting 13:53:34 (10280): No heartbeat from core client for 30 sec - exiting 13:53:35 (10280): No heartbeat from core client for 30 sec - exiting 13:53:36 (10280): No heartbeat from core client for 30 sec - exiting 13:53:37 (10280): No heartbeat from core client for 30 sec - exiting 13:53:38 (10280): No heartbeat from core client for 30 sec - exiting 13:53:39 (10280): No heartbeat from core client for 30 sec - exiting 13:53:40 (10280): No heartbeat from core client for 30 sec - exiting 13:53:41 (10280): No heartbeat from core client for 30 sec - exiting 13:53:42 (10280): No heartbeat from core client for 30 sec - exiting 13:53:43 (10280): No heartbeat from core client for 30 sec - exiting 13:53:45 (10280): No heartbeat from core client for 30 sec - exiting 13:53:46 (10280): No heartbeat from core client for 30 sec - exiting 13:53:47 (10280): No heartbeat from core client for 30 sec - exiting 13:53:48 (10280): No heartbeat from core client for 30 sec - exiting 13:53:49 (10280): No heartbeat from core client for 30 sec - exiting 13:53:50 (10280): No heartbeat from core client for 30 sec - exiting 13:53:51 (10280): No heartbeat from core client for 30 sec - exiting 13:53:52 (10280): No heartbeat from core client for 30 sec - exiting 13:53:53 (10280): No heartbeat from core client for 30 sec - exiting 13:53:54 (10280): No heartbeat from core client for 30 sec - exiting 13:53:55 (10280): No heartbeat from core client for 30 sec - exiting 13:53:56 (10280): No heartbeat from core client for 30 sec - exiting 13:53:57 (10280): No heartbeat from core client for 30 sec - exiting 13:53:58 (10280): No heartbeat from core client for 30 sec - exiting 13:53:59 (10280): No heartbeat from core client for 30 sec - exiting 13:54:00 (10280): No heartbeat from core client for 30 sec - exiting 13:54:01 (10280): No heartbeat from core client for 30 sec - exiting 13:54:02 (10280): No heartbeat from core client for 30 sec - exiting 13:54:03 (10280): No heartbeat from core client for 30 sec - exiting 13:54:04 (10280): No heartbeat from core client for 30 sec - exiting 13:54:05 (10280): No heartbeat from core client for 30 sec - exiting 13:54:06 (10280): No heartbeat from core client for 30 sec - exiting 13:54:07 (10280): No heartbeat from core client for 30 sec - exiting 13:54:08 (10280): No heartbeat from core client for 30 sec - exiting 13:54:09 (10280): No heartbeat from core client for 30 sec - exiting 13:54:10 (10280): No heartbeat from core client for 30 sec - exiting 13:54:11 (10280): No heartbeat from core client for 30 sec - exiting 13:54:12 (10280): No heartbeat from core client for 30 sec - exiting 13:54:13 (10280): No heartbeat from core client for 30 sec - exiting 13:54:14 (10280): No heartbeat from core client for 30 sec - exiting 13:54:15 (10280): No heartbeat from core client for 30 sec - exiting 13:54:16 (10280): No heartbeat from core client for 30 sec - exiting 13:54:17 (10280): No heartbeat from core client for 30 sec - exiting 13:54:18 (10280): No heartbeat from core client for 30 sec - exiting 13:54:19 (10280): No heartbeat from core client for 30 sec - exiting 13:54:20 (10280): No heartbeat from core client for 30 sec - exiting 13:54:21 (10280): No heartbeat from core client for 30 sec - exiting 13:54:22 (10280): No heartbeat from core client for 30 sec - exiting 13:54:23 (10280): No heartbeat from core client for 30 sec - exiting 13:54:24 (10280): No heartbeat from core client for 30 sec - exiting 13:54:25 (10280): No heartbeat from core client for 30 sec - exiting 13:54:26 (10280): No heartbeat from core client for 30 sec - exiting 13:54:27 (10280): No heartbeat from core client for 30 sec - exiting 13:54:28 (10280): No heartbeat from core client for 30 sec - exiting 13:54:29 (10280): No heartbeat from core client for 30 sec - exiting 13:54:30 (10280): No heartbeat from core client for 30 sec - exiting 13:54:31 (10280): No heartbeat from core client for 30 sec - exiting 13:54:32 (10280): No heartbeat from core client for 30 sec - exiting 13:54:33 (10280): No heartbeat from core client for 30 sec - exiting 13:54:34 (10280): No heartbeat from core client for 30 sec - exiting 13:54:35 (10280): No heartbeat from core client for 30 sec - exiting 13:54:36 (10280): No heartbeat from core client for 30 sec - exiting 13:54:37 (10280): No heartbeat from core client for 30 sec - exiting 13:54:38 (10280): No heartbeat from core client for 30 sec - exiting 13:54:39 (10280): No heartbeat from core client for 30 sec - exiting 13:54:40 (10280): No heartbeat from core client for 30 sec - exiting 13:54:41 (10280): No heartbeat from core client for 30 sec - exiting 13:54:42 (10280): No heartbeat from core client for 30 sec - exiting 13:54:43 (10280): No heartbeat from core client for 30 sec - exiting 13:54:44 (10280): No heartbeat from core client for 30 sec - exiting 13:54:45 (10280): No heartbeat from core client for 30 sec - exiting 13:54:46 (10280): No heartbeat from core client for 30 sec - exiting 13:54:47 (10280): No heartbeat from core client for 30 sec - exiting 13:54:48 (10280): No heartbeat from core client for 30 sec - exiting 13:54:49 (10280): No heartbeat from core client for 30 sec - exiting 13:54:50 (10280): No heartbeat from core client for 30 sec - exiting 13:54:51 (10280): No heartbeat from core client for 30 sec - exiting 13:54:52 (10280): No heartbeat from core client for 30 sec - exiting 13:54:53 (10280): No heartbeat from core client for 30 sec - exiting 13:54:54 (10280): No heartbeat from core client for 30 sec - exiting 13:54:55 (10280): No heartbeat from core client for 30 sec - exiting 13:54:56 (10280): No heartbeat from core client for 30 sec - exiting 13:54:57 (10280): No heartbeat from core client for 30 sec - exiting 13:54:58 (10280): No heartbeat from core client for 30 sec - exiting 13:54:59 (10280): No heartbeat from core client for 30 sec - exiting 13:55:00 (10280): No heartbeat from core client for 30 sec - exiting 13:55:01 (10280): No heartbeat from core client for 30 sec - exiting 13:55:02 (10280): No heartbeat from core client for 30 sec - exiting 13:55:03 (10280): No heartbeat from core client for 30 sec - exiting 13:55:04 (10280): No heartbeat from core client for 30 sec - exiting 13:55:05 (10280): No heartbeat from core client for 30 sec - exiting 13:55:06 (10280): No heartbeat from core client for 30 sec - exiting 13:55:07 (10280): No heartbeat from core client for 30 sec - exiting 13:55:08 (10280): No heartbeat from core client for 30 sec - exiting 13:55:09 (10280): No heartbeat from core client for 30 sec - exiting 13:55:10 (10280): No heartbeat from core client for 30 sec - exiting 13:55:11 (10280): No heartbeat from core client for 30 sec - exiting 13:55:12 (10280): No heartbeat from core client for 30 sec - exiting 13:55:13 (10280): No heartbeat from core client for 30 sec - exiting 13:55:14 (10280): No heartbeat from core client for 30 sec - exiting 13:55:15 (10280): No heartbeat from core client for 30 sec - exiting 13:55:16 (10280): No heartbeat from core client for 30 sec - exiting 13:55:17 (10280): No heartbeat from core client for 30 sec - exiting 13:55:18 (10280): No heartbeat from core client for 30 sec - exiting 13:55:19 (10280): No heartbeat from core client for 30 sec - exiting 13:55:20 (10280): No heartbeat from core client for 30 sec - exiting 13:55:21 (10280): No heartbeat from core client for 30 sec - exiting 13:55:22 (10280): No heartbeat from core client for 30 sec - exiting 13:55:23 (10280): No heartbeat from core client for 30 sec - exiting 13:55:24 (10280): No heartbeat from core client for 30 sec - exiting 13:55:25 (10280): No heartbeat from core client for 30 sec - exiting 13:55:26 (10280): No heartbeat from core client for 30 sec - exiting 13:55:27 (10280): No heartbeat from core client for 30 sec - exiting 13:55:28 (10280): No heartbeat from core client for 30 sec - exiting 13:55:29 (10280): No heartbeat from core client for 30 sec - exiting 13:55:30 (10280): No heartbeat from core client for 30 sec - exiting 13:55:31 (10280): No heartbeat from core client for 30 sec - exiting 13:55:32 (10280): No heartbeat from core client for 30 sec - exiting 13:55:33 (10280): No heartbeat from core client for 30 sec - exiting 13:55:34 (10280): No heartbeat from core client for 30 sec - exiting 13:55:35 (10280): No heartbeat from core client for 30 sec - exiting 13:55:36 (10280): No heartbeat from core client for 30 sec - exiting 13:55:37 (10280): No heartbeat from core client for 30 sec - exiting 13:55:38 (10280): No heartbeat from core client for 30 sec - exiting 13:55:39 (10280): No heartbeat from core client for 30 sec - exiting 13:55:40 (10280): No heartbeat from core client for 30 sec - exiting 13:55:41 (10280): No heartbeat from core client for 30 sec - exiting 13:55:42 (10280): No heartbeat from core client for 30 sec - exiting 13:55:43 (10280): No heartbeat from core client for 30 sec - exiting 13:55:44 (10280): No heartbeat from core client for 30 sec - exiting 13:55:45 (10280): No heartbeat from core client for 30 sec - exiting 13:55:46 (10280): No heartbeat from core client for 30 sec - exiting 13:55:47 (10280): No heartbeat from core client for 30 sec - exiting 13:55:48 (10280): No heartbeat from core client for 30 sec - exiting 13:55:49 (10280): No heartbeat from core client for 30 sec - exiting 13:55:50 (10280): No heartbeat from core client for 30 sec - exiting 13:55:51 (10280): No heartbeat from core client for 30 sec - exiting 13:55:52 (10280): No heartbeat from core client for 30 sec - exiting 13:55:53 (10280): No heartbeat from core client for 30 sec - exiting 13:55:54 (10280): No heartbeat from core client for 30 sec - exiting 13:55:55 (10280): No heartbeat from core client for 30 sec - exiting 13:55:56 (10280): No heartbeat from core client for 30 sec - exiting 13:55:57 (10280): No heartbeat from core client for 30 sec - exiting 13:55:58 (10280): No heartbeat from core client for 30 sec - exiting 13:55:59 (10280): No heartbeat from core client for 30 sec - exiting 13:56:00 (10280): No heartbeat from core client for 30 sec - exiting 13:56:01 (10280): No heartbeat from core client for 30 sec - exiting 13:56:02 (10280): No heartbeat from core client for 30 sec - exiting 13:56:03 (10280): No heartbeat from core client for 30 sec - exiting 13:56:04 (10280): No heartbeat from core client for 30 sec - exiting 13:56:05 (10280): No heartbeat from core client for 30 sec - exiting 13:56:06 (10280): No heartbeat from core client for 30 sec - exiting 13:56:07 (10280): No heartbeat from core client for 30 sec - exiting 13:56:08 (10280): No heartbeat from core client for 30 sec - exiting 13:56:09 (10280): No heartbeat from core client for 30 sec - exiting 13:56:10 (10280): No heartbeat from core client for 30 sec - exiting 13:56:11 (10280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:00:11 (3452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:00:13 (3452): No heartbeat from core client for 30 sec - exiting 17:00:14 (3452): No heartbeat from core client for 30 sec - exiting 17:00:15 (3452): No heartbeat from core client for 30 sec - exiting 17:00:16 (3452): No heartbeat from core client for 30 sec - exiting 17:00:17 (3452): No heartbeat from core client for 30 sec - exiting 17:00:18 (3452): No heartbeat from core client for 30 sec - exiting 17:00:19 (3452): No heartbeat from core client for 30 sec - exiting 17:00:20 (3452): No heartbeat from core client for 30 sec - exiting 17:01:38 (11052): No heartbeat from core client for 30 sec - exiting 17:01:39 (11052): No heartbeat from core client for 30 sec - exiting 17:01:40 (11052): No heartbeat from core client for 30 sec - exiting 17:01:41 (11052): No heartbeat from core client for 30 sec - exiting 17:01:42 (11052): No heartbeat from core client for 30 sec - exiting 17:01:43 (11052): No heartbeat from core client for 30 sec - exiting 17:01:44 (11052): No heartbeat from core client for 30 sec - exiting 17:01:45 (11052): No heartbeat from core client for 30 sec - exiting 17:01:46 (11052): No heartbeat from core client for 30 sec - exiting 17:01:47 (11052): No heartbeat from core client for 30 sec - exiting 17:01:48 (11052): No heartbeat from core client for 30 sec - exiting 17:01:49 (11052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5888, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9720, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 19:01:51 (7284): No heartbeat from core client for 30 sec - exiting 19:01:52 (7284): No heartbeat from core client for 30 sec - exiting 19:01:53 (7284): No heartbeat from core client for 30 sec - exiting 19:01:54 (7284): No heartbeat from core client for 30 sec - exiting 19:01:55 (7284): No heartbeat from core client for 30 sec - exiting 19:01:56 (7284): No heartbeat from core client for 30 sec - exiting 19:01:57 (7284): No heartbeat from core client for 30 sec - exiting 19:01:58 (7284): No heartbeat from core client for 30 sec - exiting 19:01:59 (7284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12168, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish 19:41:43 (3796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:24:32 (4432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:35:08 (2208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5976, selfPID=5808, iMonCtr=1 Model crash detected, will try to restart... 22:50:39 (4344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4744, selfPID=4744, iMonCtr=2 07:27:40 (4764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:27:42 (4764): No heartbeat from core client for 30 sec - exiting 07:27:43 (4764): No heartbeat from core client for 30 sec - exiting 07:27:44 (4764): No heartbeat from core client for 30 sec - exiting 07:27:45 (4764): No heartbeat from core client for 30 sec - exiting 07:27:46 (4764): No heartbeat from core client for 30 sec - exiting 07:27:47 (4764): No heartbeat from core client for 30 sec - exiting 07:27:48 (4764): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=8700, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3064, selfPID=8212, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_eu_6pao_2000_1_007500053_0_3.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6pao_2000_1_007500053_0_4.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6pao_2000_1_007500053_0_5.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6pao_2000_1_007500053_0_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6pao_2000_1_007500053_0_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6pao_2000_1_007500053_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6pao_2000_1_007500053_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6pao_2000_1_007500053_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6pao_2000_1_007500053_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_eu_6pao_2000_1_007500053_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Oct 2011 15:37:27	1170513	13492631	hadam3p_eu_6pao_2000_1_007500053_0	23,136	79,388	3.4314
31 Oct 2011 15:37:27	1170513	13492631	hadam3p_eu_6pao_2000_1_007500053_0	11,616	39,844	3.4301