Task 12624968

Name	hadam3p_pnw_344x_1989_1_007185513_0
Workunit	7383795
Created	22 Feb 2011, 13:04:43 UTC
Sent	25 Feb 2011, 19:15:17 UTC
Report deadline	8 Feb 2012, 0:35:17 UTC
Received	8 Mar 2011, 9:57:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1310772
Run time	1 days 5 hours 32 min 39 sec
CPU time	1 days 2 hours 51 min 30 sec
Validate state	Invalid
Credit	502.72
Device peak FLOPS	1.86 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.08 windows_intelx86
Stderr	<core_client_version>6.12.15</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 22:25:57 (1448): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:12:18 (4944): No heartbeat from core client for 30 sec - exiting 18:12:19 (4944): No heartbeat from core client for 30 sec - exiting 18:12:20 (4944): No heartbeat from core client for 30 sec - exiting 18:12:21 (4944): No heartbeat from core client for 30 sec - exiting 18:12:22 (4944): No heartbeat from core client for 30 sec - exiting 18:12:23 (4944): No heartbeat from core client for 30 sec - exiting 18:12:24 (4944): No heartbeat from core client for 30 sec - exiting 18:12:26 (4944): No heartbeat from core client for 30 sec - exiting 18:12:27 (4944): No heartbeat from core client for 30 sec - exiting 18:12:28 (4944): No heartbeat from core client for 30 sec - exiting 18:12:29 (4944): No heartbeat from core client for 30 sec - exiting 18:12:30 (4944): No heartbeat from core client for 30 sec - exiting 18:12:31 (4944): No heartbeat from core client for 30 sec - exiting 18:12:32 (4944): No heartbeat from core client for 30 sec - exiting 18:12:33 (4944): No heartbeat from core client for 30 sec - exiting 18:12:34 (4944): No heartbeat from core client for 30 sec - exiting 18:12:35 (4944): No heartbeat from core client for 30 sec - exiting 18:12:37 (4944): No heartbeat from core client for 30 sec - exiting 18:12:38 (4944): No heartbeat from core client for 30 sec - exiting 18:12:39 (4944): No heartbeat from core client for 30 sec - exiting 18:12:40 (4944): No heartbeat from core client for 30 sec - exiting 18:12:41 (4944): No heartbeat from core client for 30 sec - exiting 18:12:42 (4944): No heartbeat from core client for 30 sec - exiting 18:12:43 (4944): No heartbeat from core client for 30 sec - exiting 18:12:44 (4944): No heartbeat from core client for 30 sec - exiting 18:12:45 (4944): No heartbeat from core client for 30 sec - exiting 18:12:46 (4944): No heartbeat from core client for 30 sec - exiting 18:12:47 (4944): No heartbeat from core client for 30 sec - exiting 18:12:49 (4944): No heartbeat from core client for 30 sec - exiting 18:12:50 (4944): No heartbeat from core client for 30 sec - exiting 18:12:51 (4944): No heartbeat from core client for 30 sec - exiting 18:12:52 (4944): No heartbeat from core client for 30 sec - exiting 18:12:53 (4944): No heartbeat from core client for 30 sec - exiting 18:12:54 (4944): No heartbeat from core client for 30 sec - exiting 18:12:55 (4944): No heartbeat from core client for 30 sec - exiting 18:12:56 (4944): No heartbeat from core client for 30 sec - exiting 18:12:57 (4944): No heartbeat from core client for 30 sec - exiting 18:12:58 (4944): No heartbeat from core client for 30 sec - exiting 18:12:59 (4944): No heartbeat from core client for 30 sec - exiting 18:13:00 (4944): No heartbeat from core client for 30 sec - exiting 18:13:02 (4944): No heartbeat from core client for 30 sec - exiting 18:13:03 (4944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:50:29 (5536): No heartbeat from core client for 30 sec - exiting 09:50:31 (5536): No heartbeat from core client for 30 sec - exiting 09:50:32 (5536): No heartbeat from core client for 30 sec - exiting 09:50:33 (5536): No heartbeat from core client for 30 sec - exiting 09:50:34 (5536): No heartbeat from core client for 30 sec - exiting 09:50:35 (5536): No heartbeat from core client for 30 sec - exiting 09:50:36 (5536): No heartbeat from core client for 30 sec - exiting 09:50:37 (5536): No heartbeat from core client for 30 sec - exiting 09:50:38 (5536): No heartbeat from core client for 30 sec - exiting 09:50:39 (5536): No heartbeat from core client for 30 sec - exiting 09:50:40 (5536): No heartbeat from core client for 30 sec - exiting 09:50:42 (5536): No heartbeat from core client for 30 sec - exiting 09:50:43 (5536): No heartbeat from core client for 30 sec - exiting 09:50:44 (5536): No heartbeat from core client for 30 sec - exiting 09:50:45 (5536): No heartbeat from core client for 30 sec - exiting 09:50:46 (5536): No heartbeat from core client for 30 sec - exiting 09:50:47 (5536): No heartbeat from core client for 30 sec - exiting 09:50:48 (5536): No heartbeat from core client for 30 sec - exiting 09:50:49 (5536): No heartbeat from core client for 30 sec - exiting 09:50:50 (5536): No heartbeat from core client for 30 sec - exiting 09:50:51 (5536): No heartbeat from core client for 30 sec - exiting 09:50:52 (5536): No heartbeat from core client for 30 sec - exiting 09:50:54 (5536): No heartbeat from core client for 30 sec - exiting 09:50:55 (5536): No heartbeat from core client for 30 sec - exiting 09:50:56 (5536): No heartbeat from core client for 30 sec - exiting 09:50:57 (5536): No heartbeat from core client for 30 sec - exiting 09:50:58 (5536): No heartbeat from core client for 30 sec - exiting 09:50:59 (5536): No heartbeat from core client for 30 sec - exiting 09:51:00 (5536): No heartbeat from core client for 30 sec - exiting 09:51:01 (5536): No heartbeat from core client for 30 sec - exiting 09:51:02 (5536): No heartbeat from core client for 30 sec - exiting 09:51:03 (5536): No heartbeat from core client for 30 sec - exiting 09:51:04 (5536): No heartbeat from core client for 30 sec - exiting 09:51:06 (5536): No heartbeat from core client for 30 sec - exiting 09:51:07 (5536): No heartbeat from core client for 30 sec - exiting 09:51:08 (5536): No heartbeat from core client for 30 sec - exiting 09:51:09 (5536): No heartbeat from core client for 30 sec - exiting 09:51:10 (5536): No heartbeat from core client for 30 sec - exiting 09:51:11 (5536): No heartbeat from core client for 30 sec - exiting 09:51:12 (5536): No heartbeat from core client for 30 sec - exiting 09:51:13 (5536): No heartbeat from core client for 30 sec - exiting 09:51:14 (5536): No heartbeat from core client for 30 sec - exiting 09:51:15 (5536): No heartbeat from core client for 30 sec - exiting 09:51:16 (5536): No heartbeat from core client for 30 sec - exiting 09:51:18 (5536): No heartbeat from core client for 30 sec - exiting 09:51:19 (5536): No heartbeat from core client for 30 sec - exiting 09:51:20 (5536): No heartbeat from core client for 30 sec - exiting 09:51:21 (5536): No heartbeat from core client for 30 sec - exiting 09:51:22 (5536): No heartbeat from core client for 30 sec - exiting 09:51:23 (5536): No heartbeat from core client for 30 sec - exiting 09:51:24 (5536): No heartbeat from core client for 30 sec - exiting 09:51:25 (5536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:59:52 (5216): No heartbeat from core client for 30 sec - exiting 09:59:53 (5216): No heartbeat from core client for 30 sec - exiting 09:59:54 (5216): No heartbeat from core client for 30 sec - exiting 09:59:55 (5216): No heartbeat from core client for 30 sec - exiting 09:59:56 (5216): No heartbeat from core client for 30 sec - exiting 09:59:58 (5216): No heartbeat from core client for 30 sec - exiting 09:59:59 (5216): No heartbeat from core client for 30 sec - exiting 10:00:00 (5216): No heartbeat from core client for 30 sec - exiting 10:00:01 (5216): No heartbeat from core client for 30 sec - exiting 10:00:02 (5216): No heartbeat from core client for 30 sec - exiting 10:00:03 (5216): No heartbeat from core client for 30 sec - exiting 10:00:04 (5216): No heartbeat from core client for 30 sec - exiting 10:00:05 (5216): No heartbeat from core client for 30 sec - exiting 10:00:06 (5216): No heartbeat from core client for 30 sec - exiting 10:00:07 (5216): No heartbeat from core client for 30 sec - exiting 10:00:08 (5216): No heartbeat from core client for 30 sec - exiting 10:00:10 (5216): No heartbeat from core client for 30 sec - exiting 10:00:11 (5216): No heartbeat from core client for 30 sec - exiting 10:00:12 (5216): No heartbeat from core client for 30 sec - exiting 10:00:13 (5216): No heartbeat from core client for 30 sec - exiting 10:00:14 (5216): No heartbeat from core client for 30 sec - exiting 10:00:15 (5216): No heartbeat from core client for 30 sec - exiting 10:00:16 (5216): No heartbeat from core client for 30 sec - exiting 10:00:17 (5216): No heartbeat from core client for 30 sec - exiting 10:00:18 (5216): No heartbeat from core client for 30 sec - exiting 10:00:19 (5216): No heartbeat from core client for 30 sec - exiting 10:00:20 (5216): No heartbeat from core client for 30 sec - exiting 10:00:22 (5216): No heartbeat from core client for 30 sec - exiting 10:00:23 (5216): No heartbeat from core client for 30 sec - exiting 10:00:24 (5216): No heartbeat from core client for 30 sec - exiting 10:00:25 (5216): No heartbeat from core client for 30 sec - exiting 10:00:26 (5216): No heartbeat from core client for 30 sec - exiting 10:00:27 (5216): No heartbeat from core client for 30 sec - exiting 10:00:28 (5216): No heartbeat from core client for 30 sec - exiting 10:00:29 (5216): No heartbeat from core client for 30 sec - exiting 10:00:30 (5216): No heartbeat from core client for 30 sec - exiting 10:00:31 (5216): No heartbeat from core client for 30 sec - exiting 10:00:33 (5216): No heartbeat from core client for 30 sec - exiting 10:00:34 (5216): No heartbeat from core client for 30 sec - exiting 10:00:35 (5216): No heartbeat from core client for 30 sec - exiting 10:00:36 (5216): No heartbeat from core client for 30 sec - exiting 10:00:37 (5216): No heartbeat from core client for 30 sec - exiting 10:00:38 (5216): No heartbeat from core client for 30 sec - exiting 10:00:39 (5216): No heartbeat from core client for 30 sec - exiting 10:00:40 (5216): No heartbeat from core client for 30 sec - exiting 10:00:41 (5216): No heartbeat from core client for 30 sec - exiting 10:00:43 (5216): No heartbeat from core client for 30 sec - exiting 10:00:44 (5216): No heartbeat from core client for 30 sec - exiting 10:00:45 (5216): No heartbeat from core client for 30 sec - exiting 10:00:46 (5216): No heartbeat from core client for 30 sec - exiting 10:00:47 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:57:47 (5208): No heartbeat from core client for 30 sec - exiting 00:57:49 (5208): No heartbeat from core client for 30 sec - exiting 00:57:50 (5208): No heartbeat from core client for 30 sec - exiting 00:57:51 (5208): No heartbeat from core client for 30 sec - exiting 00:57:52 (5208): No heartbeat from core client for 30 sec - exiting 00:57:53 (5208): No heartbeat from core client for 30 sec - exiting 00:57:54 (5208): No heartbeat from core client for 30 sec - exiting 00:57:55 (5208): No heartbeat from core client for 30 sec - exiting 00:57:56 (5208): No heartbeat from core client for 30 sec - exiting 00:57:57 (5208): No heartbeat from core client for 30 sec - exiting 00:57:58 (5208): No heartbeat from core client for 30 sec - exiting 00:57:59 (5208): No heartbeat from core client for 30 sec - exiting 00:58:01 (5208): No heartbeat from core client for 30 sec - exiting 00:58:02 (5208): No heartbeat from core client for 30 sec - exiting 00:58:03 (5208): No heartbeat from core client for 30 sec - exiting 00:58:04 (5208): No heartbeat from core client for 30 sec - exiting 00:58:05 (5208): No heartbeat from core client for 30 sec - exiting 00:58:06 (5208): No heartbeat from core client for 30 sec - exiting 00:58:07 (5208): No heartbeat from core client for 30 sec - exiting 00:58:08 (5208): No heartbeat from core client for 30 sec - exiting 00:58:09 (5208): No heartbeat from core client for 30 sec - exiting 00:58:10 (5208): No heartbeat from core client for 30 sec - exiting 00:58:11 (5208): No heartbeat from core client for 30 sec - exiting 00:58:13 (5208): No heartbeat from core client for 30 sec - exiting 00:58:14 (5208): No heartbeat from core client for 30 sec - exiting 00:58:15 (5208): No heartbeat from core client for 30 sec - exiting 00:58:16 (5208): No heartbeat from core client for 30 sec - exiting 00:58:17 (5208): No heartbeat from core client for 30 sec - exiting 00:58:18 (5208): No heartbeat from core client for 30 sec - exiting 00:58:19 (5208): No heartbeat from core client for 30 sec - exiting 00:58:20 (5208): No heartbeat from core client for 30 sec - exiting 00:58:21 (5208): No heartbeat from core client for 30 sec - exiting 00:58:22 (5208): No heartbeat from core client for 30 sec - exiting 00:58:24 (5208): No heartbeat from core client for 30 sec - exiting 00:58:25 (5208): No heartbeat from core client for 30 sec - exiting 00:58:26 (5208): No heartbeat from core client for 30 sec - exiting 00:58:27 (5208): No heartbeat from core client for 30 sec - exiting 00:58:28 (5208): No heartbeat from core client for 30 sec - exiting 00:58:29 (5208): No heartbeat from core client for 30 sec - exiting 00:58:30 (5208): No heartbeat from core client for 30 sec - exiting 00:58:31 (5208): No heartbeat from core client for 30 sec - exiting 00:58:32 (5208): No heartbeat from core client for 30 sec - exiting 00:58:33 (5208): No heartbeat from core client for 30 sec - exiting 00:58:34 (5208): No heartbeat from core client for 30 sec - exiting 00:58:36 (5208): No heartbeat from core client for 30 sec - exiting 00:58:37 (5208): No heartbeat from core client for 30 sec - exiting 00:58:38 (5208): No heartbeat from core client for 30 sec - exiting 00:58:39 (5208): No heartbeat from core client for 30 sec - exiting 00:58:40 (5208): No heartbeat from core client for 30 sec - exiting 00:58:41 (5208): No heartbeat from core client for 30 sec - exiting 00:58:42 (5208): No heartbeat from core client for 30 sec - exiting 00:58:43 (5208): No heartbeat from core client for 30 sec - exiting 00:58:44 (5208): No heartbeat from core client for 30 sec - exiting 00:58:45 (5208): No heartbeat from core client for 30 sec - exiting 00:58:46 (5208): No heartbeat from core client for 30 sec - exiting 00:58:48 (5208): No heartbeat from core client for 30 sec - exiting 00:58:49 (5208): No heartbeat from core client for 30 sec - exiting 00:58:50 (5208): No heartbeat from core client for 30 sec - exiting 00:58:51 (5208): No heartbeat from core client for 30 sec - exiting 00:58:52 (5208): No heartbeat from core client for 30 sec - exiting 00:58:53 (5208): No heartbeat from core client for 30 sec - exiting 00:58:54 (5208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:17:14 (5136): No heartbeat from core client for 30 sec - exiting 01:17:15 (5136): No heartbeat from core client for 30 sec - exiting 01:17:16 (5136): No heartbeat from core client for 30 sec - exiting 01:17:17 (5136): No heartbeat from core client for 30 sec - exiting 01:17:18 (5136): No heartbeat from core client for 30 sec - exiting 01:17:19 (5136): No heartbeat from core client for 30 sec - exiting 01:17:20 (5136): No heartbeat from core client for 30 sec - exiting 01:17:21 (5136): No heartbeat from core client for 30 sec - exiting 01:17:22 (5136): No heartbeat from core client for 30 sec - exiting 01:17:24 (5136): No heartbeat from core client for 30 sec - exiting 01:17:25 (5136): No heartbeat from core client for 30 sec - exiting 01:17:26 (5136): No heartbeat from core client for 30 sec - exiting 01:17:27 (5136): No heartbeat from core client for 30 sec - exiting 01:17:28 (5136): No heartbeat from core client for 30 sec - exiting 01:17:29 (5136): No heartbeat from core client for 30 sec - exiting 01:17:30 (5136): No heartbeat from core client for 30 sec - exiting 01:17:31 (5136): No heartbeat from core client for 30 sec - exiting 01:17:32 (5136): No heartbeat from core client for 30 sec - exiting 01:17:33 (5136): No heartbeat from core client for 30 sec - exiting 01:17:34 (5136): No heartbeat from core client for 30 sec - exiting 01:17:36 (5136): No heartbeat from core client for 30 sec - exiting 01:17:37 (5136): No heartbeat from core client for 30 sec - exiting 01:17:38 (5136): No heartbeat from core client for 30 sec - exiting 01:17:39 (5136): No heartbeat from core client for 30 sec - exiting 01:17:40 (5136): No heartbeat from core client for 30 sec - exiting 01:17:41 (5136): No heartbeat from core client for 30 sec - exiting 01:17:42 (5136): No heartbeat from core client for 30 sec - exiting 01:17:43 (5136): No heartbeat from core client for 30 sec - exiting 01:17:44 (5136): No heartbeat from core client for 30 sec - exiting 01:17:45 (5136): No heartbeat from core client for 30 sec - exiting 01:17:47 (5136): No heartbeat from core client for 30 sec - exiting 01:17:48 (5136): No heartbeat from core client for 30 sec - exiting 01:17:49 (5136): No heartbeat from core client for 30 sec - exiting 01:17:50 (5136): No heartbeat from core client for 30 sec - exiting 01:17:51 (5136): No heartbeat from core client for 30 sec - exiting 01:17:52 (5136): No heartbeat from core client for 30 sec - exiting 01:17:53 (5136): No heartbeat from core client for 30 sec - exiting 01:17:54 (5136): No heartbeat from core client for 30 sec - exiting 01:17:55 (5136): No heartbeat from core client for 30 sec - exiting 01:17:56 (5136): No heartbeat from core client for 30 sec - exiting 01:17:57 (5136): No heartbeat from core client for 30 sec - exiting 01:17:59 (5136): No heartbeat from core client for 30 sec - exiting 01:18:00 (5136): No heartbeat from core client for 30 sec - exiting 01:18:01 (5136): No heartbeat from core client for 30 sec - exiting 01:18:02 (5136): No heartbeat from core client for 30 sec - exiting 01:18:03 (5136): No heartbeat from core client for 30 sec - exiting 01:18:35 (5136): No heartbeat from core client for 30 sec - exiting 01:18:36 (5136): No heartbeat from core client for 30 sec - exiting 01:18:37 (5136): No heartbeat from core client for 30 sec - exiting 01:18:38 (5136): No heartbeat from core client for 30 sec - exiting Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1340, selfPID=1340, iMonCtr=2 Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=200, selfPID=4824, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 2 01:19:08 (4824): called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_pnw_344x_1989_1_007185513_0_3.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_344x_1989_1_007185513_0_4.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_344x_1989_1_007185513_0_5.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_344x_1989_1_007185513_0_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_344x_1989_1_007185513_0_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_344x_1989_1_007185513_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_344x_1989_1_007185513_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_344x_1989_1_007185513_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_344x_1989_1_007185513_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_344x_1989_1_007185513_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Feb 2011 00:45:18	1079393	12624968	hadam3p_pnw_344x_1989_1_007185513_0	23,136	77,276	3.3401
26 Feb 2011 19:06:40	1079393	12624968	hadam3p_pnw_344x_1989_1_007185513_0	11,616	33,343	2.8704