Task 18824482

Name	hadam3p_anz_n9jk_2008_1_010108686_1
Workunit	10088276
Created	14 Aug 2015, 14:10:53 UTC
Sent	14 Aug 2015, 14:20:42 UTC
Report deadline	26 Jul 2016, 19:40:42 UTC
Received	3 Sep 2015, 18:00:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1353254
Run time	2 days 12 hours 44 min 47 sec
CPU time	2 days 9 hours 24 min 13 sec
Validate state	Invalid
Credit	4,981.10
Device peak FLOPS	5.23 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4308, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:51:22 (4540): No heartbeat from core client for 30 sec - exiting 13:51:23 (4540): No heartbeat from core client for 30 sec - exiting 13:51:24 (4540): No heartbeat from core client for 30 sec - exiting 13:51:25 (4540): No heartbeat from core client for 30 sec - exiting 13:51:26 (4540): No heartbeat from core client for 30 sec - exiting 13:51:27 (4540): No heartbeat from core client for 30 sec - exiting 13:51:28 (4540): No heartbeat from core client for 30 sec - exiting 13:51:29 (4540): No heartbeat from core client for 30 sec - exiting 13:51:30 (4540): No heartbeat from core client for 30 sec - exiting 13:51:31 (4540): No heartbeat from core client for 30 sec - exiting 13:51:32 (4540): No heartbeat from core client for 30 sec - exiting 13:51:33 (4540): No heartbeat from core client for 30 sec - exiting 13:51:34 (4540): No heartbeat from core client for 30 sec - exiting 13:51:35 (4540): No heartbeat from core client for 30 sec - exiting 13:51:36 (4540): No heartbeat from core client for 30 sec - exiting 13:51:37 (4540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:36:10 (4336): No heartbeat from core client for 30 sec - exiting 22:36:11 (4336): No heartbeat from core client for 30 sec - exiting 22:36:12 (4336): No heartbeat from core client for 30 sec - exiting 22:36:13 (4336): No heartbeat from core client for 30 sec - exiting 22:36:14 (4336): No heartbeat from core client for 30 sec - exiting 22:36:15 (4336): No heartbeat from core client for 30 sec - exiting 22:36:16 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:41:19 (5980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:11:56 (8288): No heartbeat from core client for 30 sec - exiting 13:11:57 (8288): No heartbeat from core client for 30 sec - exiting 13:11:58 (8288): No heartbeat from core client for 30 sec - exiting 13:11:59 (8288): No heartbeat from core client for 30 sec - exiting 13:12:00 (8288): No heartbeat from core client for 30 sec - exiting 13:12:01 (8288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:43:07 (9132): No heartbeat from core client for 30 sec - exiting 19:43:08 (9132): No heartbeat from core client for 30 sec - exiting 19:43:09 (9132): No heartbeat from core client for 30 sec - exiting 19:43:10 (9132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:11 (3476): No heartbeat from core client for 30 sec - exiting 11:33:12 (3476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:54:37 (9004): No heartbeat from core client for 30 sec - exiting 16:54:38 (9004): No heartbeat from core client for 30 sec - exiting 16:54:39 (9004): No heartbeat from core client for 30 sec - exiting 16:54:40 (9004): No heartbeat from core client for 30 sec - exiting 16:54:41 (9004): No heartbeat from core client for 30 sec - exiting 16:54:42 (9004): No heartbeat from core client for 30 sec - exiting 16:54:43 (9004): No heartbeat from core client for 30 sec - exiting 16:54:44 (9004): No heartbeat from core client for 30 sec - exiting 16:54:45 (9004): No heartbeat from core client for 30 sec - exiting 16:54:46 (9004): No heartbeat from core client for 30 sec - exiting 16:54:47 (9004): No heartbeat from core client for 30 sec - exiting 16:54:48 (9004): No heartbeat from core client for 30 sec - exiting 16:54:49 (9004): No heartbeat from core client for 30 sec - exiting 16:54:50 (9004): No heartbeat from core client for 30 sec - exiting 16:54:51 (9004): No heartbeat from core client for 30 sec - exiting 16:54:52 (9004): No heartbeat from core client for 30 sec - exiting 16:54:53 (9004): No heartbeat from core client for 30 sec - exiting 16:54:54 (9004): No heartbeat from core client for 30 sec - exiting 16:54:55 (9004): No heartbeat from core client for 30 sec - exiting 16:54:56 (9004): No heartbeat from core client for 30 sec - exiting 16:54:57 (9004): No heartbeat from core client for 30 sec - exiting 16:54:58 (9004): No heartbeat from core client for 30 sec - exiting 16:54:59 (9004): No heartbeat from core client for 30 sec - exiting 16:55:00 (9004): No heartbeat from core client for 30 sec - exiting 16:55:01 (9004): No heartbeat from core client for 30 sec - exiting 16:55:02 (9004): No heartbeat from core client for 30 sec - exiting 16:55:03 (9004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:31:46 (1236): No heartbeat from core client for 30 sec - exiting 21:31:47 (1236): No heartbeat from core client for 30 sec - exiting 21:31:48 (1236): No heartbeat from core client for 30 sec - exiting 21:31:49 (1236): No heartbeat from core client for 30 sec - exiting 21:31:50 (1236): No heartbeat from core client for 30 sec - exiting 21:31:51 (1236): No heartbeat from core client for 30 sec - exiting 21:31:52 (1236): No heartbeat from core client for 30 sec - exiting 21:31:53 (1236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8776, selfPID=8776, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:49:34 (2744): No heartbeat from core client for 30 sec - exiting 01:49:35 (2744): No heartbeat from core client for 30 sec - exiting 01:49:36 (2744): No heartbeat from core client for 30 sec - exiting 01:49:37 (2744): No heartbeat from core client for 30 sec - exiting 01:49:38 (2744): No heartbeat from core client for 30 sec - exiting 01:49:39 (2744): No heartbeat from core client for 30 sec - exiting 01:49:40 (2744): No heartbeat from core client for 30 sec - exiting 01:49:41 (2744): No heartbeat from core client for 30 sec - exiting 01:49:42 (2744): No heartbeat from core client for 30 sec - exiting 01:49:43 (2744): No heartbeat from core client for 30 sec - exiting 01:49:44 (2744): No heartbeat from core client for 30 sec - exiting 01:49:45 (2744): No heartbeat from core client for 30 sec - exiting 01:49:46 (2744): No heartbeat from core client for 30 sec - exiting 01:49:47 (2744): No heartbeat from core client for 30 sec - exiting 01:49:48 (2744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9076, selfPID=9076, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... 01:57:01 (1028): No heartbeat from core client for 30 sec - exiting 01:57:02 (1028): No heartbeat from core client for 30 sec - exiting 01:57:03 (1028): No heartbeat from core client for 30 sec - exiting 01:57:04 (1028): No heartbeat from core client for 30 sec - exiting 01:57:05 (1028): No heartbeat from core client for 30 sec - exiting 01:57:06 (1028): No heartbeat from core client for 30 sec - exiting 01:57:07 (1028): No heartbeat from core client for 30 sec - exiting 01:57:08 (1028): No heartbeat from core client for 30 sec - exiting 01:57:09 (1028): No heartbeat from core client for 30 sec - exiting 01:57:10 (1028): No heartbeat from core client for 30 sec - exiting 01:57:11 (1028): No heartbeat from core client for 30 sec - exiting 01:57:12 (1028): No heartbeat from core client for 30 sec - exiting 01:57:13 (1028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:04:47 (260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:28:45 (6668): No heartbeat from core client for 30 sec - exiting 19:28:46 (6668): No heartbeat from core client for 30 sec - exiting 19:28:47 (6668): No heartbeat from core client for 30 sec - exiting 19:28:48 (6668): No heartbeat from core client for 30 sec - exiting 19:28:49 (6668): No heartbeat from core client for 30 sec - exiting 19:28:50 (6668): No heartbeat from core client for 30 sec - exiting 19:28:51 (6668): No heartbeat from core client for 30 sec - exiting 19:28:52 (6668): No heartbeat from core client for 30 sec - exiting 19:28:53 (6668): No heartbeat from core client for 30 sec - exiting 19:28:54 (6668): No heartbeat from core client for 30 sec - exiting 19:28:55 (6668): No heartbeat from core client for 30 sec - exiting 19:28:56 (6668): No heartbeat from core client for 30 sec - exiting 19:28:57 (6668): No heartbeat from core client for 30 sec - exiting 19:28:58 (6668): No heartbeat from core client for 30 sec - exiting 19:28:59 (6668): No heartbeat from core client for 30 sec - exiting 19:29:00 (6668): No heartbeat from core client for 30 sec - exiting 19:29:01 (6668): No heartbeat from core client for 30 sec - exiting 19:29:02 (6668): No heartbeat from core client for 30 sec - exiting 19:29:03 (6668): No heartbeat from core client for 30 sec - exiting 19:29:04 (6668): No heartbeat from core client for 30 sec - exiting 19:29:05 (6668): No heartbeat from core client for 30 sec - exiting 19:29:06 (6668): No heartbeat from core client for 30 sec - exiting 19:29:07 (6668): No heartbeat from core client for 30 sec - exiting 19:29:08 (6668): No heartbeat from core client for 30 sec - exiting 19:29:09 (6668): No heartbeat from core client for 30 sec - exiting 19:29:10 (6668): No heartbeat from core client for 30 sec - exiting 19:29:11 (6668): No heartbeat from core client for 30 sec - exiting 19:29:12 (6668): No heartbeat from core client for 30 sec - exiting 19:29:13 (6668): No heartbeat from core client for 30 sec - exiting 19:29:14 (6668): No heartbeat from core client for 30 sec - exiting 19:29:15 (6668): No heartbeat from core client for 30 sec - exiting 19:29:16 (6668): No heartbeat from core client for 30 sec - exiting 19:29:17 (6668): No heartbeat from core client for 30 sec - exiting 19:29:18 (6668): No heartbeat from core client for 30 sec - exiting 19:29:19 (6668): No heartbeat from core client for 30 sec - exiting 19:29:20 (6668): No heartbeat from core client for 30 sec - exiting 19:29:21 (6668): No heartbeat from core client for 30 sec - exiting 19:29:22 (6668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3888, selfPID=3888, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 13:10:08 (7872): No heartbeat from core client for 30 sec - exiting 13:10:09 (7872): No heartbeat from core client for 30 sec - exiting 13:10:10 (7872): No heartbeat from core client for 30 sec - exiting 13:10:11 (7872): No heartbeat from core client for 30 sec - exiting 13:10:12 (7872): No heartbeat from core client for 30 sec - exiting 13:10:13 (7872): No heartbeat from core client for 30 sec - exiting 13:10:14 (7872): No heartbeat from core client for 30 sec - exiting 13:10:15 (7872): No heartbeat from core client for 30 sec - exiting 13:10:16 (7872): No heartbeat from core client for 30 sec - exiting 13:10:17 (7872): No heartbeat from core client for 30 sec - exiting 13:10:18 (7872): No heartbeat from core client for 30 sec - exiting 13:10:19 (7872): No heartbeat from core client for 30 sec - exiting 13:10:20 (7872): No heartbeat from core client for 30 sec - exiting 13:10:21 (7872): No heartbeat from core client for 30 sec - exiting 13:10:22 (7872): No heartbeat from core client for 30 sec - exiting 13:10:23 (7872): No heartbeat from core client for 30 sec - exiting 13:10:24 (7872): No heartbeat from core client for 30 sec - exiting 13:10:25 (7872): No heartbeat from core client for 30 sec - exiting 13:10:26 (7872): No heartbeat from core client for 30 sec - exiting 13:10:27 (7872): No heartbeat from core client for 30 sec - exiting 13:10:28 (7872): No heartbeat from core client for 30 sec - exiting 13:10:29 (7872): No heartbeat from core client for 30 sec - exiting 13:10:30 (7872): No heartbeat from core client for 30 sec - exiting 13:10:31 (7872): No heartbeat from core client for 30 sec - exiting 13:10:32 (7872): No heartbeat from core client for 30 sec - exiting 13:10:33 (7872): No heartbeat from core client for 30 sec - exiting 13:10:34 (7872): No heartbeat from core client for 30 sec - exiting 13:10:35 (7872): No heartbeat from core client for 30 sec - exiting 13:10:36 (7872): No heartbeat from core client for 30 sec - exiting 13:10:37 (7872): No heartbeat from core client for 30 sec - exiting 13:10:38 (7872): No heartbeat from core client for 30 sec - exiting 13:10:39 (7872): No heartbeat from core client for 30 sec - exiting 13:10:40 (7872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:59:23 (3400): No heartbeat from core client for 30 sec - exiting 16:59:25 (3400): No heartbeat from core client for 30 sec - exiting 16:59:26 (3400): No heartbeat from core client for 30 sec - exiting 16:59:27 (3400): No heartbeat from core client for 30 sec - exiting 16:59:28 (3400): No heartbeat from core client for 30 sec - exiting 16:59:29 (3400): No heartbeat from core client for 30 sec - exiting 16:59:30 (3400): No heartbeat from core client for 30 sec - exiting 16:59:31 (3400): No heartbeat from core client for 30 sec - exiting 16:59:32 (3400): No heartbeat from core client for 30 sec - exiting 16:59:33 (3400): No heartbeat from core client for 30 sec - exiting 16:59:34 (3400): No heartbeat from core client for 30 sec - exiting 16:59:35 (3400): No heartbeat from core client for 30 sec - exiting 16:59:37 (3400): No heartbeat from core client for 30 sec - exiting 16:59:38 (3400): No heartbeat from core client for 30 sec - exiting 16:59:39 (3400): No heartbeat from core client for 30 sec - exiting 16:59:40 (3400): No heartbeat from core client for 30 sec - exiting 16:59:41 (3400): No heartbeat from core client for 30 sec - exiting 16:59:42 (3400): No heartbeat from core client for 30 sec - exiting 16:59:43 (3400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2988, selfPID=2988, iMonCtr=2 21:55:57 (2436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:39:11 (1336): No heartbeat from core client for 30 sec - exiting 16:39:12 (1336): No heartbeat from core client for 30 sec - exiting 16:39:13 (1336): No heartbeat from core client for 30 sec - exiting 16:39:14 (1336): No heartbeat from core client for 30 sec - exiting 16:39:15 (1336): No heartbeat from core client for 30 sec - exiting 16:39:16 (1336): No heartbeat from core client for 30 sec - exiting 16:39:17 (1336): No heartbeat from core client for 30 sec - exiting 16:39:18 (1336): No heartbeat from core client for 30 sec - exiting 16:39:19 (1336): No heartbeat from core client for 30 sec - exiting 16:39:20 (1336): No heartbeat from core client for 30 sec - exiting 16:39:21 (1336): No heartbeat from core client for 30 sec - exiting 16:39:22 (1336): No heartbeat from core client for 30 sec - exiting 16:39:23 (1336): No heartbeat from core client for 30 sec - exiting 16:39:24 (1336): No heartbeat from core client for 30 sec - exiting 16:39:25 (1336): No heartbeat from core client for 30 sec - exiting 16:39:26 (1336): No heartbeat from core client for 30 sec - exiting 16:39:27 (1336): No heartbeat from core client for 30 sec - exiting 16:39:28 (1336): No heartbeat from core client for 30 sec - exiting 16:39:29 (1336): No heartbeat from core client for 30 sec - exiting 16:39:30 (1336): No heartbeat from core client for 30 sec - exiting 16:39:31 (1336): No heartbeat from core client for 30 sec - exiting 16:39:32 (1336): No heartbeat from core client for 30 sec - exiting 16:39:33 (1336): No heartbeat from core client for 30 sec - exiting 16:39:34 (1336): No heartbeat from core client for 30 sec - exiting 16:39:35 (1336): No heartbeat from core client for 30 sec - exiting 16:39:36 (1336): No heartbeat from core client for 30 sec - exiting 16:39:37 (1336): No heartbeat from core client for 30 sec - exiting 16:39:38 (1336): No heartbeat from core client for 30 sec - exiting 16:39:39 (1336): No heartbeat from core client for 30 sec - exiting 16:39:40 (1336): No heartbeat from core client for 30 sec - exiting 16:39:41 (1336): No heartbeat from core client for 30 sec - exiting 16:39:42 (1336): No heartbeat from core client for 30 sec - exiting 16:39:43 (1336): No heartbeat from core client for 30 sec - exiting 16:39:44 (1336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:52:16 (7476): No heartbeat from core client for 30 sec - exiting 17:52:17 (7476): No heartbeat from core client for 30 sec - exiting 17:52:18 (7476): No heartbeat from core client for 30 sec - exiting 17:52:19 (7476): No heartbeat from core client for 30 sec - exiting 17:52:20 (7476): No heartbeat from core client for 30 sec - exiting 17:52:21 (7476): No heartbeat from core client for 30 sec - exiting 17:52:22 (7476): No heartbeat from core client for 30 sec - exiting 17:52:23 (7476): No heartbeat from core client for 30 sec - exiting 17:52:24 (7476): No heartbeat from core client for 30 sec - exiting 17:52:25 (7476): No heartbeat from core client for 30 sec - exiting 17:52:26 (7476): No heartbeat from core client for 30 sec - exiting 17:52:27 (7476): No heartbeat from core client for 30 sec - exiting 17:52:28 (7476): No heartbeat from core client for 30 sec - exiting 17:52:29 (7476): No heartbeat from core client for 30 sec - exiting 17:52:30 (7476): No heartbeat from core client for 30 sec - exiting 17:52:31 (7476): No heartbeat from core client for 30 sec - exiting 17:52:32 (7476): No heartbeat from core client for 30 sec - exiting 17:52:33 (7476): No heartbeat from core client for 30 sec - exiting 17:52:34 (7476): No heartbeat from core client for 30 sec - exiting 17:52:35 (7476): No heartbeat from core client for 30 sec - exiting 17:52:36 (7476): No heartbeat from core client for 30 sec - exiting 17:52:37 (7476): No heartbeat from core client for 30 sec - exiting 17:52:38 (7476): No heartbeat from core client for 30 sec - exiting 17:52:39 (7476): No heartbeat from core client for 30 sec - exiting 17:52:40 (7476): No heartbeat from core client for 30 sec - exiting 17:52:41 (7476): No heartbeat from core client for 30 sec - exiting 17:52:42 (7476): No heartbeat from core client for 30 sec - exiting 17:52:43 (7476): No heartbeat from core client for 30 sec - exiting 17:52:44 (7476): No heartbeat from core client for 30 sec - exiting 17:52:45 (7476): No heartbeat from core client for 30 sec - exiting 17:52:46 (7476): No heartbeat from core client for 30 sec - exiting 17:52:47 (7476): No heartbeat from core client for 30 sec - exiting 17:52:48 (7476): No heartbeat from core client for 30 sec - exiting 17:52:49 (7476): No heartbeat from core client for 30 sec - exiting 17:52:50 (7476): No heartbeat from core client for 30 sec - exiting 17:52:51 (7476): No heartbeat from core client for 30 sec - exiting 17:52:52 (7476): No heartbeat from core client for 30 sec - exiting 17:52:53 (7476): No heartbeat from core client for 30 sec - exiting 17:52:54 (7476): No heartbeat from core client for 30 sec - exiting 17:52:55 (7476): No heartbeat from core client for 30 sec - exiting 17:52:56 (7476): No heartbeat from core client for 30 sec - exiting 17:52:57 (7476): No heartbeat from core client for 30 sec - exiting 17:52:58 (7476): No heartbeat from core client for 30 sec - exiting 17:52:59 (7476): No heartbeat from core client for 30 sec - exiting 17:53:00 (7476): No heartbeat from core client for 30 sec - exiting 17:53:01 (7476): No heartbeat from core client for 30 sec - exiting 17:53:02 (7476): No heartbeat from core client for 30 sec - exiting 17:53:03 (7476): No heartbeat from core client for 30 sec - exiting 17:53:04 (7476): No heartbeat from core client for 30 sec - exiting 17:53:05 (7476): No heartbeat from core client for 30 sec - exiting 17:53:06 (7476): No heartbeat from core client for 30 sec - exiting 17:53:07 (7476): No heartbeat from core client for 30 sec - exiting 17:53:08 (7476): No heartbeat from core client for 30 sec - exiting 17:53:09 (7476): No heartbeat from core client for 30 sec - exiting 17:53:10 (7476): No heartbeat from core client for 30 sec - exiting 17:53:11 (7476): No heartbeat from core client for 30 sec - exiting 17:53:12 (7476): No heartbeat from core client for 30 sec - exiting 17:53:13 (7476): No heartbeat from core client for 30 sec - exiting 17:53:14 (7476): No heartbeat from core client for 30 sec - exiting 17:53:15 (7476): No heartbeat from core client for 30 sec - exiting 17:53:16 (7476): No heartbeat from core client for 30 sec - exiting 17:53:17 (7476): No heartbeat from core client for 30 sec - exiting 17:53:18 (7476): No heartbeat from core client for 30 sec - exiting 17:53:19 (7476): No heartbeat from core client for 30 sec - exiting 17:53:20 (7476): No heartbeat from core client for 30 sec - exiting 17:53:21 (7476): No heartbeat from core client for 30 sec - exiting 17:53:22 (7476): No heartbeat from core client for 30 sec - exiting 17:53:23 (7476): No heartbeat from core client for 30 sec - exiting 17:53:24 (7476): No heartbeat from core client for 30 sec - exiting 17:53:25 (7476): No heartbeat from core client for 30 sec - exiting 17:53:26 (7476): No heartbeat from core client for 30 sec - exiting 17:53:27 (7476): No heartbeat from core client for 30 sec - exiting 17:53:28 (7476): No heartbeat from core client for 30 sec - exiting 17:53:29 (7476): No heartbeat from core client for 30 sec - exiting 17:53:30 (7476): No heartbeat from core client for 30 sec - exiting 17:53:31 (7476): No heartbeat from core client for 30 sec - exiting 17:53:32 (7476): No heartbeat from core client for 30 sec - exiting 17:53:33 (7476): No heartbeat from core client for 30 sec - exiting 17:53:34 (7476): No heartbeat from core client for 30 sec - exiting 17:53:35 (7476): No heartbeat from core client for 30 sec - exiting 17:53:36 (7476): No heartbeat from core client for 30 sec - exiting 17:53:37 (7476): No heartbeat from core client for 30 sec - exiting 17:53:38 (7476): No heartbeat from core client for 30 sec - exiting 17:53:39 (7476): No heartbeat from core client for 30 sec - exiting 17:53:40 (7476): No heartbeat from core client for 30 sec - exiting 17:53:41 (7476): No heartbeat from core client for 30 sec - exiting 17:53:42 (7476): No heartbeat from core client for 30 sec - exiting 17:53:43 (7476): No heartbeat from core client for 30 sec - exiting 17:53:44 (7476): No heartbeat from core client for 30 sec - exiting 17:53:45 (7476): No heartbeat from core client for 30 sec - exiting 17:53:46 (7476): No heartbeat from core client for 30 sec - exiting 17:53:47 (7476): No heartbeat from core client for 30 sec - exiting 17:53:48 (7476): No heartbeat from core client for 30 sec - exiting 17:53:49 (7476): No heartbeat from core client for 30 sec - exiting 17:53:50 (7476): No heartbeat from core client for 30 sec - exiting 17:53:51 (7476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5224, selfPID=5224, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7804, iMonCtr=2 Model crash detected, will try to restart... 11:52:14 (7564): No heartbeat from core client for 30 sec - exiting 11:52:15 (7564): No heartbeat from core client for 30 sec - exiting 11:52:16 (7564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 10:48:47 (7528): No heartbeat from core client for 30 sec - exiting 10:48:48 (7528): No heartbeat from core client for 30 sec - exiting 10:48:49 (7528): No heartbeat from core client for 30 sec - exiting 10:48:50 (7528): No heartbeat from core client for 30 sec - exiting 10:48:51 (7528): No heartbeat from core client for 30 sec - exiting 10:48:52 (7528): No heartbeat from core client for 30 sec - exiting 10:48:53 (7528): No heartbeat from core client for 30 sec - exiting 10:48:54 (7528): No heartbeat from core client for 30 sec - exiting 10:48:55 (7528): No heartbeat from core client for 30 sec - exiting 10:48:56 (7528): No heartbeat from core client for 30 sec - exiting 10:48:57 (7528): No heartbeat from core client for 30 sec - exiting 10:48:58 (7528): No heartbeat from core client for 30 sec - exiting 10:48:59 (7528): No heartbeat from core client for 30 sec - exiting 10:49:00 (7528): No heartbeat from core client for 30 sec - exiting 10:49:01 (7528): No heartbeat from core client for 30 sec - exiting 10:49:02 (7528): No heartbeat from core client for 30 sec - exiting 10:49:03 (7528): No heartbeat from core client for 30 sec - exiting 10:49:04 (7528): No heartbeat from core client for 30 sec - exiting 10:49:05 (7528): No heartbeat from core client for 30 sec - exiting 10:49:06 (7528): No heartbeat from core client for 30 sec - exiting 10:49:07 (7528): No heartbeat from core client for 30 sec - exiting 10:49:08 (7528): No heartbeat from core client for 30 sec - exiting 10:49:09 (7528): No heartbeat from core client for 30 sec - exiting 10:49:10 (7528): No heartbeat from core client for 30 sec - exiting 10:49:11 (7528): No heartbeat from core client for 30 sec - exiting 10:49:12 (7528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7496, selfPID=7496, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:03:56 (7280): No heartbeat from core client for 30 sec - exiting 16:03:57 (7280): No heartbeat from core client for 30 sec - exiting 16:03:58 (7280): No heartbeat from core client for 30 sec - exiting 16:03:59 (7280): No heartbeat from core client for 30 sec - exiting 16:04:00 (7280): No heartbeat from core client for 30 sec - exiting 16:04:01 (7280): No heartbeat from core client for 30 sec - exiting 16:04:02 (7280): No heartbeat from core client for 30 sec - exiting 16:04:03 (7280): No heartbeat from core client for 30 sec - exiting 16:04:04 (7280): No heartbeat from core client for 30 sec - exiting 16:04:05 (7280): No heartbeat from core client for 30 sec - exiting 16:04:06 (7280): No heartbeat from core client for 30 sec - exiting 16:04:07 (7280): No heartbeat from core client for 30 sec - exiting 16:04:08 (7280): No heartbeat from core client for 30 sec - exiting 16:04:09 (7280): No heartbeat from core client for 30 sec - exiting 16:04:10 (7280): No heartbeat from core client for 30 sec - exiting 16:04:11 (7280): No heartbeat from core client for 30 sec - exiting 16:04:12 (7280): No heartbeat from core client for 30 sec - exiting 16:04:13 (7280): No heartbeat from core client for 30 sec - exiting 16:04:14 (7280): No heartbeat from core client for 30 sec - exiting 16:04:15 (7280): No heartbeat from core client for 30 sec - exiting 16:04:16 (7280): No heartbeat from core client for 30 sec - exiting 16:04:17 (7280): No heartbeat from core client for 30 sec - exiting 16:04:18 (7280): No heartbeat from core client for 30 sec - exiting 16:04:19 (7280): No heartbeat from core client for 30 sec - exiting 16:04:20 (7280): No heartbeat from core client for 30 sec - exiting 16:04:21 (7280): No heartbeat from core client for 30 sec - exiting 16:04:22 (7280): No heartbeat from core client for 30 sec - exiting 16:04:23 (7280): No heartbeat from core client for 30 sec - exiting 16:04:24 (7280): No heartbeat from core client for 30 sec - exiting 16:04:25 (7280): No heartbeat from core client for 30 sec - exiting 16:04:26 (7280): No heartbeat from core client for 30 sec - exiting 16:04:27 (7280): No heartbeat from core client for 30 sec - exiting 16:04:28 (7280): No heartbeat from core client for 30 sec - exiting 16:04:29 (7280): No heartbeat from core client for 30 sec - exiting 16:04:30 (7280): No heartbeat from core client for 30 sec - exiting 16:04:31 (7280): No heartbeat from core client for 30 sec - exiting 16:04:32 (7280): No heartbeat from core client for 30 sec - exiting 16:04:33 (7280): No heartbeat from core client for 30 sec - exiting 16:04:34 (7280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:16:17 (4004): No heartbeat from core client for 30 sec - exiting 16:16:18 (4004): No heartbeat from core client for 30 sec - exiting 16:16:19 (4004): No heartbeat from core client for 30 sec - exiting 16:16:20 (4004): No heartbeat from core client for 30 sec - exiting 16:16:22 (4004): No heartbeat from core client for 30 sec - exiting 16:16:23 (4004): No heartbeat from core client for 30 sec - exiting 16:16:24 (4004): No heartbeat from core client for 30 sec - exiting 16:16:25 (4004): No heartbeat from core client for 30 sec - exiting 16:16:26 (4004): No heartbeat from core client for 30 sec - exiting 16:16:27 (4004): No heartbeat from core client for 30 sec - exiting 16:16:28 (4004): No heartbeat from core client for 30 sec - exiting 16:16:29 (4004): No heartbeat from core client for 30 sec - exiting 16:16:30 (4004): No heartbeat from core client for 30 sec - exiting 16:16:31 (4004): No heartbeat from core client for 30 sec - exiting 16:16:32 (4004): No heartbeat from core client for 30 sec - exiting 16:16:33 (4004): No heartbeat from core client for 30 sec - exiting 16:16:34 (4004): No heartbeat from core client for 30 sec - exiting 16:16:35 (4004): No heartbeat from core client for 30 sec - exiting 16:16:36 (4004): No heartbeat from core client for 30 sec - exiting 16:16:37 (4004): No heartbeat from core client for 30 sec - exiting 16:16:39 (4004): No heartbeat from core client for 30 sec - exiting 16:16:40 (4004): No heartbeat from core client for 30 sec - exiting 16:16:41 (4004): No heartbeat from core client for 30 sec - exiting 16:16:42 (4004): No heartbeat from core client for 30 sec - exiting 16:16:43 (4004): No heartbeat from core client for 30 sec - exiting 16:16:44 (4004): No heartbeat from core client for 30 sec - exiting 16:16:45 (4004): No heartbeat from core client for 30 sec - exiting 16:16:46 (4004): No heartbeat from core client for 30 sec - exiting 16:16:47 (4004): No heartbeat from core client for 30 sec - exiting 16:16:48 (4004): No heartbeat from core client for 30 sec - exiting 16:16:49 (4004): No heartbeat from core client for 30 sec - exiting 16:16:50 (4004): No heartbeat from core client for 30 sec - exiting 16:16:51 (4004): No heartbeat from core client for 30 sec - exiting 16:16:52 (4004): No heartbeat from core client for 30 sec - exiting 16:16:53 (4004): No heartbeat from core client for 30 sec - exiting 16:16:54 (4004): No heartbeat from core client for 30 sec - exiting 16:16:55 (4004): No heartbeat from core client for 30 sec - exiting 16:16:56 (4004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3148, selfPID=3360, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7664, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7748, selfPID=4388, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1608, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8372, selfPID=6532, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_n9jk_2008_1_010108686_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n9jk_2008_1_010108686_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Sep 2015 19:32:44	1353254	18824482	hadam3p_anz_n9jk_2008_1_010108686_1	115,499	200,060	1.7321
01 Sep 2015 21:00:32	1353254	18824482	hadam3p_anz_n9jk_2008_1_010108686_1	103,979	181,484	1.7454
30 Aug 2015 18:45:51	1353254	18824482	hadam3p_anz_n9jk_2008_1_010108686_1	92,459	163,296	1.7661
28 Aug 2015 18:32:09	1353254	18824482	hadam3p_anz_n9jk_2008_1_010108686_1	80,939	144,343	1.7834
26 Aug 2015 15:08:23	1353254	18824482	hadam3p_anz_n9jk_2008_1_010108686_1	69,419	124,840	1.7984
24 Aug 2015 19:22:49	1353254	18824482	hadam3p_anz_n9jk_2008_1_010108686_1	57,899	103,613	1.7895
24 Aug 2015 11:30:24	1353254	18824482	hadam3p_anz_n9jk_2008_1_010108686_1	46,379	82,923	1.7879
23 Aug 2015 18:44:43	1353254	18824482	hadam3p_anz_n9jk_2008_1_010108686_1	34,859	61,851	1.7743
20 Aug 2015 22:06:25	1353254	18824482	hadam3p_anz_n9jk_2008_1_010108686_1	23,339	40,396	1.7308
19 Aug 2015 18:29:29	1353254	18824482	hadam3p_anz_n9jk_2008_1_010108686_1	11,819	18,941	1.6026