Task 15690921

Name	hadcm3n_4a1z_1940_40_008307826_4
Workunit	8458961
Created	28 Mar 2013, 23:32:09 UTC
Sent	28 Mar 2013, 23:32:40 UTC
Report deadline	28 Jun 2013, 6:59:51 UTC
Received	10 Apr 2013, 12:15:23 UTC
Server state	Over
Outcome	Computation error
Client state	Done
Exit status	22 (0x00000016) Unknown error code
Computer ID	3620
Run time	4 days 9 hours 50 min 19 sec
CPU time	4 days 9 hours 50 min 19 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	2.21 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>5.10.45</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 14:56:28 (14284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:56:30 (14284): No heartbeat from core client for 30 sec - exiting 20:22:14 (13184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:22:15 (13184): No heartbeat from core client for 30 sec - exiting 20:22:16 (13184): No heartbeat from core client for 30 sec - exiting 20:22:17 (13184): No heartbeat from core client for 30 sec - exiting 04:15:43 (11784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:15:47 (11784): No heartbeat from core client for 30 sec - exiting 04:15:48 (11784): No heartbeat from core client for 30 sec - exiting 04:15:50 (11784): No heartbeat from core client for 30 sec - exiting 04:15:51 (11784): No heartbeat from core client for 30 sec - exiting 04:15:52 (11784): No heartbeat from core client for 30 sec - exiting 04:15:54 (11784): No heartbeat from core client for 30 sec - exiting 04:16:01 (11784): No heartbeat from core client for 30 sec - exiting 04:16:03 (11784): No heartbeat from core client for 30 sec - exiting 04:16:04 (11784): No heartbeat from core client for 30 sec - exiting 04:16:07 (11784): No heartbeat from core client for 30 sec - exiting 04:16:08 (11784): No heartbeat from core client for 30 sec - exiting 04:37:03 (3100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:52:55 (12504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:39:49 (13084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:39:50 (13084): No heartbeat from core client for 30 sec - exiting 08:42:14 (10932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:43:24 (12168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:43:26 (12168): No heartbeat from core client for 30 sec - exiting 13:47:11 (5860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:41:44 (9980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:41:45 (9980): No heartbeat from core client for 30 sec - exiting 20:03:29 (10280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:43:38 (7784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:14:30 (9404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:46:54 (12368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:15:57 (3664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:15:59 (3664): No heartbeat from core client for 30 sec - exiting 04:16:00 (3664): No heartbeat from core client for 30 sec - exiting 04:16:01 (3664): No heartbeat from core client for 30 sec - exiting 04:16:02 (3664): No heartbeat from core client for 30 sec - exiting 05:18:24 (6112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:36:09 (11420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:20:14 (8484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:37:22 (12756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:37:23 (12756): No heartbeat from core client for 30 sec - exiting 12:37:24 (12756): No heartbeat from core client for 30 sec - exiting 12:37:25 (12756): No heartbeat from core client for 30 sec - exiting 12:37:26 (12756): No heartbeat from core client for 30 sec - exiting 12:37:27 (12756): No heartbeat from core client for 30 sec - exiting 12:37:28 (12756): No heartbeat from core client for 30 sec - exiting 15:22:59 (10084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:25:26 (10052): No heartbeat from core client for 30 sec - exiting 15:25:27 (10052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:25:28 (10052): No heartbeat from core client for 30 sec - exiting 15:25:29 (10052): No heartbeat from core client for 30 sec - exiting 20:29:13 (9068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:29:15 (9068): No heartbeat from core client for 30 sec - exiting 20:29:16 (9068): No heartbeat from core client for 30 sec - exiting 20:29:17 (9068): No heartbeat from core client for 30 sec - exiting 20:29:18 (9068): No heartbeat from core client for 30 sec - exiting 20:29:19 (9068): No heartbeat from core client for 30 sec - exiting 20:32:29 (9408): No heartbeat from core client for 30 sec - exiting 20:33:03 (9408): No heartbeat from core client for 30 sec - exiting 20:33:04 (9408): No heartbeat from core client for 30 sec - exiting 20:33:05 (9408): No heartbeat from core client for 30 sec - exiting 20:33:06 (9408): No heartbeat from core client for 30 sec - exiting 20:33:07 (9408): No heartbeat from core client for 30 sec - exiting 20:33:08 (9408): No heartbeat from core client for 30 sec - exiting 20:33:09 (9408): No heartbeat from core client for 30 sec - exiting 20:33:10 (9408): No heartbeat from core client for 30 sec - exiting 20:33:11 (9408): No heartbeat from core client for 30 sec - exiting 20:33:12 (9408): No heartbeat from core client for 30 sec - exiting 20:33:13 (9408): No heartbeat from core client for 30 sec - exiting 20:33:14 (9408): No heartbeat from core client for 30 sec - exiting 20:33:15 (9408): No heartbeat from core client for 30 sec - exiting 20:33:16 (9408): No heartbeat from core client for 30 sec - exiting 20:33:17 (9408): No heartbeat from core client for 30 sec - exiting 20:33:18 (9408): No heartbeat from core client for 30 sec - exiting 20:33:19 (9408): No heartbeat from core client for 30 sec - exiting 20:33:20 (9408): No heartbeat from core client for 30 sec - exiting 20:33:21 (9408): No heartbeat from core client for 30 sec - exiting 20:33:22 (9408): No heartbeat from core client for 30 sec - exiting 20:33:23 (9408): No heartbeat from core client for 30 sec - exiting 20:33:24 (9408): No heartbeat from core client for 30 sec - exiting 20:33:25 (9408): No heartbeat from core client for 30 sec - exiting 20:33:26 (9408): No heartbeat from core client for 30 sec - exiting 20:33:27 (9408): No heartbeat from core client for 30 sec - exiting 20:33:28 (9408): No heartbeat from core client for 30 sec - exiting 20:33:29 (9408): No heartbeat from core client for 30 sec - exiting 20:33:30 (9408): No heartbeat from core client for 30 sec - exiting 20:33:31 (9408): No heartbeat from core client for 30 sec - exiting 20:33:32 (9408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:00 (8148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:37:01 (8148): No heartbeat from core client for 30 sec - exiting 22:48:56 (8372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:48:58 (8372): No heartbeat from core client for 30 sec - exiting 23:50:36 (12296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:10:19 (6360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:10:27 (6360): No heartbeat from core client for 30 sec - exiting 04:10:33 (6360): No heartbeat from core client for 30 sec - exiting 04:10:35 (6360): No heartbeat from core client for 30 sec - exiting 04:10:37 (6360): No heartbeat from core client for 30 sec - exiting 04:10:39 (6360): No heartbeat from core client for 30 sec - exiting 04:10:41 (6360): No heartbeat from core client for 30 sec - exiting 04:10:43 (6360): No heartbeat from core client for 30 sec - exiting 04:10:44 (6360): No heartbeat from core client for 30 sec - exiting 04:18:31 (6844): No heartbeat from core client for 30 sec - exiting 04:18:32 (6844): No heartbeat from core client for 30 sec - exiting 04:18:33 (6844): No heartbeat from core client for 30 sec - exiting 04:18:34 (6844): No heartbeat from core client for 30 sec - exiting 04:18:35 (6844): No heartbeat from core client for 30 sec - exiting 04:18:36 (6844): No heartbeat from core client for 30 sec - exiting 04:18:37 (6844): No heartbeat from core client for 30 sec - exiting 04:18:38 (6844): No heartbeat from core client for 30 sec - exiting 04:18:39 (6844): No heartbeat from core client for 30 sec - exiting 04:18:40 (6844): No heartbeat from core client for 30 sec - exiting 04:18:41 (6844): No heartbeat from core client for 30 sec - exiting 04:18:43 (6844): No heartbeat from core client for 30 sec - exiting 04:18:44 (6844): No heartbeat from core client for 30 sec - exiting 04:18:46 (6844): No heartbeat from core client for 30 sec - exiting 04:18:47 (6844): No heartbeat from core client for 30 sec - exiting 04:18:49 (6844): No heartbeat from core client for 30 sec - exiting 04:18:51 (6844): No heartbeat from core client for 30 sec - exiting 04:18:52 (6844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:26:08 (14012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:28:30 (12336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:27:20 (8604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:27:21 (8604): No heartbeat from core client for 30 sec - exiting 20:21:39 (11796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:21:41 (11796): No heartbeat from core client for 30 sec - exiting 20:21:42 (11796): No heartbeat from core client for 30 sec - exiting 20:47:37 (6932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:23:58 (8952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:23:59 (8952): No heartbeat from core client for 30 sec - exiting 21:24:00 (8952): No heartbeat from core client for 30 sec - exiting 21:24:02 (8952): No heartbeat from core client for 30 sec - exiting 21:24:03 (8952): No heartbeat from core client for 30 sec - exiting 21:24:04 (8952): No heartbeat from core client for 30 sec - exiting 21:24:05 (8952): No heartbeat from core client for 30 sec - exiting 21:24:06 (8952): No heartbeat from core client for 30 sec - exiting 21:24:07 (8952): No heartbeat from core client for 30 sec - exiting 21:24:08 (8952): No heartbeat from core client for 30 sec - exiting 21:24:09 (8952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 22:23:26 (10768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:23:29 (10768): No heartbeat from core client for 30 sec - exiting 22:27:52 (5812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:29:17 (10448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:15:21 (12336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:15:22 (12336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 10:23:15 (6740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:23:17 (6740): No heartbeat from core client for 30 sec - exiting 10:23:18 (6740): No heartbeat from core client for 30 sec - exiting 10:23:19 (6740): No heartbeat from core client for 30 sec - exiting 10:23:20 (6740): No heartbeat from core client for 30 sec - exiting 20:02:58 (13824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:00 (13824): No heartbeat from core client for 30 sec - exiting 20:38:36 (9384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:14:31 (10732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:14:35 (10732): No heartbeat from core client for 30 sec - exiting 04:14:36 (10732): No heartbeat from core client for 30 sec - exiting 04:14:37 (10732): No heartbeat from core client for 30 sec - exiting 04:14:38 (10732): No heartbeat from core client for 30 sec - exiting 04:14:41 (10732): No heartbeat from core client for 30 sec - exiting 04:20:20 (8928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:22:33 (11876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:22:35 (11876): No heartbeat from core client for 30 sec - exiting 13:30:09 (8404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:30:10 (8404): No heartbeat from core client for 30 sec - exiting 13:30:11 (8404): No heartbeat from core client for 30 sec - exiting 15:56:14 (3224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:33 (10236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:44:20 (10756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:44:24 (10756): No heartbeat from core client for 30 sec - exiting 21:58:17 (13520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:38:08 (6664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:38:10 (6664): No heartbeat from core client for 30 sec - exiting 04:38:11 (6664): No heartbeat from core client for 30 sec - exiting 12:58:38 (10980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:58:39 (10980): No heartbeat from core client for 30 sec - exiting 14:05:11 (1264): No heartbeat from core client for 30 sec - exiting 14:05:12 (1264): No heartbeat from core client for 30 sec - exiting 14:05:13 (1264): No heartbeat from core client for 30 sec - exiting 14:05:14 (1264): No heartbeat from core client for 30 sec - exiting 14:05:15 (1264): No heartbeat from core client for 30 sec - exiting 14:05:16 (1264): No heartbeat from core client for 30 sec - exiting 14:05:17 (1264): No heartbeat from core client for 30 sec - exiting 14:05:18 (1264): No heartbeat from core client for 30 sec - exiting 14:05:19 (1264): No heartbeat from core client for 30 sec - exiting 14:05:20 (1264): No heartbeat from core client for 30 sec - exiting 14:05:21 (1264): No heartbeat from core client for 30 sec - exiting 14:05:22 (1264): No heartbeat from core client for 30 sec - exiting 14:05:23 (1264): No heartbeat from core client for 30 sec - exiting 14:05:25 (1264): No heartbeat from core client for 30 sec - exiting 14:05:26 (1264): No heartbeat from core client for 30 sec - exiting 14:05:27 (1264): No heartbeat from core client for 30 sec - exiting 14:05:28 (1264): No heartbeat from core client for 30 sec - exiting 14:05:29 (1264): No heartbeat from core client for 30 sec - exiting 14:05:30 (1264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:37 (8544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:52:03 (8924): No heartbeat from core client for 30 sec - exiting 20:52:04 (8924): No heartbeat from core client for 30 sec - exiting 20:52:05 (8924): No heartbeat from core client for 30 sec - exiting 20:52:06 (8924): No heartbeat from core client for 30 sec - exiting 20:52:07 (8924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:07:58 (14024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:07:59 (14024): No heartbeat from core client for 30 sec - exiting 21:08:00 (14024): No heartbeat from core client for 30 sec - exiting 21:56:48 (9536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:03:16 (10208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:08:44 (12324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:22:48 (12912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:10:34 (7560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:24:06 (8220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:24:07 (8220): No heartbeat from core client for 30 sec - exiting 23:24:08 (8220): No heartbeat from core client for 30 sec - exiting 23:24:09 (8220): No heartbeat from core client for 30 sec - exiting 23:24:10 (8220): No heartbeat from core client for 30 sec - exiting 23:33:42 (1800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:40:17 (1272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:42:10 (7708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:47:20 (6664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:47:21 (6664): No heartbeat from core client for 30 sec - exiting 23:47:22 (6664): No heartbeat from core client for 30 sec - exiting 00:13:57 (10492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:50:20 (14264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:52:43 (13508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:11:55 (13032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:20:07 (7784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:36:49 (13184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:48:29 (9992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:48:31 (9992): No heartbeat from core client for 30 sec - exiting 01:55:38 (9376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:11:27 (5336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:44:27 (10404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:55:42 (6620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:08:12 (7732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:18:36 (5252): No heartbeat from core client for 30 sec - exiting 04:18:37 (5252): No heartbeat from core client for 30 sec - exiting 04:18:38 (5252): No heartbeat from core client for 30 sec - exiting 04:18:39 (5252): No heartbeat from core client for 30 sec - exiting 04:18:40 (5252): No heartbeat from core client for 30 sec - exiting 04:18:41 (5252): No heartbeat from core client for 30 sec - exiting 04:18:42 (5252): No heartbeat from core client for 30 sec - exiting 04:18:43 (5252): No heartbeat from core client for 30 sec - exiting 04:18:44 (5252): No heartbeat from core client for 30 sec - exiting 04:18:45 (5252): No heartbeat from core client for 30 sec - exiting 04:18:46 (5252): No heartbeat from core client for 30 sec - exiting 04:18:47 (5252): No heartbeat from core client for 30 sec - exiting 04:18:48 (5252): No heartbeat from core client for 30 sec - exiting 04:18:49 (5252): No heartbeat from core client for 30 sec - exiting 04:18:50 (5252): No heartbeat from core client for 30 sec - exiting 04:18:51 (5252): No heartbeat from core client for 30 sec - exiting 04:18:52 (5252): No heartbeat from core client for 30 sec - exiting 04:18:53 (5252): No heartbeat from core client for 30 sec - exiting 04:18:54 (5252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:23:49 (11628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:52:29 (6096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:52:31 (6096): No heartbeat from core client for 30 sec - exiting 05:07:06 (14184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:07:07 (14184): No heartbeat from core client for 30 sec - exiting 05:18:50 (13288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:20:35 (5288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:08:46 (14112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:08:48 (14112): No heartbeat from core client for 30 sec - exiting 06:11:15 (13212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:11:16 (13212): No heartbeat from core client for 30 sec - exiting 06:11:17 (13212): No heartbeat from core client for 30 sec - exiting 06:11:18 (13212): No heartbeat from core client for 30 sec - exiting 06:11:19 (13212): No heartbeat from core client for 30 sec - exiting 06:11:20 (13212): No heartbeat from core client for 30 sec - exiting 06:11:21 (13212): No heartbeat from core client for 30 sec - exiting 06:11:22 (13212): No heartbeat from core client for 30 sec - exiting 06:11:24 (13212): No heartbeat from core client for 30 sec - exiting 06:11:25 (13212): No heartbeat from core client for 30 sec - exiting 06:11:26 (13212):06:21:50 (9992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:28:42 (13092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:06:50 (11520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:22:54 (7868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:22:56 (7868): No heartbeat from core client for 30 sec - exiting 07:34:30 (9264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:21:21 (11448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:23:39 (16252): No heartbeat from core client for 30 sec - exiting 22:23:40 (16252): No heartbeat from core client for 30 sec - exiting 22:23:41 (16252): No heartbeat from core client for 30 sec - exiting 22:23:42 (16252): No heartbeat from core client for 30 sec - exiting 22:23:43 (16252): No heartbeat from core client for 30 sec - exiting 22:23:44 (16252): No heartbeat from core client for 30 sec - exiting 22:23:45 (16252): No heartbeat from core client for 30 sec - exiting 22:23:46 (16252): No heartbeat from core client for 30 sec - exiting 22:23:47 (16252): No heartbeat from core client for 30 sec - exiting 22:23:48 (16252): No heartbeat from core client for 30 sec - exiting 22:23:52 (16252): No heartbeat from core client for 30 sec - exiting 22:23:53 (16252): No heartbeat from core client for 30 sec - exiting 22:23:54 (16252): No heartbeat from core client for 30 sec - exiting 22:23:55 (16252): No heartbeat from core client for 30 sec - exiting 22:23:57 (16252): No heartbeat from core client for 30 sec - exiting 22:23:58 (16252): No heartbeat from core client for 30 sec - exiting 22:23:59 (16252): No heartbeat from core client for 30 sec - exiting 22:24:00 (16252): No heartbeat from core client for 30 sec - exiting 22:24:01 (16252): No heartbeat from core client for 30 sec - exiting 22:24:02 (16252): No heartbeat from core client for 30 sec - exiting 22:24:03 (16252): No heartbeat from core client for 30 sec - exiting 22:24:04 (16252): No heartbeat from core client for 30 sec - exiting 22:24:05 (16252): No heartbeat from core client for 30 sec - exiting 22:24:06 (16252): No heartbeat from core client for 30 sec - exiting 22:24:07 (16252): No heartbeat from core client for 30 sec - exiting 22:24:08 (16252): No heartbeat from core client for 30 sec - exiting 22:24:09 (16252): No heartbeat from core client for 30 sec - exiting 22:24:10 (16252): No heartbeat from core client for 30 sec - exiting 22:24:11 (16252): No heartbeat from core client for 30 sec - exiting 22:24:12 (16252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:25:43 (12600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:18:07 (15208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:18:08 (15208): No heartbeat from core client for 30 sec - exiting 04:18:09 (15208): No heartbeat from core client for 30 sec - exiting 04:18:10 (15208): No heartbeat from core client for 30 sec - exiting 04:33:24 (15096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:34:16 (14264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:34:19 (14264): No heartbeat from core client for 30 sec - exiting 05:34:20 (14264): No heartbeat from core client for 30 sec - exiting 05:36:26 (11240): No heartbeat from core client for 30 sec - exiting 05:36:27 (11240): No heartbeat from core client for 30 sec - exiting 05:36:28 (11240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:11:11 (3724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:11:12 (3724): No heartbeat from core client for 30 sec - exiting 06:11:13 (3724): No heartbeat from core client for 30 sec - exiting 06:11:14 (3724): No heartbeat from core client for 30 sec - exiting 06:11:15 (3724): No heartbeat from core client for 30 sec - exiting 06:11:16 (3724): No heartbeat from core client for 30 sec - exiting 06:11:17 (3724): No heartbeat from core client for 30 sec - exiting 06:11:18 (3724): No heartbeat from core client for 30 sec - exiting 06:11:19 (3724): No heartbeat from core client for 30 sec - exiting 06:11:20 (3724): No heartbeat from core client for 30 sec - exiting 06:11:21 (3724): No heartbeat from core client for 30 sec - exiting 06:15:20 (12808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:37:44 (13796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:17:44 (8824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:38:32 (3384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:07 (15476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:19:54 (10352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:21:25 (596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:05:29 (15028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:30:00 (8768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:01:59 (10476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:31:38 (14528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:31:40 (14528): No heartbeat from core client for 30 sec - exiting 11:31:41 (14528): No heartbeat from core client for 30 sec - exiting 11:45:34 (4968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:20 (8996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:22 (8996): No heartbeat from core client for 30 sec - exiting 12:28:23 (8996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 18:45:25 (15792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:45:28 (15792): No heartbeat from core client for 30 sec - exiting 18:45:30 (15792): No heartbeat from core client for 30 sec - exiting 18:45:31 (15792): No heartbeat from core client for 30 sec - exiting 19:59:28 (12604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:10:57 (16232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:46:23 (15092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:46:24 (15092): No heartbeat from core client for 30 sec - exiting 20:58:37 (1540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:30:38 (15008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:47:13 (7416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:47:18 (7416): No heartbeat from core client for 30 sec - exiting 00:47:19 (7416): No heartbeat from core client for 30 sec - exiting 00:47:20 (7416): No heartbeat from core client for 30 sec - exiting 00:47:21 (7416): No heartbeat from core client for 30 sec - exiting 00:47:23 (7416): No heartbeat from core client for 30 sec - exiting 00:47:24 (7416): No heartbeat from core client for 30 sec - exiting 00:47:25 (7416): No heartbeat from core client for 30 sec - exiting 00:47:27 (7416): No heartbeat from core client for 30 sec - exiting 00:47:28 (7416): No heartbeat from core client for 30 sec - exiting 00:47:29 (7416): No heartbeat from core client for 30 sec - exiting 01:46:23 (15708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:46:25 (15708): No heartbeat from core client for 30 sec - exiting 01:46:26 (15708): No heartbeat from core client for 30 sec - exiting 03:56:29 (13044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:56:30 (13044): No heartbeat from core client for 30 sec - exiting 03:56:31 (13044): No heartbeat from core client for 30 sec - exiting 03:56:32 (13044): No heartbeat from core client for 30 sec - exiting 04:06:41 (7568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:06:43 (7568): No heartbeat from core client for 30 sec - exiting 04:06:44 (7568): No heartbeat from core client for 30 sec - exiting 04:25:57 (15292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:25:58 (15292): No heartbeat from core client for 30 sec - exiting 04:25:59 (15292): No heartbeat from core client for 30 sec - exiting 04:38:59 (6728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:07:38 (14808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:07:39 (14808): No heartbeat from core client for 30 sec - exiting 05:07:40 (14808): No heartbeat from core client for 30 sec - exiting 05:07:41 (14808): No heartbeat from core client for 30 sec - exiting 05:21:46 (10868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:32:50 (15852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:32:51 (15852): No heartbeat from core client for 30 sec - exiting 05:32:52 (15852): No heartbeat from core client for 30 sec - exiting 05:32:58 (15852): No heartbeat from core client for 30 sec - exiting 05:32:59 (15852): No heartbeat from core client for 30 sec - exiting 05:33:00 (15852): No heartbeat from core client for 30 sec - exiting 05:44:07 (5376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:39:18 (14452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:09:35 (14468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:37:53 (13740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:46:44 (7392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:50:18 (15992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:01:24 (8516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:10:06 (6676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:12:02 (15280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:36:13 (14740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:53:07 (9240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:00:57 (8584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:32 (12664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:33 (12664): No heartbeat from core client for 30 sec - exiting 09:24:34 (12664): No heartbeat from core client for 30 sec - exiting 09:24:35 (12664): No heartbeat from core client for 30 sec - exiting 09:24:36 (12664): No heartbeat from core client for 30 sec - exiting 09:42:05 (2224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:02:28 (6328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:08:00 (12676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:44:40 (4288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:51:57 (7448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:51:58 (7448): No heartbeat from core client for 30 sec - exiting 20:20:30 (16236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:20:33 (16236): No heartbeat from core client for 30 sec - exiting 20:20:34 (16236): No heartbeat from core client for 30 sec - exiting 20:20:35 (16236): No heartbeat from core client for 30 sec - exiting 20:20:36 (16236): No heartbeat from core client for 30 sec - exiting 20:20:37 (16236): No heartbeat from core client for 30 sec - exiting 20:20:38 (16236): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6224, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6224, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6224, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6224, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6224, iMonCtr=1 Model crash detected, will try to restart... 22:12:15 (6224): No heartbeat from core client for 30 sec - exiting 22:12:16 (6224): No heartbeat from core client for 30 sec - exiting 22:12:17 (6224): No heartbeat from core client for 30 sec - exiting 22:12:18 (6224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:13:58 (12468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14908, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Apr 2013 23:39:47	3620	15690921	hadcm3n_4a1z_1940_40_008307826_4	155,520	329,962	2.1217
04 Apr 2013 03:13:33	3620	15690921	hadcm3n_4a1z_1940_40_008307826_4	129,600	277,879	2.1441
03 Apr 2013 03:53:35	3620	15690921	hadcm3n_4a1z_1940_40_008307826_4	103,680	225,602	2.1759
02 Apr 2013 05:37:24	3620	15690921	hadcm3n_4a1z_1940_40_008307826_4	77,760	174,351	2.2422
01 Apr 2013 03:37:37	3620	15690921	hadcm3n_4a1z_1940_40_008307826_4	51,840	124,133	2.3945
30 Mar 2013 19:11:01	3620	15690921	hadcm3n_4a1z_1940_40_008307826_4	25,920	65,501	2.5270