Task 16277603

Name	hadcm3n_481n_2020_40_008402407_4
Workunit	8553263
Created	1 Feb 2014, 1:14:21 UTC
Sent	1 Feb 2014, 1:14:25 UTC
Report deadline	3 May 2014, 8:41:36 UTC
Received	10 Feb 2014, 11:27:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1265980
Run time	1 days 11 hours 27 min 37 sec
CPU time	1 days 9 hours 27 min 2 sec
Validate state	Invalid
Credit	622.08
Device peak FLOPS	2.33 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:55:15 (1423): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:01:21 (22624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:10:01 (25302): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:10:19 (25302): No heartbeat from core client for 30 sec - exiting 13:16:59 (28214): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:17:00 (28214): No heartbeat from core client for 30 sec - exiting 13:23:36 (31335): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:23:40 (31335): No heartbeat from core client for 30 sec - exiting 13:23:41 (31335): No heartbeat from core client for 30 sec - exiting 13:23:42 (31335): No heartbeat from core client for 30 sec - exiting 13:23:43 (31335): No heartbeat from core client for 30 sec - exiting 13:23:44 (31335): No heartbeat from core client for 30 sec - exiting 13:23:45 (31335): No heartbeat from core client for 30 sec - exiting 13:28:54 (1016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:32:13 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:35:23 (4720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:35:30 (4720): No heartbeat from core client for 30 sec - exiting 13:35:31 (4720): No heartbeat from core client for 30 sec - exiting 13:35:32 (4720): No heartbeat from core client for 30 sec - exiting 13:35:33 (4720): No heartbeat from core client for 30 sec - exiting 13:35:34 (4720): No heartbeat from core client for 30 sec - exiting 13:35:35 (4720): No heartbeat from core client for 30 sec - exiting 13:35:36 (4720): No heartbeat from core client for 30 sec - exiting 13:35:37 (4720): No heartbeat from core client for 30 sec - exiting 13:35:38 (4720): No heartbeat from core client for 30 sec - exiting 13:35:39 (4720): No heartbeat from core client for 30 sec - exiting 13:35:42 (4720): No heartbeat from core client for 30 sec - exiting 13:35:43 (4720): No heartbeat from core client for 30 sec - exiting 13:40:58 (6253): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:41:05 (6253): No heartbeat from core client for 30 sec - exiting 13:55:41 (8406): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:02:17 (13039): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:02:23 (13039): No heartbeat from core client for 30 sec - exiting 14:02:26 (13039): No heartbeat from core client for 30 sec - exiting 14:12:52 (15275): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:13:01 (15275): No heartbeat from core client for 30 sec - exiting 14:13:02 (15275): No heartbeat from core client for 30 sec - exiting 14:13:03 (15275): No heartbeat from core client for 30 sec - exiting 14:13:04 (15275): No heartbeat from core client for 30 sec - exiting 14:13:05 (15275): No heartbeat from core client for 30 sec - exiting 14:13:06 (15275): No heartbeat from core client for 30 sec - exiting 14:18:58 (18688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:19:12 (18688): No heartbeat from core client for 30 sec - exiting 14:19:13 (18688): No heartbeat from core client for 30 sec - exiting 14:23:32 (21083): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:29:07 (23458): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:35:08 (26066): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:40:42 (28777): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:40:43 (28777): No heartbeat from core client for 30 sec - exiting 14:40:44 (28777): No heartbeat from core client for 30 sec - exiting 14:40:45 (28777): No heartbeat from core client for 30 sec - exiting 14:40:46 (28777): No heartbeat from core client for 30 sec - exiting 14:40:47 (28777): No heartbeat from core client for 30 sec - exiting 14:40:48 (28777): No heartbeat from core client for 30 sec - exiting 14:40:49 (28777): No heartbeat from core client for 30 sec - exiting 14:40:50 (28777): No heartbeat from core client for 30 sec - exiting 14:40:51 (28777): No heartbeat from core client for 30 sec - exiting 14:40:52 (28777): No heartbeat from core client for 30 sec - exiting 14:40:53 (28777): No heartbeat from core client for 30 sec - exiting 14:46:39 (31018): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:50:20 (1061): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:02:04 (3287): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:02:06 (3287): No heartbeat from core client for 30 sec - exiting 15:02:07 (3287): No heartbeat from core client for 30 sec - exiting 15:07:56 (7289): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:12:58 (9106): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:31:49 (11637): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:37:19 (18631): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:37:20 (18631): No heartbeat from core client for 30 sec - exiting 15:37:21 (18631): No heartbeat from core client for 30 sec - exiting 15:46:27 (20991): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:46:37 (20991): No heartbeat from core client for 30 sec - exiting 15:51:50 (24976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:02:03 (27195): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:06:43 (31396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:08:25 (902): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:14:09 (1441): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:19:21 (4106): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:19:23 (4106): No heartbeat from core client for 30 sec - exiting 16:19:24 (4106): No heartbeat from core client for 30 sec - exiting 16:19:25 (4106): No heartbeat from core client for 30 sec - exiting 16:19:26 (4106): No heartbeat from core client for 30 sec - exiting 16:19:27 (4106): No heartbeat from core client for 30 sec - exiting 16:24:32 (6861): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:24:41 (6861): No heartbeat from core client for 30 sec - exiting 16:31:12 (7873): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:51 (10121): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:40:57 (12523): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:58:39 (14475): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:18:05 (19710): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:19:26 (26819): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:30 (26859): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:32 (26859): No heartbeat from core client for 30 sec - exiting 17:31:08 (26990): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:38:20 (27007): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:51:47 (27025): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:52:07 (27025): No heartbeat from core client for 30 sec - exiting 17:52:08 (27025): No heartbeat from core client for 30 sec - exiting 17:52:09 (27025): No heartbeat from core client for 30 sec - exiting 17:52:10 (27025): No heartbeat from core client for 30 sec - exiting 17:52:11 (27025): No heartbeat from core client for 30 sec - exiting 17:55:59 (27051): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:56:01 (27051): No heartbeat from core client for 30 sec - exiting 17:56:02 (27051): No heartbeat from core client for 30 sec - exiting 17:56:03 (27051): No heartbeat from core client for 30 sec - exiting 17:56:04 (27051): No heartbeat from core client for 30 sec - exiting 18:00:26 (27071): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:54 (27087): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:07:16 (27087): No heartbeat from core client for 30 sec - exiting 18:15:48 (27105): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:33:56 (27123): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:33:58 (27123): No heartbeat from core client for 30 sec - exiting 18:39:33 (27158): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:44:39 (27176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:50:41 (27194): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:56:22 (27212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:06:10 (27229): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:12:15 (27253): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:37 (27269): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:40 (27269): No heartbeat from core client for 30 sec - exiting 19:25:56 (27296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:03 (27314): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:23 (27337): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:58:01 (27368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:02:49 (27406): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:08:45 (27423): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:18:21 (27441): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:28:06 (27474): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:28:08 (27474): No heartbeat from core client for 30 sec - exiting 20:50:01 (27507): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:56:05 (27557): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:56:08 (27557): No heartbeat from core client for 30 sec - exiting 21:04:05 (27577): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:12:51 (27600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:13:06 (27600): No heartbeat from core client for 30 sec - exiting 21:20:56 (27619): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:53 (27643): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:54 (27643): No heartbeat from core client for 30 sec - exiting 21:24:55 (27643): No heartbeat from core client for 30 sec - exiting 21:24:56 (27643): No heartbeat from core client for 30 sec - exiting 21:24:57 (27643): No heartbeat from core client for 30 sec - exiting 21:29:09 (27658): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:29:11 (27658): No heartbeat from core client for 30 sec - exiting 21:29:12 (27658): No heartbeat from core client for 30 sec - exiting 21:29:13 (27658): No heartbeat from core client for 30 sec - exiting 21:33:24 (27676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:33:26 (27676): No heartbeat from core client for 30 sec - exiting 21:51:29 (27690): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:51:37 (27690): No heartbeat from core client for 30 sec - exiting 21:55:07 (27728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:59:48 (27742): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:00:30 (27755): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:00:43 (27755): No heartbeat from core client for 30 sec - exiting 22:06:25 (27773): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:06:30 (27773): No heartbeat from core client for 30 sec - exiting 22:11:14 (27786): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:14:57 (27803): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:20:08 (27818): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:28:04 (27840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:35:57 (27862): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:51:40 (27883): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:55:26 (27922): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:00:52 (27938): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:06:04 (27955): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:06:05 (27955): No heartbeat from core client for 30 sec - exiting 23:10:05 (27976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:22:10 (27994): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:27:25 (28025): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:30:15 (28043): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:14 (28059): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:55:16 (28059): No heartbeat from core client for 30 sec - exiting 00:00:26 (28117): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:00:27 (28117): No heartbeat from core client for 30 sec - exiting 00:03:58 (28133): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:08:52 (28153): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:12:24 (28167): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:18:22 (28180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:28:49 (28199): No heartbeat from core client for 30 sec - exiting 00:28:52 (28199): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:28:53 (28199): No heartbeat from core client for 30 sec - exiting 00:29:57 (28225): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:34:40 (28239): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:53:31 (28261): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:59:18 (28300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:05:51 (28320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:05:55 (28320): No heartbeat from core client for 30 sec - exiting 01:09:48 (28340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:09:51 (28340): No heartbeat from core client for 30 sec - exiting 01:09:52 (28340): No heartbeat from core client for 30 sec - exiting 01:09:53 (28340): No heartbeat from core client for 30 sec - exiting 01:15:11 (28355): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:19:03 (28370): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:19:12 (28370): No heartbeat from core client for 30 sec - exiting 01:22:56 (28390): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:34:17 (28406): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:34:19 (28406): No heartbeat from core client for 30 sec - exiting 01:34:20 (28406):01:35:54 (28432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:35:57 (28432): No heartbeat from core client for 30 sec - exiting 01:36:04 (28432): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 01:43:07 (28447): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:55:31 (28453): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:01:22 (28488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:02:54 (28509): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:02:55 (28509): No heartbeat from core client for 30 sec - exiting 02:08:50 (28522): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:14:08 (28542): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:18:36 (28559): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:28:20 (28579): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:33:44 (28598): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:40:54 (28620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:40:57 (28620): No heartbeat from core client for 30 sec - exiting 02:40:58 (28620): No heartbeat from core client for 30 sec - exiting 02:40:59 (28620): No heartbeat from core client for 30 sec - exiting 02:47:22 (28635): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:47:24 (28635): No heartbeat from core client for 30 sec - exiting 02:47:25 (28635): No heartbeat from core client for 30 sec - exiting 02:53:37 (28654): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:29:41 (28671): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:29:48 (28671): No heartbeat from core client for 30 sec - exiting 03:49:49 (28749): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:56:46 (28802): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:56:48 (28802): No heartbeat from core client for 30 sec - exiting 04:23:22 (28819): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:23:33 (28819): No heartbeat from core client for 30 sec - exiting 04:31:22 (28891): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:36:58 (28910): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:40:55 (28931): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:49:25 (28949): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:49:36 (28949): No heartbeat from core client for 30 sec - exiting 04:56:49 (28971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:03:17 (28989): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:03:21 (28989): No heartbeat from core client for 30 sec - exiting 05:03:22 (28989): No heartbeat from core client for 30 sec - exiting 05:16:21 (29010): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:16:22 (29010): No heartbeat from core client for 30 sec - exiting 05:16:23 (29010): No heartbeat from core client for 30 sec - exiting 05:16:24 (29010): No heartbeat from core client for 30 sec - exiting 05:16:25 (29010): No heartbeat from core client for 30 sec - exiting 05:21:59 (29040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:28:33 (29061): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:33:32 (29095): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:33:34 (29095): No heartbeat from core client for 30 sec - exiting 05:33:35 (29095): No heartbeat from core client for 30 sec - exiting 05:33:53 (29095): No heartbeat from core client for 30 sec - exiting 05:39:21 (29114): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:46:04 (29135): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:46:08 (29135): No heartbeat from core client for 30 sec - exiting 05:53:19 (29153): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:53:23 (29153): No heartbeat from core client for 30 sec - exiting 05:55:09 (29175): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:55:17 (29175): No heartbeat from core client for 30 sec - exiting 06:01:51 (29183): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:01:52 (29183): No heartbeat from core client for 30 sec - exiting 06:05:59 (29201): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:06:01 (29201): No heartbeat from core client for 30 sec - exiting 06:11:42 (29219): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:18:28 (29239): No heartbeat from core client for 30 sec - exiting 06:18:32 (29239): No heartbeat from core client for 30 sec - exiting 06:18:33 (29239): No heartbeat from core client for 30 sec - exiting 06:18:34 (29239): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:18:35 (29239): No heartbeat from core client for 30 sec - exiting 06:23:19 (29260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:23:22 (29260): No heartbeat from core client for 30 sec - exiting 06:23:23 (29260): No heartbeat from core client for 30 sec - exiting 06:23:24 (29260): No heartbeat from core client for 30 sec - exiting SIGABRT: abort called Stack trace (9 frames): /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb76e3400] [0xb76e3424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb75061df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb7509825] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb74f14d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29270, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb7760400] [0xb7760424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb75831df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb7586825] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb756e4d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29270, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb76f7400] [0xb76f7424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb751a1df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb751d825] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75054d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29270, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb77c8400] [0xb77c8424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb75eb1df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb75ee825] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75d64d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29270, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb7749400] [0xb7749424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb756c1df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb756f825] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75574d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29270, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xb77c3400] [0xb77c3424] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb75e61df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb75e9825] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /home/yacob/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb75d14d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=29270, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Feb 2014 14:16:32	1265980	16277603	hadcm3n_481n_2020_40_008402407_4	51,840	83,660	1.6138
05 Feb 2014 20:48:00	1265980	16277603	hadcm3n_481n_2020_40_008402407_4	25,920	41,940	1.6181