Task 13122719

Name	hadcm3n_ykhy_1900_40_007359424_0
Workunit	7556854
Created	6 Jul 2011, 15:06:11 UTC
Sent	7 Jul 2011, 23:44:48 UTC
Report deadline	7 Oct 2011, 7:11:59 UTC
Received	25 Oct 2011, 14:51:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1139716
Run time	86 days 4 hours 26 min 19 sec
CPU time	83 days 22 hours 3 min 11 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	0.65 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 13:54:26 (5948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:34 (5176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:31:11 (4676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:31:12 (4676): No heartbeat from core client for 30 sec - exiting 04:24:32 (5276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:28:34 (5236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1924, iMonCtr=1 Model crash detected, will try to restart... 00:00:41 (4352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:50:10 (3144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:50:11 (3144): No heartbeat from core client for 30 sec - exiting 13:02:53 (3676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:02:54 (3676): No heartbeat from core client for 30 sec - exiting 14:35:51 (5736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:35:52 (5736): No heartbeat from core client for 30 sec - exiting 14:35:53 (5736): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4560, iMonCtr=1 Model crash detected, will try to restart... 09:57:51 (2572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:57:52 (2572): No heartbeat from core client for 30 sec - exiting 01:39:01 (6104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:31:11 (5100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:17:49 (3524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:03:28 (5384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:03:30 (5384): No heartbeat from core client for 30 sec - exiting 02:29:40 (2068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:16:33 (5200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:16:34 (5200): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 18:09:17 (6080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:09:18 (6080): No heartbeat from core client for 30 sec - exiting 00:32:28 (5448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:07:56 (2352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:07:57 (2352): No heartbeat from core client for 30 sec - exiting 10:03:50 (3792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:03:51 (3792): No heartbeat from core client for 30 sec - exiting 17:51:17 (5284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:08:29 (5964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:08:30 (5964): No heartbeat from core client for 30 sec - exiting 01:42:30 (5536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:57:50 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:57:51 (5428): No heartbeat from core client for 30 sec - exiting 09:31:48 (3512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:31:49 (3512): No heartbeat from core client for 30 sec - exiting 15:47:47 (5136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:47:48 (5136): No heartbeat from core client for 30 sec - exiting 17:23:58 (5436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:23:59 (5436): No heartbeat from core client for 30 sec - exiting 01:23:06 (4932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:23:07 (4932): No heartbeat from core client for 30 sec - exiting 01:23:08 (4932): No heartbeat from core client for 30 sec - exiting 09:13:00 (3164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:01 (3164): No heartbeat from core client for 30 sec - exiting 15:26:53 (4940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:26:54 (4940): No heartbeat from core client for 30 sec - exiting 17:03:01 (4780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:27:19 (5900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:27:20 (5900): No heartbeat from core client for 30 sec - exiting 23:27:21 (5900): No heartbeat from core client for 30 sec - exiting 01:01:31 (252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:01:32 (252): No heartbeat from core client for 30 sec - exiting 07:19:01 (6756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:19:03 (6756): No heartbeat from core client for 30 sec - exiting 08:54:56 (7272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:54:57 (7272): No heartbeat from core client for 30 sec - exiting 10:06:30 (7640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:17:55 (8028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:17:57 (8028): No heartbeat from core client for 30 sec - exiting 16:51:32 (7988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:51:34 (7988): No heartbeat from core client for 30 sec - exiting 23:05:59 (7656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:41:13 (3972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:41:14 (3972): No heartbeat from core client for 30 sec - exiting 07:01:20 (6852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:01:21 (6852): No heartbeat from core client for 30 sec - exiting 07:01:22 (6852): No heartbeat from core client for 30 sec - exiting 08:36:54 (7264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:23 (6216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:25 (6216): No heartbeat from core client for 30 sec - exiting 22:50:50 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:25:54 (3484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:25:56 (3484): No heartbeat from core client for 30 sec - exiting 01:03:40 (7632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:03:41 (7632): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7228, iMonCtr=1 Model crash detected, will try to restart... 00:03:22 (5232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:56:23 (3080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:36:13 (5204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:36:14 (5204): No heartbeat from core client for 30 sec - exiting 22:37:37 (4252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:11:17 (3248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:30:52 (5588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:30:53 (5588): No heartbeat from core client for 30 sec - exiting 12:35:06 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:35:08 (4336): No heartbeat from core client for 30 sec - exiting 12:35:09 (4336): No heartbeat from core client for 30 sec - exiting 13:37:36 (4504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:12:11 (5880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:30:01 (6124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:30:03 (6124): No heartbeat from core client for 30 sec - exiting 21:30:04 (6124): No heartbeat from core client for 30 sec - exiting 06:57:42 (5872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:57:43 (5872): No heartbeat from core client for 30 sec - exiting 06:57:44 (5872): No heartbeat from core client for 30 sec - exiting 22:35:48 (1028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:12:30 (3108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:12:31 (3108): No heartbeat from core client for 30 sec - exiting 06:37:48 (5632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:37:50 (5632): No heartbeat from core client for 30 sec - exiting 08:11:53 (7252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:11:54 (7252): No heartbeat from core client for 30 sec - exiting 16:00:56 (4308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:00:58 (4308): No heartbeat from core client for 30 sec - exiting 23:44:52 (8064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:44:54 (8064): No heartbeat from core client for 30 sec - exiting 06:00:54 (7740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:56 (7740): No heartbeat from core client for 30 sec - exiting 07:36:24 (7736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:57:39 (7168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:57:40 (7168): No heartbeat from core client for 30 sec - exiting 15:31:02 (4624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:31:04 (4624): No heartbeat from core client for 30 sec - exiting 15:31:05 (4624): No heartbeat from core client for 30 sec - exiting 23:16:32 (5192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:28:23 (3452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:28:26 (3452): No heartbeat from core client for 30 sec - exiting 05:28:27 (3452): No heartbeat from core client for 30 sec - exiting 07:02:13 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:50:04 (6568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:00:55 (7832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:00:57 (7832): No heartbeat from core client for 30 sec - exiting 22:34:17 (6788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:34:18 (6788): No heartbeat from core client for 30 sec - exiting 02:27:30 (6772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:27:31 (6772): No heartbeat from core client for 30 sec - exiting 06:14:55 (6804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:14:56 (6804): No heartbeat from core client for 30 sec - exiting 07:49:18 (2828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6368, iMonCtr=1 Model crash detected, will try to restart... 06:21:52 (4272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:21:53 (4272): No heartbeat from core client for 30 sec - exiting 06:21:54 (4272): No heartbeat from core client for 30 sec - exiting 08:24:03 (868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4452, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4424, iMonCtr=1 Model crash detected, will try to restart... 06:04:17 (3360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:04:19 (3360): No heartbeat from core client for 30 sec - exiting 12:23:45 (5952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:59:54 (5708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:59:56 (5708): No heartbeat from core client for 30 sec - exiting 21:57:16 (3240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:17 (3240): No heartbeat from core client for 30 sec - exiting 05:54:17 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:54:19 (4788): No heartbeat from core client for 30 sec - exiting 13:37:04 (3888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:37:06 (3888): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 12:50:09 (3320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:50:11 (3320): No heartbeat from core client for 30 sec - exiting 12:50:12 (3320): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1028, iMonCtr=1 Model crash detected, will try to restart... 09:11:13 (3616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:11:14 (3616): No heartbeat from core client for 30 sec - exiting 09:11:16 (3616): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5696, iMonCtr=1 Model crash detected, will try to restart... 21:48:35 (4512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:48:37 (4512): No heartbeat from core client for 30 sec - exiting 21:48:38 (4512): No heartbeat from core client for 30 sec - exiting 21:48:40 (4512): No heartbeat from core client for 30 sec - exiting 22:05:23 (5860): No heartbeat from core client for 30 sec - exiting 22:05:36 (5860): No heartbeat from core client for 30 sec - exiting 22:05:37 (5860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:05:38 (5860): No heartbeat from core client for 30 sec - exiting 00:23:07 (4676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:20:02 (6036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:20:03 (6036): No heartbeat from core client for 30 sec - exiting 06:20:04 (6036): No heartbeat from core client for 30 sec - exiting 06:34:21 (1108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:01:43 (1592): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 22:53:28 (5780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:53:29 (5780): No heartbeat from core client for 30 sec - exiting 07:23:27 (900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:23:29 (900): No heartbeat from core client for 30 sec - exiting 07:23:30 (900): No heartbeat from core client for 30 sec - exiting 11:12:33 (4876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:12:36 (4876): No heartbeat from core client for 30 sec - exiting 11:12:37 (4876): No heartbeat from core client for 30 sec - exiting 11:12:38 (4876): No heartbeat from core client for 30 sec - exiting 11:15:39 (2300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:17:35 (1128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:55:40 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:55:41 (5988): No heartbeat from core client for 30 sec - exiting 09:37:12 (1516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:37:14 (1516): No heartbeat from core client for 30 sec - exiting 09:37:15 (1516): No heartbeat from core client for 30 sec - exiting 09:37:16 (1516): No heartbeat from core client for 30 sec - exiting 09:37:17 (1516): No heartbeat from core client for 30 sec - exiting 09:37:18 (1516): No heartbeat from core client for 30 sec - exiting 09:39:31 (3260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:41:52 (5244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:14:00 (1084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:14:03 (1084): No heartbeat from core client for 30 sec - exiting 22:14:04 (1084): No heartbeat from core client for 30 sec - exiting 22:14:05 (1084): No heartbeat from core client for 30 sec - exiting 22:14:06 (1084): No heartbeat from core client for 30 sec - exiting 10:46:13 (5840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:46:15 (5840): No heartbeat from core client for 30 sec - exiting 10:46:16 (5840): No heartbeat from core client for 30 sec - exiting 10:48:34 (7744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:36:28 (6488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:36:29 (6488): No heartbeat from core client for 30 sec - exiting 14:36:30 (6488): No heartbeat from core client for 30 sec - exiting 14:38:52 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:08 (6228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:11 (6228): No heartbeat from core client for 30 sec - exiting 19:19:13 (6228): No heartbeat from core client for 30 sec - exiting 19:19:14 (6228): No heartbeat from core client for 30 sec - exiting 19:19:15 (6228): No heartbeat from core client for 30 sec - exiting 19:19:16 (6228): No heartbeat from core client for 30 sec - exiting 19:23:04 (1376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:26:27 (4084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:26:29 (4084): No heartbeat from core client for 30 sec - exiting 19:26:30 (4084): No heartbeat from core client for 30 sec - exiting 19:29:35 (6728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:18 (4712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:32:20 (4712): No heartbeat from core client for 30 sec - exiting 19:32:21 (4712): No heartbeat from core client for 30 sec - exiting 19:32:22 (4712): No heartbeat from core client for 30 sec - exiting 19:32:23 (4712): No heartbeat from core client for 30 sec - exiting 19:35:43 (5860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:38:20 (7604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:38:21 (7604): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5160, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5392, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6748, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5060, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4972, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3324, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1040, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1704, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3560, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5108, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3852, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3544, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4676, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3608, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3288, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3288, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3680, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3680, iMonCtr=1 Model crash detected, will try to restart... 02:11:47 (4812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:11:49 (4812): No heartbeat from core client for 30 sec - exiting 02:11:50 (4812): No heartbeat from core client for 30 sec - exiting 02:11:51 (4812): No heartbeat from core client for 30 sec - exiting 02:11:53 (4812): No heartbeat from core client for 30 sec - exiting 02:11:54 (4812): No heartbeat from core client for 30 sec - exiting 02:11:55 (4812): No heartbeat from core client for 30 sec - exiting 02:11:56 (4812): No heartbeat from core client for 30 sec - exiting 02:11:57 (4812): No heartbeat from core client for 30 sec - exiting 09:07:40 (3432): No heartbeat from core client for 30 sec - exiting 09:07:41 (3432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:07:42 (3432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4516, iMonCtr=1 Model crash detected, will try to restart... 15:21:32 (6080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2420, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Oct 2011 13:07:22	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	777,600	7,250,560	9.3243
12 Oct 2011 16:37:28	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	751,680	7,002,352	9.3156
09 Oct 2011 19:35:37	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	725,760	6,757,051	9.3103
07 Oct 2011 02:08:05	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	699,840	6,525,042	9.3236
04 Oct 2011 05:18:22	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	673,920	6,282,716	9.3226
01 Oct 2011 08:08:02	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	648,000	6,037,113	9.3165
28 Sep 2011 09:44:23	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	622,080	5,789,424	9.3066
25 Sep 2011 10:06:28	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	596,160	5,535,715	9.2856
22 Sep 2011 10:29:09	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	570,240	5,282,877	9.2643
17 Sep 2011 06:54:20	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	544,320	5,032,032	9.2446
14 Sep 2011 08:32:10	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	518,400	4,785,950	9.2322
10 Sep 2011 09:53:22	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	492,480	4,559,270	9.2578
07 Sep 2011 22:54:42	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	466,560	4,347,913	9.3191
05 Sep 2011 03:09:05	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	440,640	4,107,556	9.3218
01 Sep 2011 19:59:19	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	414,720	3,862,984	9.3147
30 Aug 2011 05:19:51	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	388,800	3,639,710	9.3614
26 Aug 2011 13:49:01	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	362,880	3,411,206	9.4004
24 Aug 2011 00:10:39	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	336,960	3,191,262	9.4707
21 Aug 2011 13:25:17	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	311,040	2,980,721	9.5831
14 Aug 2011 15:00:42	1139716	13122719	hadcm3n_ykhy_1900_40_007359424_0	285,120	2,733,497	9.5872