Task 15751190

Name	hadam3p_pnw_q9qi_2036_1_008357927_0
Workunit	8508786
Created	25 Apr 2013, 22:21:33 UTC
Sent	25 Apr 2013, 22:23:02 UTC
Report deadline	8 Apr 2014, 3:43:02 UTC
Received	22 May 2013, 22:04:14 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1257136
Run time	5 days 2 hours 47 min 51 sec
CPU time	5 days 2 hours 47 min 51 sec
Validate state	Invalid
Credit	2,004.61
Device peak FLOPS	1.75 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.6.36</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3328, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 01:48:13 (5376): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 01:48:14 (5376): No heartbeat from core client for 30 sec - exiting 01:48:15 (5376): No heartbeat from core client for 30 sec - exiting 01:48:16 (5376): No heartbeat from core client for 30 sec - exiting 09:15:17 (5288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:17:20 (4796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:26 (1444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:51:27 (3696): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 13:51:29 (3696): No heartbeat from core client for 30 sec - exiting 13:51:30 (3696): No heartbeat from core client for 30 sec - exiting 13:51:31 (3696): No heartbeat from core client for 30 sec - exiting 13:51:32 (3696): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:57:59 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:58:00 (4756): No heartbeat from core client for 30 sec - exiting 03:58:01 (4756): No heartbeat from core client for 30 sec - exiting 03:58:02 (4756): No heartbeat from core client for 30 sec - exiting 04:04:23 (5828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:04:24 (5828): No heartbeat from core client for 30 sec - exiting 04:05:06 (4440): No heartbeat from core client for 30 sec - exiting 04:05:07 (4440): No heartbeat from core client for 30 sec - exiting 04:05:08 (4440): No heartbeat from core client for 30 sec - exiting 04:05:09 (4440): No heartbeat from core client for 30 sec - exiting 04:05:10 (4440): No heartbeat from core client for 30 sec - exiting 04:05:12 (4440): No heartbeat from core client for 30 sec - exiting 04:05:13 (4440): No heartbeat from core client for 30 sec - exiting 04:05:14 (4440): No heartbeat from core client for 30 sec - exiting 04:05:15 (4440): No heartbeat from core client for 30 sec - exiting 04:05:16 (4440): No heartbeat from core client for 30 sec - exiting 04:05:17 (4440): No heartbeat from core client for 30 sec - exiting 04:05:18 (4440): No heartbeat from core client for 30 sec - exiting 04:05:19 (4440): No heartbeat from core client for 30 sec - exiting 04:05:20 (4440): No heartbeat from core client for 30 sec - exiting 04:05:21 (4440): No heartbeat from core client for 30 sec - exiting 04:05:22 (4440): No heartbeat from core client for 30 sec - exiting 04:05:24 (4440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:24:36 (5664): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 10:21:26 (4392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5348, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:30:22 (5664): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 06:30:24 (5664): No heartbeat from core client for 30 sec - exiting 09:55:56 (1480): No heartbeat from core client for 30 sec - exiting 09:55:57 (1480): No heartbeat from core client for 30 sec - exiting 09:55:58 (1480): No heartbeat from core client for 30 sec - exiting 09:56:00 (1480): No heartbeat from core client for 30 sec - exiting 09:56:01 (1480): No heartbeat from core client for 30 sec - exiting 09:56:02 (1480): No heartbeat from core client for 30 sec - exiting 09:56:03 (1480): No heartbeat from core client for 30 sec - exiting 09:56:04 (1480): No heartbeat from core client for 30 sec - exiting 09:56:05 (1480): No heartbeat from core client for 30 sec - exiting 09:56:06 (1480): No heartbeat from core client for 30 sec - exiting 09:56:07 (1480): No heartbeat from core client for 30 sec - exiting 09:56:08 (1480): No heartbeat from core client for 30 sec - exiting 09:56:09 (1480): No heartbeat from core client for 30 sec - exiting 09:56:11 (1480): No heartbeat from core client for 30 sec - exiting 09:56:12 (1480): No heartbeat from core client for 30 sec - exiting 09:56:13 (1480): No heartbeat from core client for 30 sec - exiting 09:56:14 (1480): No heartbeat from core client for 30 sec - exiting 09:56:15 (1480): No heartbeat from core client for 30 sec - exiting 09:56:16 (1480): No heartbeat from core client for 30 sec - exiting 09:56:17 (1480): No heartbeat from core client for 30 sec - exiting 09:56:18 (1480): No heartbeat from core client for 30 sec - exiting 09:56:19 (1480): No heartbeat from core client for 30 sec - exiting 09:56:20 (1480): No heartbeat from core client for 30 sec - exiting 09:56:21 (1480): No heartbeat from core client for 30 sec - exiting 09:56:23 (1480): No heartbeat from core client for 30 sec - exiting 09:56:24 (1480): No heartbeat from core client for 30 sec - exiting 09:56:25 (1480): No heartbeat from core client for 30 sec - exiting 09:56:26 (1480): No heartbeat from core client for 30 sec - exiting 09:56:27 (1480): No heartbeat from core client for 30 sec - exiting 09:56:28 (1480): No heartbeat from core client for 30 sec - exiting 09:56:29 (1480): No heartbeat from core client for 30 sec - exiting 09:56:30 (1480): No heartbeat from core client for 30 sec - exiting 09:56:31 (1480): No heartbeat from core client for 30 sec - exiting 09:56:32 (1480): No heartbeat from core client for 30 sec - exiting 09:56:33 (1480): No heartbeat from core client for 30 sec - exiting 09:56:35 (1480): No heartbeat from core client for 30 sec - exiting 09:56:36 (1480): No heartbeat from core client for 30 sec - exiting 09:56:37 (1480): No heartbeat from core client for 30 sec - exiting 09:56:38 (1480): No heartbeat from core client for 30 sec - exiting 09:56:39 (1480): No heartbeat from core client for 30 sec - exiting 09:56:40 (1480): No heartbeat from core client for 30 sec - exiting 09:56:41 (1480): No heartbeat from core client for 30 sec - exiting 09:56:42 (1480): No heartbeat from core client for 30 sec - exiting 09:56:43 (1480): No heartbeat from core client for 30 sec - exiting 09:56:44 (1480): No heartbeat from core client for 30 sec - exiting 09:56:45 (1480): No heartbeat from core client for 30 sec - exiting 09:56:47 (1480): No heartbeat from core client for 30 sec - exiting 09:56:48 (1480): No heartbeat from core client for 30 sec - exiting 09:56:49 (1480): No heartbeat from core client for 30 sec - exiting 09:56:50 (1480): No heartbeat from core client for 30 sec - exiting 09:56:51 (1480): No heartbeat from core client for 30 sec - exiting 09:56:52 (1480): No heartbeat from core client for 30 sec - exiting 09:56:53 (1480): No heartbeat from core client for 30 sec - exiting 09:56:54 (1480): No heartbeat from core client for 30 sec - exiting 09:56:55 (1480): No heartbeat from core client for 30 sec - exiting 09:56:56 (1480): No heartbeat from core client for 30 sec - exiting 09:56:58 (1480): No heartbeat from core client for 30 sec - exiting 09:56:59 (1480): No heartbeat from core client for 30 sec - exiting 09:57:00 (1480): No heartbeat from core client for 30 sec - exiting 09:57:01 (1480): No heartbeat from core client for 30 sec - exiting 09:57:02 (1480): No heartbeat from core client for 30 sec - exiting 09:57:03 (1480): No heartbeat from core client for 30 sec - exiting 09:57:04 (1480): No heartbeat from core client for 30 sec - exiting 09:57:05 (1480): No heartbeat from core client for 30 sec - exiting 09:57:06 (1480): No heartbeat from core client for 30 sec - exiting 09:57:07 (1480): No heartbeat from core client for 30 sec - exiting 09:57:08 (1480): No heartbeat from core client for 30 sec - exiting 09:57:10 (1480): No heartbeat from core client for 30 sec - exiting 09:57:11 (1480): No heartbeat from core client for 30 sec - exiting 09:57:12 (1480): No heartbeat from core client for 30 sec - exiting 09:57:13 (1480): No heartbeat from core client for 30 sec - exiting 09:57:14 (1480): No heartbeat from core client for 30 sec - exiting 09:57:15 (1480): No heartbeat from core client for 30 sec - exiting 09:57:16 (1480): No heartbeat from core client for 30 sec - exiting 09:57:17 (1480): No heartbeat from core client for 30 sec - exiting 09:57:18 (1480): No heartbeat from core client for 30 sec - exiting 09:57:19 (1480): No heartbeat from core client for 30 sec - exiting 09:57:20 (1480): No heartbeat from core client for 30 sec - exiting 09:57:22 (1480): No heartbeat from core client for 30 sec - exiting 09:57:23 (1480): No heartbeat from core client for 30 sec - exiting 09:57:24 (1480): No heartbeat from core client for 30 sec - exiting 09:57:25 (1480): No heartbeat from core client for 30 sec - exiting 09:57:26 (1480): No heartbeat from core client for 30 sec - exiting 09:57:27 (1480): No heartbeat from core client for 30 sec - exiting 09:57:28 (1480): No heartbeat from core client for 30 sec - exiting 09:57:29 (1480): No heartbeat from core client for 30 sec - exiting 09:57:30 (1480): No heartbeat from core client for 30 sec - exiting 09:57:32 (1480): No heartbeat from core client for 30 sec - exiting 09:57:33 (1480): No heartbeat from core client for 30 sec - exiting 09:57:34 (1480): No heartbeat from core client for 30 sec - exiting 09:57:35 (1480): No heartbeat from core client for 30 sec - exiting 09:57:36 (1480): No heartbeat from core client for 30 sec - exiting 09:57:37 (1480): No heartbeat from core client for 30 sec - exiting 09:57:38 (1480): No heartbeat from core client for 30 sec - exiting 09:57:39 (1480): No heartbeat from core client for 30 sec - exiting 09:57:40 (1480): No heartbeat from core client for 30 sec - exiting 09:57:41 (1480): No heartbeat from core client for 30 sec - exiting 09:57:42 (1480): No heartbeat from core client for 30 sec - exiting 09:57:43 (1480): No heartbeat from core client for 30 sec - exiting 09:57:45 (1480): No heartbeat from core client for 30 sec - exiting 09:57:46 (1480): No heartbeat from core client for 30 sec - exiting 09:57:47 (1480): No heartbeat from core client for 30 sec - exiting 09:57:48 (1480): No heartbeat from core client for 30 sec - exiting 09:57:49 (1480): No heartbeat from core client for 30 sec - exiting 09:57:50 (1480): No heartbeat from core client for 30 sec - exiting 09:57:51 (1480): No heartbeat from core client for 30 sec - exiting 09:57:52 (1480): No heartbeat from core client for 30 sec - exiting 09:57:53 (1480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4504, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4692, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:36:23 (4260): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 02:36:25 (4260): No heartbeat from core client for 30 sec - exiting 02:59:10 (5352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:20:59 (4932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:28:34 (4568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:29:13 (5012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:53:57 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:58:29 (2364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:08:02 (4180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:13:19 (5268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:22:30 (4232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:27:37 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1148, selfPID=1148, iMonCtr=2 04:43:01 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:54:34 (2124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:04:14 (3816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:10:15 (4036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:12:38 (4648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:27:54 (4300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:34:28 (5296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:43:21 (4476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5944, selfPID=5944, iMonCtr=2 05:53:43 (5644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5712, selfPID=5712, iMonCtr=2 05:59:38 (5908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:08:40 (5280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:11:03 (3640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:14:43 (5180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:32:19 (4392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:34:42 (6036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3064, selfPID=3064, iMonCtr=2 06:36:49 (2044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:42:45 (4080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:45:08 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:50:15 (1448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5156, selfPID=5156, iMonCtr=2 07:16:31 (4916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:22:33 (5000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:32:18 (4436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:32:57 (5548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:49:47 (5256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:43:53 (3200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:57:13 (4860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:04:45 (4676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5248, selfPID=5248, iMonCtr=2 09:05:24 (4108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:11:13 (4084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5544, selfPID=5544, iMonCtr=2 09:18:30 (2156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4612, selfPID=4612, iMonCtr=2 10:03:57 (4460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:33:11 (2096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:47:09 (3920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:51:08 (4560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3596, selfPID=3596, iMonCtr=2 11:05:14 (4844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:38:12 (3088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:36:53 (5696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:41:37 (5648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3672, selfPID=3672, iMonCtr=2 12:46:21 (3800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:24:10 (3884): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 11:24:12 (3884): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:33:42 (1248): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 08:33:43 (1248): No heartbeat from core client for 30 sec - exiting 08:33:44 (1248): No heartbeat from core client for 30 sec - exiting 19:11:19 (940): No heartbeat from core client for 30 sec - exiting 19:11:20 (940): No heartbeat from core client for 30 sec - exiting 19:11:21 (940): No heartbeat from core client for 30 sec - exiting 19:11:22 (940): No heartbeat from core client for 30 sec - exiting 19:11:23 (940): No heartbeat from core client for 30 sec - exiting 19:11:24 (940): No heartbeat from core client for 30 sec - exiting 19:11:25 (940): No heartbeat from core client for 30 sec - exiting 19:11:27 (940): No heartbeat from core client for 30 sec - exiting 19:11:28 (940): No heartbeat from core client for 30 sec - exiting 19:11:29 (940): No heartbeat from core client for 30 sec - exiting 19:11:30 (940): No heartbeat from core client for 30 sec - exiting 19:11:31 (940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:30:00 (1996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:30:01 (1996): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4196, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2360, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3520, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6116, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:06:22 (4844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:07:59 (4620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:09:50 (2792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:10:28 (3976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5020, selfPID=5020, iMonCtr=2 04:11:29 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2628, selfPID=2628, iMonCtr=2 04:13:08 (2116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:14:44 (4380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:16:23 (5216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:18:01 (5452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:19:52 (4708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:21:29 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:23:07 (4912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:24:44 (4968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5784, selfPID=5784, iMonCtr=2 04:26:21 (2376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:28:01 (488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:29:40 (4200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:18 (4884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:33:08 (2352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5328, selfPID=5328, iMonCtr=2 04:34:47 (4528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:36:25 (2192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4556, selfPID=4556, iMonCtr=2 04:38:04 (5536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5956, selfPID=5956, iMonCtr=2 04:39:42 (2860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:41:18 (1504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:43:09 (1356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:44:52 (2988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:46:31 (4068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:48:10 (4628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:49:48 (4032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:51:26 (4800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:53:05 (5468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5756, selfPID=5756, iMonCtr=2 04:54:55 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:56:34 (3976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:58:16 (748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:59:52 (2640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:01:31 (6136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:03:09 (3392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4384, selfPID=4384, iMonCtr=2 05:04:47 (4860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4332, selfPID=4332, iMonCtr=2 05:06:26 (1736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:08:04 (2692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5928, selfPID=5928, iMonCtr=2 05:09:43 (5984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:11:22 (5136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:13:00 (3976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:14:51 (6060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:16:30 (964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:18:09 (2604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:19:48 (1336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:21:25 (4088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:23:04 (6060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:24:49 (4608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4320, selfPID=4320, iMonCtr=2 05:26:27 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2756, selfPID=2756, iMonCtr=2 05:28:05 (1576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:29:48 (3388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:31:24 (4852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:33:03 (5920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:34:41 (2192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:36:20 (5908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:37:57 (6108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:39:54 (4360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:41:33 (4052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:43:12 (4900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:44:51 (5992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:46:28 (3140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4124, selfPID=4124, iMonCtr=2 05:48:06 (2576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:49:45 (2640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:51:23 (6124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:53:05 (5224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:54:47 (532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:56:22 (4936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:58:01 (1304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:59:40 (2012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:01:18 (5804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:02:57 (4092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:04:35 (4684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:06:24 (5540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:08:04 (4600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:09:43 (2756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:11:21 (5848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:13:01 (5128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:14:39 (2220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5520, selfPID=5520, iMonCtr=2 06:16:16 (1508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:17:54 (5908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:19:33 (4432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:21:10 (5940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:22:49 (6064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:24:28 (5608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:26:10 (2320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1952, selfPID=1952, iMonCtr=2 06:27:45 (5644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:29:25 (5380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:31:01 (4280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:32:52 (4160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2400, selfPID=2400, iMonCtr=2 06:34:31 (5776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1292, selfPID=1292, iMonCtr=2 06:36:09 (5512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:37:48 (4552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:39:26 (1408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=532, selfPID=532, iMonCtr=2 06:41:14 (3968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:42:54 (3676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:44:31 (4024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:46:11 (4712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=912, selfPID=912, iMonCtr=2 06:47:54 (4108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5248, selfPID=5248, iMonCtr=2 06:49:29 (4276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2016, selfPID=2016, iMonCtr=2 06:51:09 (6108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 May 2013 01:49:41	1257136	15751190	hadam3p_pnw_q9qi_2036_1_008357927_0	92,256	416,952	4.5195
16 May 2013 01:54:45	1257136	15751190	hadam3p_pnw_q9qi_2036_1_008357927_0	80,736	366,939	4.5449
12 May 2013 21:58:25	1257136	15751190	hadam3p_pnw_q9qi_2036_1_008357927_0	69,216	313,303	4.5265
09 May 2013 01:18:56	1257136	15751190	hadam3p_pnw_q9qi_2036_1_008357927_0	57,696	256,379	4.4436
06 May 2013 19:25:46	1257136	15751190	hadam3p_pnw_q9qi_2036_1_008357927_0	46,176	204,666	4.4323
04 May 2013 20:26:02	1257136	15751190	hadam3p_pnw_q9qi_2036_1_008357927_0	34,656	153,064	4.4167
02 May 2013 10:51:26	1257136	15751190	hadam3p_pnw_q9qi_2036_1_008357927_0	23,136	102,110	4.4135
30 Apr 2013 04:49:14	1257136	15751190	hadam3p_pnw_q9qi_2036_1_008357927_0	11,616	51,728	4.4532