Task 16280915

Name	hadcm3n_7e2g_1980_40_008429515_3
Workunit	8580371
Created	6 Feb 2014, 20:21:04 UTC
Sent	6 Feb 2014, 20:21:14 UTC
Report deadline	11 Aug 2023, 1:41:14 UTC
Received	14 Apr 2014, 18:33:51 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1087624
Run time	8 days 2 hours 29 min 59 sec
CPU time	7 days 22 hours 24 min 21 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.16 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 22:34:02 (4076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:13:47 (6016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:41:38 (3960): No heartbeat from core client for 30 sec - exiting 12:41:39 (3960): No heartbeat from core client for 30 sec - exiting 12:41:40 (3960): No heartbeat from core client for 30 sec - exiting 12:41:41 (3960): No heartbeat from core client for 30 sec - exiting 12:41:42 (3960): No heartbeat from core client for 30 sec - exiting 12:41:43 (3960): No heartbeat from core client for 30 sec - exiting 12:41:44 (3960): No heartbeat from core client for 30 sec - exiting 12:41:45 (3960): No heartbeat from core client for 30 sec - exiting 12:41:46 (3960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:51 (1656): No heartbeat from core client for 30 sec - exiting 19:13:52 (1656): No heartbeat from core client for 30 sec - exiting 19:13:53 (1656): No heartbeat from core client for 30 sec - exiting 19:13:54 (1656): No heartbeat from core client for 30 sec - exiting 19:13:55 (1656): No heartbeat from core client for 30 sec - exiting 19:13:56 (1656): No heartbeat from core client for 30 sec - exiting 19:13:57 (1656): No heartbeat from core client for 30 sec - exiting 19:13:58 (1656): No heartbeat from core client for 30 sec - exiting 19:13:59 (1656): No heartbeat from core client for 30 sec - exiting 19:14:00 (1656): No heartbeat from core client for 30 sec - exiting 19:14:01 (1656): No heartbeat from core client for 30 sec - exiting 19:14:02 (1656): No heartbeat from core client for 30 sec - exiting 19:14:03 (1656): No heartbeat from core client for 30 sec - exiting 19:14:04 (1656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C19:40:41 (8172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:51:51 (4772): No heartbeat from core client for 30 sec - exiting 18:51:52 (4772): No heartbeat from core client for 30 sec - exiting 18:51:53 (4772): No heartbeat from core client for 30 sec - exiting 18:51:54 (4772): No heartbeat from core client for 30 sec - exiting 18:51:55 (4772): No heartbeat from core client for 30 sec - exiting 18:51:56 (4772): No heartbeat from core client for 30 sec - exiting 18:51:57 (4772): No heartbeat from core client for 30 sec - exiting 18:51:58 (4772): No heartbeat from core client for 30 sec - exiting 18:51:59 (4772): No heartbeat from core client for 30 sec - exiting 18:52:00 (4772): No heartbeat from core client for 30 sec - exiting 18:52:01 (4772): No heartbeat from core client for 30 sec - exiting 18:52:02 (4772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:14 (10156): No heartbeat from core client for 30 sec - exiting 19:06:15 (10156): No heartbeat from core client for 30 sec - exiting 19:06:16 (10156): No heartbeat from core client for 30 sec - exiting 19:06:17 (10156): No heartbeat from core client for 30 sec - exiting 19:06:18 (10156): No heartbeat from core client for 30 sec - exiting 19:06:19 (10156): No heartbeat from core client for 30 sec - exiting 19:06:20 (10156): No heartbeat from core client for 30 sec - exiting 19:06:21 (10156): No heartbeat from core client for 30 sec - exiting 19:06:22 (10156): No heartbeat from core client for 30 sec - exiting 19:06:23 (10156): No heartbeat from core client for 30 sec - exiting 19:06:24 (10156): No heartbeat from core client for 30 sec - exiting 19:06:25 (10156): No heartbeat from core client for 30 sec - exiting 19:06:26 (10156): No heartbeat from core client for 30 sec - exiting 19:06:27 (10156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:28 (7008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:20:58 (3528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:45:23 (4092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/7e2gko.pji4c10 Error converting file to netcdf: dataout/7e2gko.pii4c10 Error converting file to netcdf: dataout/7e2gko.pfi4c10 Error converting file to netcdf: dataout/7e2gka.phi4c10 Error converting file to netcdf: dataout/7e2gka.pgi4c10 19:09:57 (6016): No heartbeat from core client for 30 sec - exiting 19:09:58 (6016): No heartbeat from core client for 30 sec - exiting 19:09:59 (6016): No heartbeat from core client for 30 sec - exiting 19:10:00 (6016): No heartbeat from core client for 30 sec - exiting 19:10:01 (6016): No heartbeat from core client for 30 sec - exiting 19:10:02 (6016): No heartbeat from core client for 30 sec - exiting 19:10:03 (6016): No heartbeat from core client for 30 sec - exiting 19:10:04 (6016): No heartbeat from core client for 30 sec - exiting 19:10:05 (6016): No heartbeat from core client for 30 sec - exiting 19:10:06 (6016): No heartbeat from core client for 30 sec - exiting 19:10:07 (6016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/7e2gko.pji4c10 Error converting file to netcdf: dataout/7e2gko.pii4c10 Error converting file to netcdf: dataout/7e2gko.pfi4c10 Error converting file to netcdf: dataout/7e2gka.phi4c10 Error converting file to netcdf: dataout/7e2gka.pgi4c10 Error converting file to netcdf: dataout/7e2gka.pei4c10 Error converting file to netcdf: dataout/7e2gka.pdi4c10 20:03:33 (6824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:13:02 (3304): No heartbeat from core client for 30 sec - exiting 20:13:03 (3304): No heartbeat from core client for 30 sec - exiting 20:13:04 (3304): No heartbeat from core client for 30 sec - exiting 20:13:05 (3304): No heartbeat from core client for 30 sec - exiting 20:13:06 (3304): No heartbeat from core client for 30 sec - exiting 20:13:07 (3304): No heartbeat from core client for 30 sec - exiting 20:13:08 (3304): No heartbeat from core client for 30 sec - exiting 20:13:09 (3304): No heartbeat from core client for 30 sec - exiting 20:13:10 (3304): No heartbeat from core client for 30 sec - exiting 20:13:11 (3304): No heartbeat from core client for 30 sec - exiting 20:13:12 (3304): No heartbeat from core client for 30 sec - exiting 20:13:13 (3304): No heartbeat from core client for 30 sec - exiting 20:13:14 (3304): No heartbeat from core client for 30 sec - exiting 20:13:15 (3304): No heartbeat from core client for 30 sec - exiting 20:13:16 (3304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:13:17 (3304): No heartbeat from core client for 30 sec - exiting 20:01:42 (3844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C19:41:23 (6372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1984, iMonCtr=1 Model crash detected, will try to restart... 20:11:26 (1540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:46:49 (6488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:56:44 (2964): No heartbeat from core client for 30 sec - exiting 11:56:45 (2964): No heartbeat from core client for 30 sec - exiting 11:56:46 (2964): No heartbeat from core client for 30 sec - exiting 11:56:47 (2964): No heartbeat from core client for 30 sec - exiting 11:56:48 (2964): No heartbeat from core client for 30 sec - exiting 11:56:49 (2964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:56:50 (2964): No heartbeat from core client for 30 sec - exiting 11:56:51 (2964): No heartbeat from core client for 30 sec - exiting 11:57:27 (8536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:02:28 (9128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:23:11 (4252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7464, iMonCtr=1 Model crash detected, will try to restart... 22:02:03 (8564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:18:16 (4248): No heartbeat from core client for 30 sec - exiting 20:18:17 (4248): No heartbeat from core client for 30 sec - exiting 20:18:18 (4248): No heartbeat from core client for 30 sec - exiting 20:18:19 (4248): No heartbeat from core client for 30 sec - exiting 20:18:20 (4248): No heartbeat from core client for 30 sec - exiting 20:18:21 (4248): No heartbeat from core client for 30 sec - exiting 20:18:22 (4248): No heartbeat from core client for 30 sec - exiting 20:18:23 (4248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:03:08 (6880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10028, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9752, iMonCtr=1 Model crash detected, will try to restart... 11:24:15 (6532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:56:55 (10100): No heartbeat from core client for 30 sec - exiting 21:56:56 (10100): No heartbeat from core client for 30 sec - exiting 21:56:57 (10100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:06:54 (8264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:07:34 (1728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:07:29 (1876): No heartbeat from core client for 30 sec - exiting 16:07:30 (1876): No heartbeat from core client for 30 sec - exiting 16:07:31 (1876): No heartbeat from core client for 30 sec - exiting 16:07:32 (1876): No heartbeat from core client for 30 sec - exiting 16:07:33 (1876): No heartbeat from core client for 30 sec - exiting 16:07:34 (1876): No heartbeat from core client for 30 sec - exiting 16:07:35 (1876): No heartbeat from core client for 30 sec - exiting 16:07:36 (1876): No heartbeat from core client for 30 sec - exiting 16:07:37 (1876): No heartbeat from core client for 30 sec - exiting 16:07:38 (1876): No heartbeat from core client for 30 sec - exiting 16:07:39 (1876): No heartbeat from core client for 30 sec - exiting 16:07:40 (1876): No heartbeat from core client for 30 sec - exiting 16:07:41 (1876): No heartbeat from core client for 30 sec - exiting 16:07:42 (1876): No heartbeat from core client for 30 sec - exiting 16:07:43 (1876): No heartbeat from core client for 30 sec - exiting 16:07:44 (1876): No heartbeat from core client for 30 sec - exiting 16:07:45 (1876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:07:46 (1876): No heartbeat from core client for 30 sec - exiting 21:42:38 (5960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:53:56 (8696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:13:10 (7172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:17:08 (5988): No heartbeat from core client for 30 sec - exiting 20:17:09 (5988): No heartbeat from core client for 30 sec - exiting 20:17:10 (5988): No heartbeat from core client for 30 sec - exiting 20:17:11 (5988): No heartbeat from core client for 30 sec - exiting 20:17:12 (5988): No heartbeat from core client for 30 sec - exiting 20:17:13 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9804, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:05:13 (8816): No heartbeat from core client for 30 sec - exiting 16:05:15 (8816): No heartbeat from core client for 30 sec - exiting 16:05:16 (8816): No heartbeat from core client for 30 sec - exiting 16:05:17 (8816): No heartbeat from core client for 30 sec - exiting 16:05:18 (8816): No heartbeat from core client for 30 sec - exiting 16:05:19 (8816): No heartbeat from core client for 30 sec - exiting 16:05:20 (8816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:06:25 (8676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:15:12 (9408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:15:51 (4816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6284, iMonCtr=1 Model crash detected, will try to restart... 19:59:05 (8796): No heartbeat from core client for 30 sec - exiting 19:59:06 (8796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:59:49 (6220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:55:07 (7036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:01:05 (4312): No heartbeat from core client for 30 sec - exiting 08:01:06 (4312): No heartbeat from core client for 30 sec - exiting 08:01:07 (4312): No heartbeat from core client for 30 sec - exiting 08:01:08 (4312): No heartbeat from core client for 30 sec - exiting 08:01:09 (4312): No heartbeat from core client for 30 sec - exiting 08:01:10 (4312): No heartbeat from core client for 30 sec - exiting 08:01:11 (4312): No heartbeat from core client for 30 sec - exiting 08:01:12 (4312): No heartbeat from core client for 30 sec - exiting 08:01:13 (4312): No heartbeat from core client for 30 sec - exiting 08:01:14 (4312): No heartbeat from core client for 30 sec - exiting 08:01:15 (4312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:55:07 (4176): No heartbeat from core client for 30 sec - exiting 19:55:08 (4176): No heartbeat from core client for 30 sec - exiting 19:55:09 (4176): No heartbeat from core client for 30 sec - exiting 19:55:10 (4176): No heartbeat from core client for 30 sec - exiting 19:55:11 (4176): No heartbeat from core client for 30 sec - exiting 19:55:12 (4176): No heartbeat from core client for 30 sec - exiting 19:55:13 (4176): No heartbeat from core client for 30 sec - exiting 19:55:14 (4176): No heartbeat from core client for 30 sec - exiting 19:55:15 (4176): No heartbeat from core client for 30 sec - exiting 19:55:16 (4176): No heartbeat from core client for 30 sec - exiting 19:55:17 (4176): No heartbeat from core client for 30 sec - exiting 19:55:18 (4176): No heartbeat from core client for 30 sec - exiting 19:55:19 (4176): No heartbeat from core client for 30 sec - exiting 19:55:20 (4176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:54:37 (8756): No heartbeat from core client for 30 sec - exiting 19:54:38 (8756): No heartbeat from core client for 30 sec - exiting 19:54:39 (8756): No heartbeat from core client for 30 sec - exiting 19:54:40 (8756): No heartbeat from core client for 30 sec - exiting 19:54:41 (8756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4968, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9728, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9728, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9728, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9728, iMonCtr=1 Model crash detected, will try to restart... 19:29:43 (8920): No heartbeat from core client for 30 sec - exiting 19:29:44 (8920): No heartbeat from core client for 30 sec - exiting 19:29:45 (8920): No heartbeat from core client for 30 sec - exiting 19:29:46 (8920): No heartbeat from core client for 30 sec - exiting 19:29:47 (8920): No heartbeat from core client for 30 sec - exiting 19:29:48 (8920): No heartbeat from core client for 30 sec - exiting 19:29:49 (8920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:53:50 (7552): No heartbeat from core client for 30 sec - exiting 19:53:51 (7552): No heartbeat from core client for 30 sec - exiting 19:53:52 (7552): No heartbeat from core client for 30 sec - exiting 19:53:53 (7552): No heartbeat from core client for 30 sec - exiting 19:53:54 (7552): No heartbeat from core client for 30 sec - exiting 19:53:55 (7552): No heartbeat from core client for 30 sec - exiting 19:53:56 (7552): No heartbeat from core client for 30 sec - exiting 19:53:57 (7552): No heartbeat from core client for 30 sec - exiting 19:53:58 (7552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:53:59 (7552): No heartbeat from core client for 30 sec - exiting 19:54:00 (7552): No heartbeat from core client for 30 sec - exiting 19:54:01 (7552): No heartbeat from core client for 30 sec - exiting 19:54:02 (7552): No heartbeat from core client for 30 sec - exiting 19:54:03 (7552): No heartbeat from core client for 30 sec - exiting 19:54:04 (7552): No heartbeat from core client for 30 sec - exiting 19:54:05 (7552): No heartbeat from core client for 30 sec - exiting 19:54:06 (7552): No heartbeat from core client for 30 sec - exiting 19:54:07 (7552): No heartbeat from core client for 30 sec - exiting 19:54:08 (7552): No heartbeat from core client for 30 sec - exiting 20:16:02 (3260): No heartbeat from core client for 30 sec - exiting 20:16:03 (3260): No heartbeat from core client for 30 sec - exiting 20:16:04 (3260): No heartbeat from core client for 30 sec - exiting 20:16:05 (3260): No heartbeat from core client for 30 sec - exiting 20:16:06 (3260): No heartbeat from core client for 30 sec - exiting 20:16:08 (3260): No heartbeat from core client for 30 sec - exiting 20:16:09 (3260): No heartbeat from core client for 30 sec - exiting 20:16:10 (3260): No heartbeat from core client for 30 sec - exiting 20:16:11 (3260): No heartbeat from core client for 30 sec - exiting 20:16:12 (3260): No heartbeat from core client for 30 sec - exiting 20:16:13 (3260): No heartbeat from core client for 30 sec - exiting 20:16:14 (3260): No heartbeat from core client for 30 sec - exiting 20:16:15 (3260): No heartbeat from core client for 30 sec - exiting 20:16:16 (3260): No heartbeat from core client for 30 sec - exiting 20:16:17 (3260): No heartbeat from core client for 30 sec - exiting 20:16:18 (3260): No heartbeat from core client for 30 sec - exiting 20:16:19 (3260): No heartbeat from core client for 30 sec - exiting 20:16:20 (3260): No heartbeat from core client for 30 sec - exiting 20:16:21 (3260): No heartbeat from core client for 30 sec - exiting 20:16:22 (3260): No heartbeat from core client for 30 sec - exiting 20:16:23 (3260): No heartbeat from core client for 30 sec - exiting 20:16:24 (3260): No heartbeat from core client for 30 sec - exiting 20:16:25 (3260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:17:12 (2548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:51:32 (8892): No heartbeat from core client for 30 sec - exiting 21:51:33 (8892): No heartbeat from core client for 30 sec - exiting 21:51:34 (8892): No heartbeat from core client for 30 sec - exiting 21:51:35 (8892): No heartbeat from core client for 30 sec - exiting 21:51:36 (8892): No heartbeat from core client for 30 sec - exiting 21:51:37 (8892): No heartbeat from core client for 30 sec - exiting 21:51:38 (8892): No heartbeat from core client for 30 sec - exiting 21:51:39 (8892): No heartbeat from core client for 30 sec - exiting 21:51:40 (8892): No heartbeat from core client for 30 sec - exiting 21:51:41 (8892): No heartbeat from core client for 30 sec - exiting 21:51:42 (8892): No heartbeat from core client for 30 sec - exiting 21:51:43 (8892): No heartbeat from core client for 30 sec - exiting 21:51:44 (8892): No heartbeat from core client for 30 sec - exiting 21:51:45 (8892): No heartbeat from core client for 30 sec - exiting 21:51:46 (8892): No heartbeat from core client for 30 sec - exiting 21:51:47 (8892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:15:51 (6300): No heartbeat from core client for 30 sec - exiting 20:15:52 (6300): No heartbeat from core client for 30 sec - exiting 20:15:53 (6300): No heartbeat from core client for 30 sec - exiting 20:15:54 (6300): No heartbeat from core client for 30 sec - exiting 20:15:55 (6300): No heartbeat from core client for 30 sec - exiting 20:15:56 (6300): No heartbeat from core client for 30 sec - exiting 20:15:57 (6300): No heartbeat from core client for 30 sec - exiting 20:15:58 (6300): No heartbeat from core client for 30 sec - exiting 20:15:59 (6300): No heartbeat from core client for 30 sec - exiting 20:16:00 (6300): No heartbeat from core client for 30 sec - exiting 20:16:01 (6300): No heartbeat from core client for 30 sec - exiting 20:16:02 (6300): No heartbeat from core client for 30 sec - exiting 20:16:03 (6300): No heartbeat from core client for 30 sec - exiting 20:16:04 (6300): No heartbeat from core client for 30 sec - exiting 20:16:05 (6300): No heartbeat from core client for 30 sec - exiting 20:16:06 (6300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8232, iMonCtr=1 Model crash detected, will try to restart... 16:54:04 (1608): No heartbeat from core client for 30 sec - exiting 16:54:05 (1608): No heartbeat from core client for 30 sec - exiting 16:54:06 (1608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:54:07 (1608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:59:55 (5904): No heartbeat from core client for 30 sec - exiting 12:59:56 (5904): No heartbeat from core client for 30 sec - exiting 12:59:57 (5904): No heartbeat from core client for 30 sec - exiting 12:59:58 (5904): No heartbeat from core client for 30 sec - exiting 12:59:59 (5904): No heartbeat from core client for 30 sec - exiting 13:00:00 (5904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:38:00 (4756): No heartbeat from core client for 30 sec - exiting 20:38:02 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:38:03 (4756): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x774EA645 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... 19:31:28 (9556): No heartbeat from core client for 30 sec - exiting 19:31:29 (9556): No heartbeat from core client for 30 sec - exiting 19:31:30 (9556): No heartbeat from core client for 30 sec - exiting 19:31:31 (9556): No heartbeat from core client for 30 sec - exiting 19:31:32 (9556): No heartbeat from core client for 30 sec - exiting 19:31:33 (9556): No heartbeat from core client for 30 sec - exiting 19:31:34 (9556): No heartbeat from core client for 30 sec - exiting 19:31:35 (9556): No heartbeat from core client for 30 sec - exiting 19:31:36 (9556): No heartbeat from core client for 30 sec - exiting 19:31:37 (9556): No heartbeat from core client for 30 sec - exiting 19:31:38 (9556): No heartbeat from core client for 30 sec - exiting 19:31:39 (9556): No heartbeat from core client for 30 sec - exiting 19:31:40 (9556): No heartbeat from core client for 30 sec - exiting 19:31:41 (9556): No heartbeat from core client for 30 sec - exiting 19:31:42 (9556): No heartbeat from core client for 30 sec - exiting 19:31:43 (9556): No heartbeat from core client for 30 sec - exiting 19:31:44 (9556): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77C95F1B read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Apr 2014 21:43:23	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	518,400	683,011	1.3175
12 Apr 2014 15:25:53	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	492,480	649,336	1.3185
07 Apr 2014 21:10:41	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	466,560	614,808	1.3177
06 Apr 2014 12:04:54	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	440,640	580,637	1.3177
02 Apr 2014 19:55:49	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	414,720	544,907	1.3139
30 Mar 2014 20:13:20	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	388,800	509,093	1.3094
24 Mar 2014 20:04:25	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	362,880	475,074	1.3092
22 Mar 2014 18:51:42	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	336,960	441,389	1.3099
20 Mar 2014 20:43:08	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	311,040	407,096	1.3088
15 Mar 2014 22:20:21	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	285,120	374,017	1.3118
10 Mar 2014 20:55:58	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	259,200	339,573	1.3101
09 Mar 2014 12:12:00	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	233,280	305,610	1.3101
05 Mar 2014 20:58:45	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	207,360	270,485	1.3044
01 Mar 2014 22:51:44	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	181,440	234,924	1.2948
27 Feb 2014 22:09:26	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	155,520	200,968	1.2922
25 Feb 2014 20:12:22	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	129,600	167,517	1.2926
21 Feb 2014 18:15:09	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	103,680	134,206	1.2944
14 Feb 2014 20:09:43	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	77,760	100,213	1.2887
12 Feb 2014 20:00:22	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	51,840	66,802	1.2886
10 Feb 2014 18:43:08	1087624	16280915	hadcm3n_7e2g_1980_40_008429515_3	25,920	33,528	1.2935