Task 15528575

Name	hadcm3n_yjvw_1980_40_008243625_3
Workunit	8398749
Created	11 Jan 2013, 1:42:34 UTC
Sent	11 Jan 2013, 1:42:39 UTC
Report deadline	12 Apr 2013, 9:09:50 UTC
Received	27 Feb 2013, 5:07:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1091586
Run time	38 days 15 hours 48 min 12 sec
CPU time	35 days 1 hours 8 min 6 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.76 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:20:11 (2268): No heartbeat from core client for 30 sec - exiting 21:20:13 (2268): No heartbeat from core client for 30 sec - exiting 21:20:14 (2268): No heartbeat from core client for 30 sec - exiting 21:20:15 (2268): No heartbeat from core client for 30 sec - exiting 21:20:16 (2268): No heartbeat from core client for 30 sec - exiting 21:20:17 (2268): No heartbeat from core client for 30 sec - exiting 21:20:18 (2268): No heartbeat from core client for 30 sec - exiting 21:20:19 (2268): No heartbeat from core client for 30 sec - exiting 21:20:20 (2268): No heartbeat from core client for 30 sec - exiting 21:20:21 (2268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5040, iMonCtr=1 Model crash detected, will try to restart... 14:44:36 (5932): No heartbeat from core client for 30 sec - exiting 14:44:37 (5932): No heartbeat from core client for 30 sec - exiting 14:44:38 (5932): No heartbeat from core client for 30 sec - exiting 14:44:39 (5932): No heartbeat from core client for 30 sec - exiting 14:44:40 (5932): No heartbeat from core client for 30 sec - exiting 14:44:41 (5932): No heartbeat from core client for 30 sec - exiting 14:44:42 (5932): No heartbeat from core client for 30 sec - exiting 14:44:43 (5932): No heartbeat from core client for 30 sec - exiting 14:44:44 (5932): No heartbeat from core client for 30 sec - exiting 14:44:45 (5932): No heartbeat from core client for 30 sec - exiting 14:44:46 (5932): No heartbeat from core client for 30 sec - exiting 14:44:47 (5932): No heartbeat from core client for 30 sec - exiting 14:44:48 (5932): No heartbeat from core client for 30 sec - exiting 14:44:49 (5932): No heartbeat from core client for 30 sec - exiting 14:44:51 (5932): No heartbeat from core client for 30 sec - exiting 14:44:52 (5932): No heartbeat from core client for 30 sec - exiting 14:44:53 (5932): No heartbeat from core client for 30 sec - exiting 14:44:54 (5932): No heartbeat from core client for 30 sec - exiting 14:44:55 (5932): No heartbeat from core client for 30 sec - exiting 14:44:56 (5932): No heartbeat from core client for 30 sec - exiting 14:44:57 (5932): No heartbeat from core client for 30 sec - exiting 14:44:58 (5932): No heartbeat from core client for 30 sec - exiting 14:44:59 (5932): No heartbeat from core client for 30 sec - exiting 14:45:00 (5932): No heartbeat from core client for 30 sec - exiting 14:45:02 (5932): No heartbeat from core client for 30 sec - exiting 14:45:03 (5932): No heartbeat from core client for 30 sec - exiting 14:45:04 (5932): No heartbeat from core client for 30 sec - exiting 14:45:05 (5932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:56:05 (2528): No heartbeat from core client for 30 sec - exiting 17:56:06 (2528): No heartbeat from core client for 30 sec - exiting 17:56:07 (2528): No heartbeat from core client for 30 sec - exiting 17:56:08 (2528): No heartbeat from core client for 30 sec - exiting 17:56:09 (2528): No heartbeat from core client for 30 sec - exiting 17:56:10 (2528): No heartbeat from core client for 30 sec - exiting 17:56:11 (2528): No heartbeat from core client for 30 sec - exiting 17:56:12 (2528): No heartbeat from core client for 30 sec - exiting 17:56:13 (2528): No heartbeat from core client for 30 sec - exiting 17:56:15 (2528): No heartbeat from core client for 30 sec - exiting 17:56:16 (2528): No heartbeat from core client for 30 sec - exiting 17:56:17 (2528): No heartbeat from core client for 30 sec - exiting 17:56:18 (2528): No heartbeat from core client for 30 sec - exiting 17:56:19 (2528): No heartbeat from core client for 30 sec - exiting 17:56:20 (2528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:44:43 (5704): No heartbeat from core client for 30 sec - exiting 22:44:44 (5704): No heartbeat from core client for 30 sec - exiting 22:44:46 (5704): No heartbeat from core client for 30 sec - exiting 22:44:47 (5704): No heartbeat from core client for 30 sec - exiting 22:44:48 (5704): No heartbeat from core client for 30 sec - exiting 22:44:49 (5704): No heartbeat from core client for 30 sec - exiting 22:44:50 (5704): No heartbeat from core client for 30 sec - exiting 22:44:51 (5704): No heartbeat from core client for 30 sec - exiting 22:44:52 (5704): No heartbeat from core client for 30 sec - exiting 22:44:53 (5704): No heartbeat from core client for 30 sec - exiting 22:44:54 (5704): No heartbeat from core client for 30 sec - exiting 22:44:55 (5704): No heartbeat from core client for 30 sec - exiting 22:44:56 (5704): No heartbeat from core client for 30 sec - exiting 22:44:58 (5704): No heartbeat from core client for 30 sec - exiting 22:44:59 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:15:11 (7048): No heartbeat from core client for 30 sec - exiting 00:15:12 (7048): No heartbeat from core client for 30 sec - exiting 00:15:13 (7048): No heartbeat from core client for 30 sec - exiting 00:15:14 (7048): No heartbeat from core client for 30 sec - exiting 00:15:15 (7048): No heartbeat from core client for 30 sec - exiting 00:15:16 (7048): No heartbeat from core client for 30 sec - exiting 00:15:18 (7048): No heartbeat from core client for 30 sec - exiting 00:15:19 (7048): No heartbeat from core client for 30 sec - exiting 00:15:20 (7048): No heartbeat from core client for 30 sec - exiting 00:15:21 (7048): No heartbeat from core client for 30 sec - exiting 00:15:22 (7048): No heartbeat from core client for 30 sec - exiting 00:15:23 (7048): No heartbeat from core client for 30 sec - exiting 00:15:24 (7048): No heartbeat from core client for 30 sec - exiting 00:15:25 (7048): No heartbeat from core client for 30 sec - exiting 00:15:26 (7048): No heartbeat from core client for 30 sec - exiting 00:15:27 (7048): No heartbeat from core client for 30 sec - exiting 00:15:29 (7048): No heartbeat from core client for 30 sec - exiting 00:15:30 (7048): No heartbeat from core client for 30 sec - exiting 00:15:31 (7048): No heartbeat from core client for 30 sec - exiting 00:15:32 (7048): No heartbeat from core client for 30 sec - exiting 00:15:33 (7048): No heartbeat from core client for 30 sec - exiting 00:15:34 (7048): No heartbeat from core client for 30 sec - exiting 00:15:35 (7048): No heartbeat from core client for 30 sec - exiting 00:15:36 (7048): No heartbeat from core client for 30 sec - exiting 00:15:37 (7048): No heartbeat from core client for 30 sec - exiting 00:15:38 (7048): No heartbeat from core client for 30 sec - exiting 00:15:40 (7048): No heartbeat from core client for 30 sec - exiting 00:15:41 (7048): No heartbeat from core client for 30 sec - exiting 00:15:42 (7048): No heartbeat from core client for 30 sec - exiting 00:15:43 (7048): No heartbeat from core client for 30 sec - exiting 00:15:44 (7048): No heartbeat from core client for 30 sec - exiting 00:15:45 (7048): No heartbeat from core client for 30 sec - exiting 00:15:46 (7048): No heartbeat from core client for 30 sec - exiting 00:15:47 (7048): No heartbeat from core client for 30 sec - exiting 00:15:48 (7048): No heartbeat from core client for 30 sec - exiting 00:15:50 (7048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:26:14 (8016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:00:35 (6672): No heartbeat from core client for 30 sec - exiting 01:00:36 (6672): No heartbeat from core client for 30 sec - exiting 01:00:38 (6672): No heartbeat from core client for 30 sec - exiting 01:00:39 (6672): No heartbeat from core client for 30 sec - exiting 01:00:40 (6672): No heartbeat from core client for 30 sec - exiting 01:00:41 (6672): No heartbeat from core client for 30 sec - exiting 01:00:42 (6672): No heartbeat from core client for 30 sec - exiting 01:00:43 (6672): No heartbeat from core client for 30 sec - exiting 01:00:44 (6672): No heartbeat from core client for 30 sec - exiting 01:00:45 (6672): No heartbeat from core client for 30 sec - exiting 01:00:46 (6672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7976, iMonCtr=1 Model crash detected, will try to restart... 18:17:37 (7292): No heartbeat from core client for 30 sec - exiting 18:17:38 (7292): No heartbeat from core client for 30 sec - exiting 18:17:39 (7292): No heartbeat from core client for 30 sec - exiting 18:17:40 (7292): No heartbeat from core client for 30 sec - exiting 18:17:41 (7292): No heartbeat from core client for 30 sec - exiting 18:17:42 (7292): No heartbeat from core client for 30 sec - exiting 18:17:43 (7292): No heartbeat from core client for 30 sec - exiting 18:17:44 (7292): No heartbeat from core client for 30 sec - exiting 18:17:46 (7292): No heartbeat from core client for 30 sec - exiting 18:17:47 (7292): No heartbeat from core client for 30 sec - exiting 18:17:48 (7292): No heartbeat from core client for 30 sec - exiting 18:17:49 (7292): No heartbeat from core client for 30 sec - exiting 18:17:50 (7292): No heartbeat from core client for 30 sec - exiting 18:17:51 (7292): No heartbeat from core client for 30 sec - exiting 18:17:52 (7292): No heartbeat from core client for 30 sec - exiting 18:17:53 (7292): No heartbeat from core client for 30 sec - exiting 18:17:54 (7292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3660, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3660, iMonCtr=1 Model crash detected, will try to restart... 18:16:01 (7836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7580, iMonCtr=1 Model crash detected, will try to restart... 05:06:44 (7772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:08:23 (1068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:57:50 (7536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:02:46 (7232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:03:26 (6852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:45:04 (7676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:45:43 (4588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F03FBB read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... 21:15:14 (8540): No heartbeat from core client for 30 sec - exiting 21:15:15 (8540): No heartbeat from core client for 30 sec - exiting 21:15:16 (8540): No heartbeat from core client for 30 sec - exiting 21:15:17 (8540): No heartbeat from core client for 30 sec - exiting 21:15:19 (8540): No heartbeat from core client for 30 sec - exiting 21:15:20 (8540): No heartbeat from core client for 30 sec - exiting 21:15:21 (8540): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x777B71F3 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Feb 2013 22:41:39	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	1,036,800	3,027,073	2.9196
24 Feb 2013 21:23:45	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	1,010,880	2,955,943	2.9241
23 Feb 2013 21:30:00	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	984,960	2,884,068	2.9281
23 Feb 2013 21:30:00	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	959,040	2,813,856	2.9340
21 Feb 2013 17:06:07	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	933,120	2,743,553	2.9402
16 Feb 2013 17:31:18	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	907,200	2,670,470	2.9436
15 Feb 2013 19:38:57	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	881,280	2,598,935	2.9490
14 Feb 2013 21:59:32	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	855,360	2,527,784	2.9552
14 Feb 2013 00:08:44	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	829,440	2,457,193	2.9625
13 Feb 2013 02:40:04	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	803,520	2,386,999	2.9707
12 Feb 2013 04:03:36	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	777,600	2,315,945	2.9783
11 Feb 2013 06:50:46	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	751,680	2,245,529	2.9873
10 Feb 2013 09:37:21	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	725,760	2,174,984	2.9968
09 Feb 2013 12:30:00	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	699,840	2,105,271	3.0082
08 Feb 2013 12:55:11	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	673,920	2,031,649	3.0147
07 Feb 2013 14:16:00	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	648,000	1,958,809	3.0229
06 Feb 2013 17:16:30	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	622,080	1,888,896	3.0364
05 Feb 2013 20:03:18	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	596,160	1,818,651	3.0506
04 Feb 2013 21:49:02	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	570,240	1,747,968	3.0653
03 Feb 2013 19:41:16	1091586	15528575	hadcm3n_yjvw_1980_40_008243625_3	544,320	1,667,426	3.0633