Task 15919683

Name	hadcm3n_o6tx_1980_40_008388614_2
Workunit	8539473
Created	14 Aug 2013, 17:28:34 UTC
Sent	14 Aug 2013, 18:54:14 UTC
Report deadline	14 Nov 2013, 2:21:25 UTC
Received	23 Sep 2013, 18:27:39 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1252582
Run time	6 days 7 hours 4 min 16 sec
CPU time	6 days 2 hours 56 min 41 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.94 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30760, iMonCtr=1 Model crash detected, will try to restart... 16:07:59 (6284): No heartbeat from core client for 30 sec - exiting 16:08:00 (6284): No heartbeat from core client for 30 sec - exiting 16:08:01 (6284): No heartbeat from core client for 30 sec - exiting 16:08:02 (6284): No heartbeat from core client for 30 sec - exiting 16:08:04 (6284): No heartbeat from core client for 30 sec - exiting 16:08:05 (6284): No heartbeat from core client for 30 sec - exiting 16:08:06 (6284): No heartbeat from core client for 30 sec - exiting 16:08:07 (6284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:44:26 (4876): No heartbeat from core client for 30 sec - exiting 11:45:06 (4876): No heartbeat from core client for 30 sec - exiting 11:45:07 (4876): No heartbeat from core client for 30 sec - exiting 11:45:08 (4876): No heartbeat from core client for 30 sec - exiting 11:45:09 (4876): No heartbeat from core client for 30 sec - exiting 11:45:10 (4876): No heartbeat from core client for 30 sec - exiting 11:45:11 (4876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:37:30 (17076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:37:35 (17076): No heartbeat from core client for 30 sec - exiting 14:38:33 (16648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:38:34 (16648): No heartbeat from core client for 30 sec - exiting 14:38:35 (16648): No heartbeat from core client for 30 sec - exiting 14:38:36 (16648): No heartbeat from core client for 30 sec - exiting 14:38:37 (16648): No heartbeat from core client for 30 sec - exiting 14:38:38 (16648): No heartbeat from core client for 30 sec - exiting 14:38:39 (16648): No heartbeat from core client for 30 sec - exiting 14:38:40 (16648): No heartbeat from core client for 30 sec - exiting 14:38:41 (16648): No heartbeat from core client for 30 sec - exiting 14:38:42 (16648): No heartbeat from core client for 30 sec - exiting 14:38:43 (16648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:16:58 (2152): No heartbeat from core client for 30 sec - exiting 09:17:00 (2152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13756, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:31:33 (2980): No heartbeat from core client for 30 sec - exiting 17:31:34 (2980): No heartbeat from core client for 30 sec - exiting 17:31:35 (2980): No heartbeat from core client for 30 sec - exiting 17:31:36 (2980): No heartbeat from core client for 30 sec - exiting 17:31:37 (2980): No heartbeat from core client for 30 sec - exiting 17:31:38 (2980): No heartbeat from core client for 30 sec - exiting 17:31:39 (2980): No heartbeat from core client for 30 sec - exiting 17:31:40 (2980): No heartbeat from core client for 30 sec - exiting 17:31:41 (2980): No heartbeat from core client for 30 sec - exiting 17:31:42 (2980): No heartbeat from core client for 30 sec - exiting 17:31:43 (2980): No heartbeat from core client for 30 sec - exiting 17:31:45 (2980): No heartbeat from core client for 30 sec - exiting 17:31:46 (2980): No heartbeat from core client for 30 sec - exiting 17:31:47 (2980): No heartbeat from core client for 30 sec - exiting 17:31:48 (2980): No heartbeat from core client for 30 sec - exiting 17:31:49 (2980): No heartbeat from core client for 30 sec - exiting 17:31:50 (2980): No heartbeat from core client for 30 sec - exiting 17:31:51 (2980): No heartbeat from core client for 30 sec - exiting 17:31:52 (2980): No heartbeat from core client for 30 sec - exiting 17:31:53 (2980): No heartbeat from core client for 30 sec - exiting 17:31:55 (2980): No heartbeat from core client for 30 sec - exiting 17:31:56 (2980): No heartbeat from core client for 30 sec - exiting 17:31:57 (2980): No heartbeat from core client for 30 sec - exiting 17:31:58 (2980): No heartbeat from core client for 30 sec - exiting 17:31:59 (2980): No heartbeat from core client for 30 sec - exiting 17:32:00 (2980): No heartbeat from core client for 30 sec - exiting 17:32:01 (2980): No heartbeat from core client for 30 sec - exiting 17:32:02 (2980): No heartbeat from core client for 30 sec - exiting 17:32:03 (2980): No heartbeat from core client for 30 sec - exiting 17:32:04 (2980): No heartbeat from core client for 30 sec - exiting 17:32:06 (2980): No heartbeat from core client for 30 sec - exiting 17:32:07 (2980): No heartbeat from core client for 30 sec - exiting 17:32:08 (2980): No heartbeat from core client for 30 sec - exiting 17:32:09 (2980): No heartbeat from core client for 30 sec - exiting 17:32:10 (2980): No heartbeat from core client for 30 sec - exiting 17:32:11 (2980): No heartbeat from core client for 30 sec - exiting 17:32:12 (2980): No heartbeat from core client for 30 sec - exiting 17:32:13 (2980): No heartbeat from core client for 30 sec - exiting 17:32:14 (2980): No heartbeat from core client for 30 sec - exiting 17:32:15 (2980): No heartbeat from core client for 30 sec - exiting 17:32:17 (2980): No heartbeat from core client for 30 sec - exiting 17:32:18 (2980): No heartbeat from core client for 30 sec - exiting 17:32:19 (2980): No heartbeat from core client for 30 sec - exiting 17:32:20 (2980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:45:37 (2688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:13:00 (6892): No heartbeat from core client for 30 sec - exiting 20:13:01 (6892): No heartbeat from core client for 30 sec - exiting 20:13:02 (6892): No heartbeat from core client for 30 sec - exiting 20:13:03 (6892): No heartbeat from core client for 30 sec - exiting 20:13:04 (6892): No heartbeat from core client for 30 sec - exiting 20:13:05 (6892): No heartbeat from core client for 30 sec - exiting 20:13:07 (6892): No heartbeat from core client for 30 sec - exiting 20:13:08 (6892): No heartbeat from core client for 30 sec - exiting 20:13:09 (6892): No heartbeat from core client for 30 sec - exiting 20:13:10 (6892): No heartbeat from core client for 30 sec - exiting 20:13:11 (6892): No heartbeat from core client for 30 sec - exiting 20:13:12 (6892): No heartbeat from core client for 30 sec - exiting 20:13:13 (6892): No heartbeat from core client for 30 sec - exiting 20:13:14 (6892): No heartbeat from core client for 30 sec - exiting 20:13:15 (6892): No heartbeat from core client for 30 sec - exiting 20:13:16 (6892): No heartbeat from core client for 30 sec - exiting 20:13:18 (6892): No heartbeat from core client for 30 sec - exiting 20:13:19 (6892): No heartbeat from core client for 30 sec - exiting 20:13:20 (6892): No heartbeat from core client for 30 sec - exiting 20:13:21 (6892): No heartbeat from core client for 30 sec - exiting 20:13:22 (6892): No heartbeat from core client for 30 sec - exiting 20:13:23 (6892): No heartbeat from core client for 30 sec - exiting 20:13:24 (6892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x779B3AC3 read attempt to address 0x40C63251 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x779B3AC3 read attempt to address 0x40C63251 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Sep 2013 18:32:11	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	518,400	520,874	1.0048
21 Sep 2013 12:16:30	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	492,480	491,210	0.9974
19 Sep 2013 04:56:23	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	466,560	464,968	0.9966
15 Sep 2013 23:53:23	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	440,640	439,128	0.9966
15 Sep 2013 09:34:06	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	414,720	412,190	0.9939
13 Sep 2013 04:45:04	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	388,800	386,621	0.9944
12 Sep 2013 09:39:42	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	362,880	360,474	0.9934
10 Sep 2013 21:07:41	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	336,960	334,969	0.9941
09 Sep 2013 15:11:56	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	311,040	310,943	0.9997
04 Sep 2013 04:24:46	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	285,120	286,529	1.0049
02 Sep 2013 20:55:29	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	259,200	261,353	1.0083
31 Aug 2013 10:20:30	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	233,280	235,559	1.0098
31 Aug 2013 00:51:21	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	207,360	208,591	1.0059
25 Aug 2013 05:07:23	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	181,440	182,555	1.0061
24 Aug 2013 04:42:07	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	155,520	156,984	1.0094
23 Aug 2013 18:13:50	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	129,600	130,935	1.0103
21 Aug 2013 22:10:16	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	103,680	103,218	0.9955
17 Aug 2013 23:22:33	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	77,760	76,040	0.9779
17 Aug 2013 03:58:47	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	51,840	51,337	0.9903
15 Aug 2013 18:04:15	1252582	15919683	hadcm3n_o6tx_1980_40_008388614_2	25,920	25,559	0.9861