Task 15888453

Name	hadcm3n_zj6u_1960_40_008401043_0
Workunit	8551899
Created	9 Jul 2013, 2:36:45 UTC
Sent	9 Jul 2013, 7:10:25 UTC
Report deadline	8 Oct 2013, 14:37:36 UTC
Received	4 Sep 2013, 11:27:23 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1273094
Run time	3 days 15 hours 18 min 38 sec
CPU time	3 days 14 hours 39 min 54 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.76 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6124, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5720, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6104, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:16:35 (2868): No heartbeat from core client for 30 sec - exiting 12:16:36 (2868): No heartbeat from core client for 30 sec - exiting 12:16:37 (2868): No heartbeat from core client for 30 sec - exiting 12:16:39 (2868): No heartbeat from core client for 30 sec - exiting 12:16:40 (2868): No heartbeat from core client for 30 sec - exiting 12:16:41 (2868): No heartbeat from core client for 30 sec - exiting 12:16:42 (2868): No heartbeat from core client for 30 sec - exiting 12:16:43 (2868): No heartbeat from core client for 30 sec - exiting 12:16:44 (2868): No heartbeat from core client for 30 sec - exiting 12:16:45 (2868): No heartbeat from core client for 30 sec - exiting 12:16:46 (2868): No heartbeat from core client for 30 sec - exiting 12:16:47 (2868): No heartbeat from core client for 30 sec - exiting 12:16:48 (2868): No heartbeat from core client for 30 sec - exiting 12:16:49 (2868): No heartbeat from core client for 30 sec - exiting 12:16:51 (2868): No heartbeat from core client for 30 sec - exiting 12:16:52 (2868): No heartbeat from core client for 30 sec - exiting 12:16:53 (2868): No heartbeat from core client for 30 sec - exiting 12:16:54 (2868): No heartbeat from core client for 30 sec - exiting 12:16:55 (2868): No heartbeat from core client for 30 sec - exiting 12:16:56 (2868): No heartbeat from core client for 30 sec - exiting 12:16:57 (2868): No heartbeat from core client for 30 sec - exiting 12:16:58 (2868): No heartbeat from core client for 30 sec - exiting 12:16:59 (2868): No heartbeat from core client for 30 sec - exiting 12:17:00 (2868): No heartbeat from core client for 30 sec - exiting 12:17:01 (2868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:17:32 (6920): No heartbeat from core client for 30 sec - exiting 12:17:33 (6920): No heartbeat from core client for 30 sec - exiting 12:17:35 (6920): No heartbeat from core client for 30 sec - exiting 12:17:36 (6920): No heartbeat from core client for 30 sec - exiting 12:17:37 (6920): No heartbeat from core client for 30 sec - exiting 12:17:38 (6920): No heartbeat from core client for 30 sec - exiting 12:17:39 (6920): No heartbeat from core client for 30 sec - exiting 12:17:40 (6920): No heartbeat from core client for 30 sec - exiting 12:17:41 (6920): No heartbeat from core client for 30 sec - exiting 12:17:42 (6920): No heartbeat from core client for 30 sec - exiting 12:17:43 (6920): No heartbeat from core client for 30 sec - exiting 12:17:44 (6920): No heartbeat from core client for 30 sec - exiting 12:17:45 (6920): No heartbeat from core client for 30 sec - exiting 12:17:47 (6920): No heartbeat from core client for 30 sec - exiting 12:17:48 (6920): No heartbeat from core client for 30 sec - exiting 12:17:49 (6920): No heartbeat from core client for 30 sec - exiting 12:17:50 (6920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:26:38 (6300): No heartbeat from core client for 30 sec - exiting 12:26:39 (6300): No heartbeat from core client for 30 sec - exiting 12:26:40 (6300): No heartbeat from core client for 30 sec - exiting 12:26:41 (6300): No heartbeat from core client for 30 sec - exiting 12:26:42 (6300): No heartbeat from core client for 30 sec - exiting 12:26:43 (6300): No heartbeat from core client for 30 sec - exiting 12:26:44 (6300): No heartbeat from core client for 30 sec - exiting 12:26:45 (6300): No heartbeat from core client for 30 sec - exiting 12:26:47 (6300): No heartbeat from core client for 30 sec - exiting 12:26:48 (6300): No heartbeat from core client for 30 sec - exiting 12:26:49 (6300): No heartbeat from core client for 30 sec - exiting 12:26:50 (6300): No heartbeat from core client for 30 sec - exiting 12:26:51 (6300): No heartbeat from core client for 30 sec - exiting 12:26:52 (6300): No heartbeat from core client for 30 sec - exiting 12:26:53 (6300): No heartbeat from core client for 30 sec - exiting 12:26:54 (6300): No heartbeat from core client for 30 sec - exiting 12:26:55 (6300): No heartbeat from core client for 30 sec - exiting 12:26:56 (6300): No heartbeat from core client for 30 sec - exiting 12:26:58 (6300): No heartbeat from core client for 30 sec - exiting 12:26:59 (6300): No heartbeat from core client for 30 sec - exiting 12:27:00 (6300): No heartbeat from core client for 30 sec - exiting 12:27:01 (6300): No heartbeat from core client for 30 sec - exiting 12:27:02 (6300): No heartbeat from core client for 30 sec - exiting 12:27:03 (6300): No heartbeat from core client for 30 sec - exiting 12:27:04 (6300): No heartbeat from core client for 30 sec - exiting 12:27:05 (6300): No heartbeat from core client for 30 sec - exiting 12:27:06 (6300): No heartbeat from core client for 30 sec - exiting 12:27:07 (6300): No heartbeat from core client for 30 sec - exiting 12:27:08 (6300): No heartbeat from core client for 30 sec - exiting 12:27:10 (6300): No heartbeat from core client for 30 sec - exiting 12:27:11 (6300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:12:37 (4260): No heartbeat from core client for 30 sec - exiting 13:12:38 (4260): No heartbeat from core client for 30 sec - exiting 13:12:39 (4260): No heartbeat from core client for 30 sec - exiting 13:12:40 (4260): No heartbeat from core client for 30 sec - exiting 13:12:41 (4260): No heartbeat from core client for 30 sec - exiting 13:12:42 (4260): No heartbeat from core client for 30 sec - exiting 13:12:43 (4260): No heartbeat from core client for 30 sec - exiting 13:12:44 (4260): No heartbeat from core client for 30 sec - exiting 13:12:46 (4260): No heartbeat from core client for 30 sec - exiting 13:12:47 (4260): No heartbeat from core client for 30 sec - exiting 13:12:48 (4260): No heartbeat from core client for 30 sec - exiting 13:12:49 (4260): No heartbeat from core client for 30 sec - exiting 13:12:50 (4260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:12:58 (2948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:17:29 (3916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:53:07 (3496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:32:22 (6104): No heartbeat from core client for 30 sec - exiting 12:32:23 (6104): No heartbeat from core client for 30 sec - exiting 12:32:24 (6104): No heartbeat from core client for 30 sec - exiting 12:32:25 (6104): No heartbeat from core client for 30 sec - exiting 12:32:26 (6104): No heartbeat from core client for 30 sec - exiting 12:32:27 (6104): No heartbeat from core client for 30 sec - exiting 12:32:29 (6104): No heartbeat from core client for 30 sec - exiting 12:32:30 (6104): No heartbeat from core client for 30 sec - exiting 12:32:31 (6104): No heartbeat from core client for 30 sec - exiting 12:32:32 (6104): No heartbeat from core client for 30 sec - exiting 12:32:33 (6104): No heartbeat from core client for 30 sec - exiting 12:32:34 (6104): No heartbeat from core client for 30 sec - exiting 12:32:35 (6104): No heartbeat from core client for 30 sec - exiting 12:32:36 (6104): No heartbeat from core client for 30 sec - exiting 12:32:37 (6104): No heartbeat from core client for 30 sec - exiting 12:32:38 (6104): No heartbeat from core client for 30 sec - exiting 12:32:39 (6104): No heartbeat from core client for 30 sec - exiting 12:32:41 (6104): No heartbeat from core client for 30 sec - exiting 12:32:42 (6104): No heartbeat from core client for 30 sec - exiting 12:32:43 (6104): No heartbeat from core client for 30 sec - exiting 12:32:44 (6104): No heartbeat from core client for 30 sec - exiting 12:32:45 (6104):CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:15:23 (5688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:39:42 (4960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3340, iMonCtr=1 Model crash detected, will try to restart... 12:28:52 (4012): No heartbeat from core client for 30 sec - exiting 12:28:54 (4012): No heartbeat from core client for 30 sec - exiting 12:28:55 (4012): No heartbeat from core client for 30 sec - exiting 12:28:56 (4012): No heartbeat from core client for 30 sec - exiting 12:28:57 (4012): No heartbeat from core client for 30 sec - exiting 12:28:58 (4012): No heartbeat from core client for 30 sec - exiting 12:28:59 (4012): No heartbeat from core client for 30 sec - exiting 12:29:00 (4012): No heartbeat from core client for 30 sec - exiting 12:29:01 (4012): No heartbeat from core client for 30 sec - exiting 12:29:02 (4012): No heartbeat from core client for 30 sec - exiting 12:29:03 (4012): No heartbeat from core client for 30 sec - exiting 12:29:05 (4012): No heartbeat from core client for 30 sec - exiting 12:29:06 (4012): No heartbeat from core client for 30 sec - exiting 12:29:07 (4012): No heartbeat from core client for 30 sec - exiting 12:29:08 (4012): No heartbeat from core client for 30 sec - exiting 12:29:09 (4012): No heartbeat from core client for 30 sec - exiting 12:29:10 (4012): No heartbeat from core client for 30 sec - exiting 12:29:11 (4012): No heartbeat from core client for 30 sec - exiting 12:29:12 (4012): No heartbeat from core client for 30 sec - exiting 12:29:13 (4012): No heartbeat from core client for 30 sec - exiting 12:29:14 (4012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 12:19:07 (6036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:04:36 (2684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:43:42 (3788): No heartbeat from core client for 30 sec - exiting 11:43:43 (3788): No heartbeat from core client for 30 sec - exiting 11:43:44 (3788): No heartbeat from core client for 30 sec - exiting 11:43:45 (3788): No heartbeat from core client for 30 sec - exiting 11:43:46 (3788): No heartbeat from core client for 30 sec - exiting 11:43:47 (3788): No heartbeat from core client for 30 sec - exiting 11:43:48 (3788): No heartbeat from core client for 30 sec - exiting 11:43:49 (3788): No heartbeat from core client for 30 sec - exiting 11:43:50 (3788): No heartbeat from core client for 30 sec - exiting 11:43:52 (3788): No heartbeat from core client for 30 sec - exiting 11:43:53 (3788): No heartbeat from core client for 30 sec - exiting 11:43:54 (3788): No heartbeat from core client for 30 sec - exiting 11:43:55 (3788): No heartbeat from core client for 30 sec - exiting 11:43:56 (3788): No heartbeat from core client for 30 sec - exiting 11:43:57 (3788): No heartbeat from core client for 30 sec - exiting 11:43:58 (3788): No heartbeat from core client for 30 sec - exiting 11:43:59 (3788): No heartbeat from core client for 30 sec - exiting 11:44:00 (3788): No heartbeat from core client for 30 sec - exiting 11:44:01 (3788): No heartbeat from core client for 30 sec - exiting 11:44:02 (3788): No heartbeat from core client for 30 sec - exiting 11:44:03 (3788): No heartbeat from core client for 30 sec - exiting 11:44:04 (3788): No heartbeat from core client for 30 sec - exiting 11:44:06 (3788): No heartbeat from core client for 30 sec - exiting 11:44:07 (3788): No heartbeat from core client for 30 sec - exiting 11:44:08 (3788): No heartbeat from core client for 30 sec - exiting 11:44:09 (3788): No heartbeat from core client for 30 sec - exiting 11:44:10 (3788): No heartbeat from core client for 30 sec - exiting 11:44:11 (3788): No heartbeat from core client for 30 sec - exiting 11:44:12 (3788): No heartbeat from core client for 30 sec - exiting 11:44:13 (3788): No heartbeat from core client for 30 sec - exiting 11:44:14 (3788): No heartbeat from core client for 30 sec - exiting 11:44:15 (3788): No heartbeat from core client for 30 sec - exiting 11:44:17 (3788): No heartbeat from core client for 30 sec - exiting 11:44:18 (3788): No heartbeat from core client for 30 sec - exiting 11:44:19 (3788): No heartbeat from core client for 30 sec - exiting 11:44:20 (3788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:45:04 (4800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6292, iMonCtr=1 Model crash detected, will try to restart... 12:13:22 (1920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77383AC3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zj6u_1960_40_008401043/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Sep 2013 11:31:33	1273094	15888453	hadcm3n_zj6u_1960_40_008401043_0	259,200	311,982	1.2036
29 Aug 2013 04:58:15	1273094	15888453	hadcm3n_zj6u_1960_40_008401043_0	233,280	281,701	1.2076
26 Aug 2013 10:54:52	1273094	15888453	hadcm3n_zj6u_1960_40_008401043_0	207,360	250,961	1.2103
21 Aug 2013 05:51:05	1273094	15888453	hadcm3n_zj6u_1960_40_008401043_0	181,440	220,826	1.2171
15 Aug 2013 08:30:34	1273094	15888453	hadcm3n_zj6u_1960_40_008401043_0	155,520	189,451	1.2182
15 Aug 2013 05:14:18	1273094	15888453	hadcm3n_zj6u_1960_40_008401043_0	129,600	157,309	1.2138
23 Jul 2013 22:13:50	1273094	15888453	hadcm3n_zj6u_1960_40_008401043_0	103,680	126,243	1.2176
23 Jul 2013 19:24:49	1273094	15888453	hadcm3n_zj6u_1960_40_008401043_0	77,760	95,061	1.2225
23 Jul 2013 19:24:49	1273094	15888453	hadcm3n_zj6u_1960_40_008401043_0	51,840	63,444	1.2238
11 Jul 2013 12:17:13	1273094	15888453	hadcm3n_zj6u_1960_40_008401043_0	25,920	32,240	1.2438