Task 14650156

Name	hadcm3n_o4kd_2060_40_007957882_0
Workunit	8112994
Created	9 May 2012, 16:09:49 UTC
Sent	9 May 2012, 16:27:29 UTC
Report deadline	8 Aug 2012, 23:54:40 UTC
Received	4 Jul 2012, 18:49:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1187786
Run time	15 days 19 hours 37 min 47 sec
CPU time	15 days 1 hours 17 min 17 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.08 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2768, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2712, iMonCtr=1 Model crash detected, will try to restart... 09:11:05 (2824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:56:02 (2844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4868, iMonCtr=1 Model crash detected, will try to restart... 07:50:31 (2832): No heartbeat from core client for 30 sec - exiting 07:50:33 (2832): No heartbeat from core client for 30 sec - exiting 07:50:34 (2832): No heartbeat from core client for 30 sec - exiting 07:50:35 (2832): No heartbeat from core client for 30 sec - exiting 07:50:36 (2832): No heartbeat from core client for 30 sec - exiting 07:50:37 (2832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:50:38 (2832): No heartbeat from core client for 30 sec - exiting 07:50:39 (2832): No heartbeat from core client for 30 sec - exiting 07:50:40 (2832): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2788, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2788, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:34:09 (2956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2868, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 09:15:59 (2976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:11:32 (2924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:31:21 (2876): No heartbeat from core client for 30 sec - exiting 01:31:23 (2876): No heartbeat from core client for 30 sec - exiting 01:31:24 (2876): No heartbeat from core client for 30 sec - exiting 01:31:25 (2876): No heartbeat from core client for 30 sec - exiting 01:31:26 (2876): No heartbeat from core client for 30 sec - exiting 01:31:27 (2876): No heartbeat from core client for 30 sec - exiting 01:31:28 (2876): No heartbeat from core client for 30 sec - exiting 01:31:29 (2876): No heartbeat from core client for 30 sec - exiting 01:31:30 (2876): No heartbeat from core client for 30 sec - exiting 01:31:31 (2876): No heartbeat from core client for 30 sec - exiting 01:31:32 (2876): No heartbeat from core client for 30 sec - exiting 01:31:34 (2876): No heartbeat from core client for 30 sec - exiting 01:31:35 (2876): No heartbeat from core client for 30 sec - exiting 01:31:36 (2876): No heartbeat from core client for 30 sec - exiting 01:31:37 (2876): No heartbeat from core client for 30 sec - exiting 01:31:38 (2876): No heartbeat from core client for 30 sec - exiting 01:31:39 (2876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:31:40 (2876): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:33:23 (1560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:55:59 (3096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3956, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2752, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4188, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1424, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2804, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3200, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2760, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2760, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2760, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5740, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2592, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5272, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2448, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4128, iMonCtr=1 Model crash detected, will try to restart... 11:10:17 (3584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:10:18 (3584): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3416, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3356, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3540, iMonCtr=1 Model crash detected, will try to restart... 17:50:27 (3536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3288, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2716, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7742BDC6 write attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77487D52 read attempt to address 0x02030F1A Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\Programy\BOINC\Boinc_DATA/projects/climateprediction.net/hadcm3n_o4kd_2060_40_007957882/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Jul 2012 18:53:47	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	777,600	1,300,633	1.6726
02 Jul 2012 15:18:31	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	751,680	1,257,370	1.6727
28 Jun 2012 12:22:19	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	725,760	1,214,309	1.6732
27 Jun 2012 06:51:09	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	699,840	1,171,213	1.6735
25 Jun 2012 10:16:59	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	673,920	1,127,965	1.6737
23 Jun 2012 18:26:05	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	648,000	1,084,829	1.6741
22 Jun 2012 10:59:16	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	622,080	1,041,786	1.6747
20 Jun 2012 07:21:34	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	596,160	998,071	1.6742
19 Jun 2012 08:25:47	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	570,240	954,840	1.6745
15 Jun 2012 17:20:58	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	544,320	911,671	1.6749
11 Jun 2012 11:58:22	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	518,400	868,729	1.6758
07 Jun 2012 19:19:52	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	492,480	825,662	1.6765
06 Jun 2012 17:35:53	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	466,560	782,181	1.6765
05 Jun 2012 09:58:22	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	440,640	738,440	1.6758
03 Jun 2012 19:06:37	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	414,720	694,393	1.6744
02 Jun 2012 10:45:35	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	388,800	650,214	1.6724
01 Jun 2012 01:27:39	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	362,880	606,825	1.6722
30 May 2012 17:32:15	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	336,960	563,360	1.6719
29 May 2012 11:02:28	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	311,040	519,469	1.6701
26 May 2012 20:46:37	1187786	14650156	hadcm3n_o4kd_2060_40_007957882_0	285,120	475,521	1.6678