Task 15594740

Name	hadcm3n_48r5_1940_40_008308052_0
Workunit	8459187
Created	7 Feb 2013, 15:57:46 UTC
Sent	7 Feb 2013, 16:03:39 UTC
Report deadline	9 May 2013, 23:30:50 UTC
Received	3 Apr 2013, 10:25:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1266710
Run time	6 days 18 hours 16 min 26 sec
CPU time	6 days 13 hours 45 min 54 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.17 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4620, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:38:12 (3264): No heartbeat from core client for 30 sec - exiting 10:38:13 (3264): No heartbeat from core client for 30 sec - exiting 10:38:14 (3264): No heartbeat from core client for 30 sec - exiting 10:38:15 (3264): No heartbeat from core client for 30 sec - exiting 10:38:16 (3264): No heartbeat from core client for 30 sec - exiting 10:38:18 (3264): No heartbeat from core client for 30 sec - exiting 10:38:19 (3264): No heartbeat from core client for 30 sec - exiting 10:38:20 (3264): No heartbeat from core client for 30 sec - exiting 10:38:21 (3264): No heartbeat from core client for 30 sec - exiting 10:38:22 (3264): No heartbeat from core client for 30 sec - exiting 10:38:23 (3264): No heartbeat from core client for 30 sec - exiting 10:38:24 (3264): No heartbeat from core client for 30 sec - exiting 10:38:25 (3264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:38:26 (3264): No heartbeat from core client for 30 sec - exiting 14:42:49 (4148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:32:52 (2584): No heartbeat from core client for 30 sec - exiting 18:32:53 (2584): No heartbeat from core client for 30 sec - exiting 18:32:54 (2584): No heartbeat from core client for 30 sec - exiting 18:32:55 (2584): No heartbeat from core client for 30 sec - exiting 18:32:56 (2584): No heartbeat from core client for 30 sec - exiting 18:32:57 (2584): No heartbeat from core client for 30 sec - exiting 18:32:59 (2584): No heartbeat from core client for 30 sec - exiting 18:33:00 (2584): No heartbeat from core client for 30 sec - exiting 18:33:01 (2584): No heartbeat from core client for 30 sec - exiting 18:33:02 (2584): No heartbeat from core client for 30 sec - exiting 18:33:03 (2584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:16:10 (2840): No heartbeat from core client for 30 sec - exiting 10:16:12 (2840): No heartbeat from core client for 30 sec - exiting 10:16:13 (2840): No heartbeat from core client for 30 sec - exiting 10:16:14 (2840): No heartbeat from core client for 30 sec - exiting 10:16:15 (2840): No heartbeat from core client for 30 sec - exiting 10:16:16 (2840): No heartbeat from core client for 30 sec - exiting 10:16:17 (2840): No heartbeat from core client for 30 sec - exiting 10:16:18 (2840): No heartbeat from core client for 30 sec - exiting 10:16:19 (2840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:16:20 (2840): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:20:51 (4024): No heartbeat from core client for 30 sec - exiting 12:20:53 (4024): No heartbeat from core client for 30 sec - exiting 12:20:54 (4024): No heartbeat from core client for 30 sec - exiting 12:20:55 (4024): No heartbeat from core client for 30 sec - exiting 12:20:56 (4024): No heartbeat from core client for 30 sec - exiting 12:20:57 (4024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2584, iMonCtr=1 Model crash detected, will try to restart... 13:02:40 (3436): No heartbeat from core client for 30 sec - exiting 13:02:41 (3436): No heartbeat from core client for 30 sec - exiting 13:02:42 (3436): No heartbeat from core client for 30 sec - exiting 13:02:43 (3436): No heartbeat from core client for 30 sec - exiting 13:02:44 (3436): No heartbeat from core client for 30 sec - exiting 13:02:45 (3436): No heartbeat from core client for 30 sec - exiting 13:02:46 (3436): No heartbeat from core client for 30 sec - exiting 13:02:47 (3436): No heartbeat from core client for 30 sec - exiting 13:02:49 (3436): No heartbeat from core client for 30 sec - exiting 13:02:50 (3436): No heartbeat from core client for 30 sec - exiting 13:02:51 (3436): No heartbeat from core client for 30 sec - exiting 13:02:52 (3436): No heartbeat from core client for 30 sec - exiting 13:02:53 (3436): No heartbeat from core client for 30 sec - exiting 13:02:54 (3436): No heartbeat from core client for 30 sec - exiting 13:02:55 (3436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:45:08 (2636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3372, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2724, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3488, iMonCtr=1 Model crash detected, will try to restart... 13:26:03 (3632): No heartbeat from core client for 30 sec - exiting 13:26:04 (3632): No heartbeat from core client for 30 sec - exiting 13:26:05 (3632): No heartbeat from core client for 30 sec - exiting 13:26:07 (3632): No heartbeat from core client for 30 sec - exiting 13:26:08 (3632): No heartbeat from core client for 30 sec - exiting 13:26:09 (3632): No heartbeat from core client for 30 sec - exiting 13:26:10 (3632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3948, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2668, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4168, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x775F3DF9 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_48r5_1940_40_008308052/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Apr 2013 10:26:15	1266710	15594740	hadcm3n_48r5_1940_40_008308052_0	259,200	567,945	2.1911
29 Mar 2013 12:17:11	1266710	15594740	hadcm3n_48r5_1940_40_008308052_0	233,280	525,451	2.2524
27 Mar 2013 12:33:48	1266710	15594740	hadcm3n_48r5_1940_40_008308052_0	207,360	480,836	2.3188
20 Mar 2013 11:22:37	1266710	15594740	hadcm3n_48r5_1940_40_008308052_0	181,440	422,302	2.3275
13 Mar 2013 11:35:38	1266710	15594740	hadcm3n_48r5_1940_40_008308052_0	155,520	360,779	2.3198
01 Mar 2013 08:52:30	1266710	15594740	hadcm3n_48r5_1940_40_008308052_0	129,600	301,363	2.3253
28 Feb 2013 16:00:34	1266710	15594740	hadcm3n_48r5_1940_40_008308052_0	103,680	240,823	2.3228
26 Feb 2013 12:11:20	1266710	15594740	hadcm3n_48r5_1940_40_008308052_0	77,760	181,100	2.3290
21 Feb 2013 10:25:01	1266710	15594740	hadcm3n_48r5_1940_40_008308052_0	51,840	120,573	2.3259
15 Feb 2013 09:42:18	1266710	15594740	hadcm3n_48r5_1940_40_008308052_0	25,920	60,353	2.3284