Task 15469303

Name	hadcm3n_yf9y_1940_40_008239648_3
Workunit	8394772
Created	2 Dec 2012, 23:39:28 UTC
Sent	2 Dec 2012, 23:39:40 UTC
Report deadline	4 Mar 2013, 7:06:51 UTC
Received	16 Dec 2012, 22:57:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1255703
Run time	8 days 17 hours 34 min 19 sec
CPU time	8 days 14 hours 28 min 34 sec
Validate state	Invalid
Credit	7,153.92
Device peak FLOPS	3.33 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=768, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:47:41 (972): No heartbeat from core client for 30 sec - exiting 01:47:42 (972): No heartbeat from core client for 30 sec - exiting 01:47:43 (972): No heartbeat from core client for 30 sec - exiting 01:47:44 (972): No heartbeat from core client for 30 sec - exiting 01:47:45 (972): No heartbeat from core client for 30 sec - exiting 01:47:46 (972): No heartbeat from core client for 30 sec - exiting 01:47:48 (972): No heartbeat from core client for 30 sec - exiting 01:47:49 (972): No heartbeat from core client for 30 sec - exiting 01:47:50 (972): No heartbeat from core client for 30 sec - exiting 01:47:51 (972): No heartbeat from core client for 30 sec - exiting 01:47:52 (972): No heartbeat from core client for 30 sec - exiting 01:47:53 (972): No heartbeat from core client for 30 sec - exiting 01:47:54 (972): No heartbeat from core client for 30 sec - exiting 01:47:55 (972): No heartbeat from core client for 30 sec - exiting 01:47:56 (972): No heartbeat from core client for 30 sec - exiting 01:47:57 (972): No heartbeat from core client for 30 sec - exiting 01:47:58 (972): No heartbeat from core client for 30 sec - exiting 01:48:00 (972): No heartbeat from core client for 30 sec - exiting 01:48:01 (972): No heartbeat from core client for 30 sec - exiting 01:48:02 (972): No heartbeat from core client for 30 sec - exiting 01:48:03 (972): No heartbeat from core client for 30 sec - exiting 01:48:04 (972): No heartbeat from core client for 30 sec - exiting 01:48:05 (972): No heartbeat from core client for 30 sec - exiting 01:48:06 (972): No heartbeat from core client for 30 sec - exiting 01:48:07 (972): No heartbeat from core client for 30 sec - exiting 01:48:08 (972): No heartbeat from core client for 30 sec - exiting 01:48:09 (972): No heartbeat from core client for 30 sec - exiting 01:48:10 (972): No heartbeat from core client for 30 sec - exiting 01:48:11 (972): No heartbeat from core client for 30 sec - exiting 01:48:12 (972): No heartbeat from core client for 30 sec - exiting 01:48:13 (972): No heartbeat from core client for 30 sec - exiting 01:48:15 (972): No heartbeat from core client for 30 sec - exiting 01:48:16 (972): No heartbeat from core client for 30 sec - exiting 01:48:17 (972): No heartbeat from core client for 30 sec - exiting 01:48:18 (972): No heartbeat from core client for 30 sec - exiting 01:48:19 (972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:23:09 (3764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:50:20 (3164): No heartbeat from core client for 30 sec - exiting 17:50:22 (3164): No heartbeat from core client for 30 sec - exiting 17:50:23 (3164): No heartbeat from core client for 30 sec - exiting 17:50:24 (3164): No heartbeat from core client for 30 sec - exiting 17:50:25 (3164): No heartbeat from core client for 30 sec - exiting 17:50:26 (3164): No heartbeat from core client for 30 sec - exiting 17:50:27 (3164): No heartbeat from core client for 30 sec - exiting 17:50:28 (3164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=940, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=940, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=940, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3928, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3928, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3928, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Dec 2012 16:21:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	596,160	731,399	1.2269
16 Dec 2012 06:19:05	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	570,240	696,245	1.2210
15 Dec 2012 20:20:55	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	544,320	661,151	1.2146
15 Dec 2012 10:33:54	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	518,400	627,203	1.2099
14 Dec 2012 13:16:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	492,480	596,731	1.2117
14 Dec 2012 13:16:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	466,560	564,088	1.2090
14 Dec 2012 13:16:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	440,640	531,294	1.2057
14 Dec 2012 13:16:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	414,720	500,393	1.2066
14 Dec 2012 13:16:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	388,800	469,786	1.2083
14 Dec 2012 13:16:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	362,880	436,987	1.2042
14 Dec 2012 13:16:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	336,960	406,513	1.2064
14 Dec 2012 13:16:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	311,040	376,245	1.2096
14 Dec 2012 13:16:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	285,120	346,061	1.2137
14 Dec 2012 13:16:10	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	259,200	315,170	1.2159
08 Dec 2012 05:20:33	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	233,280	284,953	1.2215
07 Dec 2012 20:53:04	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	207,360	254,899	1.2293
07 Dec 2012 13:07:43	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	181,440	223,395	1.2312
06 Dec 2012 20:08:41	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	155,520	192,974	1.2408
04 Dec 2012 20:55:07	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	129,600	160,314	1.2370
04 Dec 2012 11:12:45	1255703	15469303	hadcm3n_yf9y_1940_40_008239648_3	103,680	125,402	1.2095