Task 15886664

Name	hadcm3n_39vm_2020_40_008399620_0
Workunit	8550476
Created	8 Jul 2013, 21:19:44 UTC
Sent	9 Jul 2013, 17:57:58 UTC
Report deadline	9 Oct 2013, 1:25:09 UTC
Received	21 Sep 2013, 20:45:15 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1237073
Run time	2 days 13 hours 46 min 9 sec
CPU time	2 days 12 hours 59 min 47 sec
Validate state	Invalid
Credit	3,421.44
Device peak FLOPS	3.36 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:12:19 (9140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:09:15 (9672): No heartbeat from core client for 30 sec - exiting 16:09:16 (9672): No heartbeat from core client for 30 sec - exiting 16:09:17 (9672): No heartbeat from core client for 30 sec - exiting 16:09:18 (9672): No heartbeat from core client for 30 sec - exiting 16:09:19 (9672): No heartbeat from core client for 30 sec - exiting 16:09:20 (9672): No heartbeat from core client for 30 sec - exiting 16:09:21 (9672): No heartbeat from core client for 30 sec - exiting 16:09:22 (9672): No heartbeat from core client for 30 sec - exiting 16:09:23 (9672): No heartbeat from core client for 30 sec - exiting 16:09:24 (9672): No heartbeat from core client for 30 sec - exiting 16:09:25 (9672): No heartbeat from core client for 30 sec - exiting 16:09:26 (9672): No heartbeat from core client for 30 sec - exiting 16:09:27 (9672): No heartbeat from core client for 30 sec - exiting 16:09:28 (9672): No heartbeat from core client for 30 sec - exiting 16:09:29 (9672): No heartbeat from core client for 30 sec - exiting 16:09:30 (9672): No heartbeat from core client for 30 sec - exiting 16:09:31 (9672): No heartbeat from core client for 30 sec - exiting 16:09:33 (9672): No heartbeat from core client for 30 sec - exiting 16:09:34 (9672): No heartbeat from core client for 30 sec - exiting 16:09:35 (9672): No heartbeat from core client for 30 sec - exiting 16:09:36 (9672): No heartbeat from core client for 30 sec - exiting 16:09:37 (9672): No heartbeat from core client for 30 sec - exiting 16:09:38 (9672): No heartbeat from core client for 30 sec - exiting 16:09:39 (9672): No heartbeat from core client for 30 sec - exiting 16:09:40 (9672): No heartbeat from core client for 30 sec - exiting 16:09:41 (9672): No heartbeat from core client for 30 sec - exiting 16:09:42 (9672): No heartbeat from core client for 30 sec - exiting 16:09:43 (9672): No heartbeat from core client for 30 sec - exiting 16:09:44 (9672): No heartbeat from core client for 30 sec - exiting 16:09:45 (9672): No heartbeat from core client for 30 sec - exiting 16:09:46 (9672): No heartbeat from core client for 30 sec - exiting 16:09:47 (9672): No heartbeat from core client for 30 sec - exiting 16:09:48 (9672): No heartbeat from core client for 30 sec - exiting 16:09:49 (9672): No heartbeat from core client for 30 sec - exiting 16:09:50 (9672): No heartbeat from core client for 30 sec - exiting 16:09:51 (9672): No heartbeat from core client for 30 sec - exiting 16:09:52 (9672): No heartbeat from core client for 30 sec - exiting 16:09:53 (9672): No heartbeat from core client for 30 sec - exiting 16:09:54 (9672): No heartbeat from core client for 30 sec - exiting 16:09:55 (9672): No heartbeat from core client for 30 sec - exiting 16:09:56 (9672): No heartbeat from core client for 30 sec - exiting 16:09:57 (9672): No heartbeat from core client for 30 sec - exiting 16:09:58 (9672): No heartbeat from core client for 30 sec - exiting 16:09:59 (9672): No heartbeat from core client for 30 sec - exiting 16:10:00 (9672): No heartbeat from core client for 30 sec - exiting 16:10:01 (9672): No heartbeat from core client for 30 sec - exiting 16:10:02 (9672): No heartbeat from core client for 30 sec - exiting 16:10:03 (9672): No heartbeat from core client for 30 sec - exiting 16:10:04 (9672): No heartbeat from core client for 30 sec - exiting 16:10:05 (9672): No heartbeat from core client for 30 sec - exiting 16:10:06 (9672): No heartbeat from core client for 30 sec - exiting 16:10:07 (9672): No heartbeat from core client for 30 sec - exiting 16:10:08 (9672): No heartbeat from core client for 30 sec - exiting 16:10:09 (9672): No heartbeat from core client for 30 sec - exiting 16:10:10 (9672): No heartbeat from core client for 30 sec - exiting 16:10:11 (9672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5544, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5544, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5544, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5544, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5544, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5544, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Sep 2013 08:26:00	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	285,120	219,265	0.7690
03 Sep 2013 23:32:33	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	259,200	199,468	0.7696
25 Aug 2013 22:25:55	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	233,280	179,725	0.7704
19 Aug 2013 21:33:59	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	207,360	159,374	0.7686
19 Aug 2013 21:33:59	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	181,440	138,618	0.7640
26 Jul 2013 16:38:39	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	155,520	118,989	0.7651
26 Jul 2013 11:06:59	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	129,600	99,535	0.7680
26 Jul 2013 05:36:19	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	103,680	79,927	0.7709
26 Jul 2013 00:03:57	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	77,760	60,100	0.7729
25 Jul 2013 18:59:47	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	51,840	40,678	0.7847
23 Jul 2013 17:47:17	1237073	15886664	hadcm3n_39vm_2020_40_008399620_0	25,920	20,512	0.7914