Task 15473306

Name	hadcm3n_o5m2_2100_40_008254251_3
Workunit	8409375
Created	7 Dec 2012, 4:52:50 UTC
Sent	7 Dec 2012, 4:52:59 UTC
Report deadline	8 Mar 2013, 12:20:10 UTC
Received	25 Dec 2012, 23:16:06 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1252245
Run time	10 days 9 hours 35 min 2 sec
CPU time	9 days 17 hours 13 min 6 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	3.63 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4272, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o5m2ko.pju7c10 Error converting file to netcdf: dataout/o5m2ko.piu7c10 Error converting file to netcdf: dataout/o5m2ko.pfu7c10 Error converting file to netcdf: dataout/o5m2ka.phu7c10 Error converting file to netcdf: dataout/o5m2ka.pgu7c10 Error converting file to netcdf: dataout/o5m2ka.peu7c10 Error converting file to netcdf: dataout/o5m2ka.pdu7c10 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5752, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Dec 2012 14:30:46	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	855,360	825,420	0.9650
25 Dec 2012 06:42:05	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	829,440	802,419	0.9674
24 Dec 2012 20:28:25	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	803,520	778,754	0.9692
24 Dec 2012 13:55:42	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	777,600	755,482	0.9716
24 Dec 2012 07:27:36	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	751,680	732,253	0.9742
24 Dec 2012 00:54:24	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	725,760	708,822	0.9767
23 Dec 2012 17:51:27	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	699,840	685,562	0.9796
23 Dec 2012 11:03:34	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	673,920	662,434	0.9830
23 Dec 2012 04:27:13	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	648,000	638,807	0.9858
22 Dec 2012 04:03:26	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	622,080	615,819	0.9899
21 Dec 2012 20:33:50	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	596,160	592,723	0.9942
21 Dec 2012 12:06:17	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	570,240	569,358	0.9985
21 Dec 2012 02:53:49	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	544,320	546,649	1.0043
17 Dec 2012 07:09:22	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	518,400	523,469	1.0098
16 Dec 2012 23:17:51	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	492,480	498,129	1.0115
16 Dec 2012 15:15:49	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	466,560	472,242	1.0122
15 Dec 2012 07:28:20	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	440,640	446,445	1.0132
15 Dec 2012 00:31:49	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	414,720	420,370	1.0136
14 Dec 2012 14:17:07	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	388,800	394,365	1.0143
14 Dec 2012 12:20:20	1252245	15473306	hadcm3n_o5m2_2100_40_008254251_3	362,880	368,685	1.0160