Task 16081414

Name	hadcm3n_3gcs_1980_40_008318608_4
Workunit	8469743
Created	16 Nov 2013, 21:21:12 UTC
Sent	16 Nov 2013, 21:21:25 UTC
Report deadline	16 Feb 2014, 4:48:36 UTC
Received	19 Dec 2013, 2:57:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1298051
Run time	21 days 10 hours 2 min 25 sec
CPU time	11 days 12 hours 48 min 41 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.04 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3956, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 01:33:04 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:33:05 (4112): No heartbeat from core client for 30 sec - exiting 01:33:06 (4112): No heartbeat from core client for 30 sec - exiting 01:33:07 (4112): No heartbeat from core client for 30 sec - exiting 01:33:08 (4112): No heartbeat from core client for 30 sec - exiting 01:33:09 (4112): No heartbeat from core client for 30 sec - exiting 01:33:10 (4112): No heartbeat from core client for 30 sec - exiting 01:33:11 (4112): No heartbeat from core client for 30 sec - exiting 01:33:12 (4112): No heartbeat from core client for 30 sec - exiting 01:33:13 (4112): No heartbeat from core client for 30 sec - exiting 01:33:14 (4112): No heartbeat from core client for 30 sec - exiting 01:33:15 (4112): No heartbeat from core client for 30 sec - exiting 01:33:16 (4112): No heartbeat from core client for 30 sec - exiting 01:33:17 (4112): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:39:35 (10492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:39:46 (10492): No heartbeat from core client for 30 sec - exiting 04:39:47 (10492): No heartbeat from core client for 30 sec - exiting 04:39:48 (10492): No heartbeat from core client for 30 sec - exiting 04:39:49 (10492): No heartbeat from core client for 30 sec - exiting 04:39:50 (10492): No heartbeat from core client for 30 sec - exiting 04:39:51 (10492): No heartbeat from core client for 30 sec - exiting 04:39:52 (10492): No heartbeat from core client for 30 sec - exiting 04:39:53 (10492): No heartbeat from core client for 30 sec - exiting 04:39:54 (10492): No heartbeat from core client for 30 sec - exiting 04:39:55 (10492): No heartbeat from core client for 30 sec - exiting 04:39:56 (10492): No heartbeat from core client for 30 sec - exiting 04:39:57 (10492): No heartbeat from core client for 30 sec - exiting 04:39:58 (10492): No heartbeat from core client for 30 sec - exiting 04:39:59 (10492): No heartbeat from core client for 30 sec - exiting 04:40:00 (10492): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 20:17:29 (3568): No heartbeat from core client for 30 sec - exiting 20:17:30 (3568): No heartbeat from core client for 30 sec - exiting 20:17:31 (3568): No heartbeat from core client for 30 sec - exiting 20:17:32 (3568): No heartbeat from core client for 30 sec - exiting 20:17:33 (3568): No heartbeat from core client for 30 sec - exiting 20:17:34 (3568): No heartbeat from core client for 30 sec - exiting 20:17:35 (3568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3gcsko.pji5c10 Error converting file to netcdf: dataout/3gcsko.pii5c10 Error converting file to netcdf: dataout/3gcsko.pfi5c10 Error converting file to netcdf: dataout/3gcska.phi5c10 Error converting file to netcdf: dataout/3gcska.pgi5c10 Error converting file to netcdf: dataout/3gcska.pei5c10 Error converting file to netcdf: dataout/3gcska.pdi5c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8460, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17036, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7012, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Dec 2013 02:58:40	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	518,400	996,515	1.9223
17 Dec 2013 16:33:17	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	492,480	946,027	1.9209
16 Dec 2013 14:04:32	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	466,560	897,802	1.9243
14 Dec 2013 15:58:56	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	440,640	843,418	1.9141
13 Dec 2013 05:22:24	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	414,720	796,079	1.9196
11 Dec 2013 23:35:10	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	388,800	740,884	1.9056
10 Dec 2013 04:35:11	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	362,880	692,703	1.9089
08 Dec 2013 15:11:55	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	336,960	641,679	1.9043
07 Dec 2013 06:34:43	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	311,040	597,000	1.9194
06 Dec 2013 00:03:56	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	285,120	547,352	1.9197
30 Nov 2013 23:26:41	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	259,200	494,347	1.9072
30 Nov 2013 09:30:21	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	233,280	457,543	1.9613
30 Nov 2013 06:39:33	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	207,360	429,318	2.0704
28 Nov 2013 08:12:17	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	181,440	381,932	2.1050
26 Nov 2013 23:38:32	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	155,520	327,003	2.1026
24 Nov 2013 11:22:17	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	129,600	270,216	2.0850
23 Nov 2013 06:01:30	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	103,680	217,784	2.1005
21 Nov 2013 23:07:26	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	77,760	161,702	2.0795
20 Nov 2013 02:09:50	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	51,840	107,241	2.0687
18 Nov 2013 06:09:08	1298051	16081414	hadcm3n_3gcs_1980_40_008318608_4	25,920	55,230	2.1308