Task 13954053

Name	hadcm3n_ycaz_1940_40_007682590_3
Workunit	7837677
Created	23 Jan 2012, 6:02:13 UTC
Sent	23 Jan 2012, 6:02:28 UTC
Report deadline	23 Apr 2012, 13:29:39 UTC
Received	26 Jun 2012, 16:26:10 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1071380
Run time	25 days 11 hours 34 min 49 sec
CPU time	25 days 3 hours 50 min 13 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.66 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 17:05:22 (1172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:05:23 (1172): No heartbeat from core client for 30 sec - exiting 17:05:24 (1172): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2996, iMonCtr=1 Model crash detected, will try to restart... 08:30:46 (3840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:21:52 (3548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:21:53 (3548): No heartbeat from core client for 30 sec - exiting 23:21:55 (3548): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1456, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2644, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3440, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3240, iMonCtr=1 Model crash detected, will try to restart... 00:06:08 (2764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:06:09 (2764): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5684, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3456, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=204, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3568, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4376, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3604, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1268, iMonCtr=1 Model crash detected, will try to restart... 09:33:54 (2916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:52:15 (2936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3580, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3020, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3264, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 13:59:10 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2112, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2880, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2044, iMonCtr=1 Model crash detected, will try to restart... 22:58:06 (3036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:42:56 (2936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2712, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3376, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 12:11:55 (2176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:20:32 (3552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:20:34 (3552): No heartbeat from core client for 30 sec - exiting 23:20:35 (3552): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1636, iMonCtr=1 Model crash detected, will try to restart... 00:09:29 (652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:42:29 (3220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3508, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3740, iMonCtr=1 Model crash detected, will try to restart... 01:39:39 (2144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 03:18:53 (1944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2948, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1660, iMonCtr=1 Model crash detected, will try to restart... 10:13:26 (3800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:00:12 (3300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:00:13 (3300): No heartbeat from core client for 30 sec - exiting 01:00:14 (3300): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1848, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Jun 2012 15:27:19	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	1,036,800	2,173,806	2.0966
20 Jun 2012 03:28:15	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	1,010,880	2,120,461	2.0976
17 Jun 2012 23:57:26	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	984,960	2,064,619	2.0961
17 Jun 2012 08:18:51	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	959,040	2,008,428	2.0942
16 Jun 2012 16:33:11	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	933,120	1,952,029	2.0919
12 Jun 2012 18:22:43	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	907,200	1,894,566	2.0884
08 Jun 2012 00:24:26	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	881,280	1,840,453	2.0884
04 Jun 2012 18:32:48	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	855,360	1,786,017	2.0880
01 Jun 2012 18:55:59	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	829,440	1,731,205	2.0872
31 May 2012 04:19:34	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	803,520	1,675,825	2.0856
22 May 2012 03:47:06	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	777,600	1,623,231	2.0875
18 May 2012 15:56:21	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	751,680	1,569,395	2.0878
17 May 2012 14:33:19	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	725,760	1,517,066	2.0903
15 May 2012 22:50:52	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	699,840	1,461,680	2.0886
14 May 2012 23:27:32	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	673,920	1,405,551	2.0856
08 May 2012 23:33:32	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	648,000	1,351,926	2.0863
03 May 2012 04:25:34	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	622,080	1,297,314	2.0854
29 Apr 2012 18:57:31	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	596,160	1,243,602	2.0860
22 Apr 2012 17:07:09	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	570,240	1,188,883	2.0849
16 Apr 2012 19:21:32	1071380	13954053	hadcm3n_ycaz_1940_40_007682590_3	544,320	1,135,353	2.0858