Task 15490208

Name	hadcm3n_38jh_1940_40_008261903_0
Workunit	8417027
Created	20 Dec 2012, 23:46:49 UTC
Sent	20 Dec 2012, 23:49:44 UTC
Report deadline	22 Mar 2013, 7:16:55 UTC
Received	15 Jan 2013, 5:41:30 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1212245
Run time	20 days 9 hours 24 min 59 sec
CPU time	19 days 6 hours 40 min 26 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.40 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 14:01:52 (124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:59:08 (6940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:31:43 (2528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:31:44 (2528): No heartbeat from core client for 30 sec - exiting 00:31:45 (2528): No heartbeat from core client for 30 sec - exiting 00:31:46 (2528): No heartbeat from core client for 30 sec - exiting 00:31:48 (2528): No heartbeat from core client for 30 sec - exiting 00:31:49 (2528): No heartbeat from core client for 30 sec - exiting 00:31:50 (2528): No heartbeat from core client for 30 sec - exiting 00:31:51 (2528): No heartbeat from core client for 30 sec - exiting 00:31:52 (2528): No heartbeat from core client for 30 sec - exiting 00:31:53 (2528): No heartbeat from core client for 30 sec - exiting 00:31:54 (2528): No heartbeat from core client for 30 sec - exiting 04:44:09 (7848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:58:02 (3556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:58:04 (3556): No heartbeat from core client for 30 sec - exiting 00:58:05 (3556): No heartbeat from core client for 30 sec - exiting 00:58:06 (3556): No heartbeat from core client for 30 sec - exiting 00:58:07 (3556): No heartbeat from core client for 30 sec - exiting 00:58:08 (3556): No heartbeat from core client for 30 sec - exiting 00:58:09 (3556): No heartbeat from core client for 30 sec - exiting 00:58:10 (3556): No heartbeat from core client for 30 sec - exiting 00:58:11 (3556): No heartbeat from core client for 30 sec - exiting 00:58:12 (3556): No heartbeat from core client for 30 sec - exiting 00:58:13 (3556): No heartbeat from core client for 30 sec - exiting 00:59:57 (7348): No heartbeat from core client for 30 sec - exiting 00:59:58 (7348): No heartbeat from core client for 30 sec - exiting 00:59:59 (7348): No heartbeat from core client for 30 sec - exiting 01:00:00 (7348): No heartbeat from core client for 30 sec - exiting 01:00:01 (7348): No heartbeat from core client for 30 sec - exiting 01:00:02 (7348): No heartbeat from core client for 30 sec - exiting 01:00:03 (7348): No heartbeat from core client for 30 sec - exiting 01:00:04 (7348): No heartbeat from core client for 30 sec - exiting 01:00:05 (7348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Suspended CPDN Monitor - Suspend request from BOINC... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4260, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5768, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:38:39 (4984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:40:24 (4252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6020, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:55:41 (8988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:55:42 (8988): No heartbeat from core client for 30 sec - exiting 00:55:43 (8988): No heartbeat from core client for 30 sec - exiting 00:55:44 (8988): No heartbeat from core client for 30 sec - exiting 00:55:45 (8988): No heartbeat from core client for 30 sec - exiting 00:55:46 (8988): No heartbeat from core client for 30 sec - exiting 00:55:47 (8988): No heartbeat from core client for 30 sec - exiting 00:55:48 (8988): No heartbeat from core client for 30 sec - exiting 00:55:49 (8988): No heartbeat from core client for 30 sec - exiting 00:55:50 (8988): No heartbeat from core client for 30 sec - exiting 00:55:51 (8988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:53:57 (8840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:28:54 (7444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:28:55 (7444): No heartbeat from core client for 30 sec - exiting 02:28:57 (7444): No heartbeat from core client for 30 sec - exiting 02:28:58 (7444): No heartbeat from core client for 30 sec - exiting 02:28:59 (7444): No heartbeat from core client for 30 sec - exiting 02:29:00 (7444): No heartbeat from core client for 30 sec - exiting 02:29:01 (7444): No heartbeat from core client for 30 sec - exiting 02:29:02 (7444): No heartbeat from core client for 30 sec - exiting 02:29:03 (7444): No heartbeat from core client for 30 sec - exiting 02:29:04 (7444): No heartbeat from core client for 30 sec - exiting 02:29:05 (7444): No heartbeat from core client for 30 sec - exiting 02:32:02 (2956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:55:01 (8300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:03 (8300): No heartbeat from core client for 30 sec - exiting Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Jan 2013 20:18:57	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	777,600	1,665,438	2.1418
14 Jan 2013 04:02:34	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	751,680	1,610,639	2.1427
12 Jan 2013 06:28:54	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	725,760	1,500,779	2.0679
09 Jan 2013 07:40:01	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	699,840	1,412,168	2.0178
09 Jan 2013 07:40:01	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	673,920	1,359,760	2.0177
09 Jan 2013 07:40:01	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	648,000	1,305,227	2.0142
06 Jan 2013 13:22:04	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	622,080	1,248,045	2.0062
05 Jan 2013 13:20:18	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	596,160	1,193,538	2.0020
04 Jan 2013 10:55:46	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	570,240	1,133,449	1.9877
03 Jan 2013 20:15:50	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	544,320	1,079,204	1.9827
03 Jan 2013 06:35:27	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	518,400	1,022,558	1.9725
02 Jan 2013 09:05:09	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	492,480	972,751	1.9752
02 Jan 2013 09:05:09	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	466,560	918,419	1.9685
02 Jan 2013 09:05:09	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	440,640	865,589	1.9644
02 Jan 2013 09:05:09	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	414,720	812,525	1.9592
02 Jan 2013 09:05:09	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	388,800	760,394	1.9557
02 Jan 2013 09:05:09	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	362,880	708,679	1.9529
28 Dec 2012 20:31:06	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	336,960	656,604	1.9486
28 Dec 2012 04:48:37	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	311,040	603,803	1.9412
27 Dec 2012 14:05:37	1212245	15490208	hadcm3n_38jh_1940_40_008261903_0	285,120	551,574	1.9345