Task 16006922

Name	hadcm3n_7951_1980_40_008423128_1
Workunit	8573984
Created	7 Sep 2013, 14:27:03 UTC
Sent	7 Sep 2013, 15:08:15 UTC
Report deadline	7 Dec 2013, 22:35:26 UTC
Received	26 Dec 2013, 21:51:21 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1213041
Run time	33 days 2 hours 12 min 54 sec
CPU time	28 days 3 hours 33 min 15 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.38 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:49:26 (5984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5136, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4596, iMonCtr=1 Model crash detected, will try to restart... 10:39:15 (4336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:27:04 (4140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:29:19 (5408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:42:15 (5208): No heartbeat from core client for 30 sec - exiting 16:42:16 (5208): No heartbeat from core client for 30 sec - exiting 16:42:17 (5208): No heartbeat from core client for 30 sec - exiting 16:42:18 (5208): No heartbeat from core client for 30 sec - exiting 16:42:19 (5208): No heartbeat from core client for 30 sec - exiting 16:42:20 (5208): No heartbeat from core client for 30 sec - exiting 16:42:21 (5208): No heartbeat from core client for 30 sec - exiting 16:42:22 (5208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5808, iMonCtr=1 Model crash detected, will try to restart... 16:52:43 (5864): No heartbeat from core client for 30 sec - exiting 16:52:44 (5864): No heartbeat from core client for 30 sec - exiting 16:52:45 (5864): No heartbeat from core client for 30 sec - exiting 16:52:46 (5864): No heartbeat from core client for 30 sec - exiting 16:52:47 (5864): No heartbeat from core client for 30 sec - exiting 16:52:48 (5864): No heartbeat from core client for 30 sec - exiting 16:52:49 (5864): No heartbeat from core client for 30 sec - exiting 16:52:50 (5864): No heartbeat from core client for 30 sec - exiting 16:52:51 (5864): No heartbeat from core client for 30 sec - exiting 16:52:52 (5864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:26:16 (6128): No heartbeat from core client for 30 sec - exiting 16:26:17 (6128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:35:27 (5488): No heartbeat from core client for 30 sec - exiting 16:35:28 (5488): No heartbeat from core client for 30 sec - exiting 16:35:29 (5488): No heartbeat from core client for 30 sec - exiting 16:35:30 (5488): No heartbeat from core client for 30 sec - exiting 16:35:31 (5488): No heartbeat from core client for 30 sec - exiting 16:35:32 (5488): No heartbeat from core client for 30 sec - exiting 16:35:33 (5488): No heartbeat from core client for 30 sec - exiting 16:35:34 (5488): No heartbeat from core client for 30 sec - exiting 16:35:35 (5488): No heartbeat from core client for 30 sec - exiting 16:35:36 (5488): No heartbeat from core client for 30 sec - exiting 16:35:37 (5488): No heartbeat from core client for 30 sec - exiting 16:35:38 (5488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:53:01 (5504): No heartbeat from core client for 30 sec - exiting 09:53:02 (5504): No heartbeat from core client for 30 sec - exiting 09:53:03 (5504): No heartbeat from core client for 30 sec - exiting 09:53:04 (5504): No heartbeat from core client for 30 sec - exiting 09:53:05 (5504): No heartbeat from core client for 30 sec - exiting 09:53:06 (5504): No heartbeat from core client for 30 sec - exiting 09:53:08 (5504): No heartbeat from core client for 30 sec - exiting 09:53:09 (5504): No heartbeat from core client for 30 sec - exiting 09:53:10 (5504): No heartbeat from core client for 30 sec - exiting 09:53:11 (5504): No heartbeat from core client for 30 sec - exiting 09:53:12 (5504): No heartbeat from core client for 30 sec - exiting 09:53:13 (5504): No heartbeat from core client for 30 sec - exiting 09:53:14 (5504): No heartbeat from core client for 30 sec - exiting 09:53:15 (5504): No heartbeat from core client for 30 sec - exiting 09:53:16 (5504): No heartbeat from core client for 30 sec - exiting 09:53:17 (5504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:55:16 (5968): No heartbeat from core client for 30 sec - exiting 17:55:17 (5968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:59:57 (5472): No heartbeat from core client for 30 sec - exiting 17:59:58 (5472): No heartbeat from core client for 30 sec - exiting 17:59:59 (5472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2680, iMonCtr=1 Model crash detected, will try to restart... 18:09:30 (3896): No heartbeat from core client for 30 sec - exiting 18:09:31 (3896): No heartbeat from core client for 30 sec - exiting 18:09:32 (3896): No heartbeat from core client for 30 sec - exiting 18:09:33 (3896): No heartbeat from core client for 30 sec - exiting 18:09:34 (3896): No heartbeat from core client for 30 sec - exiting 18:09:35 (3896): No heartbeat from core client for 30 sec - exiting 18:09:36 (3896): No heartbeat from core client for 30 sec - exiting 18:09:37 (3896): No heartbeat from core client for 30 sec - exiting 18:09:38 (3896): No heartbeat from core client for 30 sec - exiting 18:09:39 (3896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4032, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4032, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4032, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4032, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4032, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4032, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish 12:38:21 (5256): No heartbeat from core client for 30 sec - exiting 12:38:22 (5256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:43:57 (5132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:48:41 (6036): No heartbeat from core client for 30 sec - exiting 17:48:42 (6036): No heartbeat from core client for 30 sec - exiting 17:48:43 (6036): No heartbeat from core client for 30 sec - exiting 17:48:44 (6036): No heartbeat from core client for 30 sec - exiting 17:48:45 (6036): No heartbeat from core client for 30 sec - exiting 17:48:46 (6036): No heartbeat from core client for 30 sec - exiting 17:48:48 (6036): No heartbeat from core client for 30 sec - exiting 17:48:49 (6036): No heartbeat from core client for 30 sec - exiting 17:48:50 (6036): No heartbeat from core client for 30 sec - exiting 17:48:51 (6036): No heartbeat from core client for 30 sec - exiting 17:48:52 (6036): No heartbeat from core client for 30 sec - exiting 17:48:53 (6036): No heartbeat from core client for 30 sec - exiting 17:48:54 (6036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:44:41 (3756): No heartbeat from core client for 30 sec - exiting 18:44:42 (3756): No heartbeat from core client for 30 sec - exiting 18:44:43 (3756): No heartbeat from core client for 30 sec - exiting 18:44:44 (3756): No heartbeat from core client for 30 sec - exiting 18:44:45 (3756): No heartbeat from core client for 30 sec - exiting 18:44:46 (3756): No heartbeat from core client for 30 sec - exiting 18:44:47 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:37:02 (5924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:21:13 (6468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:19:06 (1756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6620, iMonCtr=1 Model crash detected, will try to restart... 11:43:40 (5788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:20:18 (5760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:44:18 (5700): No heartbeat from core client for 30 sec - exiting 16:44:19 (5700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:16:06 (7012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:39:07 (6404): No heartbeat from core client for 30 sec - exiting 16:39:08 (6404): No heartbeat from core client for 30 sec - exiting 16:39:09 (6404): No heartbeat from core client for 30 sec - exiting 16:39:10 (6404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:28:57 (5456): No heartbeat from core client for 30 sec - exiting 16:28:58 (5456): No heartbeat from core client for 30 sec - exiting 16:28:59 (5456): No heartbeat from core client for 30 sec - exiting 16:29:00 (5456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:56:05 (5804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:51:41 (5772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:34:04 (5456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7160, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 17:04:46 (6552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5672, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Dec 2013 20:53:58	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	1,036,800	2,431,589	2.3453
25 Dec 2013 22:01:09	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	1,010,880	2,405,823	2.3799
24 Dec 2013 20:52:36	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	984,960	2,380,031	2.4164
24 Dec 2013 13:44:49	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	959,040	2,354,292	2.4548
23 Dec 2013 17:03:30	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	933,120	2,328,561	2.4955
23 Dec 2013 09:56:27	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	907,200	2,302,839	2.5384
22 Dec 2013 16:40:32	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	881,280	2,277,092	2.5838
21 Dec 2013 22:52:04	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	855,360	2,251,301	2.6320
21 Dec 2013 15:44:06	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	829,440	2,225,615	2.6833
20 Dec 2013 19:58:19	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	803,520	2,199,876	2.7378
19 Dec 2013 18:34:07	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	777,600	2,174,151	2.7960
17 Dec 2013 22:08:28	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	751,680	2,148,363	2.8581
16 Dec 2013 21:07:25	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	725,760	2,122,628	2.9247
15 Dec 2013 20:00:19	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	699,840	2,096,891	2.9962
15 Dec 2013 12:52:25	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	673,920	2,071,170	3.0733
14 Dec 2013 18:11:48	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	648,000	2,045,417	3.1565
13 Dec 2013 23:07:36	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	622,080	2,019,646	3.2466
12 Dec 2013 21:24:48	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	596,160	1,993,847	3.3445
11 Dec 2013 20:48:51	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	570,240	1,967,870	3.4510
10 Dec 2013 18:43:11	1213041	16006922	hadcm3n_7951_1980_40_008423128_1	544,320	1,940,936	3.5658