Task 15781123

Name	hadcm3n_4bzz_1980_40_008367506_1
Workunit	8518365
Created	13 May 2013, 7:10:36 UTC
Sent	13 May 2013, 7:12:23 UTC
Report deadline	12 Aug 2013, 14:39:34 UTC
Received	6 Jul 2013, 9:21:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1239712
Run time	26 days 9 hours 24 min 56 sec
CPU time	24 days 21 hours 59 min 12 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	2.44 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> Zariadenie nepozná tento príkaz. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4092, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:34:38 (1912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:18:14 (6836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:12:38 (33332): No heartbeat from core client for 30 sec - exiting 21:12:39 (33332): No heartbeat from core client for 30 sec - exiting 21:12:40 (33332): No heartbeat from core client for 30 sec - exiting 21:12:41 (33332): No heartbeat from core client for 30 sec - exiting 21:12:42 (33332): No heartbeat from core client for 30 sec - exiting 21:12:43 (33332): No heartbeat from core client for 30 sec - exiting 21:12:44 (33332): No heartbeat from core client for 30 sec - exiting 21:12:45 (33332): No heartbeat from core client for 30 sec - exiting 21:12:46 (33332): No heartbeat from core client for 30 sec - exiting 21:12:47 (33332): No heartbeat from core client for 30 sec - exiting 21:12:48 (33332): No heartbeat from core client for 30 sec - exiting 21:12:49 (33332): No heartbeat from core client for 30 sec - exiting 21:12:50 (33332): No heartbeat from core client for 30 sec - exiting 21:12:51 (33332): No heartbeat from core client for 30 sec - exiting 21:12:52 (33332): No heartbeat from core client for 30 sec - exiting 21:12:53 (33332): No heartbeat from core client for 30 sec - exiting 21:12:54 (33332): No heartbeat from core client for 30 sec - exiting 21:12:55 (33332): No heartbeat from core client for 30 sec - exiting 21:12:56 (33332): No heartbeat from core client for 30 sec - exiting 21:12:57 (33332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:58:51 (6164): No heartbeat from core client for 30 sec - exiting 23:58:52 (6164): No heartbeat from core client for 30 sec - exiting 23:58:53 (6164): No heartbeat from core client for 30 sec - exiting 23:58:54 (6164): No heartbeat from core client for 30 sec - exiting 23:58:55 (6164): No heartbeat from core client for 30 sec - exiting 23:58:56 (6164): No heartbeat from core client for 30 sec - exiting 23:58:57 (6164): No heartbeat from core client for 30 sec - exiting 23:58:58 (6164): No heartbeat from core client for 30 sec - exiting 23:58:59 (6164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4bzzko.pjj8c10 Error converting file to netcdf: dataout/4bzzko.pij8c10 Error converting file to netcdf: dataout/4bzzko.pfj8c10 Error converting file to netcdf: dataout/4bzzka.phj8c10 Error converting file to netcdf: dataout/4bzzka.pgj8c10 Error converting file to netcdf: dataout/4bzzka.pej8c10 Error converting file to netcdf: dataout/4bzzka.pdj8c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7644, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6268, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Jul 2013 14:27:33	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	855,360	2,138,239	2.4998
02 Jul 2013 21:57:53	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	829,440	2,067,722	2.4929
02 Jul 2013 11:51:39	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	803,520	1,998,342	2.4870
02 Jul 2013 10:41:59	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	777,600	1,928,582	2.4802
27 Jun 2013 16:38:02	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	751,680	1,858,938	2.4730
26 Jun 2013 18:39:05	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	725,760	1,788,894	2.4649
25 Jun 2013 22:52:39	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	699,840	1,719,606	2.4571
25 Jun 2013 05:39:50	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	673,920	1,650,736	2.4495
24 Jun 2013 03:32:41	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	648,000	1,578,413	2.4358
23 Jun 2013 07:21:07	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	622,080	1,505,675	2.4204
22 Jun 2013 14:43:50	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	596,160	1,439,989	2.4154
22 Jun 2013 14:43:50	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	570,240	1,378,377	2.4172
20 Jun 2013 18:57:38	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	544,320	1,301,241	2.3906
19 Jun 2013 16:15:22	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	518,400	1,232,327	2.3772
18 Jun 2013 00:14:30	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	492,480	1,171,545	2.3789
15 Jun 2013 05:08:18	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	466,560	1,104,837	2.3680
12 Jun 2013 15:21:55	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	440,640	1,041,581	2.3638
10 Jun 2013 18:24:10	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	414,720	977,863	2.3579
07 Jun 2013 23:08:53	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	388,800	909,381	2.3389
07 Jun 2013 04:08:29	1239712	15781123	hadcm3n_4bzz_1980_40_008367506_1	362,880	844,411	2.3270