climateprediction.net home page
Task 15529518

Task 15529518

Name hadcm3n_o78r_2140_40_008269078_4
Workunit 8424202
Created 12 Jan 2013, 9:14:55 UTC
Sent 12 Jan 2013, 9:15:03 UTC
Report deadline 13 Apr 2013, 16:42:14 UTC
Received 14 Feb 2013, 13:45:05 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1091586
Run time 30 days 6 hours 20 min 25 sec
CPU time 26 days 15 hours 40 min 19 sec
Validate state Invalid
Credit 9,331.20
Device peak FLOPS 1.76 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr 
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
The device does not recognize the command. (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
21:20:11 (2248): No heartbeat from core client for 30 sec - exiting
21:20:13 (2248): No heartbeat from core client for 30 sec - exiting
21:20:14 (2248): No heartbeat from core client for 30 sec - exiting
21:20:15 (2248): No heartbeat from core client for 30 sec - exiting
21:20:16 (2248): No heartbeat from core client for 30 sec - exiting
21:20:17 (2248): No heartbeat from core client for 30 sec - exiting
21:20:18 (2248): No heartbeat from core client for 30 sec - exiting
21:20:19 (2248): No heartbeat from core client for 30 sec - exiting
21:20:20 (2248): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5312, iMonCtr=1
Model crash detected, will try to restart...
14:44:35 (5940): No heartbeat from core client for 30 sec - exiting
14:44:37 (5940): No heartbeat from core client for 30 sec - exiting
14:44:38 (5940): No heartbeat from core client for 30 sec - exiting
14:44:39 (5940): No heartbeat from core client for 30 sec - exiting
14:44:40 (5940): No heartbeat from core client for 30 sec - exiting
14:44:41 (5940): No heartbeat from core client for 30 sec - exiting
14:44:42 (5940): No heartbeat from core client for 30 sec - exiting
14:44:43 (5940): No heartbeat from core client for 30 sec - exiting
14:44:44 (5940): No heartbeat from core client for 30 sec - exiting
14:44:45 (5940): No heartbeat from core client for 30 sec - exiting
14:44:46 (5940): No heartbeat from core client for 30 sec - exiting
14:44:47 (5940): No heartbeat from core client for 30 sec - exiting
14:44:48 (5940): No heartbeat from core client for 30 sec - exiting
14:44:49 (5940): No heartbeat from core client for 30 sec - exiting
14:44:50 (5940): No heartbeat from core client for 30 sec - exiting
14:44:51 (5940): No heartbeat from core client for 30 sec - exiting
14:44:53 (5940): No heartbeat from core client for 30 sec - exiting
14:44:54 (5940): No heartbeat from core client for 30 sec - exiting
14:44:55 (5940): No heartbeat from core client for 30 sec - exiting
14:44:56 (5940): No heartbeat from core client for 30 sec - exiting
14:44:57 (5940): No heartbeat from core client for 30 sec - exiting
14:44:58 (5940): No heartbeat from core client for 30 sec - exiting
14:44:59 (5940): No heartbeat from core client for 30 sec - exiting
14:45:00 (5940): No heartbeat from core client for 30 sec - exiting
14:45:01 (5940): No heartbeat from core client for 30 sec - exiting
14:45:02 (5940): No heartbeat from core client for 30 sec - exiting
14:45:03 (5940): No heartbeat from core client for 30 sec - exiting
14:45:05 (5940): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
17:56:06 (5760): No heartbeat from core client for 30 sec - exiting
17:56:07 (5760): No heartbeat from core client for 30 sec - exiting
17:56:08 (5760): No heartbeat from core client for 30 sec - exiting
17:56:09 (5760): No heartbeat from core client for 30 sec - exiting
17:56:10 (5760): No heartbeat from core client for 30 sec - exiting
17:56:11 (5760): No heartbeat from core client for 30 sec - exiting
17:56:12 (5760): No heartbeat from core client for 30 sec - exiting
17:56:13 (5760): No heartbeat from core client for 30 sec - exiting
17:56:14 (5760): No heartbeat from core client for 30 sec - exiting
17:56:15 (5760): No heartbeat from core client for 30 sec - exiting
17:56:16 (5760): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
22:44:43 (5712): No heartbeat from core client for 30 sec - exiting
22:44:45 (5712): No heartbeat from core client for 30 sec - exiting
22:44:46 (5712): No heartbeat from core client for 30 sec - exiting
22:44:47 (5712): No heartbeat from core client for 30 sec - exiting
22:44:48 (5712): No heartbeat from core client for 30 sec - exiting
22:44:49 (5712): No heartbeat from core client for 30 sec - exiting
22:44:50 (5712): No heartbeat from core client for 30 sec - exiting
22:44:51 (5712): No heartbeat from core client for 30 sec - exiting
22:44:52 (5712): No heartbeat from core client for 30 sec - exiting
22:44:54 (5712): No heartbeat from core client for 30 sec - exiting
22:44:55 (5712): No heartbeat from core client for 30 sec - exiting
22:44:56 (5712): No heartbeat from core client for 30 sec - exiting
22:44:57 (5712): No heartbeat from core client for 30 sec - exiting
22:44:58 (5712): No heartbeat from core client for 30 sec - exiting
22:44:59 (5712): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
MainError:	06:39:53 AM	No files match the supplied pattern.
MainError:	06:39:53 AM	No files match the supplied pattern.
CPDN Monitor - Quit request from BOINC...
MainError:	04:47:43 AM	No files match the supplied pattern.
MainError:	04:47:43 AM	No files match the supplied pattern.
MainError:	01:50:37 AM	No files match the supplied pattern.
MainError:	01:50:37 AM	No files match the supplied pattern.
MainError:	10:27:53 PM	No files match the supplied pattern.
MainError:	10:27:53 PM	No files match the supplied pattern.
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	10:36:26 PM	No files match the supplied pattern.
MainError:	10:36:26 PM	No files match the supplied pattern.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7100, iMonCtr=1
Model crash detected, will try to restart...
00:15:12 (7056): No heartbeat from core client for 30 sec - exiting
00:15:13 (7056): No heartbeat from core client for 30 sec - exiting
00:15:14 (7056): No heartbeat from core client for 30 sec - exiting
00:15:15 (7056): No heartbeat from core client for 30 sec - exiting
00:15:17 (7056): No heartbeat from core client for 30 sec - exiting
00:15:18 (7056): No heartbeat from core client for 30 sec - exiting
00:15:19 (7056): No heartbeat from core client for 30 sec - exiting
00:15:20 (7056): No heartbeat from core client for 30 sec - exiting
00:15:21 (7056): No heartbeat from core client for 30 sec - exiting
00:15:22 (7056): No heartbeat from core client for 30 sec - exiting
00:15:23 (7056): No heartbeat from core client for 30 sec - exiting
00:15:24 (7056): No heartbeat from core client for 30 sec - exiting
00:15:25 (7056): No heartbeat from core client for 30 sec - exiting
00:15:26 (7056): No heartbeat from core client for 30 sec - exiting
00:15:28 (7056): No heartbeat from core client for 30 sec - exiting
00:15:29 (7056): No heartbeat from core client for 30 sec - exiting
00:15:30 (7056): No heartbeat from core client for 30 sec - exiting
00:15:31 (7056): No heartbeat from core client for 30 sec - exiting
00:15:32 (7056): No heartbeat from core client for 30 sec - exiting
00:15:33 (7056): No heartbeat from core client for 30 sec - exiting
00:15:34 (7056): No heartbeat from core client for 30 sec - exiting
00:15:35 (7056): No heartbeat from core client for 30 sec - exiting
00:15:36 (7056): No heartbeat from core client for 30 sec - exiting
00:15:37 (7056): No heartbeat from core client for 30 sec - exiting
00:15:39 (7056): No heartbeat from core client for 30 sec - exiting
00:15:40 (7056): No heartbeat from core client for 30 sec - exiting
00:15:41 (7056): No heartbeat from core client for 30 sec - exiting
00:15:42 (7056): No heartbeat from core client for 30 sec - exiting
00:15:43 (7056): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:15:44 (7056): No heartbeat from core client for 30 sec - exiting
00:15:45 (7056): No heartbeat from core client for 30 sec - exiting
00:26:14 (8068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:00:35 (7176): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
MainError:	08:37:24 PM	No files match the supplied pattern.
MainError:	08:37:24 PM	No files match the supplied pattern.
MainError:	05:39:22 PM	No files match the supplied pattern.
MainError:	05:39:22 PM	No files match the supplied pattern.
MainError:	02:37:07 PM	No files match the supplied pattern.
MainError:	02:37:07 PM	No files match the supplied pattern.
MainError:	12:50:16 AM	No files match the supplied pattern.
MainError:	12:50:16 AM	No files match the supplied pattern.
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	01:45:35 PM	No files match the supplied pattern.
MainError:	01:45:35 PM	No files match the supplied pattern.
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3992, iMonCtr=1
Model crash detected, will try to restart...
Error converting file to netcdf: dataout/o78rka.ph11c10
Error converting file to netcdf: dataout/o78rka.pg11c10
Error converting file to netcdf: dataout/o78rka.pe11c10
MainError:	11:56:06 AM	No files match the supplied pattern.
MainError:	11:56:06 AM	No files match the supplied pattern.
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Feb 2013 12:01:23 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 777,600 2,302,780 2.9614
13 Feb 2013 13:50:18 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 751,680 2,232,123 2.9695
12 Feb 2013 12:55:14 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 725,760 2,161,257 2.9779
11 Feb 2013 14:40:35 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 699,840 2,090,553 2.9872
10 Feb 2013 17:41:37 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 673,920 2,020,662 2.9984
09 Feb 2013 20:42:20 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 648,000 1,950,724 3.0104
08 Feb 2013 22:39:06 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 622,080 1,879,999 3.0221
07 Feb 2013 22:32:51 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 596,160 1,805,117 3.0279
07 Feb 2013 01:55:39 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 570,240 1,736,006 3.0443
06 Feb 2013 04:52:19 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 544,320 1,666,006 3.0607
05 Feb 2013 06:41:28 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 518,400 1,596,228 3.0791
04 Feb 2013 09:48:01 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 492,480 1,526,817 3.1003
02 Feb 2013 14:15:32 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 466,560 1,411,926 3.0262
31 Jan 2013 12:20:08 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 440,640 1,281,029 2.9072
29 Jan 2013 14:30:24 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 414,720 1,167,294 2.8147
28 Jan 2013 16:20:43 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 388,800 1,093,535 2.8126
27 Jan 2013 18:28:12 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 362,880 1,020,382 2.8119
26 Jan 2013 20:27:24 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 336,960 947,323 2.8114
25 Jan 2013 21:57:45 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 311,040 876,748 2.8188
24 Jan 2013 23:23:24 1091586 15529518 hadcm3n_o78r_2140_40_008269078_4 285,120 801,679 2.8117


©2024 climateprediction.net