climateprediction.net home page
Task 15527597

Task 15527597

Name hadcm3n_3lbl_1940_40_008260117_1
Workunit 8415241
Created 9 Jan 2013, 23:51:45 UTC
Sent 9 Jan 2013, 23:51:55 UTC
Report deadline 11 Apr 2013, 7:19:06 UTC
Received 18 Feb 2013, 19:02:05 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1261325
Run time 14 days 5 hours 59 min 53 sec
CPU time 12 days 15 hours 14 min 11 sec
Validate state Invalid
Credit 8,398.08
Device peak FLOPS 2.87 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr
<core_client_version>7.0.28</core_client_version>
<![CDATA[
<message>
The device does not recognize the command. (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:03:27 (4816): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:03:29 (4816): No heartbeat from core client for 30 sec - exiting
20:03:30 (4816): No heartbeat from core client for 30 sec - exiting
20:03:31 (4816): No heartbeat from core client for 30 sec - exiting
20:03:32 (4816): No heartbeat from core client for 30 sec - exiting
20:06:51 (688): Can't acquire lockfile (32) - waiting 35s
20:06:57 (5200): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:37:56 (6448): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:37:58 (6448): No heartbeat from core client for 30 sec - exiting
18:37:59 (6448): No heartbeat from core client for 30 sec - exiting
18:38:00 (6448): No heartbeat from core client for 30 sec - exiting
18:38:01 (6448): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:01:10 (8552): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:01:11 (8552): No heartbeat from core client for 30 sec - exiting
19:04:30 (6416): Can't acquire lockfile (32) - waiting 35s
19:04:49 (4276): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:54:41 (9204): No heartbeat from core client for 30 sec - exiting
17:54:44 (9204): No heartbeat from core client for 30 sec - exiting
17:54:45 (9204): No heartbeat from core client for 30 sec - exiting
17:54:46 (9204): No heartbeat from core client for 30 sec - exiting
17:54:47 (9204): No heartbeat from core client for 30 sec - exiting
17:54:49 (9204): No heartbeat from core client for 30 sec - exiting
17:54:52 (9204): No heartbeat from core client for 30 sec - exiting
17:54:54 (9204): No heartbeat from core client for 30 sec - exiting
17:54:55 (9204): No heartbeat from core client for 30 sec - exiting
17:54:56 (9204): No heartbeat from core client for 30 sec - exiting
17:54:57 (9204): No heartbeat from core client for 30 sec - exiting
17:54:59 (9204): No heartbeat from core client for 30 sec - exiting
17:55:01 (9204): No heartbeat from core client for 30 sec - exiting
17:55:02 (9204): No heartbeat from core client for 30 sec - exiting
17:55:03 (9204): No heartbeat from core client for 30 sec - exiting
17:55:04 (9204): No heartbeat from core client for 30 sec - exiting
17:55:05 (9204): No heartbeat from core client for 30 sec - exiting
17:55:06 (9204): No heartbeat from core client for 30 sec - exiting
17:55:07 (9204): No heartbeat from core client for 30 sec - exiting
17:55:08 (9204): No heartbeat from core client for 30 sec - exiting
17:55:09 (9204): No heartbeat from core client for 30 sec - exiting
17:55:10 (9204): No heartbeat from core client for 30 sec - exiting
17:55:11 (9204): No heartbeat from core client for 30 sec - exiting
17:55:12 (9204): No heartbeat from core client for 30 sec - exiting
17:55:13 (9204): No heartbeat from core client for 30 sec - exiting
17:55:14 (9204): No heartbeat from core client for 30 sec - exiting
17:55:15 (9204): No heartbeat from core client for 30 sec - exiting
17:55:16 (9204): No heartbeat from core client for 30 sec - exiting
17:55:17 (9204): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
BUFFIN: C I/O Error feof - Unit 63 - Return code = 16
BUFFIN: C I/O Error feof - Unit 64 - Return code = 16
BUFFIN: C I/O Error feof - Unit 65 - Return code = 16
BUFFIN: C I/O Error feof - Unit 66 - Return code = 16
BUFFIN: C I/O Error feof - Unit 67 - Return code = 16
BUFFIN: C I/O Error feof - Unit 68 - Return code = 16
BUFFIN: C I/O Error feof - Unit 69 - Return code = 16
Error converting file to netcdf: dataout/3lblko.pjf5c10
Error converting file to netcdf: dataout/3lblko.pif5c10
Error converting file to netcdf: dataout/3lblko.pff5c10
Error converting file to netcdf: dataout/3lblka.phf5c10
Error converting file to netcdf: dataout/3lblka.pgf5c10
Error converting file to netcdf: dataout/3lblka.pef5c10
Error converting file to netcdf: dataout/3lblka.pdf5c10
18:30:36 (8232): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:30:40 (8232): No heartbeat from core client for 30 sec - exiting
18:30:42 (8232): No heartbeat from core client for 30 sec - exiting
18:30:43 (8232): No heartbeat from core client for 30 sec - exiting
18:30:44 (8232): No heartbeat from core client for 30 sec - exiting
18:30:45 (8232): No heartbeat from core client for 30 sec - exiting
18:30:46 (8232): No heartbeat from core client for 30 sec - exiting
19:03:05 (6308): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:03:14 (6308): No heartbeat from core client for 30 sec - exiting
19:03:20 (6308): No heartbeat from core client for 30 sec - exiting
19:03:25 (6308): No heartbeat from core client for 30 sec - exiting
19:41:38 (1840): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:45:38 (7152): No heartbeat from core client for 30 sec - exiting
19:45:39 (7152): No heartbeat from core client for 30 sec - exiting
19:45:40 (7152): No heartbeat from core client for 30 sec - exiting
19:45:41 (7152): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:18:30 (3664): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:18:39 (3664): No heartbeat from core client for 30 sec - exiting
20:18:44 (3664): No heartbeat from core client for 30 sec - exiting
20:18:48 (3664): No heartbeat from core client for 30 sec - exiting
20:18:57 (3664): No heartbeat from core client for 30 sec - exiting
20:18:59 (3664): No heartbeat from core client for 30 sec - exiting
20:19:03 (3664): No heartbeat from core client for 30 sec - exiting
20:19:04 (3664): No heartbeat from core client for 30 sec - exiting
20:53:15 (6376): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:53:23 (6376): No heartbeat from core client for 30 sec - exiting
20:53:26 (6376): No heartbeat from core client for 30 sec - exiting
20:53:27 (6376): No heartbeat from core client for 30 sec - exiting
20:53:28 (6376): No heartbeat from core client for 30 sec - exiting
20:53:29 (6376): No heartbeat from core client for 30 sec - exiting
20:53:30 (6376): No heartbeat from core client for 30 sec - exiting
20:53:31 (6376): No heartbeat from core client for 30 sec - exiting
21:25:33 (7752): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:26:13 (7752): No heartbeat from core client for 30 sec - exiting
21:26:15 (7752): No heartbeat from core client for 30 sec - exiting
21:26:18 (7752): No heartbeat from core client for 30 sec - exiting
21:26:31 (7752): No heartbeat from core client for 30 sec - exiting
21:26:32 (7752): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
19:41:31 (6016): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:41:33 (6016): No heartbeat from core client for 30 sec - exiting
19:41:34 (6016): No heartbeat from core client for 30 sec - exiting
19:41:35 (6016): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
21:53:17 (7864): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:53:20 (7864): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:09:47 (4244): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
02:57:51 (8500): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:57:55 (8500): No heartbeat from core client for 30 sec - exiting
02:57:57 (8500): No heartbeat from core client for 30 sec - exiting
02:58:01 (8500): No heartbeat from core client for 30 sec - exiting
02:58:11 (8500): No heartbeat from core client for 30 sec - exiting
02:58:12 (8500): No heartbeat from core client for 30 sec - exiting
02:58:13 (8500): No heartbeat from core client for 30 sec - exiting
02:58:14 (8500): No heartbeat from core client for 30 sec - exiting
02:58:16 (8500): No heartbeat from core client for 30 sec - exiting
02:58:17 (8500): No heartbeat from core client for 30 sec - exiting
03:01:05 (9788): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
18 Feb 2013 17:36:20 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 699,840 1,142,533 1.6326
18 Feb 2013 04:12:17 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 673,920 1,097,732 1.6289
17 Feb 2013 13:55:18 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 648,000 1,052,904 1.6249
17 Feb 2013 00:15:16 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 622,080 1,008,088 1.6205
16 Feb 2013 09:44:10 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 596,160 963,217 1.6157
15 Feb 2013 16:10:48 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 570,240 921,764 1.6164
14 Feb 2013 10:21:02 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 544,320 879,252 1.6153
12 Feb 2013 04:58:48 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 518,400 833,162 1.6072
10 Feb 2013 22:03:07 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 492,480 788,411 1.6009
10 Feb 2013 02:00:22 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 466,560 744,179 1.5950
09 Feb 2013 03:15:31 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 440,640 699,537 1.5875
06 Feb 2013 04:22:14 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 414,720 655,095 1.5796
04 Feb 2013 22:59:41 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 388,800 609,606 1.5679
03 Feb 2013 08:07:41 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 362,880 569,677 1.5699
02 Feb 2013 19:15:36 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 336,960 524,666 1.5571
01 Feb 2013 03:35:46 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 311,040 482,066 1.5499
30 Jan 2013 03:02:24 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 285,120 440,606 1.5453
27 Jan 2013 22:27:27 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 259,200 398,702 1.5382
27 Jan 2013 02:44:51 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 233,280 356,929 1.5300
26 Jan 2013 05:59:28 1261325 15527597 hadcm3n_3lbl_1940_40_008260117_1 207,360 314,838 1.5183


©2024 climateprediction.net