climateprediction.net home page
Task 15929981

Task 15929981

Name hadcm3n_o6cm_2020_40_008408155_0
Workunit 8559011
Created 20 Aug 2013, 14:07:55 UTC
Sent 20 Aug 2013, 16:51:38 UTC
Report deadline 20 Nov 2013, 0:18:49 UTC
Received 13 Sep 2013, 7:01:06 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 871746
Run time 1 days 13 hours 56 min 18 sec
CPU time 1 days 5 hours 14 min 42 sec
Validate state Invalid
Credit 933.12
Device peak FLOPS 3.48 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:45:04 (260): No heartbeat from core client for 30 sec - exiting
16:45:06 (260): No heartbeat from core client for 30 sec - exiting
16:45:07 (260): No heartbeat from core client for 30 sec - exiting
16:45:08 (260): No heartbeat from core client for 30 sec - exiting
16:45:09 (260): No heartbeat from core client for 30 sec - exiting
16:45:10 (260): No heartbeat from core client for 30 sec - exiting
16:45:11 (260): No heartbeat from core client for 30 sec - exiting
16:45:12 (260): No heartbeat from core client for 30 sec - exiting
16:45:13 (260): No heartbeat from core client for 30 sec - exiting
16:45:14 (260): No heartbeat from core client for 30 sec - exiting
16:45:15 (260): No heartbeat from core client for 30 sec - exiting
16:45:16 (260): No heartbeat from core client for 30 sec - exiting
16:45:17 (260): No heartbeat from core client for 30 sec - exiting
16:45:18 (260): No heartbeat from core client for 30 sec - exiting
16:45:19 (260): No heartbeat from core client for 30 sec - exiting
16:45:20 (260): No heartbeat from core client for 30 sec - exiting
16:45:21 (260): No heartbeat from core client for 30 sec - exiting
16:45:22 (260): No heartbeat from core client for 30 sec - exiting
16:45:23 (260): No heartbeat from core client for 30 sec - exiting
16:45:24 (260): No heartbeat from core client for 30 sec - exiting
16:45:25 (260): No heartbeat from core client for 30 sec - exiting
16:45:26 (260): No heartbeat from core client for 30 sec - exiting
16:45:27 (260): No heartbeat from core client for 30 sec - exiting
16:45:28 (260): No heartbeat from core client for 30 sec - exiting
16:45:29 (260): No heartbeat from core client for 30 sec - exiting
16:45:30 (260): No heartbeat from core client for 30 sec - exiting
16:45:31 (260): No heartbeat from core client for 30 sec - exiting
16:45:32 (260): No heartbeat from core client for 30 sec - exiting
16:45:33 (260): No heartbeat from core client for 30 sec - exiting
16:45:34 (260): No heartbeat from core client for 30 sec - exiting
16:45:35 (260): No heartbeat from core client for 30 sec - exiting
16:45:36 (260): No heartbeat from core client for 30 sec - exiting
16:45:37 (260): No heartbeat from core client for 30 sec - exiting
16:45:38 (260): No heartbeat from core client for 30 sec - exiting
16:45:39 (260): No heartbeat from core client for 30 sec - exiting
16:45:40 (260): No heartbeat from core client for 30 sec - exiting
16:45:41 (260): No heartbeat from core client for 30 sec - exiting
16:45:42 (260): No heartbeat from core client for 30 sec - exiting
16:45:43 (260): No heartbeat from core client for 30 sec - exiting
16:45:44 (260): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
17:27:44 (286): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashe
BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/o6cmko.pjm3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmko.pjm3c10
Error: Input file: dataout/o6cmko.pim3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmko.pim3c10
Error: Input file: dataout/o6cmko.pfm3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmko.pfm3c10
Error: Input file: dataout/o6cmka.phm3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmka.phm3c10
Error: Input file: dataout/o6cmka.pgm3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmka.pgm3c10
Error: Input file: dataout/o6cmka.pem3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmka.pem3c10
Error: Input file: dataout/o6cmka.pdm3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmka.pdm3c10
Assertion failed: (*posp == OFF_NONE || *posp == lseek(nciop->fd, 0, SEEK_CUR)), function px_pgin, file ./libsrc/posixio.c, line 189.
SIGABRT: abort called

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K549
Tue Sep 10 21:08:36 2013

Thread 0 Crashed:
0   libSystem.B.dylib                   0x9211cc5a ___kill + 10
1   libSystem.B.dylib                   0x921af5a5 _raise + 26
2   libSystem.B.dylib                   0x921c56e4 _abort + 93
3   libSystem.B.dylib                   0x921b220f ___assert_rtn + 252
4   hadcm3n_6.07_i686-apple-darwin      0x00021457 _px_pgin + 135
5   hadcm3n_6.07_i686-apple-darwin      0x00021846 _px_get + 422
6   hadcm3n_6.07_i686-apple-darwin      0x000358a3 _fault_v1hs + 131
7   hadcm3n_6.07_i686-apple-darwin      0x0003644a _ncx_put_NC + 442
8   hadcm3n_6.07_i686-apple-darwin      0x0001ba64 _write_NC + 68
9   hadcm3n_6.07_i686-apple-darwin      0x0001bdbd _NC_endef + 269
10  hadcm3n_6.07_i686-apple-darwin      0x0001cc65 _nc_enddef + 69
11  hadcm3n_6.07_i686-apple-darwin      0x0000e593 __Z12writeVarInfoPKcS0_S0_SsibiiiiiiiiiiRSt6vectorIfSaIfEERS1_IiSaIiEERS1_IdSaIdEEb + 2423
12  hadcm3n_6.07_i686-apple-darwin      0x0000ff7c __Z14gmts_generatorPSt6vectorISsSaISsEEPKciiS4_S4_bS4_S4_ffffb + 4462
13  hadcm3n_6.07_i686-apple-darwin      0x00010419 __Z18regional_generatorPSt6vectorISsSaISsEEPKcS4_ + 719
14  hadcm3n_6.07_i686-apple-darwin      0x00011963 __Z7doMeansPSt6vectorISsSaISsEEPciiiPKcdd + 193
15  hadcm3n_6.07_i686-apple-darwin      0x00011ced __Z11yearlyMeansiPKc + 529
16  hadcm3n_6.07_i686-apple-darwin      0x000067af __Z9doCM3Procv + 105
17  hadcm3n_6.07_i686-apple-darwin      0x0000791c __Z8mainLoopv + 410
18  hadcm3n_6.07_i686-apple-darwin      0x000087c7 __Z6workerv + 2989
19  hadcm3n_6.07_i686-apple-darwin      0x00008aa9 _main + 491
20  hadcm3n_6.07_i686-apple-darwin      0x00002676 start + 54

Thread 1:
0   libSystem.B.dylib                   0x920bbc0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x92143429 _nanosleep + 345
2   libSystem.B.dylib                   0x921432ca _usleep + 61
3   hadcm3n_6.07_i686-apple-darwin      0x00071a7c __Z11boinc_sleepd + 188
4   hadcm3n_6.07_i686-apple-darwin      0x00067282 __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x920e9259 __pthread_start + 345
6   libSystem.B.dylib                   0x920e90de _thread_start + 34

Thread 0 crashed 
BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/o6cmko.pjm3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmko.pjm3c10
Error: Input file: dataout/o6cmko.pim3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmko.pim3c10
Error: Input file: dataout/o6cmko.pfm3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmko.pfm3c10
Error: Input file: dataout/o6cmka.phm3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmka.phm3c10
Error: Input file: dataout/o6cmka.pgm3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmka.pgm3c10
Error: Input file: dataout/o6cmka.pem3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmka.pem3c10
Error: Input file: dataout/o6cmka.pdm3c10 is not a valid UM file.
Error converting file to netcdf: dataout/o6cmka.pdm3c10
CPDN Monitor - Quit request from BOINC...
OPEN:  File Creation Failed: No space left on device
OPEN:  Unable to Open File dataout/o6cmko.dam4130 for Read/Write

Model crashed: DUMPCTL : Fail to open output dump - may already exist                                                                                                                                                                                                          tmp/pipe_dummy                                                                  2048    
forrtl: No space left on device
forrtl: severe (38): error during write, unit -1, file stdout
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0042E622  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0042D0EB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003EF4DD  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A66F5  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A5E57  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003E1BF7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000FEDA0  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0020B19B  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00160E7A  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00164453  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0038CE7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0038A1ED  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000048E1  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  000027FB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  00002729  Unknown               Unknown  Unknown
Unknown            00000003  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=256, iMonCtr=1
Model crash detected, will try to restart...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_o6cm_2020_40_008408155/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0042E622  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0042D0EB  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003EF4DD  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A66F5  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  003A5E57  Unknown          Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
13 Sep 2013 05:00:10 871746 15929981 hadcm3n_o6cm_2020_40_008408155_0 77,760 101,981 1.3115
05 Sep 2013 00:06:06 871746 15929981 hadcm3n_o6cm_2020_40_008408155_0 51,840 72,977 1.4077
04 Sep 2013 13:03:18 871746 15929981 hadcm3n_o6cm_2020_40_008408155_0 25,920 36,629 1.4132


©2024 climateprediction.net