Task 19417889

Name	hadcm3n_ldf9_198012_480_373_010412377_0
Workunit	10412377
Created	21 Mar 2016, 14:40:22 UTC
Sent	21 Mar 2016, 14:55:17 UTC
Report deadline	3 Mar 2017, 20:15:17 UTC
Received	3 Apr 2016, 4:44:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1393067
Run time	4 days 18 hours 27 min 6 sec
CPU time	3 days 22 hours 42 min 34 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	4.14 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/ldf9ko.pji5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ko.pji5c10 Error: Input file: dataout/ldf9ko.pii5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ko.pii5c10 Error: Input file: dataout/ldf9ko.pfi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ko.pfi5c10 Error: Input file: dataout/ldf9ka.phi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.phi5c10 Error: Input file: dataout/ldf9ka.pgi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.pgi5c10 Error: Input file: dataout/ldf9ka.pei5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.pei5c10 Error: Input file: dataout/ldf9ka.pdi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.pdi5c10 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/ldf9ko.pji5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ko.pji5c10 Error: Input file: dataout/ldf9ko.pii5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ko.pii5c10 Error: Input file: dataout/ldf9ko.pfi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ko.pfi5c10 Error: Input file: dataout/ldf9ka.phi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.phi5c10 Error: Input file: dataout/ldf9ka.pgi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.pgi5c10 Error: Input file: dataout/ldf9ka.pei5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.pei5c10 Error: Input file: dataout/ldf9ka.pdi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.pdi5c10 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/ldf9ko.pji5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ko.pji5c10 Error: Input file: dataout/ldf9ko.pii5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ko.pii5c10 Error: Input file: dataout/ldf9ko.pfi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ko.pfi5c10 Error: Input file: dataout/ldf9ka.phi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.phi5c10 Error: Input file: dataout/ldf9ka.pgi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.pgi5c10 Error: Input file: dataout/ldf9ka.pei5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.pei5c10 Error: Input file: dataout/ldf9ka.pdi5c10 is not a valid UM file. Error converting file to netcdf: dataout/ldf9ka.pdi5c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(12113,0xa3db2000) malloc: * error for object 0x1033604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12113,0xa3db2000) malloc: * error for object 0x1033600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12113,0xa3db2000) malloc: * error for object 0x2824604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12113,0xa3db2000) malloc: * error for object 0x2824604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12113,0xa3db2000) malloc: * error for object 0x1033604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12113,0xa3db2000) malloc: * error for object 0x104ec04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(12113,0xa3db2000) malloc: * error for object 0x104ec00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(18330,0xa3db2000) malloc: * error for object 0x103c204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(18330,0xa3db2000) malloc: * error for object 0x103c200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Sun Apr 3 03:56:02 2016 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fcd8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/12/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x902b1000 - 0x902b1fff /usr/lib/system/libkeymgr.dylib 0x903b6000 - 0x903b6fff /usr/lib/system/liblaunch.dylib 0x905d3000 - 0x90606fff /usr/lib/system/libsystem_m.dylib 0x90607000 - 0x90607fff /usr/lib/system/libsystem_blocks.dylib 0x906f7000 - 0x9074afff /usr/lib/libstdc++.6.dylib 0x9074b000 - 0x90750fff /usr/lib/system/libcompiler_rt.dylib 0x9090d000 - 0x90926fff /usr/lib/system/libsystem_asl.dylib 0x90a8f000 - 0x90a97fff /usr/lib/system/libsystem_dnssd.dylib 0x90a9c000 - 0x90a9efff /usr/lib/system/libquarantine.dylib 0x90c2d000 - 0x90c2ffff /usr/lib/system/libsystem_coreservices.dylib 0x91447000 - 0x91460fff /usr/lib/system/libsystem_malloc.dylib 0x92e96000 - 0x92e9ffff /usr/lib/system/libcopyfile.dylib 0x92ea4000 - 0x92ecefff /usr/lib/system/libdispatch.dylib 0x9420c000 - 0x9421afff /usr/lib/libz.1.dylib 0x948bb000 - 0x948c1fff /usr/lib/system/libsystem_platform.dylib 0x94eb3000 - 0x94f1afff /usr/lib/system/libcorecrypto.dylib 0x95107000 - 0x9511efff /usr/lib/system/libsystem_coretls.dylib 0x951e2000 - 0x951f3fff /usr/lib/system/libsystem_trace.dylib 0x95e7e000 - 0x95e84fff /usr/lib/system/libunwind.dylib 0x96764000 - 0x967d2fff /usr/lib/system/libsystem_network.dylib 0x969a7000 - 0x969b0fff /usr/lib/system/libsystem_networkextension.dylib 0x96c32000 - 0x96c36fff /usr/lib/system/libcache.dylib 0x97271000 - 0x9727cfff /usr/lib/system/libcommonCrypto.dylib 0x9729e000 - 0x972a3fff /usr/lib/system/libmacho.dylib 0x9740e000 - 0x97411fff /usr/lib/system/libdyld.dylib 0x979cd000 - 0x979eefff /usr/lib/system/libsystem_kernel.dylib 0x984bc000 - 0x984bcfff /usr/lib/system/libunc.dylib 0x98a2c000 - 0x98a2efff /usr/lib/system/libsystem_configuration.dylib 0x98a61000 - 0x98a69fff /usr/lib/system/libsystem_pthread.dylib 0x98cbe000 - 0x98ce1fff /usr/lib/libc++abi.dylib 0x99d3e000 - 0x99d66fff /usr/lib/system/libxpc.dylib 0x9a167000 - 0x9a171fff /usr/lib/system/libsystem_notify.dylib 0x9aa45000 - 0x9aadafff /usr/lib/system/libsystem_c.dylib 0x9b52e000 - 0x9b530fff /usr/lib/system/libsystem_secinit.dylib 0x9c615000 - 0x9c63ffff /usr/lib/system/libsystem_info.dylib 0x9c910000 - 0x9c913fff /usr/lib/system/libsystem_sandbox.dylib 0x9cc60000 - 0x9cc61fff /usr/lib/libSystem.B.dylib 0x9d03e000 - 0x9d03ffff /usr/lib/system/libremovefile.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Apr 2016 12:00:47	1393067	19417889	hadcm3n_ldf9_198012_480_373_010412377_0	259,200	311,436	1.2015
31 Mar 2016 22:53:40	1393067	19417889	hadcm3n_ldf9_198012_480_373_010412377_0	233,280	280,224	1.2012
31 Mar 2016 11:55:05	1393067	19417889	hadcm3n_ldf9_198012_480_373_010412377_0	207,360	248,907	1.2004
30 Mar 2016 05:13:55	1393067	19417889	hadcm3n_ldf9_198012_480_373_010412377_0	181,440	217,667	1.1997
29 Mar 2016 17:21:49	1393067	19417889	hadcm3n_ldf9_198012_480_373_010412377_0	155,520	185,977	1.1958
28 Mar 2016 12:43:47	1393067	19417889	hadcm3n_ldf9_198012_480_373_010412377_0	129,600	155,032	1.1962
25 Mar 2016 17:26:33	1393067	19417889	hadcm3n_ldf9_198012_480_373_010412377_0	103,680	124,409	1.1999
24 Mar 2016 18:40:36	1393067	19417889	hadcm3n_ldf9_198012_480_373_010412377_0	77,760	93,578	1.2034
24 Mar 2016 00:45:37	1393067	19417889	hadcm3n_ldf9_198012_480_373_010412377_0	51,840	62,225	1.2003
23 Mar 2016 01:50:58	1393067	19417889	hadcm3n_ldf9_198012_480_373_010412377_0	25,920	31,326	1.2086