Task 16277933

Name	hadcm3n_7vrp_1980_40_008452456_3
Workunit	8603312
Created	2 Feb 2014, 14:48:20 UTC
Sent	2 Feb 2014, 14:48:34 UTC
Report deadline	4 May 2014, 22:15:45 UTC
Received	20 Mar 2014, 16:51:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1210673
Run time	7 days 6 hours 1 min 40 sec
CPU time	6 days 9 hours 46 min 11 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.48 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:53:29 (90556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:53:30 (90556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x9fa004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x9fa000: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0xa15604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0xa15600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0xa15604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0xa15600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x21ecc04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x2208204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x2208200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0xa15604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0xa15600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x9fa004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x11e7c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x11ca204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x11ca200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x11ca204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x11ca200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x11ca204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x11ca200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x11ca204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x9c9404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(8962,0xa19e31a8) malloc: * error for object 0x9c9400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.9.2 build 13C64 Thu Mar 20 09:40:50 2014 Thread 0 Crashed: Thread 1: -- Warning: /usr/bin/atos is moving and will be removed from a future OS X release. It is now available in the Xcode developer tools to be invoked via: `xcrun atos` To silence this warning, pass the '-d' command-line flag to this tool. -- atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8faf8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/0/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x90109000 - 0x9011bfff /usr/lib/system/libsystem_asl.dylib 0x902b9000 - 0x902bdfff /usr/lib/system/libcache.dylib 0x90325000 - 0x90330fff /usr/lib/system/libcommonCrypto.dylib 0x90aba000 - 0x90aebfff /usr/lib/system/libsystem_m.dylib 0x9144f000 - 0x9147afff /usr/lib/system/libsystem_network.dylib 0x91b0f000 - 0x91b37fff /usr/lib/system/libsystem_info.dylib 0x91da6000 - 0x91df6fff /usr/lib/system/libcorecrypto.dylib 0x92266000 - 0x9226bfff /usr/lib/system/libsystem_platform.dylib 0x92318000 - 0x92326fff /usr/lib/libz.1.dylib 0x92347000 - 0x92348fff /usr/lib/system/libremovefile.dylib 0x935ae000 - 0x935affff /usr/lib/system/libsystem_sandbox.dylib 0x94808000 - 0x9489afff /usr/lib/system/libsystem_c.dylib 0x948ea000 - 0x94907fff /usr/lib/system/libsystem_kernel.dylib 0x95312000 - 0x9532afff /usr/lib/system/libsystem_malloc.dylib 0x96c3f000 - 0x96c42fff /usr/lib/system/libdyld.dylib 0x96cb9000 - 0x96cbbfff /usr/lib/system/libsystem_configuration.dylib 0x970e6000 - 0x970e7fff /usr/lib/system/libunc.dylib 0x9879e000 - 0x987a2fff /usr/lib/system/libmacho.dylib 0x998c4000 - 0x998ccfff /usr/lib/system/liblaunch.dylib 0x9a0bf000 - 0x9a0c1fff /usr/lib/system/libquarantine.dylib 0x9a369000 - 0x9a38dfff /usr/lib/system/libxpc.dylib 0x9a3fc000 - 0x9a414fff /usr/lib/system/libdispatch.dylib 0x9b3f5000 - 0x9b3fefff /usr/lib/system/libsystem_notify.dylib 0x9b3ff000 - 0x9b404fff /usr/lib/system/libcompiler_rt.dylib 0x9b4be000 - 0x9b4c4fff /usr/lib/system/libunwind.dylib 0x9b4c8000 - 0x9b4d0fff /usr/lib/system/libcopyfile.dylib 0x9b4d1000 - 0x9b522fff /usr/lib/libstdc++.6.dylib 0x9bd25000 - 0x9bd25fff /usr/lib/system/libkeymgr.dylib 0x9bd28000 - 0x9bd29fff /usr/lib/system/libsystem_blocks.dylib 0x9bf92000 - 0x9bf99fff /usr/lib/system/libsystem_pthread.dylib 0x9bffe000 - 0x9c006fff /usr/lib/system/libsystem_dnssd.dylib 0x9c007000 - 0x9c008fff /usr/lib/libSystem.B.dylib 0x9c370000 - 0x9c394fff /usr/lib/libc++abi.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Mar 2014 16:47:25	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	518,400	553,566	1.0678
20 Mar 2014 06:40:26	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	492,480	525,129	1.0663
19 Mar 2014 22:13:49	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	466,560	496,697	1.0646
19 Mar 2014 13:32:22	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	440,640	468,261	1.0627
19 Mar 2014 04:36:07	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	414,720	439,824	1.0605
15 Mar 2014 22:10:19	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	388,800	410,908	1.0569
15 Mar 2014 13:06:29	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	362,880	382,484	1.0540
14 Mar 2014 11:29:31	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	336,960	355,165	1.0540
14 Mar 2014 03:23:02	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	311,040	329,996	1.0609
13 Mar 2014 19:00:49	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	285,120	302,672	1.0616
09 Mar 2014 12:57:14	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	259,200	273,705	1.0560
09 Mar 2014 02:59:05	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	233,280	245,017	1.0503
04 Mar 2014 19:04:15	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	207,360	216,405	1.0436
04 Mar 2014 10:44:48	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	181,440	190,131	1.0479
04 Mar 2014 02:43:17	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	155,520	164,074	1.0550
26 Feb 2014 14:28:22	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	129,600	138,062	1.0653
26 Feb 2014 06:23:55	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	103,680	112,456	1.0846
25 Feb 2014 21:52:39	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	77,760	84,829	1.0909
25 Feb 2014 21:52:39	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	51,840	56,177	1.0837
23 Feb 2014 09:55:57	1210673	16277933	hadcm3n_7vrp_1980_40_008452456_3	25,920	27,329	1.0544