Task 16055045

Name	hadcm3n_n27u_1880_40_008398565_1
Workunit	8549421
Created	3 Oct 2013, 11:52:08 UTC
Sent	3 Oct 2013, 11:57:15 UTC
Report deadline	2 Jan 2014, 19:24:26 UTC
Received	25 Oct 2013, 11:51:08 UTC
Server state	Over
Outcome	Computation error
Client state	Aborted by user
Exit status	203 (0x000000CB) EXIT_ABORTED_VIA_GUI
Computer ID	1295841
Run time	13 days 23 hours 52 min 54 sec
CPU time	13 days 4 hours 17 min 27 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.75 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> aborted by user </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:44:33 (7058): Can't acquire lockfile (-154) - waiting 35s 13:45:08 (7058): Can't acquire lockfile (-154) - exiting 13:45:41 (2218): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2149, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * Error in `../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x09b91a88 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * Error in `../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x091d10c0 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called Signal 15 received, exiting... Called boinc_finish * Error in `../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x09d960c0 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * Error in `../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x0a05b0c0 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called * Error in `../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu': double free or corruption (out): 0x08f6a0c0 * hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2369: sysmalloc: Assertion `(old_top == (((mbinptr) (((char ) &((av)->bins[((1) - 1) 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) \|\| ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Oct 2013 15:00:30	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	1,036,800	1,138,234	1.0978
22 Oct 2013 20:31:06	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	1,010,880	1,111,617	1.0997
22 Oct 2013 13:15:21	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	984,960	1,085,402	1.1020
21 Oct 2013 23:18:49	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	959,040	1,058,723	1.1039
21 Oct 2013 14:48:31	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	933,120	1,033,276	1.1073
20 Oct 2013 21:29:09	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	907,200	1,007,383	1.1104
19 Oct 2013 23:14:46	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	881,280	981,558	1.1138
19 Oct 2013 15:35:24	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	855,360	955,501	1.1171
18 Oct 2013 23:31:44	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	829,440	933,147	1.1250
18 Oct 2013 17:24:16	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	803,520	913,389	1.1367
17 Oct 2013 22:50:54	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	777,600	888,598	1.1427
17 Oct 2013 16:01:38	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	751,680	865,184	1.1510
16 Oct 2013 22:43:11	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	725,760	839,592	1.1568
16 Oct 2013 15:12:17	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	699,840	813,570	1.1625
15 Oct 2013 22:51:33	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	673,920	787,731	1.1689
15 Oct 2013 04:38:40	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	648,000	761,039	1.1744
14 Oct 2013 17:06:25	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	622,080	733,014	1.1783
14 Oct 2013 02:44:17	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	596,160	703,759	1.1805
13 Oct 2013 18:09:01	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	570,240	674,273	1.1824
13 Oct 2013 09:44:35	1295841	16055045	hadcm3n_n27u_1880_40_008398565_1	544,320	644,611	1.1843