Task 12404070

Name	famous_x8bu_1599_200_007072625_0
Workunit	7275925
Created	18 Dec 2010, 13:49:27 UTC
Sent	4 Jan 2011, 9:30:48 UTC
Report deadline	5 Apr 2011, 16:57:59 UTC
Received	23 Feb 2011, 16:32:40 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1068044
Run time	2 days 12 hours 43 min 7 sec
CPU time	2 days 8 hours 57 min 50 sec
Validate state	Invalid
Credit	1,482.40
Device peak FLOPS	2.28 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... SIGSEGV: segmentation violation Stack trace (17 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf76f5400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81f7691] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81f9bac] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82c7ed8] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x825b165] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x808e223] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x80882b1] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8263f78] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81f3f37] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81eeea2] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x831bcec] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832da0f] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf754dce7] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3345, iMonCtr=1 Model crash detected, will try to restart... (3345): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... (3552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (13 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf7792400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8137374] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8128d21] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b6e3e] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82376ab] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf75e6ce7] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3506, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (14 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf773e400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x822ceed] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b0273] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81afb87] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x812bf01] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x827a365] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8174c85] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x817613b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8325c74] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf7592ce7] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3506, iMonCtr=1 Model crash detected, will try to restart... Signal 15 received, exiting... (3506): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: SUBROUTINE POTTEM STOPPING - PRESSURE OUT OF RANGE Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (15 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf76e6400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x822ceed] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b05ee] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b2bac] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b2559] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8135fef] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b35a8] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8079c3f] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf753ace7] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12568, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf770d400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x821d710] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82db61b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82b2a6d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832affb] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf7561ce7] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12568, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (3529): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Feb 2011 16:34:04	1068044	12404070	famous_x8bu_1599_200_007072625_0	449,306	202,744	0.4512
19 Feb 2011 23:32:18	1068044	12404070	famous_x8bu_1599_200_007072625_0	439,946	198,581	0.4514
19 Feb 2011 22:05:06	1068044	12404070	famous_x8bu_1599_200_007072625_0	430,586	194,409	0.4515
19 Feb 2011 20:25:54	1068044	12404070	famous_x8bu_1599_200_007072625_0	421,226	190,223	0.4516
19 Feb 2011 20:14:19	1068044	12404070	famous_x8bu_1599_200_007072625_0	411,866	186,062	0.4518
16 Feb 2011 19:10:17	1068044	12404070	famous_x8bu_1599_200_007072625_0	402,506	181,877	0.4519
13 Feb 2011 13:38:35	1068044	12404070	famous_x8bu_1599_200_007072625_0	393,146	177,674	0.4519
13 Feb 2011 13:24:58	1068044	12404070	famous_x8bu_1599_200_007072625_0	383,786	173,459	0.4520
13 Feb 2011 13:24:58	1068044	12404070	famous_x8bu_1599_200_007072625_0	374,426	169,228	0.4520
13 Feb 2011 13:24:57	1068044	12404070	famous_x8bu_1599_200_007072625_0	365,066	164,971	0.4519
13 Feb 2011 13:24:57	1068044	12404070	famous_x8bu_1599_200_007072625_0	355,706	160,639	0.4516
13 Feb 2011 13:24:57	1068044	12404070	famous_x8bu_1599_200_007072625_0	346,346	156,447	0.4517
06 Feb 2011 17:09:16	1068044	12404070	famous_x8bu_1599_200_007072625_0	336,986	152,240	0.4518
06 Feb 2011 15:58:39	1068044	12404070	famous_x8bu_1599_200_007072625_0	327,626	148,043	0.4519
06 Feb 2011 14:51:20	1068044	12404070	famous_x8bu_1599_200_007072625_0	318,266	143,851	0.4520
06 Feb 2011 13:37:21	1068044	12404070	famous_x8bu_1599_200_007072625_0	308,906	139,652	0.4521
06 Feb 2011 12:26:51	1068044	12404070	famous_x8bu_1599_200_007072625_0	299,546	135,436	0.4521
05 Feb 2011 18:57:04	1068044	12404070	famous_x8bu_1599_200_007072625_0	290,186	131,218	0.4522
05 Feb 2011 14:20:21	1068044	12404070	famous_x8bu_1599_200_007072625_0	280,826	127,007	0.4523
05 Feb 2011 14:14:11	1068044	12404070	famous_x8bu_1599_200_007072625_0	271,466	122,771	0.4523