Task 11610493

Name	famous_u1al_1799_200_006634520_6
Workunit	6837892
Created	18 Jul 2010, 23:57:57 UTC
Sent	19 Jul 2010, 0:10:52 UTC
Report deadline	18 Oct 2010, 7:38:03 UTC
Received	8 Aug 2010, 12:18:58 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1087235
Run time	11 days 21 hours 58 min 21 sec
CPU time	11 days 16 hours 19 min 21 sec
Validate state	Invalid
Credit	3,582.36
Device peak FLOPS	0.83 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (3335): No heartbeat from core client for 30 sec - exiting (3335): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (3335): No heartbeat from core client for 30 sec - exiting (3335): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (3135): Can't acquire lockfile (-154) - waiting 35s CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 MainError: 03:54:08 AM No files match the supplied pattern. MainError: 03:54:08 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (4627): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (9478): No heartbeat from core client for 30 sec - exiting (9478): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... (9478): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (13918): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: error reading file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day cpdnmonitor: error reading file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day Could not open shared memory... Error in attaching shared memory region! Signal 11 received, exiting... (1966): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1963, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: error reading file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day cpdnmonitor: error reading file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day Could not open shared memory... Error in attaching shared memory region! Signal 11 received, exiting... (1967): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1963, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: error reading file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day cpdnmonitor: error reading file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day Could not open shared memory... Error in attaching shared memory region! Signal 11 received, exiting... (1968): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1963, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: error reading file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day cpdnmonitor: error reading file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day Could not open shared memory... Error in attaching shared memory region! Signal 11 received, exiting... (1969): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1963, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: error reading file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day cpdnmonitor: error reading file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day Could not open shared memory... Error in attaching shared memory region! Signal 11 received, exiting... (1970): called boinc_finish (1963): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day after 11 attempts Model crashed: DRLANDF1 : Error in FILE_OPEN. tmp/pipe_dummy cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day after 11 attempts Model crashed: DRLANDF1 : Error in FILE_OPEN. tmp/pipe_dummy cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day after 11 attempts Model crashed: DRLANDF1 : Error in FILE_OPEN. tmp/pipe_dummy cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day after 11 attempts Model crashed: DRLANDF1 : Error in FILE_OPEN. tmp/pipe_dummy cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day after 11 attempts Model crashed: DRLANDF1 : Error in FILE_OPEN. tmp/pipe_dummy cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file /media/PNY/BOINC/projects/climateprediction.net/famous_u1al_1799_200_006634520/dataout/ocean_restart.day after 11 attempts Model crashed: DRLANDF1 : Error in FILE_OPEN. tmp/pipe_dummy Sorry, too many model crashes! :-( (1869): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Aug 2010 03:45:34	1087235	11610493	famous_u1al_1799_200_006634520_6	1,085,786	1,002,655	0.9234
08 Aug 2010 03:45:34	1087235	11610493	famous_u1al_1799_200_006634520_6	1,076,426	993,940	0.9234
06 Aug 2010 22:22:12	1087235	11610493	famous_u1al_1799_200_006634520_6	1,067,066	985,264	0.9233
06 Aug 2010 17:07:53	1087235	11610493	famous_u1al_1799_200_006634520_6	1,057,706	976,579	0.9233
06 Aug 2010 13:42:17	1087235	11610493	famous_u1al_1799_200_006634520_6	1,048,346	967,877	0.9232
06 Aug 2010 10:10:41	1087235	11610493	famous_u1al_1799_200_006634520_6	1,038,986	959,165	0.9232
06 Aug 2010 05:49:26	1087235	11610493	famous_u1al_1799_200_006634520_6	1,029,626	950,463	0.9231
06 Aug 2010 02:11:35	1087235	11610493	famous_u1al_1799_200_006634520_6	1,020,266	941,756	0.9230
05 Aug 2010 21:26:38	1087235	11610493	famous_u1al_1799_200_006634520_6	1,010,906	933,049	0.9230
05 Aug 2010 17:53:03	1087235	11610493	famous_u1al_1799_200_006634520_6	1,001,546	924,346	0.9229
05 Aug 2010 12:45:24	1087235	11610493	famous_u1al_1799_200_006634520_6	992,186	915,617	0.9228
05 Aug 2010 09:43:07	1087235	11610493	famous_u1al_1799_200_006634520_6	982,826	906,883	0.9227
05 Aug 2010 06:19:09	1087235	11610493	famous_u1al_1799_200_006634520_6	973,466	898,184	0.9227
05 Aug 2010 03:46:57	1087235	11610493	famous_u1al_1799_200_006634520_6	964,106	889,462	0.9226
05 Aug 2010 00:04:31	1087235	11610493	famous_u1al_1799_200_006634520_6	954,746	880,768	0.9225
04 Aug 2010 22:05:48	1087235	11610493	famous_u1al_1799_200_006634520_6	945,386	872,041	0.9224
04 Aug 2010 22:05:48	1087235	11610493	famous_u1al_1799_200_006634520_6	936,026	863,349	0.9224
04 Aug 2010 22:05:48	1087235	11610493	famous_u1al_1799_200_006634520_6	926,666	854,671	0.9223
04 Aug 2010 10:44:34	1087235	11610493	famous_u1al_1799_200_006634520_6	917,306	845,971	0.9222
04 Aug 2010 07:11:30	1087235	11610493	famous_u1al_1799_200_006634520_6	907,946	837,288	0.9222