Task 12830069

Name	hadcm3n_p5rz_1900_40_007225071_0
Workunit	7423311
Created	26 Apr 2011, 15:33:55 UTC
Sent	27 Apr 2011, 23:26:58 UTC
Report deadline	28 Jul 2011, 6:54:09 UTC
Received	6 May 2011, 3:18:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1076802
Run time	6 days 21 hours 22 min 29 sec
CPU time	6 days 21 hours 14 min
Validate state	Invalid
Credit	2,177.28
Device peak FLOPS	1.70 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.9' not found (required by /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_se_6.07_i686-pc-linux-gnu.so) BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_p5rz_1900_40_007225071/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 00258E9C Unknown Unknown Unknown hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=22163, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_p5rz_1900_40_007225071/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 00258E9C Unknown Unknown Unknown hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=22163, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_p5rz_1900_40_007225071/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 00258E9C Unknown Unknown Unknown hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=22163, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_p5rz_1900_40_007225071/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 00258E9C Unknown Unknown Unknown hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=22163, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /root/BOINC/projects/climateprediction.net_index.php/hadcm3n_p5rz_1900_40_007225071/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 08451069 Unknown Unknown Unknown hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown hadcm3n_um_6.07_i 0836D10D Unknown Unknown Unknown hadcm3n_um_6.07_i 082EB086 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F66D Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 00258E9C Unknown Unknown Unknown hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=22163, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish 23:26:58 (8795): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 May 2011 05:23:53	1076802	12830069	hadcm3n_p5rz_1900_40_007225071_0	181,440	544,641	3.0018
03 May 2011 09:09:32	1076802	12830069	hadcm3n_p5rz_1900_40_007225071_0	155,520	466,149	2.9974
02 May 2011 13:33:13	1076802	12830069	hadcm3n_p5rz_1900_40_007225071_0	129,600	395,950	3.0552
01 May 2011 15:45:05	1076802	12830069	hadcm3n_p5rz_1900_40_007225071_0	103,680	317,256	3.0600
30 Apr 2011 18:19:08	1076802	12830069	hadcm3n_p5rz_1900_40_007225071_0	77,760	240,353	3.0910
29 Apr 2011 20:22:31	1076802	12830069	hadcm3n_p5rz_1900_40_007225071_0	51,840	161,366	3.1128
28 Apr 2011 23:21:24	1076802	12830069	hadcm3n_p5rz_1900_40_007225071_0	25,920	84,407	3.2564