Task 12577932

Name	famous_vnqn_1799_200_006734332_3
Workunit	6937673
Created	13 Feb 2011, 15:26:21 UTC
Sent	13 Feb 2011, 15:26:24 UTC
Report deadline	15 May 2011, 22:53:35 UTC
Received	8 Mar 2011, 16:46:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1115876
Run time	4 days 5 hours 59 min 50 sec
CPU time	4 days 5 hours 34 min 42 sec
Validate state	Invalid
Credit	4,972.03
Device peak FLOPS	2.45 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-apple-darwin
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Sat Feb 19 01:29:48 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x92e1b46b 0x92e1b46b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 12 0x00000003 0x00000003 (in famous_um_6.11_i686-apple-darwin) Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc3fc edx: 0x92db50fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc438 esp: 0xbfffc3fc ss: 0x0000001f efl: 0x00000206 eip: 0x92db50fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libirc.dylib 0x92db4000 - 0x92f5bfff /usr/lib/libSystem.B.dylib 0x97519000 - 0x97527fff /usr/lib/libz.1.dylib 0x98742000 - 0x98745fff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=78185, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Climate model /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin couldn't start (29117): called boinc_finish CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READ_FLH: I/O error tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READ_FLH: I/O error tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READ_FLH: I/O error tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READ_FLH: I/O error tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READ_FLH: I/O error tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 21 - Return code = 1 Model crashed: READ_FLH: I/O error tmp/pipe_dummy Sorry, too many model crashes! :-( (2946): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Climate model /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin couldn't start (3766): called boinc_finish Climate model /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin couldn't start (3859): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Wed Feb 23 18:47:04 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x92e1b46b 0x92e1b46b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 12 0x00000003 0x00000003 (in famous_um_6.11_i686-apple-darwin) Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc3fc edx: 0x92db50fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc438 esp: 0xbfffc3fc ss: 0x0000001f efl: 0x00000206 eip: 0x92db50fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libirc.dylib 0x92db4000 - 0x92f5bfff /usr/lib/libSystem.B.dylib 0x97519000 - 0x97527fff /usr/lib/libz.1.dylib 0x98742000 - 0x98745fff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=940, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Thu Feb 24 21:55:05 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x92e1b46b 0x92e1b46b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 12 0x00000003 0x00000003 (in famous_um_6.11_i686-apple-darwin) Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc3fc edx: 0x92db50fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc438 esp: 0xbfffc3fc ss: 0x0000001f efl: 0x00000206 eip: 0x92db50fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libirc.dylib 0x92db4000 - 0x92f5bfff /usr/lib/libSystem.B.dylib 0x97519000 - 0x97527fff /usr/lib/libz.1.dylib 0x98742000 - 0x98745fff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=38429, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Sat Feb 26 22:42:26 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x92e1b46b 0x92e1b46b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 12 0x00000003 0x00000003 (in famous_um_6.11_i686-apple-darwin) Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc3fc edx: 0x92db50fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc438 esp: 0xbfffc3fc ss: 0x0000001f efl: 0x00000206 eip: 0x92db50fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libirc.dylib 0x92db4000 - 0x92f5bfff /usr/lib/libSystem.B.dylib 0x97519000 - 0x97527fff /usr/lib/libz.1.dylib 0x98742000 - 0x98745fff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=24776, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Sat Feb 26 22:42:30 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x92e1b46b 0x92e1b46b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 12 0x00000003 0x00000003 (in famous_um_6.11_i686-apple-darwin) Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc3fc edx: 0x92db50fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc438 esp: 0xbfffc3fc ss: 0x0000001f efl: 0x00000206 eip: 0x92db50fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libirc.dylib 0x92db4000 - 0x92f5bfff /usr/lib/libSystem.B.dylib 0x97519000 - 0x97527fff /usr/lib/libz.1.dylib 0x98742000 - 0x98745fff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=24776, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Crashed executable name: famous_um_6.11_i686-apple-darwin built using BOINC library version 6.11.1 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.6.6 build 10J567 Mon Feb 28 23:55:30 2011 0 0x0048fdcf PrintBacktrace (in famous_um_6.11_i686-apple-darwin) + 1221 1 0x0049054a boinc_catch_signal (in famous_um_6.11_i686-apple-darwin) + 474 2 0x92e1b46b 0x92e1b46b 3 0xffffffff 0xffffffff 4 0x0134e6b3 0x134e6b3 5 0x0134d44f 0x134d44f 6 0x002eeac6 readcntl (in famous_um_6.11_i686-apple-darwin) + 2598 7 0x003f55ea um_setup (in famous_um_6.11_i686-apple-darwin) + 458 8 0x00452abd um_shell (in famous_um_6.11_i686-apple-darwin) + 11429 9 0x004698c0 main (in famous_um_6.11_i686-apple-darwin) + 1112 10 0x000026b6 _start (in famous_um_6.11_i686-apple-darwin) + 216 11 0x000025dd start (in famous_um_6.11_i686-apple-darwin) + 41 12 0x00000003 0x00000003 (in famous_um_6.11_i686-apple-darwin) Thread 0 crashed with X86 Thread State (32-bit): eax: 0xffffffe1 ebx: 0x00000003 ecx: 0xbfffc3fc edx: 0x92db50fa edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc438 esp: 0xbfffc3fc ss: 0x0000001f efl: 0x00000206 eip: 0x92db50fa cs: 0x00000007 ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037 Binary Images Description: 0x1000 - 0x517fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_um_6.11_i686-apple-darwin 0x12a9000 - 0x12c4fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libsvml.dylib 0x131c000 - 0x13b3fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libifcoremt.dylib 0x13fc000 - 0x1556fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libimf.dylib 0x1669000 - 0x1697fff /Library/Application Support/BOINC Data/projects/climateprediction.net/famous_vnqn_1799_200_006734332/lib/libirc.dylib 0x92db4000 - 0x92f5bfff /usr/lib/libSystem.B.dylib 0x97519000 - 0x97527fff /usr/lib/libz.1.dylib 0x98742000 - 0x98745fff /usr/lib/system/libmathCommon.A.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=77360, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... (630): called boinc_finish Signal 15 received, exiting... (630): called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (643): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,506,986	364,116	0.2416
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,497,626	362,176	0.2418
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,488,266	360,240	0.2421
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,478,906	358,258	0.2422
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,469,546	356,349	0.2425
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,460,186	354,464	0.2428
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,450,826	352,573	0.2430
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,441,466	350,684	0.2433
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,432,106	348,773	0.2435
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,422,746	346,872	0.2438
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,413,386	344,959	0.2441
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,404,026	343,041	0.2443
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,394,666	341,136	0.2446
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,385,306	339,218	0.2449
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,375,946	337,313	0.2451
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,366,586	335,402	0.2454
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,357,226	333,419	0.2457
08 Mar 2011 16:49:31	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,347,866	331,489	0.2459
08 Mar 2011 16:49:30	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,338,506	329,532	0.2462
08 Mar 2011 16:49:30	1115876	12577932	famous_vnqn_1799_200_006734332_3	1,329,146	327,563	0.2464