<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
99b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12194905.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12194905.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12194905.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12194905.xml
-- Model is restarting --
Adjusting last_iter, 931, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12194905.xml
last_cpu_time: 30898
upload_file_number: 38
last_iter: 911
last_upload: 3283200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403
The child process has been launched with process id: 3543865
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221225:223140:1671967900:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
22:32:45 STEP 912 H= 912:00 +CPU= 54.692
22:33:17 STEP 913 H= 913:00 +CPU= 29.227
22:33:49 STEP 914 H= 914:00 +CPU= 29.571
22:34:38 STEP 915 H= 915:00 +CPU= 45.823
22:35:10 STEP 916 H= 916:00 +CPU= 28.941
22:35:40 STEP 917 H= 917:00 +CPU= 28.335
22:36:30 STEP 918 H= 918:00 +CPU= 45.127
22:37:01 STEP 919 H= 919:00 +CPU= 28.996
22:37:34 STEP 920 H= 920:00 +CPU= 30.246
22:38:25 STEP 921 H= 921:00 +CPU= 47.314
22:38:57 STEP 922 H= 922:00 +CPU= 29.596
22:39:31 STEP 923 H= 923:00 +CPU= 31.512
22:40:25 STEP 924 H= 924:00 +CPU= 49.995
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000924
22:40:56 STEP 925 H= 925:00 +CPU= 28.546
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:42:03 STEP 926 H= 926:00 +CPU= 32.023
22:42:51 STEP 927 H= 927:00 +CPU= 44.427
22:43:22 STEP 928 H= 928:00 +CPU= 29.150
22:43:57 STEP 929 H= 929:00 +CPU= 31.686
22:44:47 STEP 930 H= 930:00 +CPU= 46.316
22:45:20 STEP 931 H= 931:00 +CPU= 30.565
22:45:57 STEP 932 H= 932:00 +CPU= 34.378
11:18:56 STEP 933 H= 933:00 +CPU= 47.046
11:19:34 STEP 934 H= 934:00 +CPU= 27.380
11:20:10 STEP 935 H= 935:00 +CPU= 30.558
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
ECCODES ERROR : IO ERROR: No space left on device (No space left on device)
GRIB_WRITE_BYTES_INT 50001 71872 FAILED -11
GRIB_API ERROR MSG: Input output problem
MPL_ABORT: CALLED FROM PROCESSOR 1 THRD 1
MPL_ABORT: THRD 1 GRIB_WRITE_BYTES_INT FAILED
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 2598 MB (maxheap), 4411 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.639] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.643] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.643] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.643] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.643] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.643] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.643] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GRIDFPOS
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.643] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WRMLFP
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.643] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : WROUTGPGB
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.643] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : IOSTREAM_MIX:IO_PUT
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.643] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : IOSTREAM_MIX:GRID_OUT
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.644] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : IOSTREAM_MIX:WRITE_RECORD
[EC_DRHOOK:Lian:1:1:3543865:3543865] [20221226:112148:1672014108:46208.644] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GRIB_API:IGRIB_WRITE_BYTES_INT
SDL_TRACEBACK: Calling INTEL_TRBK, THRD = 1
Process 1 thread 1 calling linux_trbk from intel_trbk()
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/8/oifs_43r3_model.exe' : sigcontextptr=0x7ffdd300e450
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 20) with addr2line-cmd
[LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.gitThe child process terminated with status: 0
double free or corruption (top)
11:22:32 (3525725): BOINC client no longer exists - exiting
11:22:32 (3525725): timer handler: client dead, exiting
11:22:37 (3525725): BOINC client no longer exists - exiting
11:22:37 (3525725): timer handler: client dead, exiting
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2009050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0261
(argv4) batchid: 978
(argv5) wuid: 12194905
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_7ec0debe29c9dbf96e5cc672009f6ea2 to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0261_2009050100_123_978_12194905.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0261_2009050100_123_978_12194905.zip
ic_ancil_file: ic_ancil_12194905
ifsdata_file: ifsdata_12194905
climate_data_file: clim_data_12194905
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_9d8bb003769847c97c157162de8ccf64 to: /var/lib/boinc-client/slots/8/ic_ancil_12194905.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12194905.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7e1ade41fb6d4a090ec6426467fdab49 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12194905.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12194905.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12194905.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12194905.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12194905.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12194905.xml
-- Model is restarting --
Adjusting last_iter, 936, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12194905.xml
last_cpu_time: 31815
upload_file_number: 38
last_iter: 935
last_upload: 3283200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403
The child process has been launched with process id: 6691
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:195332:1672131212:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
19:54:18 STEP 936 H= 936:00 +CPU= 39.996
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+000924
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_38.zip
Uploading the intermediate file: upload_file_38.zip
19:54:42 STEP 937 H= 937:00 +CPU= 23.065
19:55:09 STEP 938 H= 938:00 +CPU= 24.649
19:55:56 STEP 939 H= 939:00 +CPU= 39.579
19:56:26 STEP 940 H= 940:00 +CPU= 26.638
19:56:57 STEP 941 H= 941:00 +CPU= 25.974
19:57:44 STEP 942 H= 942:00 +CPU= 39.944
19:58:15 STEP 943 H= 943:00 +CPU= 26.106
19:58:44 STEP 944 H= 944:00 +CPU= 26.830
19:59:28 STEP 945 H= 945:00 +CPU= 39.134
19:59:56 STEP 946 H= 946:00 +CPU= 25.587
20:00:26 STEP 947 H= 947:00 +CPU= 25.808
20:01:15 STEP 948 H= 948:00 +CPU= 43.204
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000948
20:01:44 STEP 949 H= 949:00 +CPU= 25.481
20:02:13 STEP 950 H= 950:00 +CPU= 25.651
20:02:58 STEP 951 H= 951:00 +CPU= 38.875
20:03:28 STEP 952 H= 952:00 +CPU= 26.409
20:03:57 STEP 953 H= 953:00 +CPU= 25.618
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:04:55 STEP 954 H= 954:00 +CPU= 39.377
20:05:24 STEP 955 H= 955:00 +CPU= 25.836
20:05:53 STEP 956 H= 956:00 +CPU= 26.154
20:06:37 STEP 957 H= 957:00 +CPU= 38.730
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:07:24 STEP 958 H= 958:00 +CPU= 26.106
20:07:53 STEP 959 H= 959:00 +CPU= 25.849
20:08:45 STEP 960 H= 960:00 +CPU= 45.584
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_39.zip
Uploading the intermediate file: upload_file_39.zip
20:09:14 STEP 961 H= 961:00 +CPU= 25.527
20:09:44 STEP 962 H= 962:00 +CPU= 25.942
20:10:28 STEP 963 H= 963:00 +CPU= 38.749
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:11:33 STEP 964 H= 964:00 +CPU= 26.254
20:12:02 STEP 965 H= 965:00 +CPU= 25.040
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:13:43 STEP 966 H= 966:00 +CPU= 39.393
20:14:12 STEP 967 H= 967:00 +CPU= 25.515
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:14:53 STEP 968 H= 968:00 +CPU= 26.681
20:15:36 STEP 969 H= 969:00 +CPU= 38.497
20:16:06 STEP 970 H= 970:00 +CPU= 26.778
20:16:34 STEP 971 H= 971:00 +CPU= 25.242
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:18:01 STEP 972 H= 972:00 +CPU= 43.643
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000972
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:19:09 STEP 973 H= 973:00 +CPU= 25.432
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:21:31 STEP 974 H= 974:00 +CPU= 26.019
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:22:30 STEP 975 H= 975:00 +CPU= 38.388
20:23:00 STEP 976 H= 976:00 +CPU= 26.350
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:25:05 STEP 977 H= 977:00 +CPU= 27.282
20:25:49 STEP 978 H= 978:00 +CPU= 41.819
20:26:18 STEP 979 H= 979:00 +CPU= 27.744
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:27:05 STEP 980 H= 980:00 +CPU= 27.672
20:27:48 STEP 981 H= 981:00 +CPU= 41.489
20:28:18 STEP 982 H= 982:00 +CPU= 28.075
20:28:48 STEP 983 H= 983:00 +CPU= 28.466
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:30:34 STEP 984 H= 984:00 +CPU= 48.787
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+000972
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_40.zip
Uploading the intermediate file: upload_file_40.zip
20:31:06 STEP 985 H= 985:00 +CPU= 28.541
20:31:40 STEP 986 H= 986:00 +CPU= 30.334
20:32:30 STEP 987 H= 987:00 +CPU= 44.467
20:33:04 STEP 988 H= 988:00 +CPU= 31.172
20:33:35 STEP 989 H= 989:00 +CPU= 28.689
20:34:23 STEP 990 H= 990:00 +CPU= 44.192
20:34:55 STEP 991 H= 991:00 +CPU= 29.817
20:35:27 STEP 992 H= 992:00 +CPU= 30.672
20:36:14 STEP 993 H= 993:00 +CPU= 44.957
20:36:48 STEP 994 H= 994:00 +CPU= 32.761
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.754] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4411MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.762] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.762] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.762] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.762] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.762] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4411 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.764] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS_DRV
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-PHYSICS (1001)
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : EC_PHYS
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALLPAR
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203703:1672133823:2610.765] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LOCAL_ARRAYS_FIN
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203704:1672133824:2611.767] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203704:1672133824:2611.767] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203704:1672133824:2611.767] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1796] Returned from previous signal handler (at 0x1ce8bb0, signal#15, time taken = 0s), nsigs = 1
[EC_DRHOOK:Lian:1:1:6691:6691] [20221227:203704:1672133824:2611.767] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1841] Error _exit(143) upon receipt of signal#15, nsigs = 1
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2009050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0261
(argv4) batchid: 978
(argv5) wuid: 12194905
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_7ec0debe29c9dbf96e5cc672009f6ea2 to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0261_2009050100_123_978_12194905.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0261_2009050100_123_978_12194905.zip
ic_ancil_file: ic_ancil_12194905
ifsdata_file: ifsdata_12194905
climate_data_file: clim_data_12194905
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_9d8bb003769847c97c157162de8ccf64 to: /var/lib/boinc-client/slots/8/ic_ancil_12194905.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12194905.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7e1ade41fb6d4a090ec6426467fdab49 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12194905.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12194905.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12194905.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12194905.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12194905.xml
Opened progress file ok : /var/lib/boinc-client/slots/8/progress_file_12194905.xml
-- Model is restarting --
Adjusting last_iter, 995, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12194905.xml
last_cpu_time: 33701
upload_file_number: 41
last_iter: 983
last_upload: 3542400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403
The child process has been launched with process id: 4610
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:4610:4610] [20221227:203826:1672133906:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
20:39:11 STEP 984 H= 984:00 +CPU= 37.552
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:39:59 STEP 985 H= 985:00 +CPU= 23.893
20:40:26 STEP 986 H= 986:00 +CPU= 25.561
20:41:05 STEP 987 H= 987:00 +CPU= 37.994
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:41:42 STEP 988 H= 988:00 +CPU= 24.375
20:42:03 STEP 989 H= 989:00 +CPU= 20.206
20:42:34 STEP 990 H= 990:00 +CPU= 30.761
20:42:55 STEP 991 H= 991:00 +CPU= 20.087
20:43:18 STEP 992 H= 992:00 +CPU= 23.131
20:43:55 STEP 993 H= 993:00 +CPU= 35.910
20:44:18 STEP 994 H= 994:00 +CPU= 22.735
20:44:39 STEP 995 H= 995:00 +CPU= 20.646
20:45:12 STEP 996 H= 996:00 +CPU= 32.428
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+000996
20:45:31 STEP 997 H= 997:00 +CPU= 18.664
20:45:51 STEP 998 H= 998:00 +CPU= 19.252
20:46:21 STEP 999 H= 999:00 +CPU= 29.670
20:46:44 STEP 1000 H=1000:00 +CPU= 21.742
20:47:06 STEP 1001 H=1001:00 +CPU= 21.600
20:47:43 STEP 1002 H=1002:00 +CPU= 35.860
20:48:07 STEP 1003 H=1003:00 +CPU= 22.954
20:48:31 STEP 1004 H=1004:00 +CPU= 24.020
20:49:10 STEP 1005 H=1005:00 +CPU= 37.235
20:49:31 STEP 1006 H=1006:00 +CPU= 20.194
20:49:52 STEP 1007 H=1007:00 +CPU= 20.464
20:50:31 STEP 1008 H=1008:00 +CPU= 37.657
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+000996
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_41.zip
Uploading the intermediate file: upload_file_41.zip
20:50:55 STEP 1009 H=1009:00 +CPU= 23.481
20:51:20 STEP 1010 H=1010:00 +CPU= 23.296
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:52:08 STEP 1011 H=1011:00 +CPU= 35.733
20:52:33 STEP 1012 H=1012:00 +CPU= 23.656
20:52:56 STEP 1013 H=1013:00 +CPU= 22.693
20:53:31 STEP 1014 H=1014:00 +CPU= 33.426
20:53:50 STEP 1015 H=1015:00 +CPU= 18.740
20:54:10 STEP 1016 H=1016:00 +CPU= 19.670
20:54:41 STEP 1017 H=1017:00 +CPU= 30.069
20:55:02 STEP 1018 H=1018:00 +CPU= 20.580
20:55:28 STEP 1019 H=1019:00 +CPU= 25.846
20:56:08 STEP 1020 H=1020:00 +CPU= 38.491
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001020
20:56:34 STEP 1021 H=1021:00 +CPU= 24.729
20:56:58 STEP 1022 H=1022:00 +CPU= 23.483
20:57:34 STEP 1023 H=1023:00 +CPU= 35.615
20:58:00 STEP 1024 H=1024:00 +CPU= 24.381
20:58:24 STEP 1025 H=1025:00 +CPU= 23.865
20:59:00 STEP 1026 H=1026:00 +CPU= 35.100
20:59:25 STEP 1027 H=1027:00 +CPU= 23.976
20:59:48 STEP 1028 H=1028:00 +CPU= 22.204
21:00:25 STEP 1029 H=1029:00 +CPU= 35.705
21:00:51 STEP 1030 H=1030:00 +CPU= 25.189
21:01:15 STEP 1031 H=1031:00 +CPU= 23.557
21:01:59 STEP 1032 H=1032:00 +CPU= 40.529
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001020
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_42.zip
Uploading the intermediate file: upload_file_42.zip
21:02:22 STEP 1033 H=1033:00 +CPU= 22.757
21:02:47 STEP 1034 H=1034:00 +CPU= 23.950
21:03:23 STEP 1035 H=1035:00 +CPU= 34.717
21:03:48 STEP 1036 H=1036:00 +CPU= 24.503
21:04:10 STEP 1037 H=1037:00 +CPU= 22.090
21:04:47 STEP 1038 H=1038:00 +CPU= 35.357
21:05:09 STEP 1039 H=1039:00 +CPU= 21.716
21:05:33 STEP 1040 H=1040:00 +CPU= 23.811
21:06:08 STEP 1041 H=1041:00 +CPU= 33.112
21:06:32 STEP 1042 H=1042:00 +CPU= 24.048
21:06:54 STEP 1043 H=1043:00 +CPU= 21.372
21:07:32 STEP 1044 H=1044:00 +CPU= 37.362
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001044
21:07:55 STEP 1045 H=1045:00 +CPU= 21.812
21:08:18 STEP 1046 H=1046:00 +CPU= 22.466
21:08:52 STEP 1047 H=1047:00 +CPU= 33.458
21:09:17 STEP 1048 H=1048:00 +CPU= 24.102
21:09:40 STEP 1049 H=1049:00 +CPU= 22.268
21:10:18 STEP 1050 H=1050:00 +CPU= 37.053
21:10:44 STEP 1051 H=1051:00 +CPU= 24.850
21:11:11 STEP 1052 H=1052:00 +CPU= 25.619
21:11:48 STEP 1053 H=1053:00 +CPU= 35.460
21:12:12 STEP 1054 H=1054:00 +CPU= 23.432
21:12:36 STEP 1055 H=1055:00 +CPU= 23.316
21:13:19 STEP 1056 H=1056:00 +CPU= 38.828
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_43.zip
Uploading the intermediate file: upload_file_43.zip
21:13:43 STEP 1057 H=1057:00 +CPU= 23.354
21:14:05 STEP 1058 H=1058:00 +CPU= 21.489
21:14:39 STEP 1059 H=1059:00 +CPU= 33.045
21:15:01 STEP 1060 H=1060:00 +CPU= 21.889
21:15:24 STEP 1061 H=1061:00 +CPU= 22.348
21:15:57 STEP 1062 H=1062:00 +CPU= 32.708
21:16:20 STEP 1063 H=1063:00 +CPU= 22.547
21:16:42 STEP 1064 H=1064:00 +CPU= 21.864
21:17:16 STEP 1065 H=1065:00 +CPU= 32.664
21:17:38 STEP 1066 H=1066:00 +CPU= 22.131
21:18:01 STEP 1067 H=1067:00 +CPU= 22.547
21:18:37 STEP 1068 H=1068:00 +CPU= 35.273
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001068
21:19:00 STEP 1069 H=1069:00 +CPU= 21.971
21:19:22 STEP 1070 H=1070:00 +CPU= 21.503
21:19:55 STEP 1071 H=1071:00 +CPU= 32.603
21:20:17 STEP 1072 H=1072:00 +CPU= 21.802
21:20:40 STEP 1073 H=1073:00 +CPU= 22.411
21:21:13 STEP 1074 H=1074:00 +CPU= 32.689
21:21:36 STEP 1075 H=1075:00 +CPU= 22.020
21:21:58 STEP 1076 H=1076:00 +CPU= 21.766
21:22:31 STEP 1077 H=1077:00 +CPU= 31.751
21:22:53 STEP 1078 H=1078:00 +CPU= 21.563
21:23:15 STEP 1079 H=1079:00 +CPU= 22.461
21:23:53 STEP 1080 H=1080:00 +CPU= 36.690
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001068
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_44.zip
Uploading the intermediate file: upload_file_44.zip
21:24:15 STEP 1081 H=1081:00 +CPU= 21.711
21:24:36 STEP 1082 H=1082:00 +CPU= 20.844
21:25:09 STEP 1083 H=1083:00 +CPU= 31.588
21:25:30 STEP 1084 H=1084:00 +CPU= 21.394
21:25:53 STEP 1085 H=1085:00 +CPU= 21.735
21:26:25 STEP 1086 H=1086:00 +CPU= 31.832
21:26:47 STEP 1087 H=1087:00 +CPU= 21.551
21:27:09 STEP 1088 H=1088:00 +CPU= 21.231
21:27:42 STEP 1089 H=1089:00 +CPU= 32.257
21:28:04 STEP 1090 H=1090:00 +CPU= 21.687
21:28:26 STEP 1091 H=1091:00 +CPU= 21.767
21:29:02 STEP 1092 H=1092:00 +CPU= 35.012
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001092
21:29:23 STEP 1093 H=1093:00 +CPU= 21.416
21:29:45 STEP 1094 H=1094:00 +CPU= 21.005
21:30:18 STEP 1095 H=1095:00 +CPU= 32.480
21:30:41 STEP 1096 H=1096:00 +CPU= 22.116
21:31:03 STEP 1097 H=1097:00 +CPU= 22.161
21:31:38 STEP 1098 H=1098:00 +CPU= 33.277
21:31:59 STEP 1099 H=1099:00 +CPU= 20.860
21:32:21 STEP 1100 H=1100:00 +CPU= 21.607
21:32:54 STEP 1101 H=1101:00 +CPU= 31.825
21:33:16 STEP 1102 H=1102:00 +CPU= 21.731
21:33:39 STEP 1103 H=1103:00 +CPU= 22.300
21:34:17 STEP 1104 H=1104:00 +CPU= 37.232
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001092
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_45.zip
Uploading the intermediate file: upload_file_45.zip
21:34:39 STEP 1105 H=1105:00 +CPU= 21.625
21:35:01 STEP 1106 H=1106:00 +CPU= 21.325
21:35:33 STEP 1107 H=1107:00 +CPU= 31.841
21:35:55 STEP 1108 H=1108:00 +CPU= 21.463
21:36:17 STEP 1109 H=1109:00 +CPU= 21.836
21:36:50 STEP 1110 H=1110:00 +CPU= 31.977
21:37:12 STEP 1111 H=1111:00 +CPU= 21.528
21:37:34 STEP 1112 H=1112:00 +CPU= 21.295
21:38:06 STEP 1113 H=1113:00 +CPU= 32.043
21:38:29 STEP 1114 H=1114:00 +CPU= 21.821
21:38:51 STEP 1115 H=1115:00 +CPU= 21.527
21:39:27 STEP 1116 H=1116:00 +CPU= 35.299
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001116
21:39:48 STEP 1117 H=1117:00 +CPU= 21.066
21:40:10 STEP 1118 H=1118:00 +CPU= 21.142
21:40:42 STEP 1119 H=1119:00 +CPU= 31.747
21:41:05 STEP 1120 H=1120:00 +CPU= 22.125
21:41:27 STEP 1121 H=1121:00 +CPU= 22.235
21:42:04 STEP 1122 H=1122:00 +CPU= 35.475
21:42:27 STEP 1123 H=1123:00 +CPU= 22.507
21:42:48 STEP 1124 H=1124:00 +CPU= 20.597
21:43:23 STEP 1125 H=1125:00 +CPU= 33.338
21:43:46 STEP 1126 H=1126:00 +CPU= 22.939
21:44:10 STEP 1127 H=1127:00 +CPU= 23.233
21:44:49 STEP 1128 H=1128:00 +CPU= 38.181
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001116
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_46.zip
Uploading the intermediate file: upload_file_46.zip
21:45:12 STEP 1129 H=1129:00 +CPU= 22.144
21:45:34 STEP 1130 H=1130:00 +CPU= 21.661
21:46:07 STEP 1131 H=1131:00 +CPU= 31.771
21:46:28 STEP 1132 H=1132:00 +CPU= 20.936
21:46:49 STEP 1133 H=1133:00 +CPU= 20.289
21:47:21 STEP 1134 H=1134:00 +CPU= 31.307
21:47:41 STEP 1135 H=1135:00 +CPU= 19.804
21:48:03 STEP 1136 H=1136:00 +CPU= 20.682
21:48:34 STEP 1137 H=1137:00 +CPU= 30.310
21:48:55 STEP 1138 H=1138:00 +CPU= 20.548
21:49:15 STEP 1139 H=1139:00 +CPU= 19.999
21:49:50 STEP 1140 H=1140:00 +CPU= 33.587
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001140
21:50:10 STEP 1141 H=1141:00 +CPU= 19.981
21:50:31 STEP 1142 H=1142:00 +CPU= 20.461
21:51:02 STEP 1143 H=1143:00 +CPU= 30.222
21:51:23 STEP 1144 H=1144:00 +CPU= 20.647
21:51:43 STEP 1145 H=1145:00 +CPU= 19.914
21:52:15 STEP 1146 H=1146:00 +CPU= 30.952
21:52:35 STEP 1147 H=1147:00 +CPU= 19.916
21:52:57 STEP 1148 H=1148:00 +CPU= 20.675
21:53:28 STEP 1149 H=1149:00 +CPU= 30.378
21:53:48 STEP 1150 H=1150:00 +CPU= 20.294
21:54:09 STEP 1151 H=1151:00 +CPU= 20.456
21:54:46 STEP 1152 H=1152:00 +CPU= 35.631
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_47.zip
Uploading the intermediate file: upload_file_47.zip
21:55:06 STEP 1153 H=1153:00 +CPU= 19.888
21:55:27 STEP 1154 H=1154:00 +CPU= 20.315
21:55:58 STEP 1155 H=1155:00 +CPU= 30.125
21:56:19 STEP 1156 H=1156:00 +CPU= 20.481
21:56:39 STEP 1157 H=1157:00 +CPU= 19.588
21:57:10 STEP 1158 H=1158:00 +CPU= 30.838
21:57:30 STEP 1159 H=1159:00 +CPU= 19.759
21:57:51 STEP 1160 H=1160:00 +CPU= 20.243
21:58:21 STEP 1161 H=1161:00 +CPU= 29.721
21:58:41 STEP 1162 H=1162:00 +CPU= 19.467
21:59:00 STEP 1163 H=1163:00 +CPU= 18.932
21:59:33 STEP 1164 H=1164:00 +CPU= 32.368
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001164
21:59:52 STEP 1165 H=1165:00 +CPU= 18.737
22:00:12 STEP 1166 H=1166:00 +CPU= 19.673
22:00:42 STEP 1167 H=1167:00 +CPU= 29.315
22:01:03 STEP 1168 H=1168:00 +CPU= 19.953
22:01:23 STEP 1169 H=1169:00 +CPU= 19.509
22:01:54 STEP 1170 H=1170:00 +CPU= 30.288
22:02:14 STEP 1171 H=1171:00 +CPU= 19.628
22:02:34 STEP 1172 H=1172:00 +CPU= 19.534
22:03:04 STEP 1173 H=1173:00 +CPU= 29.476
22:03:24 STEP 1174 H=1174:00 +CPU= 19.450
22:03:44 STEP 1175 H=1175:00 +CPU= 19.725
22:04:20 STEP 1176 H=1176:00 +CPU= 34.967
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMGGhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMSHhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194905/ICMUAhq0f+001164
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0261_2009050100_123_978_12194905_0_r984112403_48.zip
Uploading the intermediate file: upload_file_48.zip
22:04:39 STEP 1177 H=1177:00 +CPU= 19.337
22:05:00 STEP 1178 H=1178:00 +CPU= 19.815
22:05:29 STEP 1179 H=1179:00 +CPU= 29.192
22:05:50 STEP 1180 H=1180:00 +CPU= 19.786
22:06:09 STEP 1181 H=1181:00 +CPU= 19.268
22:06:40 STEP 1182 H=1182:00 +CPU= 29.828
22:07:00 STEP 1183 H=1183:00 +CPU= 19.535
22:07:21 STEP 1184 H=1184:00 +CPU= 20.561
22:07:51 STEP 1185 H=1185:00 +CPU= 29.942
22:08:12 STEP 1186 H=1186:00 +CPU= 20.151
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
