climateprediction.net home page
Task 22307768

Task 22307768

Name oifs_43r3_bl_a0mf_2016092300_15_991_12207594_0
Workunit 12207594
Created 13 Feb 2023, 13:30:12 UTC
Sent 13 Feb 2023, 14:27:43 UTC
Report deadline 14 Apr 2023, 14:27:43 UTC
Received 13 Feb 2023, 17:25:11 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1538161
Run time 1 hours 17 min 7 sec
CPU time 1 hours 17 min 7 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 7.28 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,029.89 MB
Peak swap size 5,930.78 MB
Peak disk usage 1,291.72 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
4
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:24:21 STEP   64 H=  16:00 +CPU= 23.552
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:24:48 STEP   65 H=  16:15 +CPU= 21.908
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:25:19 STEP   66 H=  16:30 +CPU= 23.559
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:25:46 STEP   67 H=  16:45 +CPU= 22.476
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:26:16 STEP   68 H=  17:00 +CPU= 23.086
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:26:44 STEP   69 H=  17:15 +CPU= 22.494
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:27:15 STEP   70 H=  17:30 +CPU= 23.445
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:27:45 STEP   71 H=  17:45 +CPU= 24.168
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:28:23 STEP   72 H=  18:00 +CPU= 30.086
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000072
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000072
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000072
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:28:52 STEP   73 H=  18:15 +CPU= 22.735
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:29:22 STEP   74 H=  18:30 +CPU= 23.720
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM  ICE WATER           AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.201152183314113E-07 0.000000000000000E+00 0.179337981763709E-06
 GPNORM SNOW                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM RAIN                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM  CLOUD FRACTION      AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.129782114576138E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.200457913391608E-07 0.000000000000000E+00 0.180781581219392E-06
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.129708528871108E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.727452647709591E-10 0.000000000000000E+00 0.101169852434729E-08
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.102435011447748E-01 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =    72 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =    72 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000072 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000072
 MPI-TASK:   1 -    299284618 BYTES IN      6 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0087( 2.3%)
 NSTEP =    72 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000072 MODE=a
 IO-STREAM CLOSED - ./ICMSHh7zg+000072
 MPI-TASK:   1 -    707765250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0096 SECONDS 73725.5 Mbytes/s, TOTAL TIME=    0.2198( 4.4%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000072 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000072
 MPI-TASK:   1 -   4788553879 BYTES IN    125 RECORDS TRANSFERRED IN    0.0036 SECONDS******** Mbytes/s, TOTAL TIME=    0.1189( 3.0%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000072 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000072
 MPI-TASK:   1 -    290710985 BYTES IN      2 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0042( 4.8%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000072 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000072
 MPI-TASK:   1 -    299284618 BYTES IN      4 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0064( 3.1%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =    72 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =    72 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000072 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000072
 MPI-TASK:   1 -    299284609 BYTES IN      7 RECORDS TRANSFERRED IN    0.0005 SECONDS598569.2 Mbytes/s, TOTAL TIME=    0.0106( 4.7%)
 11:28:23 STEP   72 H=  18:00 +CPU= 30.086
 NSTEP =    73 STEPO   0AAA00AAA
 11:28:52 STEP   73 H=  18:15 +CPU= 22.735
 NSTEP =    74 STEPO   0AAA00AAA
 11:29:22 STEP   74 H=  18:30 +CPU= 23.720
ISTEP=      75 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=      75 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =    75 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 11:27:47 FULLPOS-B DYNFPOS     72     1.288    1.288    1.297      9:33     12:00 0.52000941564931E-07       2GB       0MB
 11:27:51 FULLPOS-V DYNFPOS     72     2.488    2.488    4.499      9:35     12:04 0.52000941564931E-07       2GB       0MB
 11:27:51 FULLPOS-T DYNFPOS     72     0.454    0.454    0.455      9:36     12:05 0.52000941564931E-07       2GB       0MB
 11:27:52 0AAA00AAA STEPO       72     0.450    0.450    0.451      9:36     12:05 0.52000941564931E-07       2GB       0MB
 11:28:23 0AAA00AAA STEPO       73    25.206   25.206   31.303     10:01     12:37 0.52000941564931E-07       2GB       0MB
 11:28:52 0AAA00AAA STEPO       74    22.735   22.735   28.854     10:24     13:05 0.52000941564931E-07       2GB       0MB
 11:29:22 0AAA00AAA STEPO       75    23.720   23.720   29.810     10:48     13:35 0.52000941564931E-07       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "      48",
 CTIME   = "00001200      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,          36, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/5/progress_file_12207594.xml

<running_values>
  <last_cpu_time>2358.780000</last_cpu_time>
  <upload_file_number>0</upload_file_number>
  <last_iter>75</last_iter>
  <last_upload>0</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0mf
(argv4) batchid: 991
(argv5) wuid: 12207594
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/5
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8bfbbbe43542e4e4ab6c5f99088fb503 to: /var/lib/boinc-client/slots/5/oifs_43r3_bl_a0mf_2016092300_15_991_12207594.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_bl_a0mf_2016092300_15_991_12207594.zip
ic_ancil_file: ic_ancil_12207594
ifsdata_file: ifsdata_12207594
climate_data_file: clim_data_12207594
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/5/ic_ancil_12207594.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12207594.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_980d64bc9beebd7ccd7101cb55e32250 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12207594.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12207594.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12207594.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12207594.zip
Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12207594.xml
Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12207594.xml
-- Model is restarting --
Adjusting last_iter, 75, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12207594.xml
last_cpu_time: 2358
upload_file_number: 0
last_iter: 49
last_upload: 0
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a0mf_2016092300_15_991_12207594_0_r1355921940
The child process has been launched with process id: 580852
Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:580852:580852] [20230213:113138:1676305898:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:32:49 STEP   48 H=  12:00 +CPU= 27.015
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:33:18 STEP   49 H=  12:15 +CPU= 24.781
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:33:51 STEP   50 H=  12:30 +CPU= 26.121
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:34:21 STEP   51 H=  12:45 +CPU= 25.915
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:34:53 STEP   52 H=  13:00 +CPU= 25.836
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:35:21 STEP   53 H=  13:15 +CPU= 22.336
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:35:53 STEP   54 H=  13:30 +CPU= 25.541
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:36:27 STEP   55 H=  13:45 +CPU= 25.526
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:37:01 STEP   56 H=  14:00 +CPU= 27.619
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:37:34 STEP   57 H=  14:15 +CPU= 26.419
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:38:07 STEP   58 H=  14:30 +CPU= 25.229
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:38:37 STEP   59 H=  14:45 +CPU= 23.850
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:39:15 STEP   60 H=  15:00 +CPU= 31.599
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000060
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000060
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000060
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:39:48 STEP   61 H=  15:15 +CPU= 24.855
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:40:19 STEP   62 H=  15:30 +CPU= 23.377
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:40:48 STEP   63 H=  15:45 +CPU= 23.212
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:41:18 STEP   64 H=  16:00 +CPU= 23.527
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:41:46 STEP   65 H=  16:15 +CPU= 23.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:42:12 STEP   66 H=  16:30 +CPU= 22.776
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:42:41 STEP   67 H=  16:45 +CPU= 22.710
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:43:12 STEP   68 H=  17:00 +CPU= 24.938
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:43:43 STEP   69 H=  17:15 +CPU= 24.852
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:44:12 STEP   70 H=  17:30 +CPU= 22.748
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:44:43 STEP   71 H=  17:45 +CPU= 24.718
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:45:23 STEP   72 H=  18:00 +CPU= 30.663
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000072
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000072
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000072
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:45:53 STEP   73 H=  18:15 +CPU= 23.985
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:46:23 STEP   74 H=  18:30 +CPU= 24.138
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:46:52 STEP   75 H=  18:45 +CPU= 23.262
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:47:23 STEP   76 H=  19:00 +CPU= 24.255
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:47:52 STEP   77 H=  19:15 +CPU= 22.867
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:48:21 STEP   78 H=  19:30 +CPU= 22.823
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:48:49 STEP   79 H=  19:45 +CPU= 21.264
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:49:20 STEP   80 H=  20:00 +CPU= 24.988
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:49:51 STEP   81 H=  20:15 +CPU= 24.119
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:50:22 STEP   82 H=  20:30 +CPU= 24.758
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:50:54 STEP   83 H=  20:45 +CPU= 25.626
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:51:34 STEP   84 H=  21:00 +CPU= 32.301
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000084
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000084
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:52:08 STEP   85 H=  21:15 +CPU= 25.339
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:52:40 STEP   86 H=  21:30 +CPU= 25.487
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:53:12 STEP   87 H=  21:45 +CPU= 25.588
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:53:44 STEP   88 H=  22:00 +CPU= 26.110
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:54:13 STEP   89 H=  22:15 +CPU= 22.808
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:54:42 STEP   90 H=  22:30 +CPU= 25.032
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:55:08 STEP   91 H=  22:45 +CPU= 21.863
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:55:38 STEP   92 H=  23:00 +CPU= 23.532
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:56:08 STEP   93 H=  23:15 +CPU= 23.467
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:56:39 STEP   94 H=  23:30 +CPU= 25.412
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:57:09 STEP   95 H=  23:45 +CPU= 23.544
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:57:51 STEP   96 H=  24:00 +CPU= 33.053
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000096
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000096
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMGGh7zg+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMSHh7zg+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMUAh7zg+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMGGh7zg+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMSHh7zg+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMUAh7zg+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMGGh7zg+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMSHh7zg+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMUAh7zg+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMGGh7zg+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMSHh7zg+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMUAh7zg+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMGGh7zg+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMSHh7zg+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMUAh7zg+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMGGh7zg+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMSHh7zg+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMUAh7zg+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMGGh7zg+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMSHh7zg+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMUAh7zg+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMGGh7zg+000084
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMSHh7zg+000084
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12207594/ICMUAh7zg+000084
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0mf_2016092300_15_991_12207594_0_r1355921940_0.zip
Uploading the intermediate file: upload_file_0.zip
  11:58:13 STEP   97 H=  24:15 +CPU= 20.487
  11:58:33 STEP   98 H=  24:30 +CPU= 19.685
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:59:04 STEP   99 H=  24:45 +CPU= 24.794
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:59:38 STEP  100 H=  25:00 +CPU= 25.596
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:00:08 STEP  101 H=  25:15 +CPU= 23.307
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:00:38 STEP  102 H=  25:30 +CPU= 24.388
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:01:08 STEP  103 H=  25:45 +CPU= 23.211
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:01:39 STEP  104 H=  26:00 +CPU= 25.275
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:02:09 STEP  105 H=  26:15 +CPU= 23.452
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:02:35 STEP  106 H=  26:30 +CPU= 22.550
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:03:05 STEP  107 H=  26:45 +CPU= 23.695
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:03:42 STEP  108 H=  27:00 +CPU= 28.597
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000108
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000108
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000108
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:04:12 STEP  109 H=  27:15 +CPU= 23.954
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:04:41 STEP  110 H=  27:30 +CPU= 22.200
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:05:12 STEP  111 H=  27:45 +CPU= 25.624
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:05:44 STEP  112 H=  28:00 +CPU= 25.204
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:06:18 STEP  113 H=  28:15 +CPU= 25.836
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:06:50 STEP  114 H=  28:30 +CPU= 25.951
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:07:20 STEP  115 H=  28:45 +CPU= 23.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:07:53 STEP  116 H=  29:00 +CPU= 26.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:08:22 STEP  117 H=  29:15 +CPU= 23.516
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:08:52 STEP  118 H=  29:30 +CPU= 23.877
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:09:23 STEP  119 H=  29:45 +CPU= 24.303
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:10:00 STEP  120 H=  30:00 +CPU= 28.882
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000120
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000120
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000120
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:10:30 STEP  121 H=  30:15 +CPU= 23.485
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:10:55 STEP  122 H=  30:30 +CPU= 21.359
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:11:23 STEP  123 H=  30:45 +CPU= 21.563
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:11:52 STEP  124 H=  31:00 +CPU= 23.042
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:12:21 STEP  125 H=  31:15 +CPU= 21.925
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:12:51 STEP  126 H=  31:30 +CPU= 23.707
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:13:15 STEP  127 H=  31:45 +CPU= 21.767
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:13:47 STEP  128 H=  32:00 +CPU= 24.486
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:14:18 STEP  129 H=  32:15 +CPU= 24.028
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:14:48 STEP  130 H=  32:30 +CPU= 23.523
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:15:20 STEP  131 H=  32:45 +CPU= 25.399
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:15:59 STEP  132 H=  33:00 +CPU= 31.032
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGh7zg+000132
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHh7zg+000132
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAh7zg+000132
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:16:30 STEP  133 H=  33:15 +CPU= 24.873
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:17:01 STEP  134 H=  33:30 +CPU= 24.143
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:17:32 STEP  135 H=  33:45 +CPU= 24.307
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:18:02 STEP  136 H=  34:00 +CPU= 23.741
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:18:25 STEP  137 H=  34:15 +CPU= 21.129
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:18:50 STEP  138 H=  34:30 +CPU= 19.196
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:19:15 STEP  139 H=  34:45 +CPU= 21.303
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
..The child process has been killed with signal: 9
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.215143545324660E-07 0.000000000000000E+00 0.180307851982886E-06
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.130056205006054E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.860948316584160E-10 0.000000000000000E+00 0.111979812079165E-08
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.167353338168242E-01 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =   132 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   132 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000132 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000132
 MPI-TASK:   1 -    320931978 BYTES IN      6 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0092( 3.3%)
 NSTEP =   132 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000132 MODE=a
 IO-STREAM CLOSED - ./ICMSHh7zg+000132
 MPI-TASK:   1 -    707765250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0104 SECONDS 68054.4 Mbytes/s, TOTAL TIME=    0.2461( 4.2%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000132 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000132
 MPI-TASK:   1 -   5134911639 BYTES IN    125 RECORDS TRANSFERRED IN    0.0035 SECONDS******** Mbytes/s, TOTAL TIME=    0.1148( 3.0%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000132 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000132
 MPI-TASK:   1 -    312358345 BYTES IN      2 RECORDS TRANSFERRED IN    0.0001 SECONDS******** Mbytes/s, TOTAL TIME=    0.0034( 2.9%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000132 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000132
 MPI-TASK:   1 -    320931978 BYTES IN      4 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0053( 5.7%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   132 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   132 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000132 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000132
 MPI-TASK:   1 -    320931969 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS802329.9 Mbytes/s, TOTAL TIME=    0.0096( 4.2%)
 12:15:59 STEP  132 H=  33:00 +CPU= 31.032
 NSTEP =   133 STEPO   0AAA00AAA
 12:16:30 STEP  133 H=  33:15 +CPU= 24.873
 NSTEP =   134 STEPO   0AAA00AAA
 12:17:01 STEP  134 H=  33:30 +CPU= 24.143
ISTEP=     135 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     135 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   135 STEPO   0AAA00AAA
 12:17:32 STEP  135 H=  33:45 +CPU= 24.307
 NSTEP =   136 STEPO   0AAA00AAA
 12:18:02 STEP  136 H=  34:00 +CPU= 23.741
 NSTEP =   137 STEPO   0AAA00AAA
 12:18:25 STEP  137 H=  34:15 +CPU= 21.129
 NSTEP =   138 STEPO   0AAA00AAA
 12:18:50 STEP  138 H=  34:30 +CPU= 19.196
 NSTEP =   139 STEPO   0AAA00AAA
 12:19:15 STEP  139 H=  34:45 +CPU= 21.303
ISTEP=     140 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     140 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   140 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 12:16:30 0AAA00AAA STEPO      134    24.873   24.873   31.204     35:23     44:52 0.80038754284485E-07       2GB       0MB
 12:17:01 0AAA00AAA STEPO      135    24.143   24.143   30.583     35:47     45:23 0.80038754284485E-07       2GB       0MB
 12:17:32 0AAA00AAA STEPO      136    24.307   24.307   30.488     36:11     45:53 0.80038754284485E-07       2GB       0MB
 12:18:02 0AAA00AAA STEPO      137    23.741   23.741   30.006     36:35     46:23 0.80038754284485E-07       2GB       0MB
 12:18:25 0AAA00AAA STEPO      138    21.129   21.129   23.274     36:56     46:46 0.80038754284485E-07       2GB       0MB
 12:18:50 0AAA00AAA STEPO      139    19.196   19.196   25.241     37:15     47:12 0.80038754284485E-07       2GB       0MB
 12:19:15 0AAA00AAA STEPO      140    21.303   21.303   25.393     37:37     47:37 0.80038754284485E-07       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "      96",
 CTIME   = "00010000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,          84, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/5/progress_file_12207594.xml

<running_values>
  <last_cpu_time>4626.980000</last_cpu_time>
  <upload_file_number>1</upload_file_number>
  <last_iter>140</last_iter>
  <last_upload>86400</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net