climateprediction.net home page
Task 22308252

Task 22308252

Name oifs_43r3_bl_a0zu_2016092300_15_991_12208077_0
Workunit 12208077
Created 13 Feb 2023, 14:23:28 UTC
Sent 13 Feb 2023, 14:27:43 UTC
Report deadline 14 Apr 2023, 14:27:43 UTC
Received 14 Feb 2023, 1:22:28 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1538161
Run time 2 hours 29 min 26 sec
CPU time 2 hours 1 min 56 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 7.28 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,058.36 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,291.42 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
ocess
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:16:18 STEP  197 H=  49:15 +CPU= 20.419
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:16:48 STEP  198 H=  49:30 +CPU= 24.364
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:17:20 STEP  199 H=  49:45 +CPU= 24.886
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:17:50 STEP  200 H=  50:00 +CPU= 24.232
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:18:21 STEP  201 H=  50:15 +CPU= 24.678
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:18:51 STEP  202 H=  50:30 +CPU= 24.257
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:19:22 STEP  203 H=  50:45 +CPU= 24.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:20:01 STEP  204 H=  51:00 +CPU= 30.323
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000204
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000204
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000204
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:20:31 STEP  205 H=  51:15 +CPU= 24.615
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:21:02 STEP  206 H=  51:30 +CPU= 24.032
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:21:32 STEP  207 H=  51:45 +CPU= 24.398
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:22:03 STEP  208 H=  52:00 +CPU= 24.474
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:22:32 STEP  209 H=  52:15 +CPU= 23.075
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:23:01 STEP  210 H=  52:30 +CPU= 22.270
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:23:30 STEP  211 H=  52:45 +CPU= 23.253
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:24:03 STEP  212 H=  53:00 +CPU= 26.593
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:24:37 STEP  213 H=  53:15 +CPU= 25.533
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:07 STEP  214 H=  53:30 +CPU= 24.020
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:37 STEP  215 H=  53:45 +CPU= 23.455
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:26:13 STEP  216 H=  54:00 +CPU= 29.206
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000216
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000216
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000216
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:26:44 STEP  217 H=  54:15 +CPU= 24.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:27:14 STEP  218 H=  54:30 +CPU= 24.073
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:27:44 STEP  219 H=  54:45 +CPU= 23.423
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:28:12 STEP  220 H=  55:00 +CPU= 22.542
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:28:41 STEP  221 H=  55:15 +CPU= 22.476
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:29:09 STEP  222 H=  55:30 +CPU= 21.727
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:29:34 STEP  223 H=  55:45 +CPU= 21.035
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:30:04 STEP  224 H=  56:00 +CPU= 23.134
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:30:32 STEP  225 H=  56:15 +CPU= 21.366
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:30:58 STEP  226 H=  56:30 +CPU= 22.689
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:31:28 STEP  227 H=  56:45 +CPU= 23.094
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:32:03 STEP  228 H=  57:00 +CPU= 28.856
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000228
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000228
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000228
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:32:32 STEP  229 H=  57:15 +CPU= 22.462
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:33:01 STEP  230 H=  57:30 +CPU= 23.134
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:33:32 STEP  231 H=  57:45 +CPU= 24.449
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:34:02 STEP  232 H=  58:00 +CPU= 24.052
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:34:33 STEP  233 H=  58:15 +CPU= 23.893
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:35:02 STEP  234 H=  58:30 +CPU= 23.220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:35:32 STEP  235 H=  58:45 +CPU= 23.755
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:36:02 STEP  236 H=  59:00 +CPU= 22.994
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:36:28 STEP  237 H=  59:15 +CPU= 21.637
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:36:54 STEP  238 H=  59:30 +CPU= 21.955
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:37:22 STEP  239 H=  59:45 +CPU= 21.826
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:38:02 STEP  240 H=  60:00 +CPU= 31.448
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000240
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000240
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000240
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:38:29 STEP  241 H=  60:15 +CPU= 21.911
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:38:59 STEP  242 H=  60:30 +CPU= 23.995
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:39:28 STEP  243 H=  60:45 +CPU= 23.456
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:39:58 STEP  244 H=  61:00 +CPU= 23.605
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:40:29 STEP  245 H=  61:15 +CPU= 24.753
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:40:57 STEP  246 H=  61:30 +CPU= 21.659
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:41:24 STEP  247 H=  61:45 +CPU= 22.846
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:41:51 STEP  248 H=  62:00 +CPU= 23.263
Quit request received from BOINC client, ending the child process
>>> Printing last 70 lines from file: NODE.001_01

I     I Sens. heat rain II      0.00000 I              I              I              I               I
I     I Sens. heat snow II      0.00000 I              I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I PREC- Pot. and kin. E.II      0.00000 I      0.00000 I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I   Orographic drag     II      0.00000 I      0.00000 I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I      I Balance        II      0.00000 I              I              I              I               I
I ADIAB- of which       II              I              I              I              I               I
I      I Hor. Flux of PEII     -0.00000 I              I              I              I               I
I      I Vert.Flux of PEII      0.00000 I              I              I              I               I
I      I Work Pres.G.F  II              I     -0.00824 I              I              I               I
I      I Conv. Ad.-Int. II      0.00824 I              I              I              I               I
DDH--------------------------------------------------------------------------------------------------I
I DYN- Horizont. Flux   II      0.00043 I     -0.00031 I      0.00765 I      0.00000 I     -0.00000  I
I    I Vertical Flux    II      0.00000 I      0.00000 I      0.00000 I      0.00000 I      0.00000  I
DDH--------------------------------------------------------------------------------------------------I
  
DDH----------------------------------------------------------------------------------------------------------------------------------
  I   - Mass balance    (10**(-6)  KG/M**2) 
  
  I INITIAL      I HORIZ. FLUX  I DYN. FLUX    I DYN. FLUX    I PHYS. FLUX   I PHYS. FLUX   I CURRENT      I I I RESIDUAL     I
  I MASS         I              I TOA          I SURFACE      I TOA          I SURFACE      I MASS         I I I              I
  ---------------------------------------------------------------------------------------------------------- I ----------------
  -              -              -              -              -              -              -              - I -              -
  - 10206523039. I           0. .           0. .           0. .           0. .           0. I 10206522993. - I -         -46. -
  -              -              -              -              -              -              -              - I -              -
  ---------------------------------------------------------------------------------------------------------- I ----------------
 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =   60.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.1414511045E-21      I       0.1658986286E-06      I
ICloud fractionI      0.0000000000E+00      I       0.5077553127E-03      I
I  Relat. Hum. I      0.4557032430E+00      I       0.4649002918E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.5978551336E+01      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   240 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000240 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000240
 MPI-TASK:   1 -    307745377 BYTES IN      7 RECORDS TRANSFERRED IN    0.0005 SECONDS615490.7 Mbytes/s, TOTAL TIME=    0.0108( 4.6%)
 16:38:02 STEP  240 H=  60:00 +CPU= 31.448
 NSTEP =   241 STEPO   0AAA00AAA
 16:38:29 STEP  241 H=  60:15 +CPU= 21.911
 NSTEP =   242 STEPO   0AAA00AAA
 16:38:59 STEP  242 H=  60:30 +CPU= 23.995
 NSTEP =   243 STEPO   0AAA00AAA
 16:39:28 STEP  243 H=  60:45 +CPU= 23.456
 NSTEP =   244 STEPO   0AAA00AAA
 16:39:58 STEP  244 H=  61:00 +CPU= 23.605
ISTEP=     245 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     245 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   245 STEPO   0AAA00AAA
 16:40:29 STEP  245 H=  61:15 +CPU= 24.753
 NSTEP =   246 STEPO   0AAA00AAA
 16:40:57 STEP  246 H=  61:30 +CPU= 21.659
 NSTEP =   247 STEPO   0AAA00AAA
 16:41:24 STEP  247 H=  61:45 +CPU= 22.846
 NSTEP =   248 STEPO   0AAA00AAA
 16:41:51 STEP  248 H=  62:00 +CPU= 23.263
 NSTEP =   249 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 16:38:59 0AAA00AAA STEPO      243    23.995   23.995   30.159     97:24  -3431:** 0.99945229889660E-07       2GB       0MB
 16:39:28 0AAA00AAA STEPO      244    23.456   23.456   29.562     97:48  -3430:** 0.99945229889660E-07       2GB       0MB
 16:39:58 0AAA00AAA STEPO      245    23.605   23.605   29.683     98:11  -3430:** 0.99945229889660E-07       2GB       0MB
 16:40:29 0AAA00AAA STEPO      246    24.753   24.753   30.984     98:36  -3429:** 0.99945229889660E-07       2GB       0MB
 16:40:57 0AAA00AAA STEPO      247    21.660   21.660   27.793     98:58  -3429:** 0.99945229889660E-07       2GB       0MB
 16:41:24 0AAA00AAA STEPO      248    22.846   22.846   26.909     99:20  -3429:** 0.99945229889660E-07       2GB       0MB
 16:41:51 0AAA00AAA STEPO      249    23.263   23.263   27.342     99:44  -3428:** 0.99945229889660E-07       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     240",
 CTIME   = "00021200      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         228, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/13/progress_file_12208077.xml

<running_values>
  <last_cpu_time>6078.870000</last_cpu_time>
  <upload_file_number>2</upload_file_number>
  <last_iter>249</last_iter>
  <last_upload>172800</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a0zu
(argv4) batchid: 991
(argv5) wuid: 12208077
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/13
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/13/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/13/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_d431967cdf9a07b90a7d41a8f727ae52 to: /var/lib/boinc-client/slots/13/oifs_43r3_bl_a0zu_2016092300_15_991_12208077.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/13/oifs_43r3_bl_a0zu_2016092300_15_991_12208077.zip
ic_ancil_file: ic_ancil_12208077
ifsdata_file: ifsdata_12208077
climate_data_file: clim_data_12208077
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/13/ic_ancil_12208077.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/13/ic_ancil_12208077.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_74b8198107a41083cc31bed7a12ef4c5 to: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12208077.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12208077.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/13/159l_2/clim_data_12208077.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/13/159l_2/clim_data_12208077.zip
Checking for progress XML file: /var/lib/boinc-client/slots/13/progress_file_12208077.xml
Opened progress file ok : /var/lib/boinc-client/slots/13/progress_file_12208077.xml
-- Model is restarting --
Adjusting last_iter, 249, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/13/progress_file_12208077.xml
last_cpu_time: 6078
upload_file_number: 2
last_iter: 241
last_upload: 172800
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a0zu_2016092300_15_991_12208077_0_r634992926
The child process has been launched with process id: 601457
Executing the command: /var/lib/boinc-client/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:47:11 STEP  240 H=  60:00 +CPU= 24.626
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:47:41 STEP  241 H=  60:15 +CPU= 23.496
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:48:11 STEP  242 H=  60:30 +CPU= 23.871
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:48:42 STEP  243 H=  60:45 +CPU= 24.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:49:12 STEP  244 H=  61:00 +CPU= 24.029
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:49:42 STEP  245 H=  61:15 +CPU= 23.048
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:50:13 STEP  246 H=  61:30 +CPU= 24.707
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:50:42 STEP  247 H=  61:45 +CPU= 22.680
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:51:12 STEP  248 H=  62:00 +CPU= 23.178
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:51:40 STEP  249 H=  62:15 +CPU= 22.213
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:52:09 STEP  250 H=  62:30 +CPU= 22.638
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:52:38 STEP  251 H=  62:45 +CPU= 23.189
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:53:15 STEP  252 H=  63:00 +CPU= 29.592
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000252
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000252
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000252
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:53:47 STEP  253 H=  63:15 +CPU= 24.098
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:54:17 STEP  254 H=  63:30 +CPU= 24.285
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:54:47 STEP  255 H=  63:45 +CPU= 23.816
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:55:19 STEP  256 H=  64:00 +CPU= 25.659
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:55:49 STEP  257 H=  64:15 +CPU= 23.460
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:56:19 STEP  258 H=  64:30 +CPU= 23.969
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:56:47 STEP  259 H=  64:45 +CPU= 24.370
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:57:12 STEP  260 H=  65:00 +CPU= 22.790
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:57:42 STEP  261 H=  65:15 +CPU= 23.475
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:58:14 STEP  262 H=  65:30 +CPU= 25.206
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:58:43 STEP  263 H=  65:45 +CPU= 24.859
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:59:22 STEP  264 H=  66:00 +CPU= 31.621
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000264
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000264
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000264
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:59:57 STEP  265 H=  66:15 +CPU= 26.968
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:00:28 STEP  266 H=  66:30 +CPU= 24.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:02:09 STEP  267 H=  66:45 +CPU= 24.851
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:02:37 STEP  268 H=  67:00 +CPU= 22.684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:03:07 STEP  269 H=  67:15 +CPU= 23.514
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:19:15 STEP  270 H=  67:30 +CPU= 25.586
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:19:43 STEP  271 H=  67:45 +CPU= 21.883
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:20:10 STEP  272 H=  68:00 +CPU= 22.394
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:30:45 STEP  273 H=  68:15 +CPU= 27.034
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:31:15 STEP  274 H=  68:30 +CPU= 23.408
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:31:42 STEP  275 H=  68:45 +CPU= 23.084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:32:46 STEP  276 H=  69:00 +CPU= 29.656
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000276
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000276
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000276
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:33:12 STEP  277 H=  69:15 +CPU= 21.914
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:33:40 STEP  278 H=  69:30 +CPU= 23.678
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:34:09 STEP  279 H=  69:45 +CPU= 24.451
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:35:05 STEP  280 H=  70:00 +CPU= 26.405
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:37:26 STEP  281 H=  70:15 +CPU= 26.325
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:37:56 STEP  282 H=  70:30 +CPU= 23.683
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:38:27 STEP  283 H=  70:45 +CPU= 25.119
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:39:16 STEP  284 H=  71:00 +CPU= 24.213
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:39:40 STEP  285 H=  71:15 +CPU= 22.004
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:40:13 STEP  286 H=  71:30 +CPU= 24.844
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:40:43 STEP  287 H=  71:45 +CPU= 24.084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:41:25 STEP  288 H=  72:00 +CPU= 32.065
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000288
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000288
Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000276
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0zu_2016092300_15_991_12208077_0_r634992926_2.zip
Uploading the intermediate file: upload_file_2.zip
  17:41:47 STEP  289 H=  72:15 +CPU= 22.220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
..The child process has been killed with signal: 9
>>> Printing last 70 lines from file: NODE.001_01

 GPNORM RAIN                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM  CLOUD FRACTION      AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.104295293200559E-01 0.000000000000000E+00 0.100000000000000E+01
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.576137522434097E-09 0.000000000000000E+00 0.526710797418912E-07
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.104677350890231E-01 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.594646321555523E-12 0.000000000000000E+00 0.203317735571648E-10
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.791136656756934E-02 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =   288 STEPO   F00000000
 CREATE RESTART FILE ./srf00030000.0001                                                                                                                                                                                                                                              
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./srf00030000.0001 MODE=w
 IO-STREAM CLOSED - ./srf00030000.0001
 MPI-TASK:   1 - ************ BYTES IN     34 RECORDS TRANSFERRED IN    2.1279 SECONDS******** Mbytes/s, TOTAL TIME=    3.4839(61.1%)
 CREATE RESTART CONTROL FILE ./rcf                                                                                                                   
 WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00021200.0001
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   288 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000288 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000288
 MPI-TASK:   1 -    311589210 BYTES IN      6 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0884( 0.3%)
 NSTEP =   288 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000288 MODE=a
 IO-STREAM CLOSED - ./ICMSHh7zg+000288
 MPI-TASK:   1 -    707765250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0098 SECONDS 72220.9 Mbytes/s, TOTAL TIME=    0.4079( 2.4%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000288 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000288
 MPI-TASK:   1 -   4985427351 BYTES IN    125 RECORDS TRANSFERRED IN    0.0023 SECONDS******** Mbytes/s, TOTAL TIME=    0.1219( 1.9%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000288 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000288
 MPI-TASK:   1 -    303015577 BYTES IN      2 RECORDS TRANSFERRED IN    0.0001 SECONDS******** Mbytes/s, TOTAL TIME=    0.0077( 1.3%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000288 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000288
 MPI-TASK:   1 -    311589210 BYTES IN      4 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0120( 1.7%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   288 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   288 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000288 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000288
 MPI-TASK:   1 -    311589201 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS778973.0 Mbytes/s, TOTAL TIME=    0.0103( 3.9%)
 17:41:25 STEP  288 H=  72:00 +CPU= 32.065
 NSTEP =   289 STEPO   0AAA00AAA
 17:41:47 STEP  289 H=  72:15 +CPU= 22.220
ISTEP=     290 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     290 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   290 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 17:40:48 0AA000000 STEPO      288     3.227    3.227    3.971     19:39     54:10 0.10830779545655E-06       2GB       0MB
 17:40:49 FULLPOS-B DYNFPOS    288     1.383    1.383    1.395     19:40     54:12 0.10830779545655E-06       2GB       0MB
 17:40:54 FULLPOS-V DYNFPOS    288     2.559    2.559    4.788     19:42     54:16 0.10830779545655E-06       2GB       0MB
 17:40:54 FULLPOS-T DYNFPOS    288     0.476    0.476    0.494     19:43     54:17 0.10830779545655E-06       2GB       0MB
 17:40:55 0AAA00AAA STEPO      288     0.447    0.447    0.454     19:43     54:17 0.10830779545655E-06       2GB       0MB
 17:41:25 0AAA00AAA STEPO      289    23.774   23.774   29.910     20:07     54:47 0.10830779545655E-06       2GB       0MB
 17:41:47 0AAA00AAA STEPO      290    22.221   22.221   22.875     20:29     55:10 0.10830779545655E-06       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     288",
 CTIME   = "00030000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         276, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/13/progress_file_12208077.xml

<running_values>
  <last_cpu_time>7286.080000</last_cpu_time>
  <upload_file_number>3</upload_file_number>
  <last_iter>289</last_iter>
  <last_upload>259200</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net