climateprediction.net home page
Task 22261310

Task 22261310

Name oifs_43r3_ps_0909_1990050100_123_959_12176553_0
Workunit 12176553
Created 21 Dec 2022, 20:24:59 UTC
Sent 21 Dec 2022, 20:44:11 UTC
Report deadline 20 Jan 2023, 20:44:11 UTC
Received 29 Dec 2022, 9:14:50 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1529561
Run time 16 hours 14 min 32 sec
CPU time 15 hours 24 min 54 sec
Validate state Invalid
Credit 2,419.20
Device peak FLOPS 3.20 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,146.24 MB
Peak swap size 4,974.80 MB
Peak disk usage 1,538.01 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
  20:16:29 STEP  107 H= 107:00 +CPU= 78.482
  20:18:53 STEP  108 H= 108:00 +CPU=140.496
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000108
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000108
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000108
  20:20:19 STEP  109 H= 109:00 +CPU= 82.909
  20:21:48 STEP  110 H= 110:00 +CPU= 85.933
  20:23:58 STEP  111 H= 111:00 +CPU=126.692
  20:25:27 STEP  112 H= 112:00 +CPU= 84.982
  20:26:52 STEP  113 H= 113:00 +CPU= 82.356
  20:29:10 STEP  114 H= 114:00 +CPU=132.690
  20:30:39 STEP  115 H= 115:00 +CPU= 84.127
  20:32:07 STEP  116 H= 116:00 +CPU= 84.807
  20:34:20 STEP  117 H= 117:00 +CPU=126.769
  20:35:47 STEP  118 H= 118:00 +CPU= 83.121
  20:37:17 STEP  119 H= 119:00 +CPU= 83.824
  20:39:59 STEP  120 H= 120:00 +CPU=153.670
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000120
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000120
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000108
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_4.zip
Uploading the intermediate file: upload_file_4.zip
  20:41:28 STEP  121 H= 121:00 +CPU= 82.611
  20:43:05 STEP  122 H= 122:00 +CPU= 85.793
  20:45:22 STEP  123 H= 123:00 +CPU=128.960
  20:46:53 STEP  124 H= 124:00 +CPU= 84.962
  20:48:16 STEP  125 H= 125:00 +CPU= 80.814
  20:50:28 STEP  126 H= 126:00 +CPU=126.780
  20:51:50 STEP  127 H= 127:00 +CPU= 78.623
  20:53:14 STEP  128 H= 128:00 +CPU= 81.264
  20:55:21 STEP  129 H= 129:00 +CPU=121.832
  20:56:47 STEP  130 H= 130:00 +CPU= 81.257
  20:58:09 STEP  131 H= 131:00 +CPU= 78.400
  21:00:28 STEP  132 H= 132:00 +CPU=135.310
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000132
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000132
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000132
  21:01:53 STEP  133 H= 133:00 +CPU= 80.239
  21:03:19 STEP  134 H= 134:00 +CPU= 82.921
  21:05:25 STEP  135 H= 135:00 +CPU=122.116
  21:06:50 STEP  136 H= 136:00 +CPU= 82.015
  21:08:14 STEP  137 H= 137:00 +CPU= 78.776
  21:10:23 STEP  138 H= 138:00 +CPU=125.059
  21:11:47 STEP  139 H= 139:00 +CPU= 78.721
  21:13:12 STEP  140 H= 140:00 +CPU= 81.463
  21:15:19 STEP  141 H= 141:00 +CPU=122.007
  21:16:41 STEP  142 H= 142:00 +CPU= 79.741
  21:18:04 STEP  143 H= 143:00 +CPU= 79.912
  21:20:38 STEP  144 H= 144:00 +CPU=144.777
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000144
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000144
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000132
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_5.zip
Uploading the intermediate file: upload_file_5.zip
  21:22:03 STEP  145 H= 145:00 +CPU= 80.034
  21:23:31 STEP  146 H= 146:00 +CPU= 83.760
  21:25:45 STEP  147 H= 147:00 +CPU=124.388
  21:27:13 STEP  148 H= 148:00 +CPU= 82.115
  21:28:36 STEP  149 H= 149:00 +CPU= 80.052
  21:30:50 STEP  150 H= 150:00 +CPU=128.827
  21:32:15 STEP  151 H= 151:00 +CPU= 81.811
  21:33:44 STEP  152 H= 152:00 +CPU= 85.035
  21:35:56 STEP  153 H= 153:00 +CPU=126.200
  21:37:25 STEP  154 H= 154:00 +CPU= 84.157
  21:38:51 STEP  155 H= 155:00 +CPU= 80.901
  21:41:15 STEP  156 H= 156:00 +CPU=139.538
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000156
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000156
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000156
  21:42:38 STEP  157 H= 157:00 +CPU= 80.279
  21:44:03 STEP  158 H= 158:00 +CPU= 81.437
  21:46:07 STEP  159 H= 159:00 +CPU=121.549
  21:47:32 STEP  160 H= 160:00 +CPU= 82.058
  21:48:55 STEP  161 H= 161:00 +CPU= 79.576
  21:51:09 STEP  162 H= 162:00 +CPU=125.689
  21:52:31 STEP  163 H= 163:00 +CPU= 79.233
  21:53:56 STEP  164 H= 164:00 +CPU= 81.291
  21:56:08 STEP  165 H= 165:00 +CPU=124.330
  21:57:35 STEP  166 H= 166:00 +CPU= 83.123
  21:59:02 STEP  167 H= 167:00 +CPU= 82.526
  22:01:41 STEP  168 H= 168:00 +CPU=149.226
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000168
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000168
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000156
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_6.zip
Uploading the intermediate file: upload_file_6.zip
  22:03:07 STEP  169 H= 169:00 +CPU= 80.214
  22:04:38 STEP  170 H= 170:00 +CPU= 85.348
  22:06:55 STEP  171 H= 171:00 +CPU=127.357
  22:08:25 STEP  172 H= 172:00 +CPU= 85.509
  22:09:54 STEP  173 H= 173:00 +CPU= 82.911
  22:12:14 STEP  174 H= 174:00 +CPU=131.776
  22:13:39 STEP  175 H= 175:00 +CPU= 81.900
  22:15:07 STEP  176 H= 176:00 +CPU= 83.869
  22:17:21 STEP  177 H= 177:00 +CPU=125.494
  22:18:50 STEP  178 H= 178:00 +CPU= 84.534
  22:20:14 STEP  179 H= 179:00 +CPU= 81.161
  22:22:46 STEP  180 H= 180:00 +CPU=142.934
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000180
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000180
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000180
  22:24:15 STEP  181 H= 181:00 +CPU= 80.456
  22:25:39 STEP  182 H= 182:00 +CPU= 80.793
  22:27:46 STEP  183 H= 183:00 +CPU=122.770
  22:29:10 STEP  184 H= 184:00 +CPU= 81.054
  22:30:30 STEP  185 H= 185:00 +CPU= 77.842
  22:32:40 STEP  186 H= 186:00 +CPU=125.759
  22:34:01 STEP  187 H= 187:00 +CPU= 77.618
  22:35:27 STEP  188 H= 188:00 +CPU= 82.742
  22:37:40 STEP  189 H= 189:00 +CPU=126.536
  22:39:05 STEP  190 H= 190:00 +CPU= 81.897
  22:40:31 STEP  191 H= 191:00 +CPU= 82.804
  22:43:12 STEP  192 H= 192:00 +CPU=150.805
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000192
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000192
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000180
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_7.zip
Uploading the intermediate file: upload_file_7.zip
  22:44:39 STEP  193 H= 193:00 +CPU= 81.210
  22:46:10 STEP  194 H= 194:00 +CPU= 85.934
  22:48:26 STEP  195 H= 195:00 +CPU=127.104
  22:49:53 STEP  196 H= 196:00 +CPU= 83.265
  22:51:20 STEP  197 H= 197:00 +CPU= 80.725
  22:53:35 STEP  198 H= 198:00 +CPU=128.787
  22:55:03 STEP  199 H= 199:00 +CPU= 79.897
  22:56:32 STEP  200 H= 200:00 +CPU= 79.911
  22:58:22 STEP  201 H= 201:00 +CPU=102.868
  22:59:20 STEP  202 H= 202:00 +CPU= 56.923
  23:00:16 STEP  203 H= 203:00 +CPU= 53.653
  23:01:55 STEP  204 H= 204:00 +CPU= 94.265
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000204
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000204
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000204
  23:02:49 STEP  205 H= 205:00 +CPU= 51.441
  23:03:43 STEP  206 H= 206:00 +CPU= 53.007
  23:05:07 STEP  207 H= 207:00 +CPU= 81.101
  23:06:03 STEP  208 H= 208:00 +CPU= 53.571
  23:06:57 STEP  209 H= 209:00 +CPU= 52.509
  23:08:24 STEP  210 H= 210:00 +CPU= 84.135
  23:09:18 STEP  211 H= 211:00 +CPU= 52.142
  23:10:13 STEP  212 H= 212:00 +CPU= 53.549
  23:11:36 STEP  213 H= 213:00 +CPU= 81.471
  23:12:32 STEP  214 H= 214:00 +CPU= 53.652
  23:13:29 STEP  215 H= 215:00 +CPU= 54.986
  23:15:16 STEP  216 H= 216:00 +CPU=101.291
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000216
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000216
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000204
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_8.zip
Uploading the intermediate file: upload_file_8.zip
  23:16:13 STEP  217 H= 217:00 +CPU= 53.516
  23:17:11 STEP  218 H= 218:00 +CPU= 55.744
  23:18:41 STEP  219 H= 219:00 +CPU= 86.450
  23:19:42 STEP  220 H= 220:00 +CPU= 58.955
  23:20:45 STEP  221 H= 221:00 +CPU= 58.564
  23:22:31 STEP  222 H= 222:00 +CPU= 98.709
  23:23:34 STEP  223 H= 223:00 +CPU= 57.985
  23:24:39 STEP  224 H= 224:00 +CPU= 61.148
  23:26:16 STEP  225 H= 225:00 +CPU= 92.994
  23:27:19 STEP  226 H= 226:00 +CPU= 60.639
  23:28:20 STEP  227 H= 227:00 +CPU= 58.609
  23:30:06 STEP  228 H= 228:00 +CPU=101.956
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000228
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000228
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000228
  23:31:08 STEP  229 H= 229:00 +CPU= 58.033
  23:32:10 STEP  230 H= 230:00 +CPU= 60.388
  23:33:46 STEP  231 H= 231:00 +CPU= 93.512
  23:34:51 STEP  232 H= 232:00 +CPU= 61.839
  23:35:51 STEP  233 H= 233:00 +CPU= 57.584
  23:37:32 STEP  234 H= 234:00 +CPU= 96.945
  23:38:33 STEP  235 H= 235:00 +CPU= 60.030
  23:39:40 STEP  236 H= 236:00 +CPU= 64.059
  23:41:17 STEP  237 H= 237:00 +CPU= 93.982
  23:42:20 STEP  238 H= 238:00 +CPU= 61.396
  23:43:23 STEP  239 H= 239:00 +CPU= 61.136
  23:45:27 STEP  240 H= 240:00 +CPU=117.241
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000240
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000240
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000228
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_9.zip
Uploading the intermediate file: upload_file_9.zip
Uploading trickle at timestep: 864000
  23:46:39 STEP  241 H= 241:00 +CPU= 66.782
  23:47:51 STEP  242 H= 242:00 +CPU= 69.739
  23:49:42 STEP  243 H= 243:00 +CPU=104.325
  23:51:00 STEP  244 H= 244:00 +CPU= 70.023
  23:52:09 STEP  245 H= 245:00 +CPU= 65.595
  23:53:50 STEP  246 H= 246:00 +CPU= 98.129
  23:54:53 STEP  247 H= 247:00 +CPU= 61.526
  23:56:00 STEP  248 H= 248:00 +CPU= 65.202
  23:57:38 STEP  249 H= 249:00 +CPU= 96.625
  23:58:45 STEP  250 H= 250:00 +CPU= 64.113
  23:59:50 STEP  251 H= 251:00 +CPU= 61.916
  00:01:51 STEP  252 H= 252:00 +CPU=113.922
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000252
  00:03:06 STEP  253 H= 253:00 +CPU= 67.878
  00:04:16 STEP  254 H= 254:00 +CPU= 68.806
  00:05:58 STEP  255 H= 255:00 +CPU= 99.443
  00:07:06 STEP  256 H= 256:00 +CPU= 66.021
  00:08:12 STEP  257 H= 257:00 +CPU= 64.213
  00:09:51 STEP  258 H= 258:00 +CPU= 96.613
  00:10:55 STEP  259 H= 259:00 +CPU= 60.222
  00:11:57 STEP  260 H= 260:00 +CPU= 60.641
  00:13:32 STEP  261 H= 261:00 +CPU= 92.892
  00:14:35 STEP  262 H= 262:00 +CPU= 61.666
  00:15:37 STEP  263 H= 263:00 +CPU= 60.357
  00:17:35 STEP  264 H= 264:00 +CPU=110.858
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000264
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000264
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000252
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_10.zip
Uploading the intermediate file: upload_file_10.zip
  00:18:38 STEP  265 H= 265:00 +CPU= 58.801
  00:19:42 STEP  266 H= 266:00 +CPU= 61.110
  00:21:24 STEP  267 H= 267:00 +CPU= 97.391
  00:22:39 STEP  268 H= 268:00 +CPU= 69.521
  00:23:49 STEP  269 H= 269:00 +CPU= 67.190
  00:25:54 STEP  270 H= 270:00 +CPU=116.753
  00:27:13 STEP  271 H= 271:00 +CPU= 72.812
  00:28:35 STEP  272 H= 272:00 +CPU= 78.652
  00:30:45 STEP  273 H= 273:00 +CPU=123.987
  00:32:14 STEP  274 H= 274:00 +CPU= 83.823
  00:33:43 STEP  275 H= 275:00 +CPU= 81.197
  00:36:10 STEP  276 H= 276:00 +CPU=141.838
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000276
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000276
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000276
  00:37:38 STEP  277 H= 277:00 +CPU= 81.947
  00:39:08 STEP  278 H= 278:00 +CPU= 84.825
  00:41:21 STEP  279 H= 279:00 +CPU=127.087
  00:42:53 STEP  280 H= 280:00 +CPU= 84.642
  00:44:17 STEP  281 H= 281:00 +CPU= 81.019
  00:46:32 STEP  282 H= 282:00 +CPU=129.945
  00:47:58 STEP  283 H= 283:00 +CPU= 81.825
  00:49:24 STEP  284 H= 284:00 +CPU= 83.101
  00:51:26 STEP  285 H= 285:00 +CPU=118.258
  00:52:44 STEP  286 H= 286:00 +CPU= 74.421
  00:54:05 STEP  287 H= 287:00 +CPU= 78.051
  00:56:51 STEP  288 H= 288:00 +CPU=155.432
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000288
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000288
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000276
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_11.zip
Uploading the intermediate file: upload_file_11.zip
  00:58:20 STEP  289 H= 289:00 +CPU= 82.068
  00:59:50 STEP  290 H= 290:00 +CPU= 85.429
  01:02:08 STEP  291 H= 291:00 +CPU=128.485
  01:03:40 STEP  292 H= 292:00 +CPU= 85.488
  01:05:06 STEP  293 H= 293:00 +CPU= 81.684
  01:07:27 STEP  294 H= 294:00 +CPU=132.506
  01:08:53 STEP  295 H= 295:00 +CPU= 81.935
  01:10:22 STEP  296 H= 296:00 +CPU= 85.636
  01:12:35 STEP  297 H= 297:00 +CPU=127.146
  01:14:03 STEP  298 H= 298:00 +CPU= 84.186
  01:15:28 STEP  299 H= 299:00 +CPU= 82.341
  01:18:02 STEP  300 H= 300:00 +CPU=146.210
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000300
  01:19:28 STEP  301 H= 301:00 +CPU= 83.333
  01:20:57 STEP  302 H= 302:00 +CPU= 85.963
  01:23:12 STEP  303 H= 303:00 +CPU=129.206
  01:24:42 STEP  304 H= 304:00 +CPU= 86.358
  01:26:08 STEP  305 H= 305:00 +CPU= 82.714
  01:28:25 STEP  306 H= 306:00 +CPU=132.604
  01:29:52 STEP  307 H= 307:00 +CPU= 82.888
  01:31:23 STEP  308 H= 308:00 +CPU= 86.268
  01:33:38 STEP  309 H= 309:00 +CPU=129.370
  01:35:10 STEP  310 H= 310:00 +CPU= 86.531
  01:36:41 STEP  311 H= 311:00 +CPU= 86.192
  01:39:25 STEP  312 H= 312:00 +CPU=155.154
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000300
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_12.zip
Uploading the intermediate file: upload_file_12.zip
  01:40:53 STEP  313 H= 313:00 +CPU= 82.318
  01:42:25 STEP  314 H= 314:00 +CPU= 85.908
  01:44:45 STEP  315 H= 315:00 +CPU=127.413
  01:46:30 STEP  316 H= 316:00 +CPU= 85.568
  01:48:05 STEP  317 H= 317:00 +CPU= 83.055
  01:50:26 STEP  318 H= 318:00 +CPU=133.513
  01:51:51 STEP  319 H= 319:00 +CPU= 82.129
  01:53:19 STEP  320 H= 320:00 +CPU= 85.190
  01:55:27 STEP  321 H= 321:00 +CPU=124.539
  01:56:55 STEP  322 H= 322:00 +CPU= 84.396
  01:58:22 STEP  323 H= 323:00 +CPU= 82.325
  02:00:54 STEP  324 H= 324:00 +CPU=145.661
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000324
  02:02:20 STEP  325 H= 325:00 +CPU= 82.718
  02:03:47 STEP  326 H= 326:00 +CPU= 83.965
  02:06:00 STEP  327 H= 327:00 +CPU=127.526
  02:07:41 STEP  328 H= 328:00 +CPU= 85.424
  02:09:11 STEP  329 H= 329:00 +CPU= 82.627
  02:11:27 STEP  330 H= 330:00 +CPU=131.898
  02:12:53 STEP  331 H= 331:00 +CPU= 82.155
  02:14:22 STEP  332 H= 332:00 +CPU= 84.772
  02:16:35 STEP  333 H= 333:00 +CPU=127.228
  02:18:05 STEP  334 H= 334:00 +CPU= 83.947
  02:19:31 STEP  335 H= 335:00 +CPU= 83.541
  02:22:15 STEP  336 H= 336:00 +CPU=152.836
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000324
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_13.zip
Uploading the intermediate file: upload_file_13.zip
  02:23:45 STEP  337 H= 337:00 +CPU= 83.174
  02:25:14 STEP  338 H= 338:00 +CPU= 85.445
  02:27:39 STEP  339 H= 339:00 +CPU=127.462
  02:29:06 STEP  340 H= 340:00 +CPU= 83.772
  02:30:29 STEP  341 H= 341:00 +CPU= 80.834
  02:32:47 STEP  342 H= 342:00 +CPU=128.799
  02:34:12 STEP  343 H= 343:00 +CPU= 79.776
  02:35:42 STEP  344 H= 344:00 +CPU= 84.169
  02:38:00 STEP  345 H= 345:00 +CPU=127.436
  02:39:32 STEP  346 H= 346:00 +CPU= 85.485
  02:40:57 STEP  347 H= 347:00 +CPU= 81.694
  02:43:28 STEP  348 H= 348:00 +CPU=142.724
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000348
  02:44:55 STEP  349 H= 349:00 +CPU= 82.221
  02:46:22 STEP  350 H= 350:00 +CPU= 83.986
  02:48:35 STEP  351 H= 351:00 +CPU=128.116
  02:50:10 STEP  352 H= 352:00 +CPU= 85.397
  02:51:48 STEP  353 H= 353:00 +CPU= 83.065
  02:54:12 STEP  354 H= 354:00 +CPU=131.894
  02:55:42 STEP  355 H= 355:00 +CPU= 83.208
  02:57:13 STEP  356 H= 356:00 +CPU= 84.748
  02:59:28 STEP  357 H= 357:00 +CPU=127.697
  03:00:57 STEP  358 H= 358:00 +CPU= 85.311
  03:02:26 STEP  359 H= 359:00 +CPU= 84.467
  03:05:11 STEP  360 H= 360:00 +CPU=154.998
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000348
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_14.zip
Uploading the intermediate file: upload_file_14.zip
  03:06:42 STEP  361 H= 361:00 +CPU= 83.421
  03:08:15 STEP  362 H= 362:00 +CPU= 86.691
  03:10:30 STEP  363 H= 363:00 +CPU=126.410
  03:11:58 STEP  364 H= 364:00 +CPU= 84.173
  03:13:25 STEP  365 H= 365:00 +CPU= 80.844
  03:15:42 STEP  366 H= 366:00 +CPU=129.850
  03:17:11 STEP  367 H= 367:00 +CPU= 82.123
  03:18:40 STEP  368 H= 368:00 +CPU= 84.109
  03:20:46 STEP  369 H= 369:00 +CPU=121.776
  03:22:09 STEP  370 H= 370:00 +CPU= 80.591
  03:23:30 STEP  371 H= 371:00 +CPU= 78.736
  03:25:56 STEP  372 H= 372:00 +CPU=142.160
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000372
  03:27:20 STEP  373 H= 373:00 +CPU= 81.977
  03:28:45 STEP  374 H= 374:00 +CPU= 83.638
  03:30:53 STEP  375 H= 375:00 +CPU=124.558
  03:32:20 STEP  376 H= 376:00 +CPU= 84.150
  03:33:41 STEP  377 H= 377:00 +CPU= 79.426
  03:35:50 STEP  378 H= 378:00 +CPU=125.321
  03:37:11 STEP  379 H= 379:00 +CPU= 78.620
  03:38:33 STEP  380 H= 380:00 +CPU= 80.197
  03:40:37 STEP  381 H= 381:00 +CPU=120.928
  03:41:58 STEP  382 H= 382:00 +CPU= 79.666
  03:43:22 STEP  383 H= 383:00 +CPU= 81.719
  03:46:01 STEP  384 H= 384:00 +CPU=149.801
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_15.zip
Uploading the intermediate file: upload_file_15.zip
  03:47:26 STEP  385 H= 385:00 +CPU= 79.858
  03:48:53 STEP  386 H= 386:00 +CPU= 82.534
  03:51:05 STEP  387 H= 387:00 +CPU=125.461
  03:52:31 STEP  388 H= 388:00 +CPU= 81.382
  03:53:52 STEP  389 H= 389:00 +CPU= 75.673
  03:55:56 STEP  390 H= 390:00 +CPU=119.431
  03:57:17 STEP  391 H= 391:00 +CPU= 75.810
  03:58:40 STEP  392 H= 392:00 +CPU= 78.300
  04:00:48 STEP  393 H= 393:00 +CPU=119.160
  04:02:11 STEP  394 H= 394:00 +CPU= 79.082
  04:03:32 STEP  395 H= 395:00 +CPU= 77.441
  04:05:56 STEP  396 H= 396:00 +CPU=138.481
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000396
  04:07:17 STEP  397 H= 397:00 +CPU= 78.364
  04:08:42 STEP  398 H= 398:00 +CPU= 81.809
  04:10:49 STEP  399 H= 399:00 +CPU=121.529
  04:12:11 STEP  400 H= 400:00 +CPU= 79.827
  04:13:33 STEP  401 H= 401:00 +CPU= 79.247
  04:15:41 STEP  402 H= 402:00 +CPU=123.688
  04:16:58 STEP  403 H= 403:00 +CPU= 75.698
  04:18:21 STEP  404 H= 404:00 +CPU= 80.263
  04:20:24 STEP  405 H= 405:00 +CPU=120.258
  04:21:47 STEP  406 H= 406:00 +CPU= 80.907
  04:23:16 STEP  407 H= 407:00 +CPU= 83.558
  04:25:54 STEP  408 H= 408:00 +CPU=149.287
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_16.zip
Uploading the intermediate file: upload_file_16.zip
  04:27:22 STEP  409 H= 409:00 +CPU= 79.793
  04:28:50 STEP  410 H= 410:00 +CPU= 83.613
  04:30:59 STEP  411 H= 411:00 +CPU=124.219
  04:32:24 STEP  412 H= 412:00 +CPU= 81.446
  04:33:45 STEP  413 H= 413:00 +CPU= 78.216
  04:35:52 STEP  414 H= 414:00 +CPU=121.962
  04:37:14 STEP  415 H= 415:00 +CPU= 76.779
  04:38:36 STEP  416 H= 416:00 +CPU= 77.870
  04:40:38 STEP  417 H= 417:00 +CPU=116.688
  04:41:59 STEP  418 H= 418:00 +CPU= 78.292
  04:43:18 STEP  419 H= 419:00 +CPU= 75.945
  04:45:37 STEP  420 H= 420:00 +CPU=132.857
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000420
  04:46:58 STEP  421 H= 421:00 +CPU= 77.078
  04:48:20 STEP  422 H= 422:00 +CPU= 79.334
  04:50:21 STEP  423 H= 423:00 +CPU=116.611
  04:51:42 STEP  424 H= 424:00 +CPU= 77.965
  04:53:02 STEP  425 H= 425:00 +CPU= 76.857
  04:55:10 STEP  426 H= 426:00 +CPU=122.995
  04:56:32 STEP  427 H= 427:00 +CPU= 78.387
  04:57:56 STEP  428 H= 428:00 +CPU= 80.307
  05:00:06 STEP  429 H= 429:00 +CPU=122.673
  05:01:32 STEP  430 H= 430:00 +CPU= 81.441
  05:02:54 STEP  431 H= 431:00 +CPU= 78.325
  05:05:26 STEP  432 H= 432:00 +CPU=144.690
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_17.zip
Uploading the intermediate file: upload_file_17.zip
  05:06:53 STEP  433 H= 433:00 +CPU= 80.513
  05:08:18 STEP  434 H= 434:00 +CPU= 82.561
  05:10:26 STEP  435 H= 435:00 +CPU=123.281
  05:11:51 STEP  436 H= 436:00 +CPU= 82.397
  05:13:12 STEP  437 H= 437:00 +CPU= 79.093
  05:15:24 STEP  438 H= 438:00 +CPU=127.906
  05:16:49 STEP  439 H= 439:00 +CPU= 80.915
  05:18:14 STEP  440 H= 440:00 +CPU= 83.194
  05:20:29 STEP  441 H= 441:00 +CPU=125.584
  05:21:59 STEP  442 H= 442:00 +CPU= 83.553
  05:23:23 STEP  443 H= 443:00 +CPU= 80.779
  05:25:51 STEP  444 H= 444:00 +CPU=142.451
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000444
  05:27:17 STEP  445 H= 445:00 +CPU= 81.216
  05:28:44 STEP  446 H= 446:00 +CPU= 83.060
  05:30:54 STEP  447 H= 447:00 +CPU=125.091
  05:32:20 STEP  448 H= 448:00 +CPU= 81.670
  05:33:43 STEP  449 H= 449:00 +CPU= 79.538
  05:35:56 STEP  450 H= 450:00 +CPU=126.825
  05:37:18 STEP  451 H= 451:00 +CPU= 78.222
  05:38:42 STEP  452 H= 452:00 +CPU= 81.082
  05:40:52 STEP  453 H= 453:00 +CPU=124.399
  05:42:19 STEP  454 H= 454:00 +CPU= 84.097
  05:43:46 STEP  455 H= 455:00 +CPU= 83.758
  05:46:28 STEP  456 H= 456:00 +CPU=152.693
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_18.zip
Uploading the intermediate file: upload_file_18.zip
  05:47:55 STEP  457 H= 457:00 +CPU= 81.366
  05:49:24 STEP  458 H= 458:00 +CPU= 84.479
  05:51:39 STEP  459 H= 459:00 +CPU=127.476
  05:53:10 STEP  460 H= 460:00 +CPU= 86.531
  05:54:38 STEP  461 H= 461:00 +CPU= 83.849
  05:56:56 STEP  462 H= 462:00 +CPU=131.701
  05:58:21 STEP  463 H= 463:00 +CPU= 81.614
  05:59:50 STEP  464 H= 464:00 +CPU= 84.780
  06:02:13 STEP  465 H= 465:00 +CPU=127.678
  06:03:47 STEP  466 H= 466:00 +CPU= 86.217
  06:05:15 STEP  467 H= 467:00 +CPU= 84.052
  06:07:48 STEP  468 H= 468:00 +CPU=142.870
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000468
  06:09:16 STEP  469 H= 469:00 +CPU= 82.590
  06:10:49 STEP  470 H= 470:00 +CPU= 84.504
  06:13:04 STEP  471 H= 471:00 +CPU=127.820
  06:14:33 STEP  472 H= 472:00 +CPU= 84.876
  06:15:59 STEP  473 H= 473:00 +CPU= 82.499
  06:18:15 STEP  474 H= 474:00 +CPU=130.816
  06:19:42 STEP  475 H= 475:00 +CPU= 83.787
  06:21:11 STEP  476 H= 476:00 +CPU= 85.685
  06:23:21 STEP  477 H= 477:00 +CPU=127.290
  06:24:48 STEP  478 H= 478:00 +CPU= 84.773
  06:26:17 STEP  479 H= 479:00 +CPU= 86.023
  06:28:58 STEP  480 H= 480:00 +CPU=153.878
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_19.zip
Uploading the intermediate file: upload_file_19.zip
Uploading trickle at timestep: 1728000
  06:30:29 STEP  481 H= 481:00 +CPU= 82.996
  06:32:01 STEP  482 H= 482:00 +CPU= 86.218
  06:34:14 STEP  483 H= 483:00 +CPU=128.545
  06:35:43 STEP  484 H= 484:00 +CPU= 85.584
  06:37:09 STEP  485 H= 485:00 +CPU= 83.053
  06:39:30 STEP  486 H= 486:00 +CPU=132.812
  06:40:55 STEP  487 H= 487:00 +CPU= 82.847
  06:42:24 STEP  488 H= 488:00 +CPU= 84.657
  06:44:40 STEP  489 H= 489:00 +CPU=129.197
  06:46:13 STEP  490 H= 490:00 +CPU= 85.889
  06:47:40 STEP  491 H= 491:00 +CPU= 82.341
  06:50:16 STEP  492 H= 492:00 +CPU=145.808
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000492
  06:51:45 STEP  493 H= 493:00 +CPU= 82.913
  06:53:16 STEP  494 H= 494:00 +CPU= 85.520
  06:55:32 STEP  495 H= 495:00 +CPU=129.324
  06:57:01 STEP  496 H= 496:00 +CPU= 84.965
  06:58:26 STEP  497 H= 497:00 +CPU= 81.936
  07:00:40 STEP  498 H= 498:00 +CPU=129.988
  07:02:05 STEP  499 H= 499:00 +CPU= 82.176
  07:03:34 STEP  500 H= 500:00 +CPU= 85.994
  07:05:47 STEP  501 H= 501:00 +CPU=127.831
  07:07:15 STEP  502 H= 502:00 +CPU= 85.238
  07:08:44 STEP  503 H= 503:00 +CPU= 85.809
  07:11:25 STEP  504 H= 504:00 +CPU=153.513
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_20.zip
Uploading the intermediate file: upload_file_20.zip
  07:12:58 STEP  505 H= 505:00 +CPU= 82.006
  07:14:26 STEP  506 H= 506:00 +CPU= 84.454
  07:16:41 STEP  507 H= 507:00 +CPU=127.740
  07:18:14 STEP  508 H= 508:00 +CPU= 86.405
  07:19:42 STEP  509 H= 509:00 +CPU= 83.568
  07:21:58 STEP  510 H= 510:00 +CPU=132.794
  07:23:26 STEP  511 H= 511:00 +CPU= 83.273
  07:24:55 STEP  512 H= 512:00 +CPU= 85.280
  07:27:14 STEP  513 H= 513:00 +CPU=129.939
  07:28:48 STEP  514 H= 514:00 +CPU= 86.922
  07:30:17 STEP  515 H= 515:00 +CPU= 83.041
  07:32:47 STEP  516 H= 516:00 +CPU=144.756
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000516
  07:34:13 STEP  517 H= 517:00 +CPU= 82.205
  07:35:44 STEP  518 H= 518:00 +CPU= 86.367
  07:38:06 STEP  519 H= 519:00 +CPU=127.222
  07:39:36 STEP  520 H= 520:00 +CPU= 86.071
  07:41:04 STEP  521 H= 521:00 +CPU= 84.263
  07:43:31 STEP  522 H= 522:00 +CPU=134.111
  07:44:59 STEP  523 H= 523:00 +CPU= 84.335
  07:46:30 STEP  524 H= 524:00 +CPU= 87.142
  07:48:45 STEP  525 H= 525:00 +CPU=128.292
  07:50:13 STEP  526 H= 526:00 +CPU= 84.712
  07:51:43 STEP  527 H= 527:00 +CPU= 84.633
  07:54:31 STEP  528 H= 528:00 +CPU=155.892
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_21.zip
Uploading the intermediate file: upload_file_21.zip
  07:56:02 STEP  529 H= 529:00 +CPU= 83.486
  07:57:32 STEP  530 H= 530:00 +CPU= 86.310
  07:59:48 STEP  531 H= 531:00 +CPU=128.824
  08:01:18 STEP  532 H= 532:00 +CPU= 85.067
  08:02:49 STEP  533 H= 533:00 +CPU= 83.152
  08:05:12 STEP  534 H= 534:00 +CPU=131.189
  08:06:40 STEP  535 H= 535:00 +CPU= 83.344
  08:08:11 STEP  536 H= 536:00 +CPU= 85.396
  08:10:29 STEP  537 H= 537:00 +CPU=129.632
  08:12:02 STEP  538 H= 538:00 +CPU= 86.272
  08:13:31 STEP  539 H= 539:00 +CPU= 84.584
  08:16:05 STEP  540 H= 540:00 +CPU=144.146
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000540
  08:17:34 STEP  541 H= 541:00 +CPU= 82.782
  08:19:02 STEP  542 H= 542:00 +CPU= 84.686
  08:21:13 STEP  543 H= 543:00 +CPU=127.069
  08:22:43 STEP  544 H= 544:00 +CPU= 85.438
  08:24:11 STEP  545 H= 545:00 +CPU= 82.671
  08:26:32 STEP  546 H= 546:00 +CPU=133.096
  08:27:56 STEP  547 H= 547:00 +CPU= 81.314
  08:29:25 STEP  548 H= 548:00 +CPU= 84.446
  08:31:34 STEP  549 H= 549:00 +CPU=123.784
  08:32:57 STEP  550 H= 550:00 +CPU= 78.691
  08:34:18 STEP  551 H= 551:00 +CPU= 78.993
  08:36:50 STEP  552 H= 552:00 +CPU=144.084
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_22.zip
Uploading the intermediate file: upload_file_22.zip
  08:38:14 STEP  553 H= 553:00 +CPU= 77.274
  08:39:38 STEP  554 H= 554:00 +CPU= 78.981
  08:41:43 STEP  555 H= 555:00 +CPU=119.302
  08:43:09 STEP  556 H= 556:00 +CPU= 81.465
  08:44:29 STEP  557 H= 557:00 +CPU= 76.965
  08:46:36 STEP  558 H= 558:00 +CPU=122.134
  08:47:58 STEP  559 H= 559:00 +CPU= 78.867
  08:49:19 STEP  560 H= 560:00 +CPU= 79.789
  08:51:27 STEP  561 H= 561:00 +CPU=121.637
  08:52:53 STEP  562 H= 562:00 +CPU= 81.648
  08:54:18 STEP  563 H= 563:00 +CPU= 79.204
  08:56:45 STEP  564 H= 564:00 +CPU=140.353
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000564
  08:58:07 STEP  565 H= 565:00 +CPU= 79.037
  08:59:35 STEP  566 H= 566:00 +CPU= 83.455
  09:01:46 STEP  567 H= 567:00 +CPU=126.650
  09:03:12 STEP  568 H= 568:00 +CPU= 82.520
  09:04:36 STEP  569 H= 569:00 +CPU= 80.431
  09:06:52 STEP  570 H= 570:00 +CPU=130.450
  09:08:17 STEP  571 H= 571:00 +CPU= 81.327
  09:09:43 STEP  572 H= 572:00 +CPU= 83.051
  09:11:57 STEP  573 H= 573:00 +CPU=127.759
  09:13:23 STEP  574 H= 574:00 +CPU= 83.021
  09:14:50 STEP  575 H= 575:00 +CPU= 83.885
  09:17:34 STEP  576 H= 576:00 +CPU=155.599
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279_23.zip
Uploading the intermediate file: upload_file_23.zip
  09:19:01 STEP  577 H= 577:00 +CPU= 81.108
  09:20:32 STEP  578 H= 578:00 +CPU= 85.379
  09:22:56 STEP  579 H= 579:00 +CPU=128.305
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.235] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3992MB (heap), 4284MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.271] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.272] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.272] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.272] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.272] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3992 MB (maxheap), 4284 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.272] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.274] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.275] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.275] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062513:1672064713:133922.276] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                CUANCAPE2 
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062514:1672064714:133923.277] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:workstation-linux-vm-1:1:1:43143:43143] [20221226:062514:1672064714:133923.277] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001247F50  cuancape2_                158  cuancape2.F90
oifs_43r3_model.e  0000000000F14871  callpar_                 1102  callpar.F90
oifs_43r3_model.e  0000000000E4940F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000E3610C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BFA3D3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1990050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0909
(argv4) batchid: 959
(argv5) wuid: 12176553
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2a662c3165dc1d5d3430ddf108342ad1 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0909_1990050100_123_959_12176553.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0909_1990050100_123_959_12176553.zip
ic_ancil_file: ic_ancil_12176553
ifsdata_file: ifsdata_12176553
climate_data_file: clim_data_12176553
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_8a4e40bab53bac03d3d7c5b7afbcff22 to: /var/lib/boinc-client/slots/1/ic_ancil_12176553.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12176553.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_65647bf993e059a40f36f0067acdb722 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12176553.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12176553.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12176553.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12176553.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12176553.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12176553.xml
-- Model is restarting --
Adjusting last_iter, 580, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12176553.xml
last_cpu_time: 55375
upload_file_number: 24
last_iter: 575
last_upload: 2073600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0909_1990050100_123_959_12176553_0_r474106279
The child process has been launched with process id: 10184
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:workstation-linux-vm-1:1:1:10184:10184] [20221229:001015:1672301415:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  00:12:01 STEP  576 H= 576:00 +CPU= 90.851
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:55:04 1529561 22261310 oifs_43r3_ps_0909_1990050100_123_959_12176553_0 1,728,000 45,442 0.0263
09 May 2023 15:54:24 1529561 22261310 oifs_43r3_ps_0909_1990050100_123_959_12176553_0 864,000 22,436 0.0260


©2024 climateprediction.net