climateprediction.net home page
Task 22262405

Task 22262405

Name oifs_43r3_ps_0187_1990050100_123_959_12175831_1
Workunit 12175831
Created 21 Dec 2022, 23:08:53 UTC
Sent 20 Jan 2023, 14:59:10 UTC
Report deadline 21 Mar 2023, 14:59:10 UTC
Received 21 Jan 2023, 14:28:23 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 9 (0x00000009) Unknown error code
Computer ID 1466689
Run time 16 hours 58 min 56 sec
CPU time 16 hours 40 min 35 sec
Validate state Invalid
Credit 7,862.40
Device peak FLOPS 4.64 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,408.58 MB
Peak swap size 4,978.08 MB
Peak disk usage 3,637.25 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
process exited with code 9 (0x9, -247)</message>
<stderr_txt>
-client/slots/0/ICMUAhq0f+001188
  02:30:40 STEP 1189 H=1189:00 +CPU= 35.906
  02:31:17 STEP 1190 H=1190:00 +CPU= 36.628
  02:32:12 STEP 1191 H=1191:00 +CPU= 54.254
  02:32:50 STEP 1192 H=1192:00 +CPU= 37.006
  02:33:24 STEP 1193 H=1193:00 +CPU= 33.658
  02:34:22 STEP 1194 H=1194:00 +CPU= 57.816
  02:34:55 STEP 1195 H=1195:00 +CPU= 32.403
  02:35:34 STEP 1196 H=1196:00 +CPU= 38.386
  02:36:29 STEP 1197 H=1197:00 +CPU= 53.633
  02:37:06 STEP 1198 H=1198:00 +CPU= 36.946
  02:37:43 STEP 1199 H=1199:00 +CPU= 36.723
  02:38:51 STEP 1200 H=1200:00 +CPU= 65.509
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001188
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_49.zip
Uploading the intermediate file: upload_file_49.zip
  02:39:27 STEP 1201 H=1201:00 +CPU= 35.284
  02:40:05 STEP 1202 H=1202:00 +CPU= 36.490
  02:41:01 STEP 1203 H=1203:00 +CPU= 55.320
  02:41:38 STEP 1204 H=1204:00 +CPU= 36.319
  02:42:16 STEP 1205 H=1205:00 +CPU= 37.505
  02:43:12 STEP 1206 H=1206:00 +CPU= 54.960
  02:43:50 STEP 1207 H=1207:00 +CPU= 37.330
  02:44:26 STEP 1208 H=1208:00 +CPU= 34.917
  02:45:23 STEP 1209 H=1209:00 +CPU= 56.218
  02:45:59 STEP 1210 H=1210:00 +CPU= 35.193
  02:46:36 STEP 1211 H=1211:00 +CPU= 36.033
  02:47:37 STEP 1212 H=1212:00 +CPU= 60.315
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001212
  02:48:13 STEP 1213 H=1213:00 +CPU= 36.158
  02:48:51 STEP 1214 H=1214:00 +CPU= 36.706
  02:49:46 STEP 1215 H=1215:00 +CPU= 54.662
  02:50:24 STEP 1216 H=1216:00 +CPU= 36.799
  02:50:58 STEP 1217 H=1217:00 +CPU= 34.085
  02:51:57 STEP 1218 H=1218:00 +CPU= 57.310
  02:52:30 STEP 1219 H=1219:00 +CPU= 33.113
  02:53:09 STEP 1220 H=1220:00 +CPU= 38.137
  02:54:03 STEP 1221 H=1221:00 +CPU= 53.575
  02:54:42 STEP 1222 H=1222:00 +CPU= 37.595
  02:55:19 STEP 1223 H=1223:00 +CPU= 36.433
  02:56:26 STEP 1224 H=1224:00 +CPU= 65.588
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001212
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_50.zip
Uploading the intermediate file: upload_file_50.zip
  02:57:02 STEP 1225 H=1225:00 +CPU= 35.137
  02:57:39 STEP 1226 H=1226:00 +CPU= 36.290
  02:58:36 STEP 1227 H=1227:00 +CPU= 55.562
  02:59:12 STEP 1228 H=1228:00 +CPU= 36.240
  02:59:50 STEP 1229 H=1229:00 +CPU= 37.431
  03:00:46 STEP 1230 H=1230:00 +CPU= 54.718
  03:01:24 STEP 1231 H=1231:00 +CPU= 37.080
  03:01:59 STEP 1232 H=1232:00 +CPU= 35.293
  03:02:56 STEP 1233 H=1233:00 +CPU= 55.894
  03:03:32 STEP 1234 H=1234:00 +CPU= 35.793
  03:04:08 STEP 1235 H=1235:00 +CPU= 35.474
  03:05:10 STEP 1236 H=1236:00 +CPU= 60.876
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001236
  03:05:46 STEP 1237 H=1237:00 +CPU= 35.700
  03:06:24 STEP 1238 H=1238:00 +CPU= 36.692
  03:07:19 STEP 1239 H=1239:00 +CPU= 54.579
  03:07:57 STEP 1240 H=1240:00 +CPU= 36.823
  03:08:31 STEP 1241 H=1241:00 +CPU= 33.976
  03:09:30 STEP 1242 H=1242:00 +CPU= 57.653
  03:10:03 STEP 1243 H=1243:00 +CPU= 32.854
  03:10:43 STEP 1244 H=1244:00 +CPU= 38.082
  03:11:37 STEP 1245 H=1245:00 +CPU= 54.127
  03:12:15 STEP 1246 H=1246:00 +CPU= 37.010
  03:12:53 STEP 1247 H=1247:00 +CPU= 36.660
  03:14:00 STEP 1248 H=1248:00 +CPU= 65.746
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_51.zip
Uploading the intermediate file: upload_file_51.zip
  03:14:36 STEP 1249 H=1249:00 +CPU= 35.265
  03:15:13 STEP 1250 H=1250:00 +CPU= 36.572
  03:16:10 STEP 1251 H=1251:00 +CPU= 55.794
  03:16:46 STEP 1252 H=1252:00 +CPU= 36.415
  03:17:25 STEP 1253 H=1253:00 +CPU= 37.523
  03:18:20 STEP 1254 H=1254:00 +CPU= 54.684
  03:18:58 STEP 1255 H=1255:00 +CPU= 37.083
  03:19:33 STEP 1256 H=1256:00 +CPU= 34.982
  03:20:30 STEP 1257 H=1257:00 +CPU= 56.034
  03:21:06 STEP 1258 H=1258:00 +CPU= 35.710
  03:21:42 STEP 1259 H=1259:00 +CPU= 35.575
  03:22:44 STEP 1260 H=1260:00 +CPU= 60.797
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001260
  03:23:21 STEP 1261 H=1261:00 +CPU= 35.938
  03:23:57 STEP 1262 H=1262:00 +CPU= 36.391
  03:24:53 STEP 1263 H=1263:00 +CPU= 54.973
  03:25:31 STEP 1264 H=1264:00 +CPU= 36.729
  03:26:05 STEP 1265 H=1265:00 +CPU= 34.213
  03:27:03 STEP 1266 H=1266:00 +CPU= 57.371
  03:27:37 STEP 1267 H=1267:00 +CPU= 33.204
  03:28:16 STEP 1268 H=1268:00 +CPU= 38.266
  03:29:11 STEP 1269 H=1269:00 +CPU= 53.870
  03:29:49 STEP 1270 H=1270:00 +CPU= 37.452
  03:30:26 STEP 1271 H=1271:00 +CPU= 36.676
  03:31:33 STEP 1272 H=1272:00 +CPU= 65.597
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001260
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_52.zip
Uploading the intermediate file: upload_file_52.zip
  03:32:09 STEP 1273 H=1273:00 +CPU= 34.772
  03:32:46 STEP 1274 H=1274:00 +CPU= 36.573
  03:33:42 STEP 1275 H=1275:00 +CPU= 55.865
  03:34:19 STEP 1276 H=1276:00 +CPU= 36.318
  03:34:57 STEP 1277 H=1277:00 +CPU= 37.245
  03:35:53 STEP 1278 H=1278:00 +CPU= 54.813
  03:36:30 STEP 1279 H=1279:00 +CPU= 37.257
  03:37:06 STEP 1280 H=1280:00 +CPU= 35.119
  03:38:02 STEP 1281 H=1281:00 +CPU= 55.875
  03:38:39 STEP 1282 H=1282:00 +CPU= 35.877
  03:39:15 STEP 1283 H=1283:00 +CPU= 35.286
  03:40:17 STEP 1284 H=1284:00 +CPU= 61.016
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001284
  03:40:53 STEP 1285 H=1285:00 +CPU= 35.648
  03:41:30 STEP 1286 H=1286:00 +CPU= 36.499
  03:42:26 STEP 1287 H=1287:00 +CPU= 54.768
  03:43:03 STEP 1288 H=1288:00 +CPU= 36.797
  03:43:37 STEP 1289 H=1289:00 +CPU= 33.748
  03:44:36 STEP 1290 H=1290:00 +CPU= 57.846
  03:45:10 STEP 1291 H=1291:00 +CPU= 33.141
  03:45:49 STEP 1292 H=1292:00 +CPU= 37.682
  03:46:44 STEP 1293 H=1293:00 +CPU= 54.317
  03:47:21 STEP 1294 H=1294:00 +CPU= 36.824
  03:47:59 STEP 1295 H=1295:00 +CPU= 36.748
  03:49:06 STEP 1296 H=1296:00 +CPU= 66.158
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001284
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_53.zip
Uploading the intermediate file: upload_file_53.zip
  03:49:43 STEP 1297 H=1297:00 +CPU= 35.462
  03:50:20 STEP 1298 H=1298:00 +CPU= 36.650
  03:51:17 STEP 1299 H=1299:00 +CPU= 56.124
  03:51:54 STEP 1300 H=1300:00 +CPU= 36.402
  03:52:32 STEP 1301 H=1301:00 +CPU= 37.489
  03:53:27 STEP 1302 H=1302:00 +CPU= 54.563
  03:54:05 STEP 1303 H=1303:00 +CPU= 37.084
  03:54:41 STEP 1304 H=1304:00 +CPU= 35.289
  03:55:38 STEP 1305 H=1305:00 +CPU= 56.006
  03:56:14 STEP 1306 H=1306:00 +CPU= 35.835
  03:56:51 STEP 1307 H=1307:00 +CPU= 35.664
  03:57:53 STEP 1308 H=1308:00 +CPU= 60.755
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001308
  03:58:29 STEP 1309 H=1309:00 +CPU= 35.937
  03:59:06 STEP 1310 H=1310:00 +CPU= 36.390
  04:00:03 STEP 1311 H=1311:00 +CPU= 55.814
  04:00:40 STEP 1312 H=1312:00 +CPU= 36.173
  04:01:15 STEP 1313 H=1313:00 +CPU= 34.865
  04:02:13 STEP 1314 H=1314:00 +CPU= 56.732
  04:02:47 STEP 1315 H=1315:00 +CPU= 33.820
  04:03:26 STEP 1316 H=1316:00 +CPU= 38.383
  04:04:21 STEP 1317 H=1317:00 +CPU= 53.881
  04:05:00 STEP 1318 H=1318:00 +CPU= 38.392
  04:05:37 STEP 1319 H=1319:00 +CPU= 37.054
  04:06:43 STEP 1320 H=1320:00 +CPU= 64.916
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001308
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_54.zip
Uploading the intermediate file: upload_file_54.zip
Uploading trickle at timestep: 4752000
  04:07:20 STEP 1321 H=1321:00 +CPU= 35.115
  04:07:56 STEP 1322 H=1322:00 +CPU= 36.209
  04:08:53 STEP 1323 H=1323:00 +CPU= 56.007
  04:09:30 STEP 1324 H=1324:00 +CPU= 36.214
  04:10:07 STEP 1325 H=1325:00 +CPU= 36.404
  04:11:03 STEP 1326 H=1326:00 +CPU= 54.666
  04:11:40 STEP 1327 H=1327:00 +CPU= 36.743
  04:12:15 STEP 1328 H=1328:00 +CPU= 34.365
  04:13:12 STEP 1329 H=1329:00 +CPU= 56.764
  04:13:47 STEP 1330 H=1330:00 +CPU= 34.428
  04:14:24 STEP 1331 H=1331:00 +CPU= 36.465
  04:15:26 STEP 1332 H=1332:00 +CPU= 60.211
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001332
  04:16:03 STEP 1333 H=1333:00 +CPU= 36.816
  04:16:40 STEP 1334 H=1334:00 +CPU= 36.632
  04:17:37 STEP 1335 H=1335:00 +CPU= 56.134
  04:18:14 STEP 1336 H=1336:00 +CPU= 36.137
  04:18:50 STEP 1337 H=1337:00 +CPU= 35.674
  04:19:47 STEP 1338 H=1338:00 +CPU= 56.079
  04:20:23 STEP 1339 H=1339:00 +CPU= 35.471
  04:21:02 STEP 1340 H=1340:00 +CPU= 38.134
  04:21:58 STEP 1341 H=1341:00 +CPU= 54.463
  04:22:36 STEP 1342 H=1342:00 +CPU= 38.029
  04:23:14 STEP 1343 H=1343:00 +CPU= 37.208
  04:24:19 STEP 1344 H=1344:00 +CPU= 64.398
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_55.zip
Uploading the intermediate file: upload_file_55.zip
  04:24:55 STEP 1345 H=1345:00 +CPU= 34.323
  04:25:32 STEP 1346 H=1346:00 +CPU= 36.476
  04:26:28 STEP 1347 H=1347:00 +CPU= 55.236
  04:27:04 STEP 1348 H=1348:00 +CPU= 35.943
  04:27:41 STEP 1349 H=1349:00 +CPU= 36.196
  04:28:36 STEP 1350 H=1350:00 +CPU= 54.442
  04:29:13 STEP 1351 H=1351:00 +CPU= 36.699
  04:29:48 STEP 1352 H=1352:00 +CPU= 34.555
  04:30:46 STEP 1353 H=1353:00 +CPU= 56.774
  04:31:21 STEP 1354 H=1354:00 +CPU= 34.488
  04:31:58 STEP 1355 H=1355:00 +CPU= 36.418
  04:32:59 STEP 1356 H=1356:00 +CPU= 60.197
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001356
  04:33:36 STEP 1357 H=1357:00 +CPU= 36.906
  04:34:13 STEP 1358 H=1358:00 +CPU= 36.378
  04:35:11 STEP 1359 H=1359:00 +CPU= 56.129
  04:35:47 STEP 1360 H=1360:00 +CPU= 36.310
  04:36:24 STEP 1361 H=1361:00 +CPU= 35.777
  04:37:21 STEP 1362 H=1362:00 +CPU= 56.363
  04:37:57 STEP 1363 H=1363:00 +CPU= 35.459
  04:38:36 STEP 1364 H=1364:00 +CPU= 38.096
  04:39:32 STEP 1365 H=1365:00 +CPU= 54.751
  04:40:11 STEP 1366 H=1366:00 +CPU= 38.250
  04:40:48 STEP 1367 H=1367:00 +CPU= 36.694
  04:41:54 STEP 1368 H=1368:00 +CPU= 64.773
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001356
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_56.zip
Uploading the intermediate file: upload_file_56.zip
  04:42:28 STEP 1369 H=1369:00 +CPU= 33.564
  04:43:05 STEP 1370 H=1370:00 +CPU= 36.817
  04:44:01 STEP 1371 H=1371:00 +CPU= 54.823
  04:44:38 STEP 1372 H=1372:00 +CPU= 36.292
  04:45:14 STEP 1373 H=1373:00 +CPU= 35.600
  04:46:10 STEP 1374 H=1374:00 +CPU= 55.443
  04:46:46 STEP 1375 H=1375:00 +CPU= 35.875
  04:47:21 STEP 1376 H=1376:00 +CPU= 34.817
  04:48:19 STEP 1377 H=1377:00 +CPU= 56.744
  04:48:54 STEP 1378 H=1378:00 +CPU= 34.247
  04:49:31 STEP 1379 H=1379:00 +CPU= 36.496
  04:50:31 STEP 1380 H=1380:00 +CPU= 60.044
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001380
  04:51:09 STEP 1381 H=1381:00 +CPU= 36.942
  04:51:46 STEP 1382 H=1382:00 +CPU= 36.334
  04:52:42 STEP 1383 H=1383:00 +CPU= 55.949
  04:53:19 STEP 1384 H=1384:00 +CPU= 36.094
  04:53:55 STEP 1385 H=1385:00 +CPU= 35.740
  04:54:52 STEP 1386 H=1386:00 +CPU= 56.428
  04:55:28 STEP 1387 H=1387:00 +CPU= 35.494
  04:56:08 STEP 1388 H=1388:00 +CPU= 38.258
  04:57:03 STEP 1389 H=1389:00 +CPU= 54.444
  04:57:42 STEP 1390 H=1390:00 +CPU= 38.430
  04:58:19 STEP 1391 H=1391:00 +CPU= 36.849
  04:59:25 STEP 1392 H=1392:00 +CPU= 64.498
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001380
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_57.zip
Uploading the intermediate file: upload_file_57.zip
  04:59:59 STEP 1393 H=1393:00 +CPU= 33.532
  05:00:36 STEP 1394 H=1394:00 +CPU= 36.829
  05:01:32 STEP 1395 H=1395:00 +CPU= 54.743
  05:02:09 STEP 1396 H=1396:00 +CPU= 36.396
  05:02:45 STEP 1397 H=1397:00 +CPU= 35.638
  05:03:42 STEP 1398 H=1398:00 +CPU= 55.578
  05:04:18 STEP 1399 H=1399:00 +CPU= 35.806
  05:04:53 STEP 1400 H=1400:00 +CPU= 35.018
  05:05:51 STEP 1401 H=1401:00 +CPU= 56.795
  05:06:25 STEP 1402 H=1402:00 +CPU= 34.026
  05:07:03 STEP 1403 H=1403:00 +CPU= 37.305
  05:08:04 STEP 1404 H=1404:00 +CPU= 60.266
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001404
  05:08:42 STEP 1405 H=1405:00 +CPU= 37.497
  05:09:20 STEP 1406 H=1406:00 +CPU= 36.761
  05:10:16 STEP 1407 H=1407:00 +CPU= 55.660
  05:10:53 STEP 1408 H=1408:00 +CPU= 36.611
  05:11:29 STEP 1409 H=1409:00 +CPU= 35.485
  05:12:27 STEP 1410 H=1410:00 +CPU= 56.430
  05:13:02 STEP 1411 H=1411:00 +CPU= 35.379
  05:13:42 STEP 1412 H=1412:00 +CPU= 38.354
  05:14:36 STEP 1413 H=1413:00 +CPU= 53.766
  05:15:15 STEP 1414 H=1414:00 +CPU= 38.223
  05:15:51 STEP 1415 H=1415:00 +CPU= 35.662
  05:16:58 STEP 1416 H=1416:00 +CPU= 65.235
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_58.zip
Uploading the intermediate file: upload_file_58.zip
  05:17:32 STEP 1417 H=1417:00 +CPU= 33.320
  05:18:11 STEP 1418 H=1418:00 +CPU= 38.020
  05:19:05 STEP 1419 H=1419:00 +CPU= 53.682
  05:19:43 STEP 1420 H=1420:00 +CPU= 37.712
  05:20:19 STEP 1421 H=1421:00 +CPU= 35.242
  05:21:18 STEP 1422 H=1422:00 +CPU= 57.229
  05:21:53 STEP 1423 H=1423:00 +CPU= 34.905
  05:22:30 STEP 1424 H=1424:00 +CPU= 36.683
  05:23:26 STEP 1425 H=1425:00 +CPU= 55.395
  05:24:03 STEP 1426 H=1426:00 +CPU= 36.092
  05:24:41 STEP 1427 H=1427:00 +CPU= 37.452
  05:25:42 STEP 1428 H=1428:00 +CPU= 60.286
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001428
  05:26:20 STEP 1429 H=1429:00 +CPU= 37.435
  05:26:57 STEP 1430 H=1430:00 +CPU= 36.234
  05:27:53 STEP 1431 H=1431:00 +CPU= 55.201
  05:28:30 STEP 1432 H=1432:00 +CPU= 36.409
  05:29:06 STEP 1433 H=1433:00 +CPU= 35.164
  05:30:03 STEP 1434 H=1434:00 +CPU= 56.608
  05:30:39 STEP 1435 H=1435:00 +CPU= 35.093
  05:31:17 STEP 1436 H=1436:00 +CPU= 37.836
  05:32:12 STEP 1437 H=1437:00 +CPU= 53.767
  05:32:51 STEP 1438 H=1438:00 +CPU= 38.156
  05:33:26 STEP 1439 H=1439:00 +CPU= 34.577
  05:34:32 STEP 1440 H=1440:00 +CPU= 64.880
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_59.zip
Uploading the intermediate file: upload_file_59.zip
  05:35:06 STEP 1441 H=1441:00 +CPU= 33.662
  05:35:45 STEP 1442 H=1442:00 +CPU= 38.114
  05:36:40 STEP 1443 H=1443:00 +CPU= 53.760
  05:37:18 STEP 1444 H=1444:00 +CPU= 37.452
  05:37:54 STEP 1445 H=1445:00 +CPU= 35.336
  05:38:52 STEP 1446 H=1446:00 +CPU= 57.314
  05:39:28 STEP 1447 H=1447:00 +CPU= 35.030
  05:40:05 STEP 1448 H=1448:00 +CPU= 36.829
  05:41:00 STEP 1449 H=1449:00 +CPU= 54.931
  05:41:38 STEP 1450 H=1450:00 +CPU= 36.501
  05:42:16 STEP 1451 H=1451:00 +CPU= 37.499
  05:43:17 STEP 1452 H=1452:00 +CPU= 60.168
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001452
  05:43:55 STEP 1453 H=1453:00 +CPU= 37.554
  05:44:31 STEP 1454 H=1454:00 +CPU= 35.834
  05:45:27 STEP 1455 H=1455:00 +CPU= 55.340
  05:46:04 STEP 1456 H=1456:00 +CPU= 36.363
  05:46:40 STEP 1457 H=1457:00 +CPU= 35.049
  05:47:37 STEP 1458 H=1458:00 +CPU= 56.615
  05:48:13 STEP 1459 H=1459:00 +CPU= 35.062
  05:48:51 STEP 1460 H=1460:00 +CPU= 36.934
  05:49:46 STEP 1461 H=1461:00 +CPU= 54.289
  05:50:25 STEP 1462 H=1462:00 +CPU= 37.828
  05:51:00 STEP 1463 H=1463:00 +CPU= 35.033
  05:52:07 STEP 1464 H=1464:00 +CPU= 64.719
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001452
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_60.zip
Uploading the intermediate file: upload_file_60.zip
  05:52:41 STEP 1465 H=1465:00 +CPU= 34.047
  05:53:20 STEP 1466 H=1466:00 +CPU= 38.069
  05:54:15 STEP 1467 H=1467:00 +CPU= 53.884
  05:54:53 STEP 1468 H=1468:00 +CPU= 37.770
  05:55:29 STEP 1469 H=1469:00 +CPU= 35.506
  05:56:28 STEP 1470 H=1470:00 +CPU= 57.431
  05:57:03 STEP 1471 H=1471:00 +CPU= 35.392
  05:57:41 STEP 1472 H=1472:00 +CPU= 36.794
  05:58:36 STEP 1473 H=1473:00 +CPU= 54.899
  05:59:14 STEP 1474 H=1474:00 +CPU= 36.849
  05:59:52 STEP 1475 H=1475:00 +CPU= 37.549
  06:00:52 STEP 1476 H=1476:00 +CPU= 59.702
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001476
  06:01:30 STEP 1477 H=1477:00 +CPU= 37.350
  06:02:07 STEP 1478 H=1478:00 +CPU= 35.611
  06:03:02 STEP 1479 H=1479:00 +CPU= 55.096
  06:03:39 STEP 1480 H=1480:00 +CPU= 36.369
  06:04:15 STEP 1481 H=1481:00 +CPU= 35.074
  06:05:13 STEP 1482 H=1482:00 +CPU= 56.610
  06:05:48 STEP 1483 H=1483:00 +CPU= 35.115
  06:06:26 STEP 1484 H=1484:00 +CPU= 37.017
  06:07:21 STEP 1485 H=1485:00 +CPU= 54.578
  06:08:00 STEP 1486 H=1486:00 +CPU= 37.574
  06:08:35 STEP 1487 H=1487:00 +CPU= 35.281
  06:09:41 STEP 1488 H=1488:00 +CPU= 64.826
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001476
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_61.zip
Uploading the intermediate file: upload_file_61.zip
  06:10:17 STEP 1489 H=1489:00 +CPU= 34.854
  06:10:56 STEP 1490 H=1490:00 +CPU= 38.250
  06:11:50 STEP 1491 H=1491:00 +CPU= 53.727
  06:12:29 STEP 1492 H=1492:00 +CPU= 38.330
  06:13:05 STEP 1493 H=1493:00 +CPU= 35.729
  06:14:03 STEP 1494 H=1494:00 +CPU= 57.406
  06:14:39 STEP 1495 H=1495:00 +CPU= 35.755
  06:15:17 STEP 1496 H=1496:00 +CPU= 36.699
  06:16:13 STEP 1497 H=1497:00 +CPU= 55.307
  06:16:50 STEP 1498 H=1498:00 +CPU= 36.792
  06:17:28 STEP 1499 H=1499:00 +CPU= 37.547
  06:18:28 STEP 1500 H=1500:00 +CPU= 59.144
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001500
  06:19:05 STEP 1501 H=1501:00 +CPU= 37.385
  06:19:41 STEP 1502 H=1502:00 +CPU= 35.190
  06:20:38 STEP 1503 H=1503:00 +CPU= 55.479
  06:21:14 STEP 1504 H=1504:00 +CPU= 35.774
  06:21:50 STEP 1505 H=1505:00 +CPU= 35.289
  06:22:47 STEP 1506 H=1506:00 +CPU= 56.744
  06:23:23 STEP 1507 H=1507:00 +CPU= 34.944
  06:24:00 STEP 1508 H=1508:00 +CPU= 36.486
  06:24:56 STEP 1509 H=1509:00 +CPU= 55.057
  06:25:33 STEP 1510 H=1510:00 +CPU= 36.633
  06:26:09 STEP 1511 H=1511:00 +CPU= 35.164
  06:27:15 STEP 1512 H=1512:00 +CPU= 65.111
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001500
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_62.zip
Uploading the intermediate file: upload_file_62.zip
  06:27:51 STEP 1513 H=1513:00 +CPU= 35.304
  06:28:30 STEP 1514 H=1514:00 +CPU= 38.378
  06:29:25 STEP 1515 H=1515:00 +CPU= 54.009
  06:30:04 STEP 1516 H=1516:00 +CPU= 38.325
  06:30:40 STEP 1517 H=1517:00 +CPU= 35.767
  06:31:38 STEP 1518 H=1518:00 +CPU= 56.781
  06:32:15 STEP 1519 H=1519:00 +CPU= 35.726
  06:32:52 STEP 1520 H=1520:00 +CPU= 36.590
  06:33:48 STEP 1521 H=1521:00 +CPU= 56.010
  06:34:25 STEP 1522 H=1522:00 +CPU= 36.345
  06:35:03 STEP 1523 H=1523:00 +CPU= 36.766
  06:36:03 STEP 1524 H=1524:00 +CPU= 59.593
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001524
  06:36:40 STEP 1525 H=1525:00 +CPU= 36.110
  06:37:15 STEP 1526 H=1526:00 +CPU= 34.832
  06:38:12 STEP 1527 H=1527:00 +CPU= 56.308
  06:38:48 STEP 1528 H=1528:00 +CPU= 34.732
  06:39:24 STEP 1529 H=1529:00 +CPU= 36.309
  06:40:21 STEP 1530 H=1530:00 +CPU= 55.563
  06:40:57 STEP 1531 H=1531:00 +CPU= 35.798
  06:41:34 STEP 1532 H=1532:00 +CPU= 35.845
  06:42:30 STEP 1533 H=1533:00 +CPU= 55.881
  06:43:07 STEP 1534 H=1534:00 +CPU= 36.183
  06:43:44 STEP 1535 H=1535:00 +CPU= 36.575
  06:44:51 STEP 1536 H=1536:00 +CPU= 65.791
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_63.zip
Uploading the intermediate file: upload_file_63.zip
  06:45:27 STEP 1537 H=1537:00 +CPU= 35.443
  06:46:06 STEP 1538 H=1538:00 +CPU= 38.645
  06:47:00 STEP 1539 H=1539:00 +CPU= 53.146
  06:47:39 STEP 1540 H=1540:00 +CPU= 38.376
  06:48:15 STEP 1541 H=1541:00 +CPU= 35.345
  06:49:12 STEP 1542 H=1542:00 +CPU= 56.564
  06:49:48 STEP 1543 H=1543:00 +CPU= 35.348
  06:50:26 STEP 1544 H=1544:00 +CPU= 36.451
  06:51:23 STEP 1545 H=1545:00 +CPU= 55.911
  06:52:00 STEP 1546 H=1546:00 +CPU= 36.380
  06:52:37 STEP 1547 H=1547:00 +CPU= 36.649
  06:53:38 STEP 1548 H=1548:00 +CPU= 59.704
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001548
  06:54:14 STEP 1549 H=1549:00 +CPU= 36.154
  06:54:50 STEP 1550 H=1550:00 +CPU= 35.143
  06:55:48 STEP 1551 H=1551:00 +CPU= 55.984
  06:56:27 STEP 1552 H=1552:00 +CPU= 33.100
  06:57:03 STEP 1553 H=1553:00 +CPU= 35.565
  06:58:00 STEP 1554 H=1554:00 +CPU= 56.341
  06:58:36 STEP 1555 H=1555:00 +CPU= 34.580
  06:59:13 STEP 1556 H=1556:00 +CPU= 36.315
  07:00:08 STEP 1557 H=1557:00 +CPU= 54.876
  07:00:46 STEP 1558 H=1558:00 +CPU= 36.842
  07:01:22 STEP 1559 H=1559:00 +CPU= 35.390
  07:02:28 STEP 1560 H=1560:00 +CPU= 64.887
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001548
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_64.zip
Uploading the intermediate file: upload_file_64.zip
Uploading trickle at timestep: 5616000
  07:03:04 STEP 1561 H=1561:00 +CPU= 35.187
  07:03:42 STEP 1562 H=1562:00 +CPU= 37.984
  07:04:36 STEP 1563 H=1563:00 +CPU= 53.529
  07:05:15 STEP 1564 H=1564:00 +CPU= 38.231
  07:05:51 STEP 1565 H=1565:00 +CPU= 35.716
  07:06:49 STEP 1566 H=1566:00 +CPU= 57.222
  07:07:26 STEP 1567 H=1567:00 +CPU= 35.587
  07:08:03 STEP 1568 H=1568:00 +CPU= 36.495
  07:08:59 STEP 1569 H=1569:00 +CPU= 55.719
  07:09:36 STEP 1570 H=1570:00 +CPU= 36.517
  07:10:14 STEP 1571 H=1571:00 +CPU= 37.537
  07:11:14 STEP 1572 H=1572:00 +CPU= 59.054
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001572
  07:11:52 STEP 1573 H=1573:00 +CPU= 37.211
  07:12:28 STEP 1574 H=1574:00 +CPU= 34.900
  07:13:24 STEP 1575 H=1575:00 +CPU= 55.593
  07:14:00 STEP 1576 H=1576:00 +CPU= 35.747
  07:14:36 STEP 1577 H=1577:00 +CPU= 35.171
  07:15:33 STEP 1578 H=1578:00 +CPU= 56.554
  07:16:08 STEP 1579 H=1579:00 +CPU= 34.823
  07:16:46 STEP 1580 H=1580:00 +CPU= 36.556
  07:17:41 STEP 1581 H=1581:00 +CPU= 54.466
  07:18:19 STEP 1582 H=1582:00 +CPU= 36.968
  07:18:54 STEP 1583 H=1583:00 +CPU= 35.039
  07:20:00 STEP 1584 H=1584:00 +CPU= 64.874
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001572
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_65.zip
Uploading the intermediate file: upload_file_65.zip
  07:20:36 STEP 1585 H=1585:00 +CPU= 34.327
  07:21:14 STEP 1586 H=1586:00 +CPU= 38.302
  07:22:09 STEP 1587 H=1587:00 +CPU= 53.633
  07:22:47 STEP 1588 H=1588:00 +CPU= 37.404
  07:23:23 STEP 1589 H=1589:00 +CPU= 35.741
  07:24:21 STEP 1590 H=1590:00 +CPU= 57.277
  07:24:57 STEP 1591 H=1591:00 +CPU= 35.029
  07:25:34 STEP 1592 H=1592:00 +CPU= 36.909
  07:26:31 STEP 1593 H=1593:00 +CPU= 55.410
  07:27:08 STEP 1594 H=1594:00 +CPU= 36.634
  07:27:47 STEP 1595 H=1595:00 +CPU= 37.474
  07:28:47 STEP 1596 H=1596:00 +CPU= 59.874
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001596
  07:29:25 STEP 1597 H=1597:00 +CPU= 37.512
  07:30:02 STEP 1598 H=1598:00 +CPU= 35.647
  07:30:58 STEP 1599 H=1599:00 +CPU= 55.220
  07:31:34 STEP 1600 H=1600:00 +CPU= 36.269
  07:32:10 STEP 1601 H=1601:00 +CPU= 34.943
  07:33:07 STEP 1602 H=1602:00 +CPU= 56.623
  07:33:43 STEP 1603 H=1603:00 +CPU= 34.883
  07:34:21 STEP 1604 H=1604:00 +CPU= 37.255
  07:35:15 STEP 1605 H=1605:00 +CPU= 53.927
  07:35:54 STEP 1606 H=1606:00 +CPU= 37.701
  07:36:29 STEP 1607 H=1607:00 +CPU= 35.248
  07:37:35 STEP 1608 H=1608:00 +CPU= 64.718
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001596
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_66.zip
Uploading the intermediate file: upload_file_66.zip
  07:38:10 STEP 1609 H=1609:00 +CPU= 33.894
  07:38:48 STEP 1610 H=1610:00 +CPU= 37.772
  07:39:43 STEP 1611 H=1611:00 +CPU= 54.136
  07:40:21 STEP 1612 H=1612:00 +CPU= 37.317
  07:40:57 STEP 1613 H=1613:00 +CPU= 35.471
  07:41:55 STEP 1614 H=1614:00 +CPU= 57.295
  07:42:31 STEP 1615 H=1615:00 +CPU= 34.864
  07:43:08 STEP 1616 H=1616:00 +CPU= 36.949
  07:44:05 STEP 1617 H=1617:00 +CPU= 55.600
  07:44:42 STEP 1618 H=1618:00 +CPU= 36.605
  07:45:20 STEP 1619 H=1619:00 +CPU= 37.550
  07:46:21 STEP 1620 H=1620:00 +CPU= 60.018
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001620
  07:47:00 STEP 1621 H=1621:00 +CPU= 37.509
  07:47:36 STEP 1622 H=1622:00 +CPU= 35.811
  07:48:32 STEP 1623 H=1623:00 +CPU= 55.194
  07:49:09 STEP 1624 H=1624:00 +CPU= 36.361
  07:49:44 STEP 1625 H=1625:00 +CPU= 34.985
  07:50:42 STEP 1626 H=1626:00 +CPU= 56.682
  07:51:18 STEP 1627 H=1627:00 +CPU= 35.064
  07:51:56 STEP 1628 H=1628:00 +CPU= 36.939
  07:52:50 STEP 1629 H=1629:00 +CPU= 53.862
  07:53:29 STEP 1630 H=1630:00 +CPU= 37.663
  07:54:04 STEP 1631 H=1631:00 +CPU= 34.921
  07:55:10 STEP 1632 H=1632:00 +CPU= 64.580
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_67.zip
Uploading the intermediate file: upload_file_67.zip
  07:55:45 STEP 1633 H=1633:00 +CPU= 33.970
  07:56:24 STEP 1634 H=1634:00 +CPU= 38.477
  07:57:18 STEP 1635 H=1635:00 +CPU= 53.947
  07:57:57 STEP 1636 H=1636:00 +CPU= 37.737
  07:58:33 STEP 1637 H=1637:00 +CPU= 35.413
  07:59:31 STEP 1638 H=1638:00 +CPU= 57.273
  08:00:07 STEP 1639 H=1639:00 +CPU= 35.138
  08:00:44 STEP 1640 H=1640:00 +CPU= 36.916
  08:01:41 STEP 1641 H=1641:00 +CPU= 55.042
  08:02:17 STEP 1642 H=1642:00 +CPU= 36.224
  08:02:55 STEP 1643 H=1643:00 +CPU= 37.530
  08:03:56 STEP 1644 H=1644:00 +CPU= 60.276
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001644
  08:04:34 STEP 1645 H=1645:00 +CPU= 37.437
  08:05:11 STEP 1646 H=1646:00 +CPU= 36.025
  08:06:07 STEP 1647 H=1647:00 +CPU= 55.368
  08:06:44 STEP 1648 H=1648:00 +CPU= 36.355
  08:07:20 STEP 1649 H=1649:00 +CPU= 35.024
  08:08:17 STEP 1650 H=1650:00 +CPU= 56.605
  08:08:53 STEP 1651 H=1651:00 +CPU= 34.953
  08:09:31 STEP 1652 H=1652:00 +CPU= 37.102
  08:10:25 STEP 1653 H=1653:00 +CPU= 53.854
  08:11:04 STEP 1654 H=1654:00 +CPU= 37.655
  08:11:39 STEP 1655 H=1655:00 +CPU= 34.835
  08:12:45 STEP 1656 H=1656:00 +CPU= 64.735
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001644
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_68.zip
Uploading the intermediate file: upload_file_68.zip
  08:13:20 STEP 1657 H=1657:00 +CPU= 34.030
  08:13:58 STEP 1658 H=1658:00 +CPU= 38.240
  08:14:53 STEP 1659 H=1659:00 +CPU= 53.976
  08:15:32 STEP 1660 H=1660:00 +CPU= 38.059
  08:16:08 STEP 1661 H=1661:00 +CPU= 35.619
  08:17:06 STEP 1662 H=1662:00 +CPU= 57.264
  08:17:42 STEP 1663 H=1663:00 +CPU= 35.440
  08:18:20 STEP 1664 H=1664:00 +CPU= 36.936
  08:19:17 STEP 1665 H=1665:00 +CPU= 55.981
  08:19:53 STEP 1666 H=1666:00 +CPU= 36.414
  08:20:31 STEP 1667 H=1667:00 +CPU= 37.562
  08:21:32 STEP 1668 H=1668:00 +CPU= 59.637
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001668
  08:22:10 STEP 1669 H=1669:00 +CPU= 37.304
  08:22:46 STEP 1670 H=1670:00 +CPU= 35.119
  08:23:42 STEP 1671 H=1671:00 +CPU= 55.738
  08:24:19 STEP 1672 H=1672:00 +CPU= 35.612
  08:24:55 STEP 1673 H=1673:00 +CPU= 35.485
  08:25:52 STEP 1674 H=1674:00 +CPU= 56.518
  08:26:28 STEP 1675 H=1675:00 +CPU= 34.949
  08:27:05 STEP 1676 H=1676:00 +CPU= 36.396
  08:28:01 STEP 1677 H=1677:00 +CPU= 55.475
  08:28:38 STEP 1678 H=1678:00 +CPU= 36.472
  08:29:14 STEP 1679 H=1679:00 +CPU= 35.218
  08:30:20 STEP 1680 H=1680:00 +CPU= 65.301
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001680
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001680
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001668
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_69.zip
Uploading the intermediate file: upload_file_69.zip
  08:30:56 STEP 1681 H=1681:00 +CPU= 35.206
  08:31:35 STEP 1682 H=1682:00 +CPU= 38.505
  08:32:31 STEP 1683 H=1683:00 +CPU= 54.025
  08:33:10 STEP 1684 H=1684:00 +CPU= 38.488
  08:33:46 STEP 1685 H=1685:00 +CPU= 35.383
  08:34:44 STEP 1686 H=1686:00 +CPU= 56.798
  08:35:20 STEP 1687 H=1687:00 +CPU= 35.372
  08:35:57 STEP 1688 H=1688:00 +CPU= 36.770
  08:36:54 STEP 1689 H=1689:00 +CPU= 55.933
  08:37:31 STEP 1690 H=1690:00 +CPU= 36.259
  08:38:08 STEP 1691 H=1691:00 +CPU= 36.971
  08:39:09 STEP 1692 H=1692:00 +CPU= 59.747
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001692
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001692
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001692
  08:39:46 STEP 1693 H=1693:00 +CPU= 36.047
  08:40:21 STEP 1694 H=1694:00 +CPU= 35.040
  08:41:18 STEP 1695 H=1695:00 +CPU= 56.228
  08:41:54 STEP 1696 H=1696:00 +CPU= 34.829
  08:42:31 STEP 1697 H=1697:00 +CPU= 36.394
  08:43:27 STEP 1698 H=1698:00 +CPU= 55.864
  08:44:03 STEP 1699 H=1699:00 +CPU= 35.464
  08:44:39 STEP 1700 H=1700:00 +CPU= 35.656
  08:45:36 STEP 1701 H=1701:00 +CPU= 55.664
  08:46:12 STEP 1702 H=1702:00 +CPU= 36.279
  08:46:49 STEP 1703 H=1703:00 +CPU= 35.590
  08:47:57 STEP 1704 H=1704:00 +CPU= 64.861
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001704
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001704
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001692
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_70.zip
Uploading the intermediate file: upload_file_70.zip
  08:48:35 STEP 1705 H=1705:00 +CPU= 32.338
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1990050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0187
(argv4) batchid: 959
(argv5) wuid: 12175831
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_593897677bd4d1335e7262afe194b9df to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0187_1990050100_123_959_12175831.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0187_1990050100_123_959_12175831.zip
ic_ancil_file: ic_ancil_12175831
ifsdata_file: ifsdata_12175831
climate_data_file: clim_data_12175831
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_0c4a0214bf1644545d53396bfb47dc9b to: /var/lib/boinc-client/slots/0/ic_ancil_12175831.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12175831.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_083e480ed5258eb91dd6947b745e68e4 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12175831.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12175831.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12175831.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12175831.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12175831.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12175831.xml
-- Model is restarting --
Adjusting last_iter, 1706, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12175831.xml
last_cpu_time: 58816
upload_file_number: 71
last_iter: 1703
last_upload: 6134400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071
The child process has been launched with process id: 108589
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:clement-M3970:1:1:108589:108589] [20230121:084955:1674308995:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:50:58 STEP 1704 H=1704:00 +CPU= 57.179
  08:51:32 STEP 1705 H=1705:00 +CPU= 31.985
  08:52:08 STEP 1706 H=1706:00 +CPU= 29.724
  08:53:00 STEP 1707 H=1707:00 +CPU= 45.698
  08:53:41 STEP 1708 H=1708:00 +CPU= 33.628
  08:54:23 STEP 1709 H=1709:00 +CPU= 34.326
  08:55:22 STEP 1710 H=1710:00 +CPU= 52.093
  08:56:02 STEP 1711 H=1711:00 +CPU= 35.148
  08:56:38 STEP 1712 H=1712:00 +CPU= 34.264
  08:57:35 STEP 1713 H=1713:00 +CPU= 55.163
  08:58:13 STEP 1714 H=1714:00 +CPU= 36.001
  08:58:47 STEP 1715 H=1715:00 +CPU= 33.182
  08:59:51 STEP 1716 H=1716:00 +CPU= 61.734
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001716
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001716
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001716
  09:00:26 STEP 1717 H=1717:00 +CPU= 33.563
  09:01:05 STEP 1718 H=1718:00 +CPU= 37.403
  09:02:01 STEP 1719 H=1719:00 +CPU= 53.454
  09:02:40 STEP 1720 H=1720:00 +CPU= 37.825
  09:03:14 STEP 1721 H=1721:00 +CPU= 33.417
  09:04:14 STEP 1722 H=1722:00 +CPU= 56.868
  09:04:50 STEP 1723 H=1723:00 +CPU= 34.404
  09:05:26 STEP 1724 H=1724:00 +CPU= 35.348
  09:06:25 STEP 1725 H=1725:00 +CPU= 55.370
  09:07:01 STEP 1726 H=1726:00 +CPU= 34.493
  09:07:40 STEP 1727 H=1727:00 +CPU= 37.599
  09:08:56 STEP 1728 H=1728:00 +CPU= 63.011
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+001728
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+001728
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMGGhq0f+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMSHhq0f+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12175831/ICMUAhq0f+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0187_1990050100_123_959_12175831_1_r1425391071_71.zip
Uploading the intermediate file: upload_file_71.zip
  09:09:34 STEP 1729 H=1729:00 +CPU= 36.310
  09:10:09 STEP 1730 H=1730:00 +CPU= 33.417
  09:11:09 STEP 1731 H=1731:00 +CPU= 56.869
  09:11:44 STEP 1732 H=1732:00 +CPU= 34.071
double free or corruption (out)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:33:08 1466689 22262405 oifs_43r3_ps_0187_1990050100_123_959_12175831_1 5,616,000 52,562 0.0094
09 May 2023 16:32:51 1466689 22262405 oifs_43r3_ps_0187_1990050100_123_959_12175831_1 4,752,000 42,188 0.0089
09 May 2023 16:31:52 1466689 22262405 oifs_43r3_ps_0187_1990050100_123_959_12175831_1 3,456,000 27,627 0.0080
09 May 2023 16:31:47 1466689 22262405 oifs_43r3_ps_0187_1990050100_123_959_12175831_1 2,592,000 20,694 0.0080
09 May 2023 16:31:29 1466689 22262405 oifs_43r3_ps_0187_1990050100_123_959_12175831_1 1,728,000 13,864 0.0080
09 May 2023 16:31:21 1466689 22262405 oifs_43r3_ps_0187_1990050100_123_959_12175831_1 864,000 7,007 0.0081


©2024 climateprediction.net