climateprediction.net home page
Task 22264553

Task 22264553

Name oifs_43r3_ps_0990_1993050100_123_962_12179634_0
Workunit 12179634
Created 22 Dec 2022, 4:33:47 UTC
Sent 22 Dec 2022, 4:37:58 UTC
Report deadline 21 Jan 2023, 4:37:58 UTC
Received 23 Dec 2022, 4:19:41 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID 1517859
Run time 21 hours 22 min 57 sec
CPU time 20 hours 35 min 49 sec
Validate state Invalid
Credit 8,467.20
Device peak FLOPS 6.98 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,796.31 MB
Peak swap size 4,974.26 MB
Peak disk usage 2,324.43 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Process still present 5 min after writing finish file; aborting</message>
<stderr_txt>
/ICMSHhq0f+002268
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002268
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_94.zip
Uploading the intermediate file: upload_file_94.zip
  14:47:40 STEP 2281 H=2281:00 +CPU= 15.909
  14:48:19 STEP 2282 H=2282:00 +CPU= 35.194
  14:49:03 STEP 2283 H=2283:00 +CPU= 27.386
  14:49:32 STEP 2284 H=2284:00 +CPU= 24.703
  14:49:52 STEP 2285 H=2285:00 +CPU= 16.509
  14:50:27 STEP 2286 H=2286:00 +CPU= 27.191
  14:50:48 STEP 2287 H=2287:00 +CPU= 16.433
  14:51:10 STEP 2288 H=2288:00 +CPU= 16.973
  14:51:42 STEP 2289 H=2289:00 +CPU= 24.609
  14:52:04 STEP 2290 H=2290:00 +CPU= 17.367
  14:52:26 STEP 2291 H=2291:00 +CPU= 16.955
  14:53:11 STEP 2292 H=2292:00 +CPU= 35.883
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002292
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002292
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002292
  14:53:35 STEP 2293 H=2293:00 +CPU= 18.713
  14:54:08 STEP 2294 H=2294:00 +CPU= 27.513
  14:54:52 STEP 2295 H=2295:00 +CPU= 37.142
  14:55:25 STEP 2296 H=2296:00 +CPU= 25.326
  14:55:49 STEP 2297 H=2297:00 +CPU= 18.474
  14:56:30 STEP 2298 H=2298:00 +CPU= 32.802
  14:56:53 STEP 2299 H=2299:00 +CPU= 17.281
  14:57:24 STEP 2300 H=2300:00 +CPU= 22.028
  14:58:01 STEP 2301 H=2301:00 +CPU= 29.098
  14:58:24 STEP 2302 H=2302:00 +CPU= 18.241
  14:58:50 STEP 2303 H=2303:00 +CPU= 20.133
  14:59:34 STEP 2304 H=2304:00 +CPU= 35.468
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002304
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002304
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002304
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002280
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002280
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002280
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002292
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002292
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002292
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_95.zip
Uploading the intermediate file: upload_file_95.zip
  14:59:57 STEP 2305 H=2305:00 +CPU= 17.346
  15:00:19 STEP 2306 H=2306:00 +CPU= 17.867
  15:00:57 STEP 2307 H=2307:00 +CPU= 28.653
  15:01:17 STEP 2308 H=2308:00 +CPU= 17.625
  15:01:37 STEP 2309 H=2309:00 +CPU= 16.225
  15:02:10 STEP 2310 H=2310:00 +CPU= 25.795
  15:02:29 STEP 2311 H=2311:00 +CPU= 15.321
  15:02:52 STEP 2312 H=2312:00 +CPU= 17.475
  15:03:26 STEP 2313 H=2313:00 +CPU= 26.307
  15:03:50 STEP 2314 H=2314:00 +CPU= 20.794
  15:04:11 STEP 2315 H=2315:00 +CPU= 16.995
  15:04:50 STEP 2316 H=2316:00 +CPU= 31.355
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002316
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002316
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002316
  15:05:12 STEP 2317 H=2317:00 +CPU= 16.521
  15:05:40 STEP 2318 H=2318:00 +CPU= 23.911
  15:06:14 STEP 2319 H=2319:00 +CPU= 27.285
  15:06:47 STEP 2320 H=2320:00 +CPU= 23.666
  15:07:10 STEP 2321 H=2321:00 +CPU= 18.142
  15:07:57 STEP 2322 H=2322:00 +CPU= 40.743
  15:08:22 STEP 2323 H=2323:00 +CPU= 20.705
  15:08:59 STEP 2324 H=2324:00 +CPU= 31.771
  15:09:35 STEP 2325 H=2325:00 +CPU= 29.382
  15:09:57 STEP 2326 H=2326:00 +CPU= 17.282
  15:10:21 STEP 2327 H=2327:00 +CPU= 18.424
  15:11:20 STEP 2328 H=2328:00 +CPU= 45.601
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002328
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002328
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002328
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002304
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002304
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002304
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002316
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002316
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002316
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_96.zip
Uploading the intermediate file: upload_file_96.zip
  15:11:54 STEP 2329 H=2329:00 +CPU= 26.343
  15:12:48 STEP 2330 H=2330:00 +CPU= 47.019
  15:13:28 STEP 2331 H=2331:00 +CPU= 28.026
  15:13:48 STEP 2332 H=2332:00 +CPU= 16.642
  15:14:25 STEP 2333 H=2333:00 +CPU= 32.232
  15:15:02 STEP 2334 H=2334:00 +CPU= 28.843
  15:15:27 STEP 2335 H=2335:00 +CPU= 19.329
  15:15:49 STEP 2336 H=2336:00 +CPU= 18.586
  15:16:40 STEP 2337 H=2337:00 +CPU= 42.436
  15:17:17 STEP 2338 H=2338:00 +CPU= 31.976
  15:17:50 STEP 2339 H=2339:00 +CPU= 26.921
  15:18:54 STEP 2340 H=2340:00 +CPU= 56.551
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002340
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002340
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002340
  15:19:33 STEP 2341 H=2341:00 +CPU= 34.458
  15:19:57 STEP 2342 H=2342:00 +CPU= 19.628
  15:20:33 STEP 2343 H=2343:00 +CPU= 28.505
  15:20:56 STEP 2344 H=2344:00 +CPU= 20.095
  15:21:18 STEP 2345 H=2345:00 +CPU= 18.002
  15:21:58 STEP 2346 H=2346:00 +CPU= 31.353
  15:22:22 STEP 2347 H=2347:00 +CPU= 19.069
  15:22:44 STEP 2348 H=2348:00 +CPU= 18.086
  15:23:17 STEP 2349 H=2349:00 +CPU= 26.187
  15:23:55 STEP 2350 H=2350:00 +CPU= 25.631
  15:24:26 STEP 2351 H=2351:00 +CPU= 25.513
  15:25:10 STEP 2352 H=2352:00 +CPU= 35.896
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002352
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002352
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002352
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002328
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002328
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002328
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002340
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002340
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002340
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_97.zip
Uploading the intermediate file: upload_file_97.zip
  15:25:35 STEP 2353 H=2353:00 +CPU= 18.912
  15:25:58 STEP 2354 H=2354:00 +CPU= 18.340
  15:26:36 STEP 2355 H=2355:00 +CPU= 29.521
  15:27:03 STEP 2356 H=2356:00 +CPU= 23.638
  15:27:27 STEP 2357 H=2357:00 +CPU= 18.649
  15:27:56 STEP 2358 H=2358:00 +CPU= 26.673
  15:28:20 STEP 2359 H=2359:00 +CPU= 18.399
  15:28:43 STEP 2360 H=2360:00 +CPU= 18.296
  15:29:16 STEP 2361 H=2361:00 +CPU= 27.920
  15:29:48 STEP 2362 H=2362:00 +CPU= 27.827
  15:30:09 STEP 2363 H=2363:00 +CPU= 17.437
  15:30:53 STEP 2364 H=2364:00 +CPU= 29.725
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002364
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002364
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002364
  15:31:15 STEP 2365 H=2365:00 +CPU= 17.442
  15:31:35 STEP 2366 H=2366:00 +CPU= 16.538
  15:32:15 STEP 2367 H=2367:00 +CPU= 33.299
  15:32:41 STEP 2368 H=2368:00 +CPU= 20.520
  15:33:02 STEP 2369 H=2369:00 +CPU= 16.331
  15:33:36 STEP 2370 H=2370:00 +CPU= 26.022
  15:33:56 STEP 2371 H=2371:00 +CPU= 16.254
  15:34:17 STEP 2372 H=2372:00 +CPU= 15.964
  15:35:20 STEP 2373 H=2373:00 +CPU= 55.621
  15:35:49 STEP 2374 H=2374:00 +CPU= 20.866
  15:36:13 STEP 2375 H=2375:00 +CPU= 19.299
  15:36:54 STEP 2376 H=2376:00 +CPU= 31.854
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002376
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002376
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002376
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002352
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002352
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002352
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002364
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002364
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002364
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_98.zip
Uploading the intermediate file: upload_file_98.zip
  15:37:14 STEP 2377 H=2377:00 +CPU= 16.315
  15:37:38 STEP 2378 H=2378:00 +CPU= 17.270
  15:38:12 STEP 2379 H=2379:00 +CPU= 26.626
  15:38:34 STEP 2380 H=2380:00 +CPU= 16.381
  15:39:01 STEP 2381 H=2381:00 +CPU= 20.889
  15:39:36 STEP 2382 H=2382:00 +CPU= 26.511
  15:39:56 STEP 2383 H=2383:00 +CPU= 16.157
  15:40:16 STEP 2384 H=2384:00 +CPU= 16.525
  15:41:03 STEP 2385 H=2385:00 +CPU= 38.982
  15:41:25 STEP 2386 H=2386:00 +CPU= 16.240
  15:41:45 STEP 2387 H=2387:00 +CPU= 16.374
  15:42:57 STEP 2388 H=2388:00 +CPU= 52.522
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002388
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002388
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002388
  15:43:17 STEP 2389 H=2389:00 +CPU= 16.016
  15:43:42 STEP 2390 H=2390:00 +CPU= 19.381
  15:45:38 STEP 2391 H=2391:00 +CPU=107.806
  15:46:00 STEP 2392 H=2392:00 +CPU= 17.570
  15:46:37 STEP 2393 H=2393:00 +CPU= 32.133
  15:47:11 STEP 2394 H=2394:00 +CPU= 27.699
  15:47:46 STEP 2395 H=2395:00 +CPU= 30.982
  15:48:10 STEP 2396 H=2396:00 +CPU= 19.055
  15:48:47 STEP 2397 H=2397:00 +CPU= 27.779
  15:49:10 STEP 2398 H=2398:00 +CPU= 18.350
  15:49:32 STEP 2399 H=2399:00 +CPU= 17.565
  15:50:16 STEP 2400 H=2400:00 +CPU= 30.836
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002400
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002400
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002400
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002376
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002376
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002376
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002388
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002388
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002388
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_99.zip
Uploading the intermediate file: upload_file_99.zip
Uploading trickle at timestep: 8640000
  15:50:43 STEP 2401 H=2401:00 +CPU= 22.789
  15:51:13 STEP 2402 H=2402:00 +CPU= 23.366
  15:51:43 STEP 2403 H=2403:00 +CPU= 25.586
  15:52:04 STEP 2404 H=2404:00 +CPU= 16.708
  15:52:27 STEP 2405 H=2405:00 +CPU= 16.315
  15:52:58 STEP 2406 H=2406:00 +CPU= 25.835
  15:53:18 STEP 2407 H=2407:00 +CPU= 15.979
  15:53:39 STEP 2408 H=2408:00 +CPU= 16.873
  15:54:13 STEP 2409 H=2409:00 +CPU= 26.770
  15:54:42 STEP 2410 H=2410:00 +CPU= 24.515
  15:55:10 STEP 2411 H=2411:00 +CPU= 16.834
  15:56:03 STEP 2412 H=2412:00 +CPU= 43.805
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002412
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002412
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002412
  15:56:41 STEP 2413 H=2413:00 +CPU= 33.939
  15:57:02 STEP 2414 H=2414:00 +CPU= 18.807
  15:58:02 STEP 2415 H=2415:00 +CPU= 52.747
  15:58:23 STEP 2416 H=2416:00 +CPU= 16.719
  15:58:45 STEP 2417 H=2417:00 +CPU= 17.004
  15:59:19 STEP 2418 H=2418:00 +CPU= 26.560
  15:59:41 STEP 2419 H=2419:00 +CPU= 17.654
  16:00:06 STEP 2420 H=2420:00 +CPU= 18.755
  16:00:46 STEP 2421 H=2421:00 +CPU= 32.107
  16:01:10 STEP 2422 H=2422:00 +CPU= 18.225
  16:01:32 STEP 2423 H=2423:00 +CPU= 17.306
  16:02:18 STEP 2424 H=2424:00 +CPU= 34.181
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002424
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002424
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002424
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002400
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002400
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002400
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002412
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002412
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002412
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_100.zip
Uploading the intermediate file: upload_file_100.zip
  16:02:46 STEP 2425 H=2425:00 +CPU= 22.873
  16:03:13 STEP 2426 H=2426:00 +CPU= 20.047
  16:03:48 STEP 2427 H=2427:00 +CPU= 27.644
  16:04:12 STEP 2428 H=2428:00 +CPU= 18.229
  16:04:36 STEP 2429 H=2429:00 +CPU= 19.406
  16:05:11 STEP 2430 H=2430:00 +CPU= 28.424
  16:05:34 STEP 2431 H=2431:00 +CPU= 18.081
  16:05:58 STEP 2432 H=2432:00 +CPU= 18.348
  16:06:34 STEP 2433 H=2433:00 +CPU= 27.806
  16:06:55 STEP 2434 H=2434:00 +CPU= 16.958
  16:07:18 STEP 2435 H=2435:00 +CPU= 18.507
  16:07:59 STEP 2436 H=2436:00 +CPU= 32.634
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002436
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002436
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002436
  16:08:21 STEP 2437 H=2437:00 +CPU= 17.926
  16:08:53 STEP 2438 H=2438:00 +CPU= 25.611
  16:09:32 STEP 2439 H=2439:00 +CPU= 31.062
  16:09:56 STEP 2440 H=2440:00 +CPU= 19.864
  16:10:21 STEP 2441 H=2441:00 +CPU= 21.613
  16:11:02 STEP 2442 H=2442:00 +CPU= 30.630
  16:11:27 STEP 2443 H=2443:00 +CPU= 19.681
  16:12:05 STEP 2444 H=2444:00 +CPU= 31.320
  16:12:49 STEP 2445 H=2445:00 +CPU= 37.911
  16:13:14 STEP 2446 H=2446:00 +CPU= 19.001
  16:13:40 STEP 2447 H=2447:00 +CPU= 19.463
  16:14:25 STEP 2448 H=2448:00 +CPU= 34.416
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002448
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002448
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002448
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002424
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002424
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002424
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002436
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002436
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002436
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_101.zip
Uploading the intermediate file: upload_file_101.zip
  16:15:03 STEP 2449 H=2449:00 +CPU= 27.776
  16:15:34 STEP 2450 H=2450:00 +CPU= 25.798
  16:16:24 STEP 2451 H=2451:00 +CPU= 40.450
  16:16:51 STEP 2452 H=2452:00 +CPU= 22.511
  16:17:17 STEP 2453 H=2453:00 +CPU= 20.285
  16:17:53 STEP 2454 H=2454:00 +CPU= 28.506
  16:18:14 STEP 2455 H=2455:00 +CPU= 17.456
  16:18:47 STEP 2456 H=2456:00 +CPU= 28.108
  16:19:25 STEP 2457 H=2457:00 +CPU= 29.679
  16:19:50 STEP 2458 H=2458:00 +CPU= 19.732
  16:20:32 STEP 2459 H=2459:00 +CPU= 31.239
  16:21:23 STEP 2460 H=2460:00 +CPU= 32.378
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002460
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002460
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002460
  16:21:49 STEP 2461 H=2461:00 +CPU= 19.516
  16:22:19 STEP 2462 H=2462:00 +CPU= 23.142
  16:22:56 STEP 2463 H=2463:00 +CPU= 28.116
  16:23:19 STEP 2464 H=2464:00 +CPU= 18.557
  16:23:54 STEP 2465 H=2465:00 +CPU= 27.080
  16:24:51 STEP 2466 H=2466:00 +CPU= 31.622
  16:25:12 STEP 2467 H=2467:00 +CPU= 17.594
  16:25:36 STEP 2468 H=2468:00 +CPU= 20.943
  16:26:25 STEP 2469 H=2469:00 +CPU= 39.730
  16:26:48 STEP 2470 H=2470:00 +CPU= 17.909
  16:27:12 STEP 2471 H=2471:00 +CPU= 19.339
  16:27:55 STEP 2472 H=2472:00 +CPU= 33.366
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002472
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002472
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002472
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002448
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002448
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002448
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002460
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002460
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002460
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_102.zip
Uploading the intermediate file: upload_file_102.zip
  16:28:20 STEP 2473 H=2473:00 +CPU= 19.689
  16:28:47 STEP 2474 H=2474:00 +CPU= 20.634
  16:29:25 STEP 2475 H=2475:00 +CPU= 30.378
  16:29:51 STEP 2476 H=2476:00 +CPU= 21.147
  16:30:16 STEP 2477 H=2477:00 +CPU= 19.528
  16:30:51 STEP 2478 H=2478:00 +CPU= 28.447
  16:31:15 STEP 2479 H=2479:00 +CPU= 19.340
  16:31:45 STEP 2480 H=2480:00 +CPU= 25.282
  16:32:24 STEP 2481 H=2481:00 +CPU= 31.070
  16:32:47 STEP 2482 H=2482:00 +CPU= 18.505
  16:33:28 STEP 2483 H=2483:00 +CPU= 34.625
  16:34:14 STEP 2484 H=2484:00 +CPU= 34.186
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002484
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002484
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002484
  16:34:44 STEP 2485 H=2485:00 +CPU= 21.904
  16:35:10 STEP 2486 H=2486:00 +CPU= 19.821
  16:35:56 STEP 2487 H=2487:00 +CPU= 29.024
  16:36:21 STEP 2488 H=2488:00 +CPU= 21.066
  16:36:43 STEP 2489 H=2489:00 +CPU= 16.553
  16:37:23 STEP 2490 H=2490:00 +CPU= 32.500
  16:37:49 STEP 2491 H=2491:00 +CPU= 21.086
  16:38:14 STEP 2492 H=2492:00 +CPU= 20.191
  16:38:52 STEP 2493 H=2493:00 +CPU= 30.903
  16:39:19 STEP 2494 H=2494:00 +CPU= 20.053
  16:39:46 STEP 2495 H=2495:00 +CPU= 21.869
  16:40:34 STEP 2496 H=2496:00 +CPU= 39.181
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002496
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002496
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002496
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002472
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002472
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002472
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002484
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002484
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002484
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_103.zip
Uploading the intermediate file: upload_file_103.zip
  16:40:59 STEP 2497 H=2497:00 +CPU= 19.641
  16:41:25 STEP 2498 H=2498:00 +CPU= 18.732
  16:42:07 STEP 2499 H=2499:00 +CPU= 35.480
  16:42:33 STEP 2500 H=2500:00 +CPU= 19.985
  16:42:59 STEP 2501 H=2501:00 +CPU= 21.008
  16:43:44 STEP 2502 H=2502:00 +CPU= 32.800
  16:44:14 STEP 2503 H=2503:00 +CPU= 21.333
  16:44:37 STEP 2504 H=2504:00 +CPU= 20.630
  16:45:12 STEP 2505 H=2505:00 +CPU= 27.744
  16:45:45 STEP 2506 H=2506:00 +CPU= 18.684
  16:46:11 STEP 2507 H=2507:00 +CPU= 20.543
  16:46:52 STEP 2508 H=2508:00 +CPU= 34.437
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002508
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002508
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002508
  16:47:26 STEP 2509 H=2509:00 +CPU= 27.709
  16:47:49 STEP 2510 H=2510:00 +CPU= 18.676
  16:48:35 STEP 2511 H=2511:00 +CPU= 37.110
  16:49:00 STEP 2512 H=2512:00 +CPU= 19.449
  16:49:24 STEP 2513 H=2513:00 +CPU= 20.430
  16:50:01 STEP 2514 H=2514:00 +CPU= 30.170
  16:50:24 STEP 2515 H=2515:00 +CPU= 17.859
  16:50:47 STEP 2516 H=2516:00 +CPU= 18.008
  16:51:36 STEP 2517 H=2517:00 +CPU= 40.613
  16:52:00 STEP 2518 H=2518:00 +CPU= 19.921
  16:52:26 STEP 2519 H=2519:00 +CPU= 20.034
  16:53:09 STEP 2520 H=2520:00 +CPU= 33.545
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002520
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002520
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002520
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002496
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002496
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002496
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002508
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002508
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002508
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_104.zip
Uploading the intermediate file: upload_file_104.zip
  16:53:33 STEP 2521 H=2521:00 +CPU= 19.199
  16:53:55 STEP 2522 H=2522:00 +CPU= 18.370
  16:54:41 STEP 2523 H=2523:00 +CPU= 37.036
  16:55:07 STEP 2524 H=2524:00 +CPU= 19.620
  16:55:35 STEP 2525 H=2525:00 +CPU= 22.494
  16:56:18 STEP 2526 H=2526:00 +CPU= 32.648
  16:56:46 STEP 2527 H=2527:00 +CPU= 21.427
  16:57:14 STEP 2528 H=2528:00 +CPU= 22.456
  16:57:51 STEP 2529 H=2529:00 +CPU= 29.121
  16:58:28 STEP 2530 H=2530:00 +CPU= 31.107
  16:58:53 STEP 2531 H=2531:00 +CPU= 19.298
  16:59:41 STEP 2532 H=2532:00 +CPU= 37.727
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002532
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002532
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002532
  17:00:18 STEP 2533 H=2533:00 +CPU= 31.300
  17:00:45 STEP 2534 H=2534:00 +CPU= 21.848
  17:01:42 STEP 2535 H=2535:00 +CPU= 44.468
  17:02:14 STEP 2536 H=2536:00 +CPU= 27.270
  17:02:38 STEP 2537 H=2537:00 +CPU= 19.167
  17:03:21 STEP 2538 H=2538:00 +CPU= 34.574
  17:03:48 STEP 2539 H=2539:00 +CPU= 21.471
  17:04:13 STEP 2540 H=2540:00 +CPU= 20.234
  17:04:51 STEP 2541 H=2541:00 +CPU= 30.406
  17:05:16 STEP 2542 H=2542:00 +CPU= 21.542
  17:05:51 STEP 2543 H=2543:00 +CPU= 29.221
  17:06:48 STEP 2544 H=2544:00 +CPU= 47.987
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002544
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002544
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002544
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002520
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002520
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002520
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002532
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002532
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002532
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_105.zip
Uploading the intermediate file: upload_file_105.zip
  17:07:12 STEP 2545 H=2545:00 +CPU= 18.869
  17:07:37 STEP 2546 H=2546:00 +CPU= 18.971
  17:08:20 STEP 2547 H=2547:00 +CPU= 35.493
  17:08:53 STEP 2548 H=2548:00 +CPU= 22.284
  17:09:19 STEP 2549 H=2549:00 +CPU= 22.048
  17:10:04 STEP 2550 H=2550:00 +CPU= 35.922
  17:10:30 STEP 2551 H=2551:00 +CPU= 21.336
  17:10:54 STEP 2552 H=2552:00 +CPU= 17.675
  17:11:34 STEP 2553 H=2553:00 +CPU= 31.737
  17:11:59 STEP 2554 H=2554:00 +CPU= 19.421
  17:12:31 STEP 2555 H=2555:00 +CPU= 27.178
  17:13:17 STEP 2556 H=2556:00 +CPU= 35.946
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002556
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002556
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002556
  17:13:45 STEP 2557 H=2557:00 +CPU= 22.510
  17:14:12 STEP 2558 H=2558:00 +CPU= 21.734
  17:14:58 STEP 2559 H=2559:00 +CPU= 39.931
  17:15:23 STEP 2560 H=2560:00 +CPU= 20.474
  17:15:53 STEP 2561 H=2561:00 +CPU= 21.765
  17:16:35 STEP 2562 H=2562:00 +CPU= 33.757
  17:16:58 STEP 2563 H=2563:00 +CPU= 18.911
  17:17:37 STEP 2564 H=2564:00 +CPU= 32.609
  17:18:27 STEP 2565 H=2565:00 +CPU= 31.724
  17:18:50 STEP 2566 H=2566:00 +CPU= 19.282
  17:19:18 STEP 2567 H=2567:00 +CPU= 23.427
  17:20:16 STEP 2568 H=2568:00 +CPU= 47.300
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002568
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002568
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002568
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002544
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002544
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002544
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002556
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002556
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002556
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_106.zip
Uploading the intermediate file: upload_file_106.zip
  17:20:40 STEP 2569 H=2569:00 +CPU= 19.171
  17:21:06 STEP 2570 H=2570:00 +CPU= 20.457
  17:21:57 STEP 2571 H=2571:00 +CPU= 42.595
  17:22:37 STEP 2572 H=2572:00 +CPU= 33.538
  17:23:14 STEP 2573 H=2573:00 +CPU= 32.258
  17:23:54 STEP 2574 H=2574:00 +CPU= 32.100
  17:24:17 STEP 2575 H=2575:00 +CPU= 19.036
  17:24:50 STEP 2576 H=2576:00 +CPU= 27.459
  17:25:24 STEP 2577 H=2577:00 +CPU= 28.310
  17:25:54 STEP 2578 H=2578:00 +CPU= 23.037
  17:26:19 STEP 2579 H=2579:00 +CPU= 19.825
  17:27:06 STEP 2580 H=2580:00 +CPU= 36.835
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002580
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002580
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002580
  17:27:30 STEP 2581 H=2581:00 +CPU= 19.499
  17:27:59 STEP 2582 H=2582:00 +CPU= 23.159
  17:28:42 STEP 2583 H=2583:00 +CPU= 33.621
  17:29:21 STEP 2584 H=2584:00 +CPU= 28.781
  17:29:48 STEP 2585 H=2585:00 +CPU= 21.981
  17:30:33 STEP 2586 H=2586:00 +CPU= 35.499
  17:31:01 STEP 2587 H=2587:00 +CPU= 21.826
  17:31:34 STEP 2588 H=2588:00 +CPU= 27.022
  17:32:20 STEP 2589 H=2589:00 +CPU= 38.756
  17:32:47 STEP 2590 H=2590:00 +CPU= 21.234
  17:33:10 STEP 2591 H=2591:00 +CPU= 18.836
  17:34:08 STEP 2592 H=2592:00 +CPU= 40.193
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002592
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002592
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002592
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002568
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002568
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002568
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002580
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002580
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002580
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_107.zip
Uploading the intermediate file: upload_file_107.zip
  17:34:40 STEP 2593 H=2593:00 +CPU= 26.399
  17:35:08 STEP 2594 H=2594:00 +CPU= 25.045
  17:36:23 STEP 2595 H=2595:00 +CPU= 57.800
  17:36:48 STEP 2596 H=2596:00 +CPU= 19.302
  17:37:13 STEP 2597 H=2597:00 +CPU= 18.917
  17:37:56 STEP 2598 H=2598:00 +CPU= 33.812
  17:38:21 STEP 2599 H=2599:00 +CPU= 19.299
  17:38:45 STEP 2600 H=2600:00 +CPU= 18.939
  17:39:23 STEP 2601 H=2601:00 +CPU= 30.114
  17:39:58 STEP 2602 H=2602:00 +CPU= 29.871
  17:40:19 STEP 2603 H=2603:00 +CPU= 17.262
  17:41:01 STEP 2604 H=2604:00 +CPU= 33.271
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002604
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002604
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002604
  17:41:26 STEP 2605 H=2605:00 +CPU= 20.751
  17:41:56 STEP 2606 H=2606:00 +CPU= 22.540
  17:42:41 STEP 2607 H=2607:00 +CPU= 35.947
  17:43:12 STEP 2608 H=2608:00 +CPU= 24.809
  17:43:35 STEP 2609 H=2609:00 +CPU= 18.336
  17:44:24 STEP 2610 H=2610:00 +CPU= 42.275
  17:44:51 STEP 2611 H=2611:00 +CPU= 21.491
  17:45:18 STEP 2612 H=2612:00 +CPU= 21.807
  17:45:57 STEP 2613 H=2613:00 +CPU= 30.332
  17:46:23 STEP 2614 H=2614:00 +CPU= 20.066
  17:46:48 STEP 2615 H=2615:00 +CPU= 20.462
  17:47:35 STEP 2616 H=2616:00 +CPU= 37.435
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002616
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002616
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002616
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002592
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002592
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002592
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002604
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002604
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002604
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_108.zip
Uploading the intermediate file: upload_file_108.zip
  17:47:58 STEP 2617 H=2617:00 +CPU= 19.463
  17:48:31 STEP 2618 H=2618:00 +CPU= 27.130
  17:49:34 STEP 2619 H=2619:00 +CPU= 40.311
  17:50:09 STEP 2620 H=2620:00 +CPU= 31.645
  17:50:47 STEP 2621 H=2621:00 +CPU= 32.288
  17:51:34 STEP 2622 H=2622:00 +CPU= 32.984
  17:51:57 STEP 2623 H=2623:00 +CPU= 17.648
  17:52:25 STEP 2624 H=2624:00 +CPU= 21.405
  17:53:06 STEP 2625 H=2625:00 +CPU= 31.858
  17:53:35 STEP 2626 H=2626:00 +CPU= 23.533
  17:54:01 STEP 2627 H=2627:00 +CPU= 20.665
  17:54:43 STEP 2628 H=2628:00 +CPU= 34.552
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002628
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002628
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002628
  17:55:10 STEP 2629 H=2629:00 +CPU= 20.485
  17:55:36 STEP 2630 H=2630:00 +CPU= 20.577
  17:56:16 STEP 2631 H=2631:00 +CPU= 32.329
  17:56:44 STEP 2632 H=2632:00 +CPU= 21.911
  17:57:12 STEP 2633 H=2633:00 +CPU= 20.749
  17:57:53 STEP 2634 H=2634:00 +CPU= 31.518
  17:58:24 STEP 2635 H=2635:00 +CPU= 18.941
  17:58:52 STEP 2636 H=2636:00 +CPU= 22.925
  17:59:43 STEP 2637 H=2637:00 +CPU= 41.457
  18:00:18 STEP 2638 H=2638:00 +CPU= 27.912
  18:00:42 STEP 2639 H=2639:00 +CPU= 19.509
  18:01:25 STEP 2640 H=2640:00 +CPU= 33.805
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002640
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002640
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002640
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002616
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002616
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002616
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002628
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002628
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002628
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_109.zip
Uploading the intermediate file: upload_file_109.zip
Uploading trickle at timestep: 9504000
  18:01:51 STEP 2641 H=2641:00 +CPU= 23.126
  18:02:23 STEP 2642 H=2642:00 +CPU= 27.421
  18:03:01 STEP 2643 H=2643:00 +CPU= 30.589
  18:03:39 STEP 2644 H=2644:00 +CPU= 32.543
  18:04:15 STEP 2645 H=2645:00 +CPU= 27.727
  18:04:54 STEP 2646 H=2646:00 +CPU= 31.274
  18:05:20 STEP 2647 H=2647:00 +CPU= 20.775
  18:05:46 STEP 2648 H=2648:00 +CPU= 20.062
  18:06:47 STEP 2649 H=2649:00 +CPU= 52.929
  18:07:21 STEP 2650 H=2650:00 +CPU= 25.808
  18:07:50 STEP 2651 H=2651:00 +CPU= 21.009
  18:08:36 STEP 2652 H=2652:00 +CPU= 34.381
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002652
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002652
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002652
  18:09:02 STEP 2653 H=2653:00 +CPU= 20.338
  18:09:31 STEP 2654 H=2654:00 +CPU= 23.489
  18:10:14 STEP 2655 H=2655:00 +CPU= 33.768
  18:10:40 STEP 2656 H=2656:00 +CPU= 20.763
  18:11:06 STEP 2657 H=2657:00 +CPU= 19.253
  18:12:09 STEP 2658 H=2658:00 +CPU= 45.089
  18:12:36 STEP 2659 H=2659:00 +CPU= 20.475
  18:13:02 STEP 2660 H=2660:00 +CPU= 21.369
  18:13:43 STEP 2661 H=2661:00 +CPU= 31.679
  18:14:10 STEP 2662 H=2662:00 +CPU= 21.017
  18:14:36 STEP 2663 H=2663:00 +CPU= 20.501
  18:15:34 STEP 2664 H=2664:00 +CPU= 47.316
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002664
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002664
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002664
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002640
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002640
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002640
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002652
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002652
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002652
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_110.zip
Uploading the intermediate file: upload_file_110.zip
  18:16:07 STEP 2665 H=2665:00 +CPU= 26.951
  18:16:39 STEP 2666 H=2666:00 +CPU= 25.666
  18:17:21 STEP 2667 H=2667:00 +CPU= 33.266
  18:17:48 STEP 2668 H=2668:00 +CPU= 19.811
  18:18:22 STEP 2669 H=2669:00 +CPU= 29.515
  18:19:05 STEP 2670 H=2670:00 +CPU= 34.472
  18:19:38 STEP 2671 H=2671:00 +CPU= 26.696
  18:20:14 STEP 2672 H=2672:00 +CPU= 30.534
  18:20:56 STEP 2673 H=2673:00 +CPU= 33.658
  18:21:40 STEP 2674 H=2674:00 +CPU= 38.435
  18:22:03 STEP 2675 H=2675:00 +CPU= 17.314
  18:22:43 STEP 2676 H=2676:00 +CPU= 31.591
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002676
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002676
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002676
  18:23:10 STEP 2677 H=2677:00 +CPU= 20.954
  18:23:37 STEP 2678 H=2678:00 +CPU= 22.212
  18:24:20 STEP 2679 H=2679:00 +CPU= 34.669
  18:25:04 STEP 2680 H=2680:00 +CPU= 37.398
  18:25:33 STEP 2681 H=2681:00 +CPU= 22.339
  18:26:13 STEP 2682 H=2682:00 +CPU= 31.807
  18:26:38 STEP 2683 H=2683:00 +CPU= 20.219
  18:27:06 STEP 2684 H=2684:00 +CPU= 21.327
  18:27:50 STEP 2685 H=2685:00 +CPU= 35.101
  18:28:22 STEP 2686 H=2686:00 +CPU= 24.245
  18:28:51 STEP 2687 H=2687:00 +CPU= 21.672
  18:30:15 STEP 2688 H=2688:00 +CPU= 62.746
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002688
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002688
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002688
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002664
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002664
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002664
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002676
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002676
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002676
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_111.zip
Uploading the intermediate file: upload_file_111.zip
  18:30:50 STEP 2689 H=2689:00 +CPU= 23.823
  18:31:23 STEP 2690 H=2690:00 +CPU= 20.598
  18:31:59 STEP 2691 H=2691:00 +CPU= 29.415
  18:32:29 STEP 2692 H=2692:00 +CPU= 23.649
  18:33:02 STEP 2693 H=2693:00 +CPU= 21.356
  18:33:48 STEP 2694 H=2694:00 +CPU= 32.676
  18:34:10 STEP 2695 H=2695:00 +CPU= 19.681
  18:34:34 STEP 2696 H=2696:00 +CPU= 19.049
  18:35:30 STEP 2697 H=2697:00 +CPU= 45.128
  18:36:06 STEP 2698 H=2698:00 +CPU= 29.120
  18:36:36 STEP 2699 H=2699:00 +CPU= 24.102
  18:37:25 STEP 2700 H=2700:00 +CPU= 37.807
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002700
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002700
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002700
  18:37:48 STEP 2701 H=2701:00 +CPU= 18.053
  18:38:12 STEP 2702 H=2702:00 +CPU= 19.065
  18:38:51 STEP 2703 H=2703:00 +CPU= 30.851
  18:39:19 STEP 2704 H=2704:00 +CPU= 22.167
  18:39:42 STEP 2705 H=2705:00 +CPU= 17.943
  18:40:32 STEP 2706 H=2706:00 +CPU= 38.478
  18:40:59 STEP 2707 H=2707:00 +CPU= 20.732
  18:41:29 STEP 2708 H=2708:00 +CPU= 23.818
  18:42:18 STEP 2709 H=2709:00 +CPU= 37.566
  18:42:48 STEP 2710 H=2710:00 +CPU= 22.687
  18:43:13 STEP 2711 H=2711:00 +CPU= 20.160
  18:44:01 STEP 2712 H=2712:00 +CPU= 38.228
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002712
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002712
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002712
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002688
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002688
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002688
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002700
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002700
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002700
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_112.zip
Uploading the intermediate file: upload_file_112.zip
  18:44:40 STEP 2713 H=2713:00 +CPU= 33.203
  18:45:05 STEP 2714 H=2714:00 +CPU= 20.490
  18:45:42 STEP 2715 H=2715:00 +CPU= 30.000
  18:46:10 STEP 2716 H=2716:00 +CPU= 22.317
  18:46:36 STEP 2717 H=2717:00 +CPU= 19.413
  18:47:32 STEP 2718 H=2718:00 +CPU= 39.824
  18:47:56 STEP 2719 H=2719:00 +CPU= 19.342
  18:48:23 STEP 2720 H=2720:00 +CPU= 21.227
  18:49:02 STEP 2721 H=2721:00 +CPU= 30.632
  18:49:30 STEP 2722 H=2722:00 +CPU= 23.573
  18:49:55 STEP 2723 H=2723:00 +CPU= 19.573
  18:50:36 STEP 2724 H=2724:00 +CPU= 33.728
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002724
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002724
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002724
  18:51:01 STEP 2725 H=2725:00 +CPU= 20.154
  18:51:29 STEP 2726 H=2726:00 +CPU= 23.699
  18:52:10 STEP 2727 H=2727:00 +CPU= 34.150
  18:52:34 STEP 2728 H=2728:00 +CPU= 18.418
  18:52:59 STEP 2729 H=2729:00 +CPU= 21.847
  18:53:46 STEP 2730 H=2730:00 +CPU= 39.601
  18:54:17 STEP 2731 H=2731:00 +CPU= 25.683
  18:54:44 STEP 2732 H=2732:00 +CPU= 22.136
  18:55:26 STEP 2733 H=2733:00 +CPU= 31.107
  18:55:48 STEP 2734 H=2734:00 +CPU= 18.224
  18:56:17 STEP 2735 H=2735:00 +CPU= 23.769
  18:57:05 STEP 2736 H=2736:00 +CPU= 36.394
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002736
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002736
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002736
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002712
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002712
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002712
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002724
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002724
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002724
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_113.zip
Uploading the intermediate file: upload_file_113.zip
  18:57:42 STEP 2737 H=2737:00 +CPU= 25.352
  18:58:10 STEP 2738 H=2738:00 +CPU= 21.143
  18:58:47 STEP 2739 H=2739:00 +CPU= 28.625
  18:59:14 STEP 2740 H=2740:00 +CPU= 23.488
  18:59:37 STEP 2741 H=2741:00 +CPU= 18.239
  19:00:19 STEP 2742 H=2742:00 +CPU= 32.104
  19:00:43 STEP 2743 H=2743:00 +CPU= 19.313
  19:01:16 STEP 2744 H=2744:00 +CPU= 23.984
  19:01:55 STEP 2745 H=2745:00 +CPU= 30.483
  19:02:21 STEP 2746 H=2746:00 +CPU= 19.979
  19:02:47 STEP 2747 H=2747:00 +CPU= 20.390
  19:05:13 STEP 2748 H=2748:00 +CPU=137.293
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002748
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002748
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002748
  19:06:31 STEP 2749 H=2749:00 +CPU= 56.504
  19:07:56 STEP 2750 H=2750:00 +CPU= 79.748
  19:10:26 STEP 2751 H=2751:00 +CPU=104.689
  19:10:52 STEP 2752 H=2752:00 +CPU= 19.629
  19:11:27 STEP 2753 H=2753:00 +CPU= 27.414
  19:12:07 STEP 2754 H=2754:00 +CPU= 31.415
  19:12:30 STEP 2755 H=2755:00 +CPU= 18.235
  19:12:55 STEP 2756 H=2756:00 +CPU= 19.359
  19:13:35 STEP 2757 H=2757:00 +CPU= 30.830
  19:14:13 STEP 2758 H=2758:00 +CPU= 31.166
  19:14:37 STEP 2759 H=2759:00 +CPU= 19.308
  19:15:22 STEP 2760 H=2760:00 +CPU= 36.347
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002760
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002760
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002760
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002736
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002736
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002736
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002748
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002748
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002748
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_114.zip
Uploading the intermediate file: upload_file_114.zip
  19:15:46 STEP 2761 H=2761:00 +CPU= 18.587
  19:16:13 STEP 2762 H=2762:00 +CPU= 21.267
  19:16:49 STEP 2763 H=2763:00 +CPU= 28.792
  19:17:30 STEP 2764 H=2764:00 +CPU= 35.315
  19:17:55 STEP 2765 H=2765:00 +CPU= 18.971
  19:18:36 STEP 2766 H=2766:00 +CPU= 33.184
  19:19:13 STEP 2767 H=2767:00 +CPU= 25.015
  19:19:45 STEP 2768 H=2768:00 +CPU= 21.092
  19:20:21 STEP 2769 H=2769:00 +CPU= 28.724
  19:20:47 STEP 2770 H=2770:00 +CPU= 18.821
  19:21:11 STEP 2771 H=2771:00 +CPU= 18.661
  19:21:53 STEP 2772 H=2772:00 +CPU= 33.759
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002772
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002772
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002772
  19:22:30 STEP 2773 H=2773:00 +CPU= 21.728
  19:22:56 STEP 2774 H=2774:00 +CPU= 21.289
  19:23:37 STEP 2775 H=2775:00 +CPU= 33.430
  19:24:06 STEP 2776 H=2776:00 +CPU= 24.621
  19:24:29 STEP 2777 H=2777:00 +CPU= 18.346
  19:25:20 STEP 2778 H=2778:00 +CPU= 43.226
  19:25:45 STEP 2779 H=2779:00 +CPU= 19.911
  19:26:10 STEP 2780 H=2780:00 +CPU= 20.000
  19:28:20 STEP 2781 H=2781:00 +CPU=107.584
  19:32:18 STEP 2782 H=2782:00 +CPU=191.985
  19:34:21 STEP 2783 H=2783:00 +CPU= 79.061
  19:36:54 STEP 2784 H=2784:00 +CPU=141.291
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002784
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002784
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002784
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002760
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002760
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002760
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002772
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002772
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002772
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_115.zip
Uploading the intermediate file: upload_file_115.zip
  19:37:30 STEP 2785 H=2785:00 +CPU= 32.266
  19:41:26 STEP 2786 H=2786:00 +CPU= 72.305
  19:42:15 STEP 2787 H=2787:00 +CPU= 40.632
  19:42:43 STEP 2788 H=2788:00 +CPU= 22.405
  19:44:51 STEP 2789 H=2789:00 +CPU=104.441
  19:45:26 STEP 2790 H=2790:00 +CPU= 30.099
  19:45:51 STEP 2791 H=2791:00 +CPU= 20.146
  19:47:25 STEP 2792 H=2792:00 +CPU= 48.717
  19:48:52 STEP 2793 H=2793:00 +CPU= 35.770
  19:49:19 STEP 2794 H=2794:00 +CPU= 19.627
  19:49:43 STEP 2795 H=2795:00 +CPU= 19.395
  19:50:26 STEP 2796 H=2796:00 +CPU= 32.387
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002796
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002796
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002796
  19:50:46 STEP 2797 H=2797:00 +CPU= 16.975
  19:51:13 STEP 2798 H=2798:00 +CPU= 21.305
  19:51:56 STEP 2799 H=2799:00 +CPU= 34.753
  19:52:21 STEP 2800 H=2800:00 +CPU= 20.911
  19:52:45 STEP 2801 H=2801:00 +CPU= 19.913
  19:54:27 STEP 2802 H=2802:00 +CPU= 69.418
  19:59:02 STEP 2803 H=2803:00 +CPU=170.536
  19:59:41 STEP 2804 H=2804:00 +CPU= 33.444
  20:00:22 STEP 2805 H=2805:00 +CPU= 32.359
  20:00:56 STEP 2806 H=2806:00 +CPU= 28.549
  20:01:31 STEP 2807 H=2807:00 +CPU= 23.606
  20:03:12 STEP 2808 H=2808:00 +CPU= 84.495
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002808
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002808
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002808
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002784
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002784
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002784
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002796
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002796
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002796
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_116.zip
Uploading the intermediate file: upload_file_116.zip
  20:03:42 STEP 2809 H=2809:00 +CPU= 19.382
  20:04:23 STEP 2810 H=2810:00 +CPU= 35.611
  20:05:41 STEP 2811 H=2811:00 +CPU= 51.320
  20:06:24 STEP 2812 H=2812:00 +CPU= 34.136
  20:08:33 STEP 2813 H=2813:00 +CPU= 81.544
  20:09:21 STEP 2814 H=2814:00 +CPU= 37.661
  20:09:56 STEP 2815 H=2815:00 +CPU= 30.007
  20:10:26 STEP 2816 H=2816:00 +CPU= 21.523
  20:11:04 STEP 2817 H=2817:00 +CPU= 30.230
  20:11:27 STEP 2818 H=2818:00 +CPU= 18.769
  20:12:01 STEP 2819 H=2819:00 +CPU= 30.090
  20:12:47 STEP 2820 H=2820:00 +CPU= 37.881
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002820
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002820
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002820
  20:13:10 STEP 2821 H=2821:00 +CPU= 18.041
  20:13:41 STEP 2822 H=2822:00 +CPU= 21.684
  20:14:23 STEP 2823 H=2823:00 +CPU= 34.800
  20:14:47 STEP 2824 H=2824:00 +CPU= 18.568
  20:15:24 STEP 2825 H=2825:00 +CPU= 34.303
  20:16:04 STEP 2826 H=2826:00 +CPU= 30.956
  20:16:36 STEP 2827 H=2827:00 +CPU= 26.912
  20:17:26 STEP 2828 H=2828:00 +CPU= 19.661
  20:18:13 STEP 2829 H=2829:00 +CPU= 39.410
  20:18:46 STEP 2830 H=2830:00 +CPU= 22.533
  20:19:25 STEP 2831 H=2831:00 +CPU= 32.318
  20:20:06 STEP 2832 H=2832:00 +CPU= 35.272
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002832
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002832
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002832
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002808
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002808
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002808
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002820
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002820
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002820
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_117.zip
Uploading the intermediate file: upload_file_117.zip
  20:20:47 STEP 2833 H=2833:00 +CPU= 36.387
  20:21:21 STEP 2834 H=2834:00 +CPU= 25.956
  20:23:22 STEP 2835 H=2835:00 +CPU= 98.310
  20:24:56 STEP 2836 H=2836:00 +CPU= 89.029
  20:25:22 STEP 2837 H=2837:00 +CPU= 20.765
  20:31:11 STEP 2838 H=2838:00 +CPU=341.468
  20:32:20 STEP 2839 H=2839:00 +CPU= 42.098
  20:32:46 STEP 2840 H=2840:00 +CPU= 20.031
  20:33:31 STEP 2841 H=2841:00 +CPU= 35.497
  20:34:00 STEP 2842 H=2842:00 +CPU= 24.323
  20:36:55 STEP 2843 H=2843:00 +CPU=137.917
  20:37:45 STEP 2844 H=2844:00 +CPU= 38.337
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002844
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002844
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002844
  20:40:11 STEP 2845 H=2845:00 +CPU=116.740
  20:40:36 STEP 2846 H=2846:00 +CPU= 20.086
  20:41:19 STEP 2847 H=2847:00 +CPU= 28.933
  20:41:42 STEP 2848 H=2848:00 +CPU= 18.349
  20:42:16 STEP 2849 H=2849:00 +CPU= 22.174
  20:42:55 STEP 2850 H=2850:00 +CPU= 30.739
  20:43:20 STEP 2851 H=2851:00 +CPU= 19.668
  20:43:48 STEP 2852 H=2852:00 +CPU= 20.976
  20:44:34 STEP 2853 H=2853:00 +CPU= 37.148
  20:45:04 STEP 2854 H=2854:00 +CPU= 24.384
  20:45:30 STEP 2855 H=2855:00 +CPU= 21.470
  20:46:32 STEP 2856 H=2856:00 +CPU= 49.653
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002856
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002856
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002856
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002832
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002832
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002832
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002844
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002844
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002844
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_118.zip
Uploading the intermediate file: upload_file_118.zip
  20:46:57 STEP 2857 H=2857:00 +CPU= 18.403
  20:50:11 STEP 2858 H=2858:00 +CPU=188.688
  20:50:53 STEP 2859 H=2859:00 +CPU= 32.545
  20:51:18 STEP 2860 H=2860:00 +CPU= 18.397
  20:51:41 STEP 2861 H=2861:00 +CPU= 17.521
  20:52:27 STEP 2862 H=2862:00 +CPU= 38.064
  20:52:54 STEP 2863 H=2863:00 +CPU= 21.143
  20:53:17 STEP 2864 H=2864:00 +CPU= 17.627
  20:55:03 STEP 2865 H=2865:00 +CPU= 87.319
  20:55:30 STEP 2866 H=2866:00 +CPU= 21.279
  20:55:55 STEP 2867 H=2867:00 +CPU= 19.999
  20:56:44 STEP 2868 H=2868:00 +CPU= 36.092
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002868
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002868
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002868
  20:57:11 STEP 2869 H=2869:00 +CPU= 22.245
  20:57:38 STEP 2870 H=2870:00 +CPU= 19.374
  20:58:26 STEP 2871 H=2871:00 +CPU= 39.636
  20:59:08 STEP 2872 H=2872:00 +CPU= 24.268
  20:59:35 STEP 2873 H=2873:00 +CPU= 20.772
  21:00:32 STEP 2874 H=2874:00 +CPU= 45.405
  21:00:58 STEP 2875 H=2875:00 +CPU= 20.515
  21:01:22 STEP 2876 H=2876:00 +CPU= 18.544
  21:03:01 STEP 2877 H=2877:00 +CPU= 87.910
  21:03:38 STEP 2878 H=2878:00 +CPU= 33.831
  21:04:05 STEP 2879 H=2879:00 +CPU= 20.618
  21:06:22 STEP 2880 H=2880:00 +CPU=121.549
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002880
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002880
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002880
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002856
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002856
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002856
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002868
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002868
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002868
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_119.zip
Uploading the intermediate file: upload_file_119.zip
  21:06:46 STEP 2881 H=2881:00 +CPU= 18.732
Uploading trickle at timestep: 10368000
  21:07:13 STEP 2882 H=2882:00 +CPU= 24.789
  21:07:48 STEP 2883 H=2883:00 +CPU= 27.906
  21:09:17 STEP 2884 H=2884:00 +CPU= 84.739
  21:09:45 STEP 2885 H=2885:00 +CPU= 24.084
  21:10:18 STEP 2886 H=2886:00 +CPU= 27.025
  21:11:55 STEP 2887 H=2887:00 +CPU= 49.467
  21:12:18 STEP 2888 H=2888:00 +CPU= 18.094
  21:12:49 STEP 2889 H=2889:00 +CPU= 26.067
  21:13:11 STEP 2890 H=2890:00 +CPU= 17.702
  21:13:37 STEP 2891 H=2891:00 +CPU= 22.551
  21:16:46 STEP 2892 H=2892:00 +CPU=108.317
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002892
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002892
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002892
  21:17:18 STEP 2893 H=2893:00 +CPU= 22.089
  21:17:40 STEP 2894 H=2894:00 +CPU= 18.564
  21:18:26 STEP 2895 H=2895:00 +CPU= 41.260
  21:20:48 STEP 2896 H=2896:00 +CPU=137.550
  21:24:17 STEP 2897 H=2897:00 +CPU=141.803
  21:29:11 STEP 2898 H=2898:00 +CPU=234.471
  21:30:00 STEP 2899 H=2899:00 +CPU= 38.113
  21:30:28 STEP 2900 H=2900:00 +CPU= 16.793
  21:31:59 STEP 2901 H=2901:00 +CPU= 86.060
  21:34:20 STEP 2902 H=2902:00 +CPU=113.232
  21:35:41 STEP 2903 H=2903:00 +CPU= 47.276
  21:36:17 STEP 2904 H=2904:00 +CPU= 30.226
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002904
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002904
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002904
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002880
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002880
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002880
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002892
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002892
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002892
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_120.zip
Uploading the intermediate file: upload_file_120.zip
  21:36:36 STEP 2905 H=2905:00 +CPU= 15.649
  21:38:13 STEP 2906 H=2906:00 +CPU= 79.975
  21:39:19 STEP 2907 H=2907:00 +CPU= 54.775
  21:39:39 STEP 2908 H=2908:00 +CPU= 16.288
  21:40:59 STEP 2909 H=2909:00 +CPU= 64.888
  21:42:11 STEP 2910 H=2910:00 +CPU= 35.724
  21:43:05 STEP 2911 H=2911:00 +CPU= 45.474
  21:43:31 STEP 2912 H=2912:00 +CPU= 22.467
  21:44:10 STEP 2913 H=2913:00 +CPU= 32.017
  21:44:45 STEP 2914 H=2914:00 +CPU= 29.556
  21:45:06 STEP 2915 H=2915:00 +CPU= 16.543
  21:45:41 STEP 2916 H=2916:00 +CPU= 27.970
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002916
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002916
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002916
  21:46:07 STEP 2917 H=2917:00 +CPU= 21.563
  21:46:28 STEP 2918 H=2918:00 +CPU= 15.808
  21:47:37 STEP 2919 H=2919:00 +CPU= 55.113
  21:49:14 STEP 2920 H=2920:00 +CPU= 92.569
  21:50:10 STEP 2921 H=2921:00 +CPU= 40.097
  21:51:47 STEP 2922 H=2922:00 +CPU= 76.078
  21:52:08 STEP 2923 H=2923:00 +CPU= 16.524
  21:52:31 STEP 2924 H=2924:00 +CPU= 16.794
  21:53:03 STEP 2925 H=2925:00 +CPU= 25.618
  21:53:23 STEP 2926 H=2926:00 +CPU= 15.584
  21:54:24 STEP 2927 H=2927:00 +CPU= 29.745
  21:57:06 STEP 2928 H=2928:00 +CPU=123.906
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002928
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002928
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002928
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002904
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002904
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002904
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002916
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002916
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002916
Zipping up the intermediate file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_121.zip
Uploading the intermediate file: upload_file_121.zip
  21:57:46 STEP 2929 H=2929:00 +CPU= 25.459
  21:59:11 STEP 2930 H=2930:00 +CPU= 81.096
  22:00:25 STEP 2931 H=2931:00 +CPU= 56.086
  22:01:00 STEP 2932 H=2932:00 +CPU= 27.127
  22:01:16 STEP 2933 H=2933:00 +CPU= 14.329
  22:01:54 STEP 2934 H=2934:00 +CPU= 31.063
  22:02:15 STEP 2935 H=2935:00 +CPU= 15.814
  22:02:44 STEP 2936 H=2936:00 +CPU= 22.996
  22:03:17 STEP 2937 H=2937:00 +CPU= 24.865
  22:03:40 STEP 2938 H=2938:00 +CPU= 16.430
  22:04:15 STEP 2939 H=2939:00 +CPU= 32.167
  22:05:20 STEP 2940 H=2940:00 +CPU= 39.954
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002940
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002940
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002940
  22:05:41 STEP 2941 H=2941:00 +CPU= 15.791
  22:06:39 STEP 2942 H=2942:00 +CPU= 50.122
  22:07:42 STEP 2943 H=2943:00 +CPU= 60.113
  22:08:48 STEP 2944 H=2944:00 +CPU= 31.783
  22:10:40 STEP 2945 H=2945:00 +CPU= 89.570
  22:14:18 STEP 2946 H=2946:00 +CPU=161.240
  22:14:49 STEP 2947 H=2947:00 +CPU= 21.240
  22:15:44 STEP 2948 H=2948:00 +CPU= 54.598
  22:16:18 STEP 2949 H=2949:00 +CPU= 25.954
  22:16:39 STEP 2950 H=2950:00 +CPU= 16.290
  22:17:01 STEP 2951 H=2951:00 +CPU= 17.664
  22:17:43 STEP 2952 H=2952:00 +CPU= 34.352
The child process terminated with status: 0
Moving to projects directory: /home/roland/slots/0/ICMGGhq0f+002952
Moving to projects directory: /home/roland/slots/0/ICMSHhq0f+002952
Moving to projects directory: /home/roland/slots/0/ICMUAhq0f+002952
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002952
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002940
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002928
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002952
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002940
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMUAhq0f+002928
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMGGhq0f+002952
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002940
Adding to the zip: /home/roland/projects/climateprediction.net/oifs_43r3_ps_12179634/ICMSHhq0f+002928
Zipping up the final file: /home/roland/projects/climateprediction.net/oifs_43r3_ps_0990_1993050100_123_962_12179634_0_r595671008_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
22:21:21 (1046): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:48:15 1517859 22264553 oifs_43r3_ps_0990_1993050100_123_962_12179634_0 6,048,000 39,709 0.0066
09 May 2023 15:48:10 1517859 22264553 oifs_43r3_ps_0990_1993050100_123_962_12179634_0 5,184,000 34,270 0.0066
09 May 2023 15:47:59 1517859 22264553 oifs_43r3_ps_0990_1993050100_123_962_12179634_0 4,320,000 28,926 0.0067
09 May 2023 15:47:53 1517859 22264553 oifs_43r3_ps_0990_1993050100_123_962_12179634_0 3,456,000 23,738 0.0069
09 May 2023 15:47:42 1517859 22264553 oifs_43r3_ps_0990_1993050100_123_962_12179634_0 2,592,000 18,072 0.0070
09 May 2023 15:47:35 1517859 22264553 oifs_43r3_ps_0990_1993050100_123_962_12179634_0 1,728,000 12,088 0.0070
09 May 2023 15:47:30 1517859 22264553 oifs_43r3_ps_0990_1993050100_123_962_12179634_0 864,000 6,008 0.0070


©2024 climateprediction.net