climateprediction.net home page
Task 22265257

Task 22265257

Name oifs_43r3_ps_0655_1994050100_123_963_12180299_0
Workunit 12180299
Created 22 Dec 2022, 6:18:27 UTC
Sent 26 Dec 2022, 7:49:30 UTC
Report deadline 25 Jan 2023, 7:49:30 UTC
Received 4 Jan 2023, 22:30:26 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1443559
Run time 1 days 0 hours 42 min 15 sec
CPU time 23 hours 34 min 48 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 4.39 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,292.59 MB
Peak swap size 4,977.84 MB
Peak disk usage 3,748.14 MB
Stderr 
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<stderr_txt>
0f+002400
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_99.zip
Uploading the intermediate file: upload_file_99.zip
  07:23:34 STEP 2401 H=2401:00 +CPU= 17.871
Uploading trickle at timestep: 8640000
  07:23:53 STEP 2402 H=2402:00 +CPU= 19.198
  07:24:24 STEP 2403 H=2403:00 +CPU= 30.608
  07:24:44 STEP 2404 H=2404:00 +CPU= 20.144
  07:25:05 STEP 2405 H=2405:00 +CPU= 21.111
  07:25:32 STEP 2406 H=2406:00 +CPU= 27.058
  07:25:50 STEP 2407 H=2407:00 +CPU= 17.170
  07:26:07 STEP 2408 H=2408:00 +CPU= 17.675
  07:26:34 STEP 2409 H=2409:00 +CPU= 26.499
  07:26:51 STEP 2410 H=2410:00 +CPU= 17.393
  07:27:08 STEP 2411 H=2411:00 +CPU= 16.996
  07:27:38 STEP 2412 H=2412:00 +CPU= 29.362
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002412
  07:27:55 STEP 2413 H=2413:00 +CPU= 17.267
  07:28:13 STEP 2414 H=2414:00 +CPU= 17.721
  07:28:40 STEP 2415 H=2415:00 +CPU= 26.820
  07:28:58 STEP 2416 H=2416:00 +CPU= 17.874
  07:29:15 STEP 2417 H=2417:00 +CPU= 17.160
  07:29:42 STEP 2418 H=2418:00 +CPU= 27.106
  07:30:00 STEP 2419 H=2419:00 +CPU= 17.312
  07:30:25 STEP 2420 H=2420:00 +CPU= 24.919
  07:30:56 STEP 2421 H=2421:00 +CPU= 31.559
  07:31:14 STEP 2422 H=2422:00 +CPU= 17.428
  07:31:31 STEP 2423 H=2423:00 +CPU= 17.835
  07:32:16 STEP 2424 H=2424:00 +CPU= 43.798
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_100.zip
Uploading the intermediate file: upload_file_100.zip
  07:32:41 STEP 2425 H=2425:00 +CPU= 24.627
  07:33:06 STEP 2426 H=2426:00 +CPU= 25.325
  07:33:44 STEP 2427 H=2427:00 +CPU= 37.851
  07:34:10 STEP 2428 H=2428:00 +CPU= 25.551
  07:34:34 STEP 2429 H=2429:00 +CPU= 24.122
  07:35:12 STEP 2430 H=2430:00 +CPU= 38.679
  07:35:37 STEP 2431 H=2431:00 +CPU= 24.313
  07:36:02 STEP 2432 H=2432:00 +CPU= 25.308
  07:36:40 STEP 2433 H=2433:00 +CPU= 37.839
  07:37:04 STEP 2434 H=2434:00 +CPU= 24.022
  07:37:22 STEP 2435 H=2435:00 +CPU= 17.673
  07:37:51 STEP 2436 H=2436:00 +CPU= 29.291
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002436
  07:38:09 STEP 2437 H=2437:00 +CPU= 17.478
  07:38:26 STEP 2438 H=2438:00 +CPU= 17.246
  07:38:53 STEP 2439 H=2439:00 +CPU= 27.231
  07:39:12 STEP 2440 H=2440:00 +CPU= 18.863
  07:39:36 STEP 2441 H=2441:00 +CPU= 24.058
  07:40:14 STEP 2442 H=2442:00 +CPU= 37.652
  07:40:31 STEP 2443 H=2443:00 +CPU= 17.028
  07:40:49 STEP 2444 H=2444:00 +CPU= 17.418
  07:41:15 STEP 2445 H=2445:00 +CPU= 26.377
  07:41:33 STEP 2446 H=2446:00 +CPU= 17.752
  07:41:51 STEP 2447 H=2447:00 +CPU= 17.916
  07:42:35 STEP 2448 H=2448:00 +CPU= 43.718
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_101.zip
Uploading the intermediate file: upload_file_101.zip
  07:42:55 STEP 2449 H=2449:00 +CPU= 20.008
  07:43:13 STEP 2450 H=2450:00 +CPU= 17.772
  07:43:41 STEP 2451 H=2451:00 +CPU= 27.857
  07:43:59 STEP 2452 H=2452:00 +CPU= 17.995
  07:44:21 STEP 2453 H=2453:00 +CPU= 22.408
  07:45:00 STEP 2454 H=2454:00 +CPU= 38.501
  07:45:18 STEP 2455 H=2455:00 +CPU= 18.394
  07:45:44 STEP 2456 H=2456:00 +CPU= 25.045
  07:46:20 STEP 2457 H=2457:00 +CPU= 36.661
  07:46:39 STEP 2458 H=2458:00 +CPU= 18.556
  07:46:57 STEP 2459 H=2459:00 +CPU= 17.700
  07:47:33 STEP 2460 H=2460:00 +CPU= 36.444
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002460
  07:47:58 STEP 2461 H=2461:00 +CPU= 24.230
  07:48:23 STEP 2462 H=2462:00 +CPU= 25.443
  07:48:59 STEP 2463 H=2463:00 +CPU= 35.508
  07:49:17 STEP 2464 H=2464:00 +CPU= 17.994
  07:49:33 STEP 2465 H=2465:00 +CPU= 16.861
  07:50:02 STEP 2466 H=2466:00 +CPU= 28.773
  07:50:25 STEP 2467 H=2467:00 +CPU= 22.662
  07:50:49 STEP 2468 H=2468:00 +CPU= 23.916
  07:51:23 STEP 2469 H=2469:00 +CPU= 34.300
  07:51:41 STEP 2470 H=2470:00 +CPU= 17.927
  07:52:03 STEP 2471 H=2471:00 +CPU= 21.426
  07:52:44 STEP 2472 H=2472:00 +CPU= 40.370
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_102.zip
Uploading the intermediate file: upload_file_102.zip
  07:53:08 STEP 2473 H=2473:00 +CPU= 23.930
  07:53:33 STEP 2474 H=2474:00 +CPU= 25.310
  07:54:07 STEP 2475 H=2475:00 +CPU= 33.453
  07:54:26 STEP 2476 H=2476:00 +CPU= 18.979
  07:54:45 STEP 2477 H=2477:00 +CPU= 19.571
  07:55:12 STEP 2478 H=2478:00 +CPU= 27.017
  07:55:30 STEP 2479 H=2479:00 +CPU= 18.046
  07:55:49 STEP 2480 H=2480:00 +CPU= 18.470
  07:56:15 STEP 2481 H=2481:00 +CPU= 26.405
  07:56:33 STEP 2482 H=2482:00 +CPU= 18.010
  07:56:50 STEP 2483 H=2483:00 +CPU= 17.035
  07:57:20 STEP 2484 H=2484:00 +CPU= 29.276
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002484
  07:57:37 STEP 2485 H=2485:00 +CPU= 17.167
  07:57:55 STEP 2486 H=2486:00 +CPU= 17.636
  07:58:21 STEP 2487 H=2487:00 +CPU= 26.451
  07:58:40 STEP 2488 H=2488:00 +CPU= 18.420
  07:58:57 STEP 2489 H=2489:00 +CPU= 17.343
  07:59:26 STEP 2490 H=2490:00 +CPU= 28.882
  07:59:48 STEP 2491 H=2491:00 +CPU= 22.677
  08:00:14 STEP 2492 H=2492:00 +CPU= 25.257
  08:00:50 STEP 2493 H=2493:00 +CPU= 35.919
  08:01:13 STEP 2494 H=2494:00 +CPU= 23.548
  08:01:38 STEP 2495 H=2495:00 +CPU= 24.781
  08:02:21 STEP 2496 H=2496:00 +CPU= 42.481
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_103.zip
Uploading the intermediate file: upload_file_103.zip
  08:02:44 STEP 2497 H=2497:00 +CPU= 22.846
  08:03:07 STEP 2498 H=2498:00 +CPU= 23.432
  08:03:42 STEP 2499 H=2499:00 +CPU= 35.099
  08:04:04 STEP 2500 H=2500:00 +CPU= 21.221
  08:04:24 STEP 2501 H=2501:00 +CPU= 19.996
  08:05:01 STEP 2502 H=2502:00 +CPU= 36.763
  08:05:24 STEP 2503 H=2503:00 +CPU= 23.726
  08:05:49 STEP 2504 H=2504:00 +CPU= 25.010
  08:06:27 STEP 2505 H=2505:00 +CPU= 37.798
  08:06:52 STEP 2506 H=2506:00 +CPU= 25.289
  08:07:13 STEP 2507 H=2507:00 +CPU= 20.791
  08:07:44 STEP 2508 H=2508:00 +CPU= 30.401
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002508
  08:08:06 STEP 2509 H=2509:00 +CPU= 22.455
  08:08:31 STEP 2510 H=2510:00 +CPU= 24.993
  08:09:09 STEP 2511 H=2511:00 +CPU= 37.687
  08:09:34 STEP 2512 H=2512:00 +CPU= 25.232
  08:09:59 STEP 2513 H=2513:00 +CPU= 24.430
  08:10:38 STEP 2514 H=2514:00 +CPU= 38.911
  08:11:02 STEP 2515 H=2515:00 +CPU= 24.381
  08:11:27 STEP 2516 H=2516:00 +CPU= 25.032
  08:12:03 STEP 2517 H=2517:00 +CPU= 35.615
  08:12:27 STEP 2518 H=2518:00 +CPU= 23.453
  08:12:50 STEP 2519 H=2519:00 +CPU= 23.724
  08:13:35 STEP 2520 H=2520:00 +CPU= 44.441
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_104.zip
Uploading the intermediate file: upload_file_104.zip
  08:13:59 STEP 2521 H=2521:00 +CPU= 24.479
  08:14:23 STEP 2522 H=2522:00 +CPU= 23.232
  08:15:00 STEP 2523 H=2523:00 +CPU= 37.359
  08:15:26 STEP 2524 H=2524:00 +CPU= 25.410
  08:15:50 STEP 2525 H=2525:00 +CPU= 24.592
  08:16:25 STEP 2526 H=2526:00 +CPU= 34.452
  08:16:42 STEP 2527 H=2527:00 +CPU= 17.075
  08:17:00 STEP 2528 H=2528:00 +CPU= 17.820
  08:17:27 STEP 2529 H=2529:00 +CPU= 26.812
  08:17:44 STEP 2530 H=2530:00 +CPU= 17.634
  08:18:02 STEP 2531 H=2531:00 +CPU= 17.052
  08:18:32 STEP 2532 H=2532:00 +CPU= 30.781
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002532
  08:18:50 STEP 2533 H=2533:00 +CPU= 17.537
  08:19:15 STEP 2534 H=2534:00 +CPU= 25.057
  08:19:53 STEP 2535 H=2535:00 +CPU= 37.781
  08:20:14 STEP 2536 H=2536:00 +CPU= 20.706
  08:20:31 STEP 2537 H=2537:00 +CPU= 16.999
  08:20:58 STEP 2538 H=2538:00 +CPU= 26.625
  08:21:15 STEP 2539 H=2539:00 +CPU= 17.460
  08:21:39 STEP 2540 H=2540:00 +CPU= 23.688
  08:22:17 STEP 2541 H=2541:00 +CPU= 38.187
  08:22:42 STEP 2542 H=2542:00 +CPU= 25.121
  08:23:07 STEP 2543 H=2543:00 +CPU= 25.368
  08:23:42 STEP 2544 H=2544:00 +CPU= 34.302
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_105.zip
Uploading the intermediate file: upload_file_105.zip
  08:24:00 STEP 2545 H=2545:00 +CPU= 17.778
  08:24:17 STEP 2546 H=2546:00 +CPU= 17.643
  08:24:44 STEP 2547 H=2547:00 +CPU= 26.661
  08:25:02 STEP 2548 H=2548:00 +CPU= 17.720
  08:25:19 STEP 2549 H=2549:00 +CPU= 16.817
  08:25:46 STEP 2550 H=2550:00 +CPU= 27.309
  08:26:03 STEP 2551 H=2551:00 +CPU= 17.419
  08:26:27 STEP 2552 H=2552:00 +CPU= 23.370
  08:27:03 STEP 2553 H=2553:00 +CPU= 36.052
  08:27:27 STEP 2554 H=2554:00 +CPU= 23.833
  08:27:49 STEP 2555 H=2555:00 +CPU= 22.494
  08:28:18 STEP 2556 H=2556:00 +CPU= 29.025
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002556
  08:28:36 STEP 2557 H=2557:00 +CPU= 17.909
  08:29:00 STEP 2558 H=2558:00 +CPU= 23.486
  08:29:35 STEP 2559 H=2559:00 +CPU= 35.594
  08:29:54 STEP 2560 H=2560:00 +CPU= 18.805
  08:30:11 STEP 2561 H=2561:00 +CPU= 16.992
  08:30:38 STEP 2562 H=2562:00 +CPU= 27.061
  08:30:56 STEP 2563 H=2563:00 +CPU= 17.026
  08:31:14 STEP 2564 H=2564:00 +CPU= 17.925
  08:31:40 STEP 2565 H=2565:00 +CPU= 26.394
  08:31:58 STEP 2566 H=2566:00 +CPU= 17.820
  08:32:17 STEP 2567 H=2567:00 +CPU= 19.026
  08:32:48 STEP 2568 H=2568:00 +CPU= 31.281
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_106.zip
Uploading the intermediate file: upload_file_106.zip
  08:33:07 STEP 2569 H=2569:00 +CPU= 19.200
  08:33:25 STEP 2570 H=2570:00 +CPU= 17.815
  08:33:57 STEP 2571 H=2571:00 +CPU= 31.342
  08:34:20 STEP 2572 H=2572:00 +CPU= 23.073
  08:34:42 STEP 2573 H=2573:00 +CPU= 22.348
  08:35:20 STEP 2574 H=2574:00 +CPU= 38.048
  08:35:39 STEP 2575 H=2575:00 +CPU= 18.932
  08:36:04 STEP 2576 H=2576:00 +CPU= 24.772
  08:36:39 STEP 2577 H=2577:00 +CPU= 35.357
  08:37:00 STEP 2578 H=2578:00 +CPU= 20.776
  08:37:23 STEP 2579 H=2579:00 +CPU= 22.644
  08:38:04 STEP 2580 H=2580:00 +CPU= 40.960
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002580
  08:38:23 STEP 2581 H=2581:00 +CPU= 19.434
  08:38:42 STEP 2582 H=2582:00 +CPU= 18.382
  08:39:09 STEP 2583 H=2583:00 +CPU= 27.142
  08:39:27 STEP 2584 H=2584:00 +CPU= 17.490
  08:39:44 STEP 2585 H=2585:00 +CPU= 17.094
  08:40:11 STEP 2586 H=2586:00 +CPU= 27.287
  08:40:28 STEP 2587 H=2587:00 +CPU= 17.101
  08:40:46 STEP 2588 H=2588:00 +CPU= 17.635
  08:41:27 STEP 2589 H=2589:00 +CPU= 39.713
  08:41:50 STEP 2590 H=2590:00 +CPU= 22.408
  08:42:07 STEP 2591 H=2591:00 +CPU= 17.551
  08:42:50 STEP 2592 H=2592:00 +CPU= 42.927
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_107.zip
Uploading the intermediate file: upload_file_107.zip
  08:43:12 STEP 2593 H=2593:00 +CPU= 22.302
  08:43:32 STEP 2594 H=2594:00 +CPU= 19.386
  08:44:11 STEP 2595 H=2595:00 +CPU= 38.048
  08:44:32 STEP 2596 H=2596:00 +CPU= 20.650
  08:44:52 STEP 2597 H=2597:00 +CPU= 20.334
  08:45:34 STEP 2598 H=2598:00 +CPU= 41.426
  08:46:00 STEP 2599 H=2599:00 +CPU= 24.857
  08:46:23 STEP 2600 H=2600:00 +CPU= 23.007
  08:47:02 STEP 2601 H=2601:00 +CPU= 37.653
  08:47:29 STEP 2602 H=2602:00 +CPU= 27.232
  08:47:56 STEP 2603 H=2603:00 +CPU= 26.126
  08:48:38 STEP 2604 H=2604:00 +CPU= 41.667
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002604
  08:49:02 STEP 2605 H=2605:00 +CPU= 24.140
  08:49:25 STEP 2606 H=2606:00 +CPU= 22.857
  08:49:52 STEP 2607 H=2607:00 +CPU= 26.765
  08:50:11 STEP 2608 H=2608:00 +CPU= 19.084
  08:50:34 STEP 2609 H=2609:00 +CPU= 23.096
  08:51:12 STEP 2610 H=2610:00 +CPU= 38.025
  08:51:35 STEP 2611 H=2611:00 +CPU= 22.965
  08:52:00 STEP 2612 H=2612:00 +CPU= 24.236
  08:52:36 STEP 2613 H=2613:00 +CPU= 35.302
  08:52:59 STEP 2614 H=2614:00 +CPU= 22.955
  08:53:22 STEP 2615 H=2615:00 +CPU= 22.352
  08:54:04 STEP 2616 H=2616:00 +CPU= 42.233
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_108.zip
Uploading the intermediate file: upload_file_108.zip
  08:54:28 STEP 2617 H=2617:00 +CPU= 23.906
  08:54:52 STEP 2618 H=2618:00 +CPU= 23.418
  08:55:26 STEP 2619 H=2619:00 +CPU= 34.481
  08:55:49 STEP 2620 H=2620:00 +CPU= 22.443
  08:56:06 STEP 2621 H=2621:00 +CPU= 17.039
  08:56:42 STEP 2622 H=2622:00 +CPU= 35.332
  08:56:59 STEP 2623 H=2623:00 +CPU= 17.134
  08:57:20 STEP 2624 H=2624:00 +CPU= 21.425
  08:57:54 STEP 2625 H=2625:00 +CPU= 33.532
  08:58:16 STEP 2626 H=2626:00 +CPU= 21.946
  08:58:38 STEP 2627 H=2627:00 +CPU= 22.047
  08:59:18 STEP 2628 H=2628:00 +CPU= 39.053
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002628
  08:59:42 STEP 2629 H=2629:00 +CPU= 24.203
  09:00:07 STEP 2630 H=2630:00 +CPU= 24.748
  09:00:41 STEP 2631 H=2631:00 +CPU= 33.371
  09:01:00 STEP 2632 H=2632:00 +CPU= 18.588
  09:01:26 STEP 2633 H=2633:00 +CPU= 25.803
  09:01:58 STEP 2634 H=2634:00 +CPU= 31.346
  09:02:17 STEP 2635 H=2635:00 +CPU= 18.970
  09:02:41 STEP 2636 H=2636:00 +CPU= 22.670
  09:03:20 STEP 2637 H=2637:00 +CPU= 36.669
  09:03:39 STEP 2638 H=2638:00 +CPU= 19.081
  09:04:01 STEP 2639 H=2639:00 +CPU= 21.672
  09:04:34 STEP 2640 H=2640:00 +CPU= 32.558
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_109.zip
Uploading the intermediate file: upload_file_109.zip
  09:04:55 STEP 2641 H=2641:00 +CPU= 21.104
Uploading trickle at timestep: 9504000
  09:05:18 STEP 2642 H=2642:00 +CPU= 22.616
  09:05:57 STEP 2643 H=2643:00 +CPU= 37.976
  09:06:21 STEP 2644 H=2644:00 +CPU= 23.651
  09:06:45 STEP 2645 H=2645:00 +CPU= 24.315
  09:07:23 STEP 2646 H=2646:00 +CPU= 37.247
  09:07:47 STEP 2647 H=2647:00 +CPU= 23.789
  09:08:11 STEP 2648 H=2648:00 +CPU= 23.197
  09:08:51 STEP 2649 H=2649:00 +CPU= 39.402
  09:09:13 STEP 2650 H=2650:00 +CPU= 21.541
  09:09:34 STEP 2651 H=2651:00 +CPU= 21.090
  09:10:08 STEP 2652 H=2652:00 +CPU= 32.988
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002652
  09:10:31 STEP 2653 H=2653:00 +CPU= 22.872
  09:10:53 STEP 2654 H=2654:00 +CPU= 21.908
  09:11:29 STEP 2655 H=2655:00 +CPU= 34.917
  09:11:53 STEP 2656 H=2656:00 +CPU= 23.943
  09:12:15 STEP 2657 H=2657:00 +CPU= 22.059
  09:12:54 STEP 2658 H=2658:00 +CPU= 38.362
  09:13:15 STEP 2659 H=2659:00 +CPU= 21.302
  09:13:40 STEP 2660 H=2660:00 +CPU= 24.048
  09:14:16 STEP 2661 H=2661:00 +CPU= 36.133
  09:14:41 STEP 2662 H=2662:00 +CPU= 24.924
  09:15:07 STEP 2663 H=2663:00 +CPU= 25.889
  09:15:48 STEP 2664 H=2664:00 +CPU= 41.154
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_110.zip
Uploading the intermediate file: upload_file_110.zip
  09:16:09 STEP 2665 H=2665:00 +CPU= 20.569
  09:16:28 STEP 2666 H=2666:00 +CPU= 18.944
  09:17:05 STEP 2667 H=2667:00 +CPU= 36.768
  09:17:26 STEP 2668 H=2668:00 +CPU= 20.755
  09:17:45 STEP 2669 H=2669:00 +CPU= 18.804
  09:18:22 STEP 2670 H=2670:00 +CPU= 37.615
  09:18:45 STEP 2671 H=2671:00 +CPU= 23.044
  09:19:07 STEP 2672 H=2672:00 +CPU= 21.545
  09:19:45 STEP 2673 H=2673:00 +CPU= 37.961
  09:20:11 STEP 2674 H=2674:00 +CPU= 24.877
  09:20:36 STEP 2675 H=2675:00 +CPU= 24.767
  09:21:13 STEP 2676 H=2676:00 +CPU= 36.269
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002676
  09:21:35 STEP 2677 H=2677:00 +CPU= 21.863
  09:21:59 STEP 2678 H=2678:00 +CPU= 23.972
  09:22:34 STEP 2679 H=2679:00 +CPU= 34.174
  09:22:52 STEP 2680 H=2680:00 +CPU= 18.528
  09:23:15 STEP 2681 H=2681:00 +CPU= 23.187
  09:23:52 STEP 2682 H=2682:00 +CPU= 36.318
  09:24:17 STEP 2683 H=2683:00 +CPU= 24.952
  09:24:36 STEP 2684 H=2684:00 +CPU= 19.349
  09:25:12 STEP 2685 H=2685:00 +CPU= 35.405
  09:25:35 STEP 2686 H=2686:00 +CPU= 22.478
  09:25:53 STEP 2687 H=2687:00 +CPU= 17.996
  09:26:31 STEP 2688 H=2688:00 +CPU= 38.239
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_111.zip
Uploading the intermediate file: upload_file_111.zip
  09:26:54 STEP 2689 H=2689:00 +CPU= 22.913
  09:27:16 STEP 2690 H=2690:00 +CPU= 21.608
  09:27:42 STEP 2691 H=2691:00 +CPU= 26.043
  09:28:02 STEP 2692 H=2692:00 +CPU= 19.774
  09:28:22 STEP 2693 H=2693:00 +CPU= 19.650
  09:28:56 STEP 2694 H=2694:00 +CPU= 33.921
  09:29:18 STEP 2695 H=2695:00 +CPU= 21.853
  09:29:42 STEP 2696 H=2696:00 +CPU= 23.476
  09:30:17 STEP 2697 H=2697:00 +CPU= 35.530
  09:30:42 STEP 2698 H=2698:00 +CPU= 24.769
  09:31:07 STEP 2699 H=2699:00 +CPU= 24.620
  09:31:46 STEP 2700 H=2700:00 +CPU= 39.191
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002700
  09:32:10 STEP 2701 H=2701:00 +CPU= 23.156
  09:32:32 STEP 2702 H=2702:00 +CPU= 22.873
  09:33:09 STEP 2703 H=2703:00 +CPU= 36.551
  09:33:34 STEP 2704 H=2704:00 +CPU= 25.174
  09:33:55 STEP 2705 H=2705:00 +CPU= 20.730
  09:34:24 STEP 2706 H=2706:00 +CPU= 28.106
  09:34:43 STEP 2707 H=2707:00 +CPU= 18.924
  09:35:01 STEP 2708 H=2708:00 +CPU= 18.489
  09:35:28 STEP 2709 H=2709:00 +CPU= 26.696
  09:35:47 STEP 2710 H=2710:00 +CPU= 18.372
  09:36:07 STEP 2711 H=2711:00 +CPU= 19.878
  09:36:49 STEP 2712 H=2712:00 +CPU= 41.898
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_112.zip
Uploading the intermediate file: upload_file_112.zip
  09:37:12 STEP 2713 H=2713:00 +CPU= 22.477
  09:37:37 STEP 2714 H=2714:00 +CPU= 24.503
  09:38:13 STEP 2715 H=2715:00 +CPU= 35.391
  09:38:36 STEP 2716 H=2716:00 +CPU= 23.066
  09:38:59 STEP 2717 H=2717:00 +CPU= 22.663
  09:39:37 STEP 2718 H=2718:00 +CPU= 37.619
  09:39:56 STEP 2719 H=2719:00 +CPU= 19.456
  09:40:17 STEP 2720 H=2720:00 +CPU= 21.023
  09:40:46 STEP 2721 H=2721:00 +CPU= 28.360
  09:41:10 STEP 2722 H=2722:00 +CPU= 23.851
  09:41:31 STEP 2723 H=2723:00 +CPU= 21.050
  09:42:09 STEP 2724 H=2724:00 +CPU= 37.865
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002724
  09:42:31 STEP 2725 H=2725:00 +CPU= 21.479
  09:42:50 STEP 2726 H=2726:00 +CPU= 18.537
  09:43:26 STEP 2727 H=2727:00 +CPU= 36.200
  09:43:48 STEP 2728 H=2728:00 +CPU= 21.145
  09:44:07 STEP 2729 H=2729:00 +CPU= 18.807
  09:44:48 STEP 2730 H=2730:00 +CPU= 38.921
  09:45:09 STEP 2731 H=2731:00 +CPU= 20.552
  09:45:34 STEP 2732 H=2732:00 +CPU= 25.054
  09:46:07 STEP 2733 H=2733:00 +CPU= 33.265
  09:46:32 STEP 2734 H=2734:00 +CPU= 23.498
  09:46:58 STEP 2735 H=2735:00 +CPU= 25.427
  09:47:45 STEP 2736 H=2736:00 +CPU= 44.839
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_113.zip
Uploading the intermediate file: upload_file_113.zip
  09:48:09 STEP 2737 H=2737:00 +CPU= 22.788
  09:48:36 STEP 2738 H=2738:00 +CPU= 26.363
  09:49:11 STEP 2739 H=2739:00 +CPU= 33.827
  09:49:33 STEP 2740 H=2740:00 +CPU= 22.165
  09:49:56 STEP 2741 H=2741:00 +CPU= 22.215
  09:50:31 STEP 2742 H=2742:00 +CPU= 35.298
  09:50:54 STEP 2743 H=2743:00 +CPU= 22.170
  09:51:18 STEP 2744 H=2744:00 +CPU= 24.459
  09:51:46 STEP 2745 H=2745:00 +CPU= 27.110
  09:52:04 STEP 2746 H=2746:00 +CPU= 18.741
  09:52:23 STEP 2747 H=2747:00 +CPU= 18.785
  09:52:57 STEP 2748 H=2748:00 +CPU= 33.587
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002748
  09:53:19 STEP 2749 H=2749:00 +CPU= 22.331
  09:53:42 STEP 2750 H=2750:00 +CPU= 22.355
  09:54:09 STEP 2751 H=2751:00 +CPU= 27.527
  09:54:31 STEP 2752 H=2752:00 +CPU= 21.452
  09:54:53 STEP 2753 H=2753:00 +CPU= 22.376
  09:55:31 STEP 2754 H=2754:00 +CPU= 38.213
  09:55:56 STEP 2755 H=2755:00 +CPU= 24.090
  09:56:20 STEP 2756 H=2756:00 +CPU= 24.078
  09:56:46 STEP 2757 H=2757:00 +CPU= 26.141
  09:57:07 STEP 2758 H=2758:00 +CPU= 20.953
  09:57:32 STEP 2759 H=2759:00 +CPU= 24.661
  09:58:12 STEP 2760 H=2760:00 +CPU= 39.505
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_114.zip
Uploading the intermediate file: upload_file_114.zip
  09:58:29 STEP 2761 H=2761:00 +CPU= 17.183
  09:58:47 STEP 2762 H=2762:00 +CPU= 17.918
  09:59:13 STEP 2763 H=2763:00 +CPU= 26.094
  09:59:32 STEP 2764 H=2764:00 +CPU= 18.863
  09:59:56 STEP 2765 H=2765:00 +CPU= 23.567
  10:00:31 STEP 2766 H=2766:00 +CPU= 34.758
  10:00:55 STEP 2767 H=2767:00 +CPU= 23.086
  10:01:19 STEP 2768 H=2768:00 +CPU= 22.940
  10:01:53 STEP 2769 H=2769:00 +CPU= 33.942
  10:02:17 STEP 2770 H=2770:00 +CPU= 23.127
  10:02:39 STEP 2771 H=2771:00 +CPU= 21.314
  10:03:19 STEP 2772 H=2772:00 +CPU= 40.115
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002772
  10:03:42 STEP 2773 H=2773:00 +CPU= 23.347
  10:04:07 STEP 2774 H=2774:00 +CPU= 23.655
  10:04:45 STEP 2775 H=2775:00 +CPU= 36.305
  10:05:11 STEP 2776 H=2776:00 +CPU= 25.926
  10:05:36 STEP 2777 H=2777:00 +CPU= 24.171
  10:06:17 STEP 2778 H=2778:00 +CPU= 39.949
  10:06:41 STEP 2779 H=2779:00 +CPU= 24.000
  10:07:04 STEP 2780 H=2780:00 +CPU= 23.179
  10:07:32 STEP 2781 H=2781:00 +CPU= 27.484
  10:07:56 STEP 2782 H=2782:00 +CPU= 23.518
  10:08:20 STEP 2783 H=2783:00 +CPU= 24.081
  10:09:05 STEP 2784 H=2784:00 +CPU= 44.019
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_115.zip
Uploading the intermediate file: upload_file_115.zip
  10:09:29 STEP 2785 H=2785:00 +CPU= 23.987
  10:09:54 STEP 2786 H=2786:00 +CPU= 24.472
  10:10:31 STEP 2787 H=2787:00 +CPU= 36.641
  10:10:56 STEP 2788 H=2788:00 +CPU= 25.439
  10:11:18 STEP 2789 H=2789:00 +CPU= 21.564
  10:11:56 STEP 2790 H=2790:00 +CPU= 37.228
  10:12:18 STEP 2791 H=2791:00 +CPU= 22.107
  10:12:41 STEP 2792 H=2792:00 +CPU= 22.789
  10:13:10 STEP 2793 H=2793:00 +CPU= 29.073
  10:13:28 STEP 2794 H=2794:00 +CPU= 17.549
  10:13:54 STEP 2795 H=2795:00 +CPU= 25.770
  10:14:35 STEP 2796 H=2796:00 +CPU= 41.031
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002796
  10:14:59 STEP 2797 H=2797:00 +CPU= 23.403
  10:15:23 STEP 2798 H=2798:00 +CPU= 24.460
  10:15:58 STEP 2799 H=2799:00 +CPU= 34.455
  10:16:23 STEP 2800 H=2800:00 +CPU= 24.421
  10:16:46 STEP 2801 H=2801:00 +CPU= 22.772
  10:17:25 STEP 2802 H=2802:00 +CPU= 38.838
  10:17:46 STEP 2803 H=2803:00 +CPU= 20.215
  10:18:09 STEP 2804 H=2804:00 +CPU= 23.352
  10:18:45 STEP 2805 H=2805:00 +CPU= 35.368
  10:19:04 STEP 2806 H=2806:00 +CPU= 18.341
  10:19:28 STEP 2807 H=2807:00 +CPU= 23.247
  10:20:14 STEP 2808 H=2808:00 +CPU= 43.904
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_116.zip
Uploading the intermediate file: upload_file_116.zip
  10:20:39 STEP 2809 H=2809:00 +CPU= 24.573
  10:21:02 STEP 2810 H=2810:00 +CPU= 22.939
  10:21:35 STEP 2811 H=2811:00 +CPU= 33.143
  10:22:01 STEP 2812 H=2812:00 +CPU= 24.477
  10:22:26 STEP 2813 H=2813:00 +CPU= 23.979
  10:23:02 STEP 2814 H=2814:00 +CPU= 35.848
  10:23:24 STEP 2815 H=2815:00 +CPU= 21.547
  10:23:50 STEP 2816 H=2816:00 +CPU= 24.805
  10:24:34 STEP 2817 H=2817:00 +CPU= 39.954
  10:25:01 STEP 2818 H=2818:00 +CPU= 26.144
  10:25:20 STEP 2819 H=2819:00 +CPU= 18.854
  10:25:57 STEP 2820 H=2820:00 +CPU= 35.552
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002820
  10:26:22 STEP 2821 H=2821:00 +CPU= 24.502
  10:26:48 STEP 2822 H=2822:00 +CPU= 24.882
  10:27:28 STEP 2823 H=2823:00 +CPU= 39.797
  10:27:55 STEP 2824 H=2824:00 +CPU= 26.266
  10:28:14 STEP 2825 H=2825:00 +CPU= 19.413
  10:28:45 STEP 2826 H=2826:00 +CPU= 30.575
  10:29:09 STEP 2827 H=2827:00 +CPU= 23.331
  10:29:33 STEP 2828 H=2828:00 +CPU= 23.807
  10:30:10 STEP 2829 H=2829:00 +CPU= 37.174
  10:30:33 STEP 2830 H=2830:00 +CPU= 21.939
  10:30:57 STEP 2831 H=2831:00 +CPU= 23.889
  10:31:39 STEP 2832 H=2832:00 +CPU= 40.483
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_117.zip
Uploading the intermediate file: upload_file_117.zip
  10:32:03 STEP 2833 H=2833:00 +CPU= 23.385
  10:32:24 STEP 2834 H=2834:00 +CPU= 21.026
  10:33:03 STEP 2835 H=2835:00 +CPU= 36.756
  10:33:47 STEP 2836 H=2836:00 +CPU= 33.973
  10:34:22 STEP 2837 H=2837:00 +CPU= 31.433
  10:35:08 STEP 2838 H=2838:00 +CPU= 41.905
  10:35:30 STEP 2839 H=2839:00 +CPU= 21.608
  10:35:53 STEP 2840 H=2840:00 +CPU= 23.163
  10:36:26 STEP 2841 H=2841:00 +CPU= 32.649
  10:36:53 STEP 2842 H=2842:00 +CPU= 25.913
  10:37:18 STEP 2843 H=2843:00 +CPU= 23.566
  10:38:00 STEP 2844 H=2844:00 +CPU= 40.941
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002844
  10:38:23 STEP 2845 H=2845:00 +CPU= 23.233
  10:38:44 STEP 2846 H=2846:00 +CPU= 21.030
  10:39:17 STEP 2847 H=2847:00 +CPU= 32.959
  10:39:43 STEP 2848 H=2848:00 +CPU= 25.693
  10:40:07 STEP 2849 H=2849:00 +CPU= 24.055
  10:40:34 STEP 2850 H=2850:00 +CPU= 26.952
  10:40:54 STEP 2851 H=2851:00 +CPU= 19.054
  10:41:19 STEP 2852 H=2852:00 +CPU= 24.979
  10:41:48 STEP 2853 H=2853:00 +CPU= 28.168
  10:42:15 STEP 2854 H=2854:00 +CPU= 26.561
  10:42:41 STEP 2855 H=2855:00 +CPU= 25.065
  10:43:29 STEP 2856 H=2856:00 +CPU= 45.164
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_118.zip
Uploading the intermediate file: upload_file_118.zip
  10:43:54 STEP 2857 H=2857:00 +CPU= 23.878
  10:44:16 STEP 2858 H=2858:00 +CPU= 22.261
  10:44:52 STEP 2859 H=2859:00 +CPU= 34.628
  10:45:16 STEP 2860 H=2860:00 +CPU= 23.255
  10:45:33 STEP 2861 H=2861:00 +CPU= 16.740
  10:45:59 STEP 2862 H=2862:00 +CPU= 26.434
  10:46:20 STEP 2863 H=2863:00 +CPU= 19.905
  10:46:43 STEP 2864 H=2864:00 +CPU= 22.705
  10:47:24 STEP 2865 H=2865:00 +CPU= 38.256
  10:47:51 STEP 2866 H=2866:00 +CPU= 25.857
  10:48:18 STEP 2867 H=2867:00 +CPU= 25.395
  10:48:55 STEP 2868 H=2868:00 +CPU= 36.919
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002868
  10:49:15 STEP 2869 H=2869:00 +CPU= 19.379
  10:49:37 STEP 2870 H=2870:00 +CPU= 22.573
  10:50:09 STEP 2871 H=2871:00 +CPU= 30.740
  10:50:35 STEP 2872 H=2872:00 +CPU= 25.337
  10:50:56 STEP 2873 H=2873:00 +CPU= 20.933
  10:51:31 STEP 2874 H=2874:00 +CPU= 34.153
  10:51:55 STEP 2875 H=2875:00 +CPU= 23.650
  10:52:16 STEP 2876 H=2876:00 +CPU= 20.600
  10:52:48 STEP 2877 H=2877:00 +CPU= 31.539
  10:53:12 STEP 2878 H=2878:00 +CPU= 23.580
  10:53:35 STEP 2879 H=2879:00 +CPU= 23.175
  10:54:18 STEP 2880 H=2880:00 +CPU= 41.509
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_119.zip
Uploading the intermediate file: upload_file_119.zip
  10:54:41 STEP 2881 H=2881:00 +CPU= 22.947
Uploading trickle at timestep: 10368000
  10:54:59 STEP 2882 H=2882:00 +CPU= 17.489
  10:55:35 STEP 2883 H=2883:00 +CPU= 35.679
  10:55:57 STEP 2884 H=2884:00 +CPU= 21.261
  10:56:19 STEP 2885 H=2885:00 +CPU= 22.666
  10:56:53 STEP 2886 H=2886:00 +CPU= 32.708
  10:57:14 STEP 2887 H=2887:00 +CPU= 20.653
  10:57:36 STEP 2888 H=2888:00 +CPU= 22.100
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1994050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0655
(argv4) batchid: 963
(argv5) wuid: 12180299
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/3
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f028d6861fa3d7d9e3f9931dbf0009a6 to: /var/lib/boinc/slots/3/oifs_43r3_ps_0655_1994050100_123_963_12180299.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/3/oifs_43r3_ps_0655_1994050100_123_963_12180299.zip
ic_ancil_file: ic_ancil_12180299
ifsdata_file: ifsdata_12180299
climate_data_file: clim_data_12180299
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_3a03b987c75139e61264c810a4c604dc to: /var/lib/boinc/slots/3/ic_ancil_12180299.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/3/ic_ancil_12180299.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_4338196b9ae848d79155ef758a24f427 to: /var/lib/boinc/slots/3/ifsdata/ifsdata_12180299.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/3/ifsdata/ifsdata_12180299.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/3/159l_2/clim_data_12180299.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/3/159l_2/clim_data_12180299.zip
Checking for progress XML file: /var/lib/boinc/slots/3/progress_file_12180299.xml
Opened progress file ok : /var/lib/boinc/slots/3/progress_file_12180299.xml
-- Model is restarting --
Adjusting last_iter, 2889, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/3/progress_file_12180299.xml
last_cpu_time: 82953
upload_file_number: 120
last_iter: 2879
last_upload: 10368000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744
The child process has been launched with process id: 19645
Executing the command: /var/lib/boinc/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:cftw.ddns.ms:1:1:19645:19645] [20221227:112611:1672136771:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  11:27:03 STEP 2880 H=2880:00 +CPU= 44.204
  11:27:35 STEP 2881 H=2881:00 +CPU= 28.662
  11:28:08 STEP 2882 H=2882:00 +CPU= 29.544
  11:28:56 STEP 2883 H=2883:00 +CPU= 44.020
  11:29:24 STEP 2884 H=2884:00 +CPU= 25.713
  11:29:51 STEP 2885 H=2885:00 +CPU= 25.891
  11:30:31 STEP 2886 H=2886:00 +CPU= 37.302
  11:30:58 STEP 2887 H=2887:00 +CPU= 25.761
  11:31:21 STEP 2888 H=2888:00 +CPU= 21.426
  11:32:01 STEP 2889 H=2889:00 +CPU= 38.565
  11:32:28 STEP 2890 H=2890:00 +CPU= 24.861
  11:32:52 STEP 2891 H=2891:00 +CPU= 22.492
  11:33:39 STEP 2892 H=2892:00 +CPU= 43.259
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002892
  11:34:01 STEP 2893 H=2893:00 +CPU= 21.754
  11:34:26 STEP 2894 H=2894:00 +CPU= 23.568
  11:35:07 STEP 2895 H=2895:00 +CPU= 39.297
  11:35:37 STEP 2896 H=2896:00 +CPU= 27.686
  11:36:00 STEP 2897 H=2897:00 +CPU= 22.371
  11:36:36 STEP 2898 H=2898:00 +CPU= 34.671
  11:36:58 STEP 2899 H=2899:00 +CPU= 21.837
  11:37:26 STEP 2900 H=2900:00 +CPU= 25.893
  11:38:09 STEP 2901 H=2901:00 +CPU= 39.343
  11:38:32 STEP 2902 H=2902:00 +CPU= 23.027
  11:38:51 STEP 2903 H=2903:00 +CPU= 19.133
  11:39:24 STEP 2904 H=2904:00 +CPU= 30.775
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_120.zip
Uploading the intermediate file: upload_file_120.zip
  11:39:40 STEP 2905 H=2905:00 +CPU= 16.755
  11:39:58 STEP 2906 H=2906:00 +CPU= 17.356
  11:40:23 STEP 2907 H=2907:00 +CPU= 25.351
  11:40:41 STEP 2908 H=2908:00 +CPU= 18.036
  11:40:59 STEP 2909 H=2909:00 +CPU= 18.225
  11:41:28 STEP 2910 H=2910:00 +CPU= 28.388
  11:41:45 STEP 2911 H=2911:00 +CPU= 16.829
  11:42:02 STEP 2912 H=2912:00 +CPU= 16.875
  11:42:27 STEP 2913 H=2913:00 +CPU= 25.202
  11:42:43 STEP 2914 H=2914:00 +CPU= 16.627
  11:43:00 STEP 2915 H=2915:00 +CPU= 16.208
  11:43:28 STEP 2916 H=2916:00 +CPU= 28.402
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002916
  11:43:50 STEP 2917 H=2917:00 +CPU= 22.071
  11:44:15 STEP 2918 H=2918:00 +CPU= 24.276
  11:44:50 STEP 2919 H=2919:00 +CPU= 33.641
  11:45:10 STEP 2920 H=2920:00 +CPU= 19.684
  11:45:29 STEP 2921 H=2921:00 +CPU= 19.679
  11:45:57 STEP 2922 H=2922:00 +CPU= 28.051
  11:46:15 STEP 2923 H=2923:00 +CPU= 17.075
  11:46:32 STEP 2924 H=2924:00 +CPU= 17.375
  11:47:00 STEP 2925 H=2925:00 +CPU= 28.404
  11:47:21 STEP 2926 H=2926:00 +CPU= 20.941
  11:47:42 STEP 2927 H=2927:00 +CPU= 20.655
  11:48:17 STEP 2928 H=2928:00 +CPU= 34.311
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_121.zip
Uploading the intermediate file: upload_file_121.zip
  11:48:37 STEP 2929 H=2929:00 +CPU= 19.290
  11:48:54 STEP 2930 H=2930:00 +CPU= 17.633
  11:49:29 STEP 2931 H=2931:00 +CPU= 34.144
  11:49:52 STEP 2932 H=2932:00 +CPU= 23.160
  11:50:15 STEP 2933 H=2933:00 +CPU= 22.415
  11:50:51 STEP 2934 H=2934:00 +CPU= 36.286
  11:51:14 STEP 2935 H=2935:00 +CPU= 22.650
  11:51:39 STEP 2936 H=2936:00 +CPU= 25.094
  11:52:06 STEP 2937 H=2937:00 +CPU= 26.795
  11:52:29 STEP 2938 H=2938:00 +CPU= 22.885
  11:52:53 STEP 2939 H=2939:00 +CPU= 23.498
  11:53:35 STEP 2940 H=2940:00 +CPU= 41.613
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002940
  11:53:59 STEP 2941 H=2941:00 +CPU= 23.983
  11:54:26 STEP 2942 H=2942:00 +CPU= 26.004
  11:55:03 STEP 2943 H=2943:00 +CPU= 37.079
  11:55:26 STEP 2944 H=2944:00 +CPU= 22.831
  11:55:48 STEP 2945 H=2945:00 +CPU= 22.251
  11:56:24 STEP 2946 H=2946:00 +CPU= 35.862
  11:56:47 STEP 2947 H=2947:00 +CPU= 22.764
  11:57:11 STEP 2948 H=2948:00 +CPU= 23.860
  11:57:48 STEP 2949 H=2949:00 +CPU= 37.022
  11:58:14 STEP 2950 H=2950:00 +CPU= 25.413
  11:58:40 STEP 2951 H=2951:00 +CPU= 25.802
  11:59:23 STEP 2952 H=2952:00 +CPU= 41.987
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12180299/ICMUAhq0f+002928
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0655_1994050100_123_963_12180299_0_r1117914744_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
12:02:46 (19639): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:58:36 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 10,623,600 84,888 0.0080
09 May 2023 15:58:32 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 10,368,000 82,754 0.0080
09 May 2023 15:58:22 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 9,504,000 76,284 0.0080
09 May 2023 15:58:16 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 8,640,000 70,231 0.0081
09 May 2023 15:58:10 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 7,776,000 64,346 0.0083
09 May 2023 15:58:02 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 6,912,000 58,128 0.0084
09 May 2023 15:57:55 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 6,048,000 51,410 0.0085
09 May 2023 15:57:46 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 5,184,000 44,489 0.0086
09 May 2023 15:57:37 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 4,320,000 37,362 0.0086
09 May 2023 15:57:28 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 3,456,000 30,173 0.0087
09 May 2023 15:57:17 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 2,592,000 22,549 0.0087
09 May 2023 15:57:05 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 1,728,000 15,378 0.0089
09 May 2023 15:56:54 1443559 22265257 oifs_43r3_ps_0655_1994050100_123_963_12180299_0 864,000 7,941 0.0092


©2024 climateprediction.net