climateprediction.net home page
Task 22267769

Task 22267769

Name oifs_43r3_ps_0011_1997050100_123_966_12182655_0
Workunit 12182655
Created 22 Dec 2022, 12:28:25 UTC
Sent 20 Jan 2023, 19:55:22 UTC
Report deadline 21 Mar 2023, 19:55:22 UTC
Received 26 Jan 2023, 0:05:10 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1538241
Run time 22 hours 49 min 39 sec
CPU time 22 hours 43 min 38 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 6.24 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,414.62 MB
Peak swap size 4,978.07 MB
Peak disk usage 4,481.86 MB
Stderr 
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<stderr_txt>
c-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_104.zip
Uploading the intermediate file: upload_file_104.zip
  07:54:57 STEP 2521 H=2521:00 +CPU= 20.164
Uploading trickle at timestep: 9072000
  07:55:18 STEP 2522 H=2522:00 +CPU= 20.444
  07:55:47 STEP 2523 H=2523:00 +CPU= 29.669
  07:56:09 STEP 2524 H=2524:00 +CPU= 21.465
  07:56:29 STEP 2525 H=2525:00 +CPU= 20.214
  07:57:01 STEP 2526 H=2526:00 +CPU= 31.937
  07:57:21 STEP 2527 H=2527:00 +CPU= 19.704
  07:57:40 STEP 2528 H=2528:00 +CPU= 19.058
  07:58:04 STEP 2529 H=2529:00 +CPU= 24.052
  07:58:21 STEP 2530 H=2530:00 +CPU= 17.004
  07:58:36 STEP 2531 H=2531:00 +CPU= 15.726
  07:59:08 STEP 2532 H=2532:00 +CPU= 31.117
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002532
  07:59:27 STEP 2533 H=2533:00 +CPU= 19.395
  07:59:49 STEP 2534 H=2534:00 +CPU= 21.841
  08:00:20 STEP 2535 H=2535:00 +CPU= 31.289
  08:00:42 STEP 2536 H=2536:00 +CPU= 21.611
  08:01:01 STEP 2537 H=2537:00 +CPU= 18.941
  08:01:26 STEP 2538 H=2538:00 +CPU= 25.822
  08:01:46 STEP 2539 H=2539:00 +CPU= 19.715
  08:02:05 STEP 2540 H=2540:00 +CPU= 18.736
  08:02:32 STEP 2541 H=2541:00 +CPU= 26.916
  08:02:52 STEP 2542 H=2542:00 +CPU= 19.799
  08:03:13 STEP 2543 H=2543:00 +CPU= 21.428
  08:03:51 STEP 2544 H=2544:00 +CPU= 38.199
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_105.zip
Uploading the intermediate file: upload_file_105.zip
  08:04:10 STEP 2545 H=2545:00 +CPU= 18.251
  08:04:31 STEP 2546 H=2546:00 +CPU= 21.185
  08:05:01 STEP 2547 H=2547:00 +CPU= 30.541
  08:05:20 STEP 2548 H=2548:00 +CPU= 19.165
  08:05:41 STEP 2549 H=2549:00 +CPU= 20.152
  08:06:13 STEP 2550 H=2550:00 +CPU= 31.981
  08:06:33 STEP 2551 H=2551:00 +CPU= 20.034
  08:06:54 STEP 2552 H=2552:00 +CPU= 21.239
  08:07:25 STEP 2553 H=2553:00 +CPU= 30.929
  08:07:46 STEP 2554 H=2554:00 +CPU= 20.717
  08:08:05 STEP 2555 H=2555:00 +CPU= 19.824
  08:08:40 STEP 2556 H=2556:00 +CPU= 34.594
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002556
  08:08:59 STEP 2557 H=2557:00 +CPU= 19.516
  08:09:19 STEP 2558 H=2558:00 +CPU= 19.895
  08:09:50 STEP 2559 H=2559:00 +CPU= 30.894
  08:10:11 STEP 2560 H=2560:00 +CPU= 20.868
  08:10:30 STEP 2561 H=2561:00 +CPU= 19.312
  08:11:02 STEP 2562 H=2562:00 +CPU= 31.126
  08:11:20 STEP 2563 H=2563:00 +CPU= 18.558
  08:11:35 STEP 2564 H=2564:00 +CPU= 14.443
  08:12:01 STEP 2565 H=2565:00 +CPU= 26.466
  08:12:23 STEP 2566 H=2566:00 +CPU= 21.746
  08:12:43 STEP 2567 H=2567:00 +CPU= 20.472
  08:13:20 STEP 2568 H=2568:00 +CPU= 37.202
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_106.zip
Uploading the intermediate file: upload_file_106.zip
  08:13:41 STEP 2569 H=2569:00 +CPU= 20.941
  08:14:02 STEP 2570 H=2570:00 +CPU= 20.788
  08:14:27 STEP 2571 H=2571:00 +CPU= 24.556
  08:14:47 STEP 2572 H=2572:00 +CPU= 20.498
  08:15:07 STEP 2573 H=2573:00 +CPU= 19.384
  08:15:31 STEP 2574 H=2574:00 +CPU= 23.949
  08:15:44 STEP 2575 H=2575:00 +CPU= 13.857
  08:16:00 STEP 2576 H=2576:00 +CPU= 15.761
  08:16:31 STEP 2577 H=2577:00 +CPU= 30.526
  08:16:52 STEP 2578 H=2578:00 +CPU= 20.941
  08:17:12 STEP 2579 H=2579:00 +CPU= 20.659
  08:17:49 STEP 2580 H=2580:00 +CPU= 36.285
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002580
  08:18:10 STEP 2581 H=2581:00 +CPU= 21.092
  08:18:31 STEP 2582 H=2582:00 +CPU= 20.864
  08:19:02 STEP 2583 H=2583:00 +CPU= 31.787
  08:19:24 STEP 2584 H=2584:00 +CPU= 21.825
  08:19:42 STEP 2585 H=2585:00 +CPU= 17.515
  08:20:12 STEP 2586 H=2586:00 +CPU= 29.854
  08:20:33 STEP 2587 H=2587:00 +CPU= 21.131
  08:20:50 STEP 2588 H=2588:00 +CPU= 17.638
  08:21:14 STEP 2589 H=2589:00 +CPU= 23.736
  08:21:31 STEP 2590 H=2590:00 +CPU= 16.630
  08:21:51 STEP 2591 H=2591:00 +CPU= 19.861
  08:22:20 STEP 2592 H=2592:00 +CPU= 29.704
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_107.zip
Uploading the intermediate file: upload_file_107.zip
  08:22:38 STEP 2593 H=2593:00 +CPU= 18.142
  08:22:57 STEP 2594 H=2594:00 +CPU= 18.221
  08:23:28 STEP 2595 H=2595:00 +CPU= 31.322
  08:23:44 STEP 2596 H=2596:00 +CPU= 15.834
  08:24:04 STEP 2597 H=2597:00 +CPU= 20.217
  08:24:35 STEP 2598 H=2598:00 +CPU= 30.867
  08:24:54 STEP 2599 H=2599:00 +CPU= 18.627
  08:25:13 STEP 2600 H=2600:00 +CPU= 19.438
  08:25:45 STEP 2601 H=2601:00 +CPU= 31.764
  08:26:05 STEP 2602 H=2602:00 +CPU= 20.487
  08:26:24 STEP 2603 H=2603:00 +CPU= 18.714
  08:26:52 STEP 2604 H=2604:00 +CPU= 28.239
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002604
  08:27:10 STEP 2605 H=2605:00 +CPU= 18.059
  08:27:29 STEP 2606 H=2606:00 +CPU= 18.285
  08:27:54 STEP 2607 H=2607:00 +CPU= 25.882
  08:28:12 STEP 2608 H=2608:00 +CPU= 17.373
  08:28:29 STEP 2609 H=2609:00 +CPU= 16.897
  08:28:56 STEP 2610 H=2610:00 +CPU= 27.094
  08:29:14 STEP 2611 H=2611:00 +CPU= 18.315
  08:29:33 STEP 2612 H=2612:00 +CPU= 18.880
  08:30:00 STEP 2613 H=2613:00 +CPU= 27.360
  08:30:21 STEP 2614 H=2614:00 +CPU= 20.597
  08:30:41 STEP 2615 H=2615:00 +CPU= 20.242
  08:31:17 STEP 2616 H=2616:00 +CPU= 35.405
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_108.zip
Uploading the intermediate file: upload_file_108.zip
  08:31:37 STEP 2617 H=2617:00 +CPU= 20.797
  08:31:58 STEP 2618 H=2618:00 +CPU= 20.568
  08:32:26 STEP 2619 H=2619:00 +CPU= 27.526
  08:32:46 STEP 2620 H=2620:00 +CPU= 20.827
  08:33:05 STEP 2621 H=2621:00 +CPU= 18.427
  08:33:36 STEP 2622 H=2622:00 +CPU= 30.771
  08:33:56 STEP 2623 H=2623:00 +CPU= 20.016
  08:34:14 STEP 2624 H=2624:00 +CPU= 18.720
  08:34:38 STEP 2625 H=2625:00 +CPU= 23.256
  08:34:57 STEP 2626 H=2626:00 +CPU= 19.403
  08:35:16 STEP 2627 H=2627:00 +CPU= 18.950
  08:35:51 STEP 2628 H=2628:00 +CPU= 35.012
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002628
  08:36:12 STEP 2629 H=2629:00 +CPU= 20.963
  08:36:34 STEP 2630 H=2630:00 +CPU= 21.866
  08:37:05 STEP 2631 H=2631:00 +CPU= 31.307
  08:37:25 STEP 2632 H=2632:00 +CPU= 19.836
  08:37:46 STEP 2633 H=2633:00 +CPU= 21.253
  08:38:18 STEP 2634 H=2634:00 +CPU= 31.584
  08:38:38 STEP 2635 H=2635:00 +CPU= 19.663
  08:38:58 STEP 2636 H=2636:00 +CPU= 20.603
  08:39:25 STEP 2637 H=2637:00 +CPU= 26.695
  08:39:45 STEP 2638 H=2638:00 +CPU= 20.518
  08:40:06 STEP 2639 H=2639:00 +CPU= 20.960
  08:40:38 STEP 2640 H=2640:00 +CPU= 31.862
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_109.zip
Uploading the intermediate file: upload_file_109.zip
  08:40:56 STEP 2641 H=2641:00 +CPU= 17.766
  08:41:17 STEP 2642 H=2642:00 +CPU= 21.439
  08:41:49 STEP 2643 H=2643:00 +CPU= 31.049
  08:42:09 STEP 2644 H=2644:00 +CPU= 20.488
  08:42:25 STEP 2645 H=2645:00 +CPU= 16.206
  08:42:58 STEP 2646 H=2646:00 +CPU= 32.899
  08:43:19 STEP 2647 H=2647:00 +CPU= 20.768
  08:43:40 STEP 2648 H=2648:00 +CPU= 21.552
  08:44:03 STEP 2649 H=2649:00 +CPU= 22.556
  08:44:19 STEP 2650 H=2650:00 +CPU= 15.968
  08:44:38 STEP 2651 H=2651:00 +CPU= 19.385
  08:45:11 STEP 2652 H=2652:00 +CPU= 32.939
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002652
  08:45:30 STEP 2653 H=2653:00 +CPU= 18.523
  08:45:50 STEP 2654 H=2654:00 +CPU= 19.788
  08:46:20 STEP 2655 H=2655:00 +CPU= 30.721
  08:46:39 STEP 2656 H=2656:00 +CPU= 18.910
  08:46:56 STEP 2657 H=2657:00 +CPU= 17.064
  08:47:25 STEP 2658 H=2658:00 +CPU= 28.307
  08:47:40 STEP 2659 H=2659:00 +CPU= 15.189
  08:47:55 STEP 2660 H=2660:00 +CPU= 14.753
  08:48:23 STEP 2661 H=2661:00 +CPU= 28.310
  08:48:43 STEP 2662 H=2662:00 +CPU= 20.365
  08:49:04 STEP 2663 H=2663:00 +CPU= 21.024
  08:49:40 STEP 2664 H=2664:00 +CPU= 35.776
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_110.zip
Uploading the intermediate file: upload_file_110.zip
  08:49:58 STEP 2665 H=2665:00 +CPU= 17.405
  08:50:19 STEP 2666 H=2666:00 +CPU= 21.244
  08:50:51 STEP 2667 H=2667:00 +CPU= 32.097
  08:51:13 STEP 2668 H=2668:00 +CPU= 21.629
  08:51:30 STEP 2669 H=2669:00 +CPU= 17.733
  08:51:58 STEP 2670 H=2670:00 +CPU= 27.850
  08:52:18 STEP 2671 H=2671:00 +CPU= 19.916
  08:52:39 STEP 2672 H=2672:00 +CPU= 20.444
  08:53:04 STEP 2673 H=2673:00 +CPU= 25.940
  08:53:20 STEP 2674 H=2674:00 +CPU= 15.567
  08:53:36 STEP 2675 H=2675:00 +CPU= 15.539
  08:54:07 STEP 2676 H=2676:00 +CPU= 31.793
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002676
  08:54:24 STEP 2677 H=2677:00 +CPU= 16.670
  08:54:45 STEP 2678 H=2678:00 +CPU= 20.885
  08:55:16 STEP 2679 H=2679:00 +CPU= 30.763
  08:55:35 STEP 2680 H=2680:00 +CPU= 19.765
  08:55:56 STEP 2681 H=2681:00 +CPU= 20.046
  08:56:21 STEP 2682 H=2682:00 +CPU= 25.207
  08:56:38 STEP 2683 H=2683:00 +CPU= 17.598
  08:56:53 STEP 2684 H=2684:00 +CPU= 14.447
  08:57:19 STEP 2685 H=2685:00 +CPU= 26.529
  08:57:40 STEP 2686 H=2686:00 +CPU= 20.991
  08:58:01 STEP 2687 H=2687:00 +CPU= 21.020
  08:58:38 STEP 2688 H=2688:00 +CPU= 36.351
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_111.zip
Uploading the intermediate file: upload_file_111.zip
  08:58:57 STEP 2689 H=2689:00 +CPU= 19.004
  08:59:17 STEP 2690 H=2690:00 +CPU= 20.333
  08:59:46 STEP 2691 H=2691:00 +CPU= 29.286
  09:00:06 STEP 2692 H=2692:00 +CPU= 19.764
  09:00:26 STEP 2693 H=2693:00 +CPU= 19.706
  09:00:58 STEP 2694 H=2694:00 +CPU= 32.045
  09:01:12 STEP 2695 H=2695:00 +CPU= 14.543
  09:01:27 STEP 2696 H=2696:00 +CPU= 14.845
  09:02:00 STEP 2697 H=2697:00 +CPU= 32.389
  09:02:21 STEP 2698 H=2698:00 +CPU= 20.908
  09:02:39 STEP 2699 H=2699:00 +CPU= 18.825
  09:03:05 STEP 2700 H=2700:00 +CPU= 25.475
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002700
  09:03:21 STEP 2701 H=2701:00 +CPU= 16.576
  09:03:41 STEP 2702 H=2702:00 +CPU= 19.324
  09:04:11 STEP 2703 H=2703:00 +CPU= 29.902
  09:04:30 STEP 2704 H=2704:00 +CPU= 19.650
  09:04:49 STEP 2705 H=2705:00 +CPU= 19.055
  09:05:17 STEP 2706 H=2706:00 +CPU= 27.681
  09:05:37 STEP 2707 H=2707:00 +CPU= 19.677
  09:05:58 STEP 2708 H=2708:00 +CPU= 21.756
  09:06:28 STEP 2709 H=2709:00 +CPU= 29.317
  09:06:49 STEP 2710 H=2710:00 +CPU= 21.002
  09:07:10 STEP 2711 H=2711:00 +CPU= 21.089
  09:07:47 STEP 2712 H=2712:00 +CPU= 37.382
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_112.zip
Uploading the intermediate file: upload_file_112.zip
  09:08:07 STEP 2713 H=2713:00 +CPU= 19.714
  09:08:26 STEP 2714 H=2714:00 +CPU= 19.197
  09:08:55 STEP 2715 H=2715:00 +CPU= 28.649
  09:09:15 STEP 2716 H=2716:00 +CPU= 20.220
  09:09:35 STEP 2717 H=2717:00 +CPU= 19.830
  09:10:06 STEP 2718 H=2718:00 +CPU= 31.405
  09:10:22 STEP 2719 H=2719:00 +CPU= 16.133
  09:10:39 STEP 2720 H=2720:00 +CPU= 16.182
  09:11:00 STEP 2721 H=2721:00 +CPU= 21.623
  09:11:21 STEP 2722 H=2722:00 +CPU= 20.260
  09:11:40 STEP 2723 H=2723:00 +CPU= 19.851
  09:12:14 STEP 2724 H=2724:00 +CPU= 33.979
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002724
  09:12:36 STEP 2725 H=2725:00 +CPU= 21.357
  09:12:57 STEP 2726 H=2726:00 +CPU= 21.465
  09:13:24 STEP 2727 H=2727:00 +CPU= 27.290
  09:13:40 STEP 2728 H=2728:00 +CPU= 15.122
  09:13:53 STEP 2729 H=2729:00 +CPU= 13.446
  09:14:15 STEP 2730 H=2730:00 +CPU= 21.743
  09:14:28 STEP 2731 H=2731:00 +CPU= 13.694
  09:14:46 STEP 2732 H=2732:00 +CPU= 17.165
  09:15:16 STEP 2733 H=2733:00 +CPU= 30.318
  09:15:37 STEP 2734 H=2734:00 +CPU= 20.711
  09:15:53 STEP 2735 H=2735:00 +CPU= 16.665
  09:16:30 STEP 2736 H=2736:00 +CPU= 36.856
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_113.zip
Uploading the intermediate file: upload_file_113.zip
  09:16:51 STEP 2737 H=2737:00 +CPU= 20.910
  09:17:13 STEP 2738 H=2738:00 +CPU= 21.731
  09:17:45 STEP 2739 H=2739:00 +CPU= 31.649
  09:17:59 STEP 2740 H=2740:00 +CPU= 14.911
  09:18:19 STEP 2741 H=2741:00 +CPU= 19.630
  09:18:50 STEP 2742 H=2742:00 +CPU= 31.155
  09:19:05 STEP 2743 H=2743:00 +CPU= 15.156
  09:19:23 STEP 2744 H=2744:00 +CPU= 17.349
  09:19:49 STEP 2745 H=2745:00 +CPU= 26.054
  09:20:07 STEP 2746 H=2746:00 +CPU= 18.382
  09:20:27 STEP 2747 H=2747:00 +CPU= 19.840
  09:20:59 STEP 2748 H=2748:00 +CPU= 32.157
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002748
  09:21:13 STEP 2749 H=2749:00 +CPU= 14.050
  09:21:32 STEP 2750 H=2750:00 +CPU= 18.495
  09:22:00 STEP 2751 H=2751:00 +CPU= 28.294
  09:22:21 STEP 2752 H=2752:00 +CPU= 20.607
  09:22:41 STEP 2753 H=2753:00 +CPU= 19.841
  09:23:13 STEP 2754 H=2754:00 +CPU= 32.192
  09:23:32 STEP 2755 H=2755:00 +CPU= 19.042
  09:23:53 STEP 2756 H=2756:00 +CPU= 21.654
  09:24:24 STEP 2757 H=2757:00 +CPU= 30.525
  09:24:43 STEP 2758 H=2758:00 +CPU= 19.203
  09:25:05 STEP 2759 H=2759:00 +CPU= 21.354
  09:25:40 STEP 2760 H=2760:00 +CPU= 35.917
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_114.zip
Uploading the intermediate file: upload_file_114.zip
  09:26:00 STEP 2761 H=2761:00 +CPU= 19.923
Uploading trickle at timestep: 9936000
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1997050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0011
(argv4) batchid: 966
(argv5) wuid: 12182655
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client_40001/slots/4
Project directory is: /var/lib/boinc-client_40001/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client_40001/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client_40001/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_d6bceb8aba4c39ad527298a126cca069 to: /var/lib/boinc-client_40001/slots/4/oifs_43r3_ps_0011_1997050100_123_966_12182655.zip
Unzipping the namelist zip file: /var/lib/boinc-client_40001/slots/4/oifs_43r3_ps_0011_1997050100_123_966_12182655.zip
ic_ancil_file: ic_ancil_12182655
ifsdata_file: ifsdata_12182655
climate_data_file: clim_data_12182655
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_d10acdf1f7278af2042e2ba7dbb0f95a to: /var/lib/boinc-client_40001/slots/4/ic_ancil_12182655.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client_40001/slots/4/ic_ancil_12182655.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_350637d6dfff3b140315c80528ad950f to: /var/lib/boinc-client_40001/slots/4/ifsdata/ifsdata_12182655.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client_40001/slots/4/ifsdata/ifsdata_12182655.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client_40001/slots/4/159l_2/clim_data_12182655.zip
Unzipping the climate data zip file: /var/lib/boinc-client_40001/slots/4/159l_2/clim_data_12182655.zip
Checking for progress XML file: /var/lib/boinc-client_40001/slots/4/progress_file_12182655.xml
Opened progress file ok : /var/lib/boinc-client_40001/slots/4/progress_file_12182655.xml
-- Model is restarting --
Adjusting last_iter, 2762, to previous model restart step.
Creating progress file: /var/lib/boinc-client_40001/slots/4/progress_file_12182655.xml
last_cpu_time: 76422
upload_file_number: 115
last_iter: 2759
last_upload: 9936000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540
The child process has been launched with process id: 1422
Executing the command: /var/lib/boinc-client_40001/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1422:1422] [20230124:101008:1674555008:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  10:11:23 STEP 2760 H=2760:00 +CPU= 36.629
  10:11:43 STEP 2761 H=2761:00 +CPU= 20.023
  10:12:02 STEP 2762 H=2762:00 +CPU= 18.873
  10:12:33 STEP 2763 H=2763:00 +CPU= 30.603
  10:12:54 STEP 2764 H=2764:00 +CPU= 21.162
  10:13:14 STEP 2765 H=2765:00 +CPU= 20.512
  10:13:48 STEP 2766 H=2766:00 +CPU= 33.938
  10:14:10 STEP 2767 H=2767:00 +CPU= 22.178
  10:14:38 STEP 2768 H=2768:00 +CPU= 27.744
  10:15:19 STEP 2769 H=2769:00 +CPU= 40.831
  10:15:47 STEP 2770 H=2770:00 +CPU= 27.447
  10:16:08 STEP 2771 H=2771:00 +CPU= 21.821
  10:16:43 STEP 2772 H=2772:00 +CPU= 34.653
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002772
  10:17:03 STEP 2773 H=2773:00 +CPU= 20.043
  10:17:27 STEP 2774 H=2774:00 +CPU= 24.033
  10:18:01 STEP 2775 H=2775:00 +CPU= 33.749
  10:18:25 STEP 2776 H=2776:00 +CPU= 24.523
  10:18:48 STEP 2777 H=2777:00 +CPU= 22.830
  10:19:18 STEP 2778 H=2778:00 +CPU= 29.473
  10:19:39 STEP 2779 H=2779:00 +CPU= 21.654
  10:20:04 STEP 2780 H=2780:00 +CPU= 24.313
  10:20:42 STEP 2781 H=2781:00 +CPU= 38.444
  10:21:05 STEP 2782 H=2782:00 +CPU= 22.726
  10:21:30 STEP 2783 H=2783:00 +CPU= 25.547
  10:22:15 STEP 2784 H=2784:00 +CPU= 44.542
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_115.zip
Uploading the intermediate file: upload_file_115.zip
  10:22:40 STEP 2785 H=2785:00 +CPU= 24.833
  10:23:02 STEP 2786 H=2786:00 +CPU= 22.522
  10:23:32 STEP 2787 H=2787:00 +CPU= 29.413
  10:23:49 STEP 2788 H=2788:00 +CPU= 17.292
  10:24:14 STEP 2789 H=2789:00 +CPU= 25.256
  10:24:53 STEP 2790 H=2790:00 +CPU= 39.028
  10:25:16 STEP 2791 H=2791:00 +CPU= 22.951
  10:25:41 STEP 2792 H=2792:00 +CPU= 24.542
  10:26:18 STEP 2793 H=2793:00 +CPU= 37.529
  10:26:42 STEP 2794 H=2794:00 +CPU= 23.183
  10:27:07 STEP 2795 H=2795:00 +CPU= 25.007
  10:27:53 STEP 2796 H=2796:00 +CPU= 45.987
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002796
  10:28:17 STEP 2797 H=2797:00 +CPU= 24.189
  10:28:44 STEP 2798 H=2798:00 +CPU= 27.786
  10:29:21 STEP 2799 H=2799:00 +CPU= 36.410
  10:29:40 STEP 2800 H=2800:00 +CPU= 19.207
  10:30:03 STEP 2801 H=2801:00 +CPU= 23.011
  10:30:50 STEP 2802 H=2802:00 +CPU= 47.212
  10:31:17 STEP 2803 H=2803:00 +CPU= 26.466
  10:31:38 STEP 2804 H=2804:00 +CPU= 21.398
  10:32:11 STEP 2805 H=2805:00 +CPU= 32.727
  10:32:32 STEP 2806 H=2806:00 +CPU= 21.134
  10:33:01 STEP 2807 H=2807:00 +CPU= 28.571
  10:33:58 STEP 2808 H=2808:00 +CPU= 57.393
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_116.zip
Uploading the intermediate file: upload_file_116.zip
  10:34:22 STEP 2809 H=2809:00 +CPU= 24.340
  10:34:46 STEP 2810 H=2810:00 +CPU= 23.969
  10:35:21 STEP 2811 H=2811:00 +CPU= 34.647
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1997050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0011
(argv4) batchid: 966
(argv5) wuid: 12182655
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client_40001/slots/4
Project directory is: /var/lib/boinc-client_40001/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client_40001/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client_40001/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_d6bceb8aba4c39ad527298a126cca069 to: /var/lib/boinc-client_40001/slots/4/oifs_43r3_ps_0011_1997050100_123_966_12182655.zip
Unzipping the namelist zip file: /var/lib/boinc-client_40001/slots/4/oifs_43r3_ps_0011_1997050100_123_966_12182655.zip
ic_ancil_file: ic_ancil_12182655
ifsdata_file: ifsdata_12182655
climate_data_file: clim_data_12182655
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_d10acdf1f7278af2042e2ba7dbb0f95a to: /var/lib/boinc-client_40001/slots/4/ic_ancil_12182655.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client_40001/slots/4/ic_ancil_12182655.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_350637d6dfff3b140315c80528ad950f to: /var/lib/boinc-client_40001/slots/4/ifsdata/ifsdata_12182655.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client_40001/slots/4/ifsdata/ifsdata_12182655.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client_40001/slots/4/159l_2/clim_data_12182655.zip
Unzipping the climate data zip file: /var/lib/boinc-client_40001/slots/4/159l_2/clim_data_12182655.zip
Checking for progress XML file: /var/lib/boinc-client_40001/slots/4/progress_file_12182655.xml
Opened progress file ok : /var/lib/boinc-client_40001/slots/4/progress_file_12182655.xml
-- Model is restarting --
Adjusting last_iter, 2812, to previous model restart step.
Creating progress file: /var/lib/boinc-client_40001/slots/4/progress_file_12182655.xml
last_cpu_time: 77909
upload_file_number: 117
last_iter: 2807
last_upload: 10108800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540
The child process has been launched with process id: 1416
Executing the command: /var/lib/boinc-client_40001/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:e3:1:1:1416:1416] [20230124:104956:1674557396:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  10:51:23 STEP 2808 H=2808:00 +CPU= 43.094
  10:51:46 STEP 2809 H=2809:00 +CPU= 23.257
  10:52:11 STEP 2810 H=2810:00 +CPU= 25.078
  10:52:45 STEP 2811 H=2811:00 +CPU= 34.337
  10:53:07 STEP 2812 H=2812:00 +CPU= 21.730
  10:53:28 STEP 2813 H=2813:00 +CPU= 21.083
  10:54:04 STEP 2814 H=2814:00 +CPU= 35.413
  10:54:24 STEP 2815 H=2815:00 +CPU= 20.602
  10:54:41 STEP 2816 H=2816:00 +CPU= 17.023
  10:55:10 STEP 2817 H=2817:00 +CPU= 29.215
  10:55:31 STEP 2818 H=2818:00 +CPU= 20.446
  10:55:51 STEP 2819 H=2819:00 +CPU= 20.087
  10:56:24 STEP 2820 H=2820:00 +CPU= 32.720
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002820
  10:56:46 STEP 2821 H=2821:00 +CPU= 22.748
  10:57:07 STEP 2822 H=2822:00 +CPU= 20.321
  10:57:34 STEP 2823 H=2823:00 +CPU= 27.559
  10:57:58 STEP 2824 H=2824:00 +CPU= 23.903
  10:58:21 STEP 2825 H=2825:00 +CPU= 22.459
  10:58:58 STEP 2826 H=2826:00 +CPU= 37.856
  10:59:21 STEP 2827 H=2827:00 +CPU= 22.588
  10:59:47 STEP 2828 H=2828:00 +CPU= 25.583
  11:00:26 STEP 2829 H=2829:00 +CPU= 38.936
  11:00:51 STEP 2830 H=2830:00 +CPU= 25.123
  11:01:15 STEP 2831 H=2831:00 +CPU= 24.464
  11:01:53 STEP 2832 H=2832:00 +CPU= 37.632
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_117.zip
Uploading the intermediate file: upload_file_117.zip
  11:02:08 STEP 2833 H=2833:00 +CPU= 14.749
  11:02:27 STEP 2834 H=2834:00 +CPU= 19.541
  11:02:58 STEP 2835 H=2835:00 +CPU= 31.307
  11:03:21 STEP 2836 H=2836:00 +CPU= 22.658
  11:03:44 STEP 2837 H=2837:00 +CPU= 22.541
  11:04:23 STEP 2838 H=2838:00 +CPU= 38.946
  11:04:45 STEP 2839 H=2839:00 +CPU= 22.543
  11:05:02 STEP 2840 H=2840:00 +CPU= 16.596
  11:05:35 STEP 2841 H=2841:00 +CPU= 33.362
  11:05:53 STEP 2842 H=2842:00 +CPU= 17.831
  11:06:19 STEP 2843 H=2843:00 +CPU= 25.936
  11:07:05 STEP 2844 H=2844:00 +CPU= 45.711
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002844
  11:07:26 STEP 2845 H=2845:00 +CPU= 21.756
  11:07:48 STEP 2846 H=2846:00 +CPU= 21.931
  11:08:17 STEP 2847 H=2847:00 +CPU= 29.126
  11:08:39 STEP 2848 H=2848:00 +CPU= 21.554
  11:09:00 STEP 2849 H=2849:00 +CPU= 21.195
  11:09:32 STEP 2850 H=2850:00 +CPU= 31.787
  11:09:47 STEP 2851 H=2851:00 +CPU= 14.600
  11:10:01 STEP 2852 H=2852:00 +CPU= 14.845
  11:10:26 STEP 2853 H=2853:00 +CPU= 24.125
  11:10:48 STEP 2854 H=2854:00 +CPU= 22.701
  11:11:12 STEP 2855 H=2855:00 +CPU= 23.383
  11:11:53 STEP 2856 H=2856:00 +CPU= 41.044
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_118.zip
Uploading the intermediate file: upload_file_118.zip
  11:12:14 STEP 2857 H=2857:00 +CPU= 21.482
  11:12:36 STEP 2858 H=2858:00 +CPU= 22.156
  11:13:09 STEP 2859 H=2859:00 +CPU= 32.288
  11:13:31 STEP 2860 H=2860:00 +CPU= 22.380
  11:13:49 STEP 2861 H=2861:00 +CPU= 17.975
  11:14:17 STEP 2862 H=2862:00 +CPU= 28.161
  11:14:38 STEP 2863 H=2863:00 +CPU= 20.416
  11:15:00 STEP 2864 H=2864:00 +CPU= 22.340
  11:15:33 STEP 2865 H=2865:00 +CPU= 33.350
  11:15:50 STEP 2866 H=2866:00 +CPU= 16.551
  11:16:09 STEP 2867 H=2867:00 +CPU= 19.258
  11:16:46 STEP 2868 H=2868:00 +CPU= 36.627
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002868
  11:17:07 STEP 2869 H=2869:00 +CPU= 21.103
  11:17:29 STEP 2870 H=2870:00 +CPU= 22.075
  11:18:00 STEP 2871 H=2871:00 +CPU= 31.462
  11:18:16 STEP 2872 H=2872:00 +CPU= 15.781
  11:18:33 STEP 2873 H=2873:00 +CPU= 16.776
  11:19:00 STEP 2874 H=2874:00 +CPU= 27.262
  11:19:23 STEP 2875 H=2875:00 +CPU= 22.762
  11:19:48 STEP 2876 H=2876:00 +CPU= 24.651
  11:20:26 STEP 2877 H=2877:00 +CPU= 38.394
  11:20:48 STEP 2878 H=2878:00 +CPU= 22.014
  11:21:11 STEP 2879 H=2879:00 +CPU= 22.704
  11:21:59 STEP 2880 H=2880:00 +CPU= 47.842
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_119.zip
Uploading the intermediate file: upload_file_119.zip
  11:22:25 STEP 2881 H=2881:00 +CPU= 25.914
  11:22:51 STEP 2882 H=2882:00 +CPU= 26.545
  11:23:32 STEP 2883 H=2883:00 +CPU= 41.145
  11:23:57 STEP 2884 H=2884:00 +CPU= 24.494
  11:24:20 STEP 2885 H=2885:00 +CPU= 23.507
  11:25:00 STEP 2886 H=2886:00 +CPU= 39.936
  11:25:26 STEP 2887 H=2887:00 +CPU= 25.677
  11:25:53 STEP 2888 H=2888:00 +CPU= 26.902
  11:26:27 STEP 2889 H=2889:00 +CPU= 33.984
  11:26:51 STEP 2890 H=2890:00 +CPU= 24.725
  11:27:17 STEP 2891 H=2891:00 +CPU= 25.015
  11:27:54 STEP 2892 H=2892:00 +CPU= 37.948
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002892
  11:28:16 STEP 2893 H=2893:00 +CPU= 21.949
  11:28:38 STEP 2894 H=2894:00 +CPU= 21.243
  11:29:08 STEP 2895 H=2895:00 +CPU= 30.617
  11:29:28 STEP 2896 H=2896:00 +CPU= 20.000
  11:29:50 STEP 2897 H=2897:00 +CPU= 21.835
  11:30:27 STEP 2898 H=2898:00 +CPU= 37.362
  11:30:52 STEP 2899 H=2899:00 +CPU= 24.277
  11:31:19 STEP 2900 H=2900:00 +CPU= 27.220
  11:31:58 STEP 2901 H=2901:00 +CPU= 38.551
  11:32:21 STEP 2902 H=2902:00 +CPU= 23.248
  11:32:47 STEP 2903 H=2903:00 +CPU= 25.872
  11:33:23 STEP 2904 H=2904:00 +CPU= 36.241
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_120.zip
Uploading the intermediate file: upload_file_120.zip
  11:33:45 STEP 2905 H=2905:00 +CPU= 21.844
  11:34:07 STEP 2906 H=2906:00 +CPU= 22.393
  11:34:47 STEP 2907 H=2907:00 +CPU= 40.282
  11:35:05 STEP 2908 H=2908:00 +CPU= 17.026
  11:35:29 STEP 2909 H=2909:00 +CPU= 24.316
  11:36:05 STEP 2910 H=2910:00 +CPU= 35.855
  11:36:28 STEP 2911 H=2911:00 +CPU= 23.350
  11:36:49 STEP 2912 H=2912:00 +CPU= 20.559
  11:37:29 STEP 2913 H=2913:00 +CPU= 40.086
  11:37:51 STEP 2914 H=2914:00 +CPU= 22.332
  11:38:15 STEP 2915 H=2915:00 +CPU= 24.438
  11:38:56 STEP 2916 H=2916:00 +CPU= 40.228
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002916
  11:39:13 STEP 2917 H=2917:00 +CPU= 17.159
  11:39:33 STEP 2918 H=2918:00 +CPU= 20.360
  11:40:13 STEP 2919 H=2919:00 +CPU= 39.591
  11:40:40 STEP 2920 H=2920:00 +CPU= 27.006
  11:41:06 STEP 2921 H=2921:00 +CPU= 26.344
  11:41:47 STEP 2922 H=2922:00 +CPU= 40.765
  11:42:08 STEP 2923 H=2923:00 +CPU= 21.473
  11:42:35 STEP 2924 H=2924:00 +CPU= 26.546
  11:43:12 STEP 2925 H=2925:00 +CPU= 36.679
  11:43:36 STEP 2926 H=2926:00 +CPU= 24.086
  11:44:01 STEP 2927 H=2927:00 +CPU= 24.823
  11:44:48 STEP 2928 H=2928:00 +CPU= 46.967
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_121.zip
Uploading the intermediate file: upload_file_121.zip
  11:45:13 STEP 2929 H=2929:00 +CPU= 25.684
  11:45:40 STEP 2930 H=2930:00 +CPU= 26.288
  11:46:16 STEP 2931 H=2931:00 +CPU= 36.462
  11:46:42 STEP 2932 H=2932:00 +CPU= 26.391
  11:47:05 STEP 2933 H=2933:00 +CPU= 22.742
  11:47:42 STEP 2934 H=2934:00 +CPU= 36.740
  11:48:05 STEP 2935 H=2935:00 +CPU= 23.550
  11:48:27 STEP 2936 H=2936:00 +CPU= 22.073
  11:49:00 STEP 2937 H=2937:00 +CPU= 32.489
  11:49:22 STEP 2938 H=2938:00 +CPU= 22.424
  11:49:42 STEP 2939 H=2939:00 +CPU= 19.300
  11:50:20 STEP 2940 H=2940:00 +CPU= 37.968
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002940
  11:50:43 STEP 2941 H=2941:00 +CPU= 23.812
  11:51:05 STEP 2942 H=2942:00 +CPU= 21.990
  11:51:34 STEP 2943 H=2943:00 +CPU= 28.297
  11:51:51 STEP 2944 H=2944:00 +CPU= 16.864
  11:52:10 STEP 2945 H=2945:00 +CPU= 19.117
  11:52:45 STEP 2946 H=2946:00 +CPU= 35.481
  11:53:11 STEP 2947 H=2947:00 +CPU= 26.080
  11:53:33 STEP 2948 H=2948:00 +CPU= 21.908
  11:54:12 STEP 2949 H=2949:00 +CPU= 38.609
  11:54:38 STEP 2950 H=2950:00 +CPU= 26.531
  11:55:04 STEP 2951 H=2951:00 +CPU= 25.347
  11:55:45 STEP 2952 H=2952:00 +CPU= 41.541
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client_40001/slots/4/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_12182655/ICMSHhq0f+002928
Zipping up the final file: /var/lib/boinc-client_40001/projects/climateprediction.net/oifs_43r3_ps_0011_1997050100_123_966_12182655_0_r476913540_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
11:59:09 (1115): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:39:16 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 10,623,600 81,818 0.0077
09 May 2023 16:39:16 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 10,623,600 81,818 0.0077
09 May 2023 16:39:02 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 9,072,000 70,934 0.0078
09 May 2023 16:38:58 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 8,208,000 65,650 0.0080
09 May 2023 16:38:58 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 8,208,000 65,650 0.0080
09 May 2023 16:38:50 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 7,344,000 60,314 0.0082
09 May 2023 16:38:41 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 6,048,000 50,849 0.0084
09 May 2023 16:38:41 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 6,048,000 50,849 0.0084
09 May 2023 16:38:34 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 5,184,000 43,985 0.0085
09 May 2023 16:38:34 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 5,184,000 43,985 0.0085
09 May 2023 16:38:26 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 4,320,000 36,521 0.0085
09 May 2023 16:38:26 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 4,320,000 36,521 0.0085
09 May 2023 16:38:17 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 3,456,000 29,578 0.0086
09 May 2023 16:38:09 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 2,592,000 22,384 0.0086
09 May 2023 16:37:58 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 1,728,000 14,531 0.0084
09 May 2023 16:37:48 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 864,000 7,589 0.0088
09 May 2023 16:39:09 1538241 22267769 oifs_43r3_ps_0011_1997050100_123_966_12182655_0 0 74,891 0.0000


©2024 climateprediction.net