climateprediction.net home page
Task 22295621

Task 22295621

Name oifs_43r3_ps_0348_1997050100_123_966_12182992_2
Workunit 12182992
Created 4 Jan 2023, 22:43:49 UTC
Sent 8 Jan 2023, 23:06:49 UTC
Report deadline 7 Feb 2023, 23:06:49 UTC
Received 9 Jan 2023, 22:48:25 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1537854
Run time 22 hours 42 min
CPU time 21 hours 25 min 37 sec
Validate state Invalid
Credit 7,257.60
Device peak FLOPS 3.60 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,403.36 MB
Peak swap size 4,974.27 MB
Peak disk usage 2,266.98 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
hq0f+000744
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000744
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000756
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000756
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000756
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_31.zip
Uploading the intermediate file: upload_file_31.zip
  13:14:23 STEP  769 H= 769:00 +CPU= 43.290
  13:15:11 STEP  770 H= 770:00 +CPU= 44.996
  13:16:24 STEP  771 H= 771:00 +CPU= 68.351
  13:17:11 STEP  772 H= 772:00 +CPU= 44.922
  13:17:57 STEP  773 H= 773:00 +CPU= 43.565
  13:19:12 STEP  774 H= 774:00 +CPU= 70.237
  13:19:58 STEP  775 H= 775:00 +CPU= 43.708
  13:20:46 STEP  776 H= 776:00 +CPU= 44.856
  13:21:58 STEP  777 H= 777:00 +CPU= 68.941
  13:22:46 STEP  778 H= 778:00 +CPU= 44.913
  13:23:32 STEP  779 H= 779:00 +CPU= 43.724
  13:24:53 STEP  780 H= 780:00 +CPU= 76.980
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000780
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000780
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000780
  13:25:40 STEP  781 H= 781:00 +CPU= 43.778
  13:26:27 STEP  782 H= 782:00 +CPU= 44.909
  13:27:40 STEP  783 H= 783:00 +CPU= 68.842
  13:28:27 STEP  784 H= 784:00 +CPU= 44.930
  13:29:13 STEP  785 H= 785:00 +CPU= 43.596
  13:30:27 STEP  786 H= 786:00 +CPU= 70.077
  13:31:14 STEP  787 H= 787:00 +CPU= 43.699
  13:32:01 STEP  788 H= 788:00 +CPU= 44.787
  13:33:14 STEP  789 H= 789:00 +CPU= 68.915
  13:34:01 STEP  790 H= 790:00 +CPU= 44.698
  13:34:49 STEP  791 H= 791:00 +CPU= 45.005
  13:36:17 STEP  792 H= 792:00 +CPU= 80.678
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000792
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000792
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000792
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000768
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000768
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000768
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000780
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000780
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000780
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_32.zip
Uploading the intermediate file: upload_file_32.zip
  13:37:03 STEP  793 H= 793:00 +CPU= 43.674
  13:37:51 STEP  794 H= 794:00 +CPU= 44.947
  13:39:04 STEP  795 H= 795:00 +CPU= 68.884
  13:39:51 STEP  796 H= 796:00 +CPU= 44.963
  13:40:38 STEP  797 H= 797:00 +CPU= 43.641
  13:41:52 STEP  798 H= 798:00 +CPU= 70.486
  13:42:38 STEP  799 H= 799:00 +CPU= 43.548
  13:43:26 STEP  800 H= 800:00 +CPU= 44.882
  13:44:39 STEP  801 H= 801:00 +CPU= 68.864
  13:45:26 STEP  802 H= 802:00 +CPU= 44.859
  13:46:12 STEP  803 H= 803:00 +CPU= 43.549
  13:47:34 STEP  804 H= 804:00 +CPU= 77.080
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000804
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000804
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000804
  13:48:20 STEP  805 H= 805:00 +CPU= 43.702
  13:49:07 STEP  806 H= 806:00 +CPU= 45.005
  13:50:21 STEP  807 H= 807:00 +CPU= 69.216
  13:51:08 STEP  808 H= 808:00 +CPU= 45.279
  13:51:55 STEP  809 H= 809:00 +CPU= 43.727
  13:53:09 STEP  810 H= 810:00 +CPU= 70.150
  13:53:55 STEP  811 H= 811:00 +CPU= 43.674
  13:54:42 STEP  812 H= 812:00 +CPU= 44.672
  13:55:55 STEP  813 H= 813:00 +CPU= 68.807
  13:56:43 STEP  814 H= 814:00 +CPU= 44.674
  13:57:31 STEP  815 H= 815:00 +CPU= 45.176
  13:58:58 STEP  816 H= 816:00 +CPU= 80.625
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000816
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000816
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000816
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000792
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000792
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000792
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000804
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000804
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000804
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_33.zip
Uploading the intermediate file: upload_file_33.zip
  13:59:44 STEP  817 H= 817:00 +CPU= 43.578
  14:00:32 STEP  818 H= 818:00 +CPU= 44.855
  14:01:45 STEP  819 H= 819:00 +CPU= 68.979
  14:02:33 STEP  820 H= 820:00 +CPU= 45.038
  14:03:19 STEP  821 H= 821:00 +CPU= 43.608
  14:04:33 STEP  822 H= 822:00 +CPU= 70.435
  14:05:19 STEP  823 H= 823:00 +CPU= 43.649
  14:06:07 STEP  824 H= 824:00 +CPU= 44.947
  14:07:20 STEP  825 H= 825:00 +CPU= 69.015
  14:08:07 STEP  826 H= 826:00 +CPU= 44.934
  14:08:53 STEP  827 H= 827:00 +CPU= 43.697
  14:10:15 STEP  828 H= 828:00 +CPU= 77.096
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000828
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000828
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000828
  14:11:01 STEP  829 H= 829:00 +CPU= 43.906
  14:11:49 STEP  830 H= 830:00 +CPU= 44.988
  14:13:02 STEP  831 H= 831:00 +CPU= 69.015
  14:13:49 STEP  832 H= 832:00 +CPU= 45.016
  14:14:35 STEP  833 H= 833:00 +CPU= 43.477
  14:15:50 STEP  834 H= 834:00 +CPU= 70.317
  14:16:37 STEP  835 H= 835:00 +CPU= 43.197
  14:17:24 STEP  836 H= 836:00 +CPU= 44.826
  14:18:37 STEP  837 H= 837:00 +CPU= 68.882
  14:19:24 STEP  838 H= 838:00 +CPU= 44.731
  14:20:12 STEP  839 H= 839:00 +CPU= 44.531
  14:21:38 STEP  840 H= 840:00 +CPU= 80.998
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000840
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000840
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000840
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000816
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000816
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000816
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000828
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000828
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000828
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_34.zip
Uploading the intermediate file: upload_file_34.zip
  14:22:24 STEP  841 H= 841:00 +CPU= 43.433
  14:23:12 STEP  842 H= 842:00 +CPU= 44.916
  14:24:25 STEP  843 H= 843:00 +CPU= 68.768
  14:25:12 STEP  844 H= 844:00 +CPU= 44.922
  14:25:58 STEP  845 H= 845:00 +CPU= 43.733
  14:27:13 STEP  846 H= 846:00 +CPU= 70.373
  14:27:59 STEP  847 H= 847:00 +CPU= 43.539
  14:28:47 STEP  848 H= 848:00 +CPU= 45.090
  14:29:59 STEP  849 H= 849:00 +CPU= 68.904
  14:30:47 STEP  850 H= 850:00 +CPU= 45.053
  14:31:33 STEP  851 H= 851:00 +CPU= 43.590
  14:32:55 STEP  852 H= 852:00 +CPU= 77.025
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000852
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000852
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000852
  14:33:41 STEP  853 H= 853:00 +CPU= 43.664
  14:34:29 STEP  854 H= 854:00 +CPU= 44.957
  14:35:42 STEP  855 H= 855:00 +CPU= 69.048
  14:36:30 STEP  856 H= 856:00 +CPU= 44.531
  14:37:16 STEP  857 H= 857:00 +CPU= 43.665
  14:38:30 STEP  858 H= 858:00 +CPU= 70.248
  14:39:16 STEP  859 H= 859:00 +CPU= 43.587
  14:40:04 STEP  860 H= 860:00 +CPU= 44.805
  14:41:17 STEP  861 H= 861:00 +CPU= 69.011
  14:42:04 STEP  862 H= 862:00 +CPU= 44.819
  14:42:52 STEP  863 H= 863:00 +CPU= 45.094
  14:44:19 STEP  864 H= 864:00 +CPU= 81.118
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000864
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000864
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000864
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000840
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000840
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000840
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000852
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000852
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000852
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_35.zip
Uploading the intermediate file: upload_file_35.zip
  14:45:05 STEP  865 H= 865:00 +CPU= 43.464
  14:45:52 STEP  866 H= 866:00 +CPU= 44.850
  14:47:05 STEP  867 H= 867:00 +CPU= 68.828
  14:47:52 STEP  868 H= 868:00 +CPU= 44.970
  14:48:38 STEP  869 H= 869:00 +CPU= 43.570
  14:49:52 STEP  870 H= 870:00 +CPU= 70.272
  14:50:39 STEP  871 H= 871:00 +CPU= 43.724
  14:51:26 STEP  872 H= 872:00 +CPU= 45.077
  14:52:39 STEP  873 H= 873:00 +CPU= 68.791
  14:53:26 STEP  874 H= 874:00 +CPU= 44.753
  14:54:13 STEP  875 H= 875:00 +CPU= 43.375
  14:55:34 STEP  876 H= 876:00 +CPU= 76.957
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000876
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000876
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000876
  14:56:20 STEP  877 H= 877:00 +CPU= 43.218
  14:57:08 STEP  878 H= 878:00 +CPU= 44.912
  14:58:20 STEP  879 H= 879:00 +CPU= 68.786
  14:59:08 STEP  880 H= 880:00 +CPU= 44.955
  14:59:54 STEP  881 H= 881:00 +CPU= 43.605
  15:01:09 STEP  882 H= 882:00 +CPU= 70.008
  15:01:55 STEP  883 H= 883:00 +CPU= 43.626
  15:02:42 STEP  884 H= 884:00 +CPU= 44.828
  15:03:55 STEP  885 H= 885:00 +CPU= 68.625
  15:04:42 STEP  886 H= 886:00 +CPU= 44.838
  15:05:30 STEP  887 H= 887:00 +CPU= 44.978
  15:06:57 STEP  888 H= 888:00 +CPU= 81.068
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000888
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000888
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000888
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000864
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000864
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000864
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000876
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000876
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000876
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_36.zip
Uploading the intermediate file: upload_file_36.zip
  15:07:43 STEP  889 H= 889:00 +CPU= 43.633
  15:08:30 STEP  890 H= 890:00 +CPU= 44.993
  15:09:43 STEP  891 H= 891:00 +CPU= 68.850
  15:10:30 STEP  892 H= 892:00 +CPU= 44.853
  15:11:16 STEP  893 H= 893:00 +CPU= 43.564
  15:12:31 STEP  894 H= 894:00 +CPU= 70.359
  15:13:17 STEP  895 H= 895:00 +CPU= 43.845
  15:14:05 STEP  896 H= 896:00 +CPU= 44.567
  15:15:18 STEP  897 H= 897:00 +CPU= 69.005
  15:16:05 STEP  898 H= 898:00 +CPU= 44.526
  15:16:51 STEP  899 H= 899:00 +CPU= 43.663
  15:18:13 STEP  900 H= 900:00 +CPU= 76.850
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000900
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000900
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000900
  15:18:59 STEP  901 H= 901:00 +CPU= 43.801
  15:19:47 STEP  902 H= 902:00 +CPU= 45.046
  15:21:00 STEP  903 H= 903:00 +CPU= 68.335
  15:21:47 STEP  904 H= 904:00 +CPU= 44.904
  15:22:33 STEP  905 H= 905:00 +CPU= 43.558
  15:23:48 STEP  906 H= 906:00 +CPU= 70.311
  15:24:34 STEP  907 H= 907:00 +CPU= 43.727
  15:25:21 STEP  908 H= 908:00 +CPU= 44.892
  15:26:34 STEP  909 H= 909:00 +CPU= 69.033
  15:27:22 STEP  910 H= 910:00 +CPU= 44.936
  15:28:09 STEP  911 H= 911:00 +CPU= 45.063
  15:29:37 STEP  912 H= 912:00 +CPU= 81.106
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000912
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000912
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000912
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000888
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000888
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000888
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000900
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000900
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000900
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_37.zip
Uploading the intermediate file: upload_file_37.zip
  15:30:23 STEP  913 H= 913:00 +CPU= 43.528
  15:31:10 STEP  914 H= 914:00 +CPU= 44.984
  15:32:23 STEP  915 H= 915:00 +CPU= 68.848
  15:33:10 STEP  916 H= 916:00 +CPU= 44.698
  15:33:56 STEP  917 H= 917:00 +CPU= 43.505
  15:35:11 STEP  918 H= 918:00 +CPU= 70.274
  15:35:57 STEP  919 H= 919:00 +CPU= 43.667
  15:36:44 STEP  920 H= 920:00 +CPU= 44.511
  15:37:57 STEP  921 H= 921:00 +CPU= 68.732
  15:38:45 STEP  922 H= 922:00 +CPU= 44.964
  15:39:31 STEP  923 H= 923:00 +CPU= 43.625
  15:40:52 STEP  924 H= 924:00 +CPU= 76.347
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000924
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000924
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000924
  15:41:38 STEP  925 H= 925:00 +CPU= 43.630
  15:42:26 STEP  926 H= 926:00 +CPU= 44.872
  15:43:38 STEP  927 H= 927:00 +CPU= 68.869
  15:44:26 STEP  928 H= 928:00 +CPU= 45.003
  15:45:12 STEP  929 H= 929:00 +CPU= 43.672
  15:46:27 STEP  930 H= 930:00 +CPU= 70.273
  15:47:13 STEP  931 H= 931:00 +CPU= 43.617
  15:48:00 STEP  932 H= 932:00 +CPU= 44.899
  15:49:13 STEP  933 H= 933:00 +CPU= 68.725
  15:50:00 STEP  934 H= 934:00 +CPU= 44.917
  15:50:48 STEP  935 H= 935:00 +CPU= 44.974
  15:52:15 STEP  936 H= 936:00 +CPU= 80.629
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000936
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000936
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000936
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000912
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000912
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000912
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000924
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000924
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000924
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_38.zip
Uploading the intermediate file: upload_file_38.zip
  15:53:01 STEP  937 H= 937:00 +CPU= 43.640
  15:53:48 STEP  938 H= 938:00 +CPU= 45.000
  15:55:01 STEP  939 H= 939:00 +CPU= 68.309
  15:55:48 STEP  940 H= 940:00 +CPU= 44.790
  15:56:34 STEP  941 H= 941:00 +CPU= 43.012
  15:57:48 STEP  942 H= 942:00 +CPU= 70.216
  15:58:34 STEP  943 H= 943:00 +CPU= 43.471
  15:59:22 STEP  944 H= 944:00 +CPU= 44.991
  16:00:34 STEP  945 H= 945:00 +CPU= 68.498
  16:01:22 STEP  946 H= 946:00 +CPU= 45.055
  16:02:08 STEP  947 H= 947:00 +CPU= 43.628
  16:03:29 STEP  948 H= 948:00 +CPU= 76.915
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000948
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000948
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000948
  16:04:15 STEP  949 H= 949:00 +CPU= 43.588
  16:05:03 STEP  950 H= 950:00 +CPU= 45.029
  16:06:16 STEP  951 H= 951:00 +CPU= 68.642
  16:07:03 STEP  952 H= 952:00 +CPU= 44.793
  16:07:49 STEP  953 H= 953:00 +CPU= 43.660
  16:09:03 STEP  954 H= 954:00 +CPU= 70.218
  16:09:49 STEP  955 H= 955:00 +CPU= 43.585
  16:10:37 STEP  956 H= 956:00 +CPU= 44.776
  16:11:50 STEP  957 H= 957:00 +CPU= 68.932
  16:12:37 STEP  958 H= 958:00 +CPU= 44.823
  16:13:25 STEP  959 H= 959:00 +CPU= 44.895
  16:14:52 STEP  960 H= 960:00 +CPU= 80.558
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000960
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000960
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000960
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000936
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000936
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000936
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000948
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000948
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000948
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_39.zip
Uploading the intermediate file: upload_file_39.zip
Uploading trickle at timestep: 3456000
  16:15:38 STEP  961 H= 961:00 +CPU= 43.613
  16:16:26 STEP  962 H= 962:00 +CPU= 44.620
  16:17:39 STEP  963 H= 963:00 +CPU= 68.817
  16:18:26 STEP  964 H= 964:00 +CPU= 44.773
  16:19:12 STEP  965 H= 965:00 +CPU= 43.777
  16:20:26 STEP  966 H= 966:00 +CPU= 69.634
  16:21:13 STEP  967 H= 967:00 +CPU= 43.621
  16:22:00 STEP  968 H= 968:00 +CPU= 44.909
  16:23:13 STEP  969 H= 969:00 +CPU= 69.000
  16:24:00 STEP  970 H= 970:00 +CPU= 44.963
  16:24:46 STEP  971 H= 971:00 +CPU= 43.731
  16:26:08 STEP  972 H= 972:00 +CPU= 76.798
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000972
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000972
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000972
  16:26:54 STEP  973 H= 973:00 +CPU= 43.734
  16:27:41 STEP  974 H= 974:00 +CPU= 44.923
  16:28:54 STEP  975 H= 975:00 +CPU= 68.541
  16:29:41 STEP  976 H= 976:00 +CPU= 44.937
  16:30:27 STEP  977 H= 977:00 +CPU= 43.609
  16:31:42 STEP  978 H= 978:00 +CPU= 70.269
  16:32:28 STEP  979 H= 979:00 +CPU= 43.696
  16:33:15 STEP  980 H= 980:00 +CPU= 44.903
  16:34:28 STEP  981 H= 981:00 +CPU= 68.460
  16:35:16 STEP  982 H= 982:00 +CPU= 44.946
  16:36:03 STEP  983 H= 983:00 +CPU= 44.807
  16:37:31 STEP  984 H= 984:00 +CPU= 80.852
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000984
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000984
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000984
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000960
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000960
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000960
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000972
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000972
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000972
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_40.zip
Uploading the intermediate file: upload_file_40.zip
  16:38:17 STEP  985 H= 985:00 +CPU= 43.414
  16:39:04 STEP  986 H= 986:00 +CPU= 44.910
  16:40:17 STEP  987 H= 987:00 +CPU= 68.579
  16:41:04 STEP  988 H= 988:00 +CPU= 44.830
  16:41:50 STEP  989 H= 989:00 +CPU= 43.629
  16:43:05 STEP  990 H= 990:00 +CPU= 70.141
  16:43:51 STEP  991 H= 991:00 +CPU= 43.599
  16:44:38 STEP  992 H= 992:00 +CPU= 44.904
  16:45:51 STEP  993 H= 993:00 +CPU= 68.935
  16:46:39 STEP  994 H= 994:00 +CPU= 44.923
  16:47:25 STEP  995 H= 995:00 +CPU= 43.582
  16:48:46 STEP  996 H= 996:00 +CPU= 76.829
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+000996
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+000996
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+000996
  16:49:32 STEP  997 H= 997:00 +CPU= 43.650
  16:50:20 STEP  998 H= 998:00 +CPU= 44.998
  16:51:32 STEP  999 H= 999:00 +CPU= 68.679
  16:52:19 STEP 1000 H=1000:00 +CPU= 44.685
  16:53:05 STEP 1001 H=1001:00 +CPU= 43.591
  16:54:19 STEP 1002 H=1002:00 +CPU= 69.733
  16:55:05 STEP 1003 H=1003:00 +CPU= 43.420
  16:55:53 STEP 1004 H=1004:00 +CPU= 44.899
  16:57:06 STEP 1005 H=1005:00 +CPU= 68.712
  16:57:53 STEP 1006 H=1006:00 +CPU= 44.878
  16:58:41 STEP 1007 H=1007:00 +CPU= 44.958
  17:00:08 STEP 1008 H=1008:00 +CPU= 80.512
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001008
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001008
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001008
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000984
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000984
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000984
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+000996
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+000996
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+000996
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_41.zip
Uploading the intermediate file: upload_file_41.zip
  17:00:54 STEP 1009 H=1009:00 +CPU= 43.623
  17:01:42 STEP 1010 H=1010:00 +CPU= 44.830
  17:02:54 STEP 1011 H=1011:00 +CPU= 68.749
  17:03:42 STEP 1012 H=1012:00 +CPU= 45.124
  17:04:29 STEP 1013 H=1013:00 +CPU= 43.785
  17:05:43 STEP 1014 H=1014:00 +CPU= 70.344
  17:06:29 STEP 1015 H=1015:00 +CPU= 43.735
  17:07:17 STEP 1016 H=1016:00 +CPU= 44.971
  17:08:30 STEP 1017 H=1017:00 +CPU= 68.784
  17:09:17 STEP 1018 H=1018:00 +CPU= 45.033
  17:10:03 STEP 1019 H=1019:00 +CPU= 43.709
  17:11:25 STEP 1020 H=1020:00 +CPU= 76.774
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001020
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001020
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001020
  17:12:10 STEP 1021 H=1021:00 +CPU= 43.298
  17:12:58 STEP 1022 H=1022:00 +CPU= 44.882
  17:14:11 STEP 1023 H=1023:00 +CPU= 68.593
  17:14:58 STEP 1024 H=1024:00 +CPU= 44.956
  17:15:44 STEP 1025 H=1025:00 +CPU= 43.547
  17:16:58 STEP 1026 H=1026:00 +CPU= 69.686
  17:17:44 STEP 1027 H=1027:00 +CPU= 43.547
  17:18:32 STEP 1028 H=1028:00 +CPU= 44.680
  17:19:44 STEP 1029 H=1029:00 +CPU= 68.903
  17:20:32 STEP 1030 H=1030:00 +CPU= 44.748
  17:21:20 STEP 1031 H=1031:00 +CPU= 45.117
  17:22:47 STEP 1032 H=1032:00 +CPU= 81.025
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001032
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001032
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001032
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001008
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001008
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001008
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001020
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001020
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001020
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_42.zip
Uploading the intermediate file: upload_file_42.zip
  17:23:33 STEP 1033 H=1033:00 +CPU= 43.488
  17:24:21 STEP 1034 H=1034:00 +CPU= 44.928
  17:25:33 STEP 1035 H=1035:00 +CPU= 68.895
  17:26:21 STEP 1036 H=1036:00 +CPU= 44.856
  17:27:07 STEP 1037 H=1037:00 +CPU= 43.790
  17:28:21 STEP 1038 H=1038:00 +CPU= 70.201
  17:29:07 STEP 1039 H=1039:00 +CPU= 43.512
  17:29:55 STEP 1040 H=1040:00 +CPU= 44.949
  17:31:07 STEP 1041 H=1041:00 +CPU= 68.720
  17:31:55 STEP 1042 H=1042:00 +CPU= 44.978
  17:32:41 STEP 1043 H=1043:00 +CPU= 43.486
  17:34:02 STEP 1044 H=1044:00 +CPU= 76.832
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001044
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001044
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001044
  17:34:48 STEP 1045 H=1045:00 +CPU= 43.785
  17:35:36 STEP 1046 H=1046:00 +CPU= 45.025
  17:36:49 STEP 1047 H=1047:00 +CPU= 68.692
  17:37:36 STEP 1048 H=1048:00 +CPU= 44.949
  17:38:22 STEP 1049 H=1049:00 +CPU= 43.554
  17:39:36 STEP 1050 H=1050:00 +CPU= 70.158
  17:40:23 STEP 1051 H=1051:00 +CPU= 43.407
  17:41:10 STEP 1052 H=1052:00 +CPU= 44.891
  17:42:23 STEP 1053 H=1053:00 +CPU= 68.603
  17:43:10 STEP 1054 H=1054:00 +CPU= 44.873
  17:43:58 STEP 1055 H=1055:00 +CPU= 44.948
  17:45:24 STEP 1056 H=1056:00 +CPU= 80.961
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001056
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001056
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001056
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001032
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001032
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001032
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001044
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001044
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001044
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_43.zip
Uploading the intermediate file: upload_file_43.zip
  17:46:10 STEP 1057 H=1057:00 +CPU= 43.471
  17:46:58 STEP 1058 H=1058:00 +CPU= 45.082
  17:48:11 STEP 1059 H=1059:00 +CPU= 68.964
  17:48:58 STEP 1060 H=1060:00 +CPU= 44.914
  17:49:44 STEP 1061 H=1061:00 +CPU= 43.753
  17:50:59 STEP 1062 H=1062:00 +CPU= 70.397
  17:51:45 STEP 1063 H=1063:00 +CPU= 43.689
  17:52:33 STEP 1064 H=1064:00 +CPU= 45.013
  17:53:45 STEP 1065 H=1065:00 +CPU= 68.751
  17:54:33 STEP 1066 H=1066:00 +CPU= 44.888
  17:55:19 STEP 1067 H=1067:00 +CPU= 43.667
  17:56:40 STEP 1068 H=1068:00 +CPU= 76.539
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001068
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001068
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001068
  17:57:26 STEP 1069 H=1069:00 +CPU= 43.585
  17:58:13 STEP 1070 H=1070:00 +CPU= 44.852
  17:59:26 STEP 1071 H=1071:00 +CPU= 68.801
  18:00:13 STEP 1072 H=1072:00 +CPU= 44.752
  18:00:59 STEP 1073 H=1073:00 +CPU= 43.504
  18:02:13 STEP 1074 H=1074:00 +CPU= 70.290
  18:02:59 STEP 1075 H=1075:00 +CPU= 43.414
  18:03:47 STEP 1076 H=1076:00 +CPU= 45.075
  18:04:59 STEP 1077 H=1077:00 +CPU= 68.614
  18:05:47 STEP 1078 H=1078:00 +CPU= 44.747
  18:06:34 STEP 1079 H=1079:00 +CPU= 44.993
  18:08:01 STEP 1080 H=1080:00 +CPU= 80.671
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001080
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001080
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001080
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001056
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001056
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001056
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001068
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001068
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001068
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_44.zip
Uploading the intermediate file: upload_file_44.zip
  18:08:47 STEP 1081 H=1081:00 +CPU= 43.530
  18:09:35 STEP 1082 H=1082:00 +CPU= 45.036
  18:10:47 STEP 1083 H=1083:00 +CPU= 68.560
  18:11:35 STEP 1084 H=1084:00 +CPU= 44.790
  18:12:20 STEP 1085 H=1085:00 +CPU= 43.462
  18:13:34 STEP 1086 H=1086:00 +CPU= 70.212
  18:14:21 STEP 1087 H=1087:00 +CPU= 43.617
  18:15:08 STEP 1088 H=1088:00 +CPU= 45.040
  18:16:21 STEP 1089 H=1089:00 +CPU= 68.909
  18:17:09 STEP 1090 H=1090:00 +CPU= 45.006
  18:17:55 STEP 1091 H=1091:00 +CPU= 43.623
  18:19:16 STEP 1092 H=1092:00 +CPU= 76.857
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001092
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001092
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001092
  18:20:02 STEP 1093 H=1093:00 +CPU= 43.600
  18:20:50 STEP 1094 H=1094:00 +CPU= 44.911
  18:22:03 STEP 1095 H=1095:00 +CPU= 68.841
  18:22:50 STEP 1096 H=1096:00 +CPU= 44.860
  18:23:36 STEP 1097 H=1097:00 +CPU= 43.591
  18:24:50 STEP 1098 H=1098:00 +CPU= 70.124
  18:25:36 STEP 1099 H=1099:00 +CPU= 43.582
  18:26:23 STEP 1100 H=1100:00 +CPU= 44.759
  18:27:36 STEP 1101 H=1101:00 +CPU= 68.873
  18:28:23 STEP 1102 H=1102:00 +CPU= 44.739
  18:29:11 STEP 1103 H=1103:00 +CPU= 45.007
  18:30:38 STEP 1104 H=1104:00 +CPU= 80.676
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001104
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001104
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001104
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001080
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001080
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001080
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001092
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001092
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001092
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_45.zip
Uploading the intermediate file: upload_file_45.zip
  18:31:24 STEP 1105 H=1105:00 +CPU= 43.605
  18:32:12 STEP 1106 H=1106:00 +CPU= 44.776
  18:33:24 STEP 1107 H=1107:00 +CPU= 68.813
  18:34:12 STEP 1108 H=1108:00 +CPU= 44.782
  18:34:58 STEP 1109 H=1109:00 +CPU= 43.597
  18:36:12 STEP 1110 H=1110:00 +CPU= 70.294
  18:36:58 STEP 1111 H=1111:00 +CPU= 43.758
  18:37:46 STEP 1112 H=1112:00 +CPU= 44.995
  18:38:59 STEP 1113 H=1113:00 +CPU= 68.779
  18:39:46 STEP 1114 H=1114:00 +CPU= 44.951
  18:40:32 STEP 1115 H=1115:00 +CPU= 43.590
  18:41:53 STEP 1116 H=1116:00 +CPU= 76.816
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001116
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001116
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001116
  18:42:40 STEP 1117 H=1117:00 +CPU= 43.808
  18:43:27 STEP 1118 H=1118:00 +CPU= 44.981
  18:44:40 STEP 1119 H=1119:00 +CPU= 68.782
  18:45:27 STEP 1120 H=1120:00 +CPU= 44.860
  18:46:13 STEP 1121 H=1121:00 +CPU= 43.634
  18:47:28 STEP 1122 H=1122:00 +CPU= 70.241
  18:48:14 STEP 1123 H=1123:00 +CPU= 43.595
  18:49:01 STEP 1124 H=1124:00 +CPU= 44.782
  18:50:14 STEP 1125 H=1125:00 +CPU= 68.601
  18:51:01 STEP 1126 H=1126:00 +CPU= 44.696
  18:51:49 STEP 1127 H=1127:00 +CPU= 45.105
  18:53:16 STEP 1128 H=1128:00 +CPU= 81.001
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001128
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001128
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001128
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001104
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001104
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001104
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001116
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001116
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001116
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_46.zip
Uploading the intermediate file: upload_file_46.zip
  18:54:02 STEP 1129 H=1129:00 +CPU= 43.359
  18:54:50 STEP 1130 H=1130:00 +CPU= 44.886
  18:56:02 STEP 1131 H=1131:00 +CPU= 68.865
  18:56:50 STEP 1132 H=1132:00 +CPU= 44.966
  18:57:36 STEP 1133 H=1133:00 +CPU= 43.532
  18:58:50 STEP 1134 H=1134:00 +CPU= 70.193
  18:59:36 STEP 1135 H=1135:00 +CPU= 43.625
  19:00:23 STEP 1136 H=1136:00 +CPU= 44.622
  19:01:36 STEP 1137 H=1137:00 +CPU= 68.708
  19:02:23 STEP 1138 H=1138:00 +CPU= 44.808
  19:03:10 STEP 1139 H=1139:00 +CPU= 43.640
  19:04:31 STEP 1140 H=1140:00 +CPU= 76.584
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001140
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001140
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001140
  19:05:17 STEP 1141 H=1141:00 +CPU= 43.637
  19:06:04 STEP 1142 H=1142:00 +CPU= 44.758
  19:07:17 STEP 1143 H=1143:00 +CPU= 68.818
  19:08:04 STEP 1144 H=1144:00 +CPU= 44.888
  19:08:51 STEP 1145 H=1145:00 +CPU= 43.744
  19:10:05 STEP 1146 H=1146:00 +CPU= 70.203
  19:10:51 STEP 1147 H=1147:00 +CPU= 43.516
  19:11:38 STEP 1148 H=1148:00 +CPU= 44.894
  19:12:51 STEP 1149 H=1149:00 +CPU= 68.677
  19:13:39 STEP 1150 H=1150:00 +CPU= 44.730
  19:14:26 STEP 1151 H=1151:00 +CPU= 44.954
  19:15:54 STEP 1152 H=1152:00 +CPU= 80.827
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001152
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001152
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001152
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001128
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001128
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001128
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001140
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001140
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001140
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_47.zip
Uploading the intermediate file: upload_file_47.zip
  19:16:40 STEP 1153 H=1153:00 +CPU= 43.504
  19:17:28 STEP 1154 H=1154:00 +CPU= 44.969
  19:18:40 STEP 1155 H=1155:00 +CPU= 68.485
  19:19:28 STEP 1156 H=1156:00 +CPU= 44.950
  19:20:14 STEP 1157 H=1157:00 +CPU= 43.565
  19:21:28 STEP 1158 H=1158:00 +CPU= 70.353
  19:22:14 STEP 1159 H=1159:00 +CPU= 43.699
  19:23:02 STEP 1160 H=1160:00 +CPU= 44.882
  19:24:14 STEP 1161 H=1161:00 +CPU= 68.764
  19:25:02 STEP 1162 H=1162:00 +CPU= 44.968
  19:25:48 STEP 1163 H=1163:00 +CPU= 43.585
  19:27:09 STEP 1164 H=1164:00 +CPU= 76.582
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001164
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001164
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001164
  19:27:55 STEP 1165 H=1165:00 +CPU= 43.504
  19:28:42 STEP 1166 H=1166:00 +CPU= 44.925
  19:29:55 STEP 1167 H=1167:00 +CPU= 68.678
  19:30:42 STEP 1168 H=1168:00 +CPU= 44.856
  19:31:28 STEP 1169 H=1169:00 +CPU= 43.556
  19:32:42 STEP 1170 H=1170:00 +CPU= 69.940
  19:33:29 STEP 1171 H=1171:00 +CPU= 43.657
  19:34:16 STEP 1172 H=1172:00 +CPU= 44.683
  19:35:28 STEP 1173 H=1173:00 +CPU= 68.746
  19:36:16 STEP 1174 H=1174:00 +CPU= 44.658
  19:37:03 STEP 1175 H=1175:00 +CPU= 44.847
  19:38:31 STEP 1176 H=1176:00 +CPU= 80.571
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001176
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001176
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001176
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001152
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001152
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001152
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001164
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001164
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001164
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_48.zip
Uploading the intermediate file: upload_file_48.zip
  19:39:17 STEP 1177 H=1177:00 +CPU= 43.361
  19:40:04 STEP 1178 H=1178:00 +CPU= 44.777
  19:41:17 STEP 1179 H=1179:00 +CPU= 68.682
  19:42:05 STEP 1180 H=1180:00 +CPU= 44.949
  19:42:51 STEP 1181 H=1181:00 +CPU= 43.557
  19:44:05 STEP 1182 H=1182:00 +CPU= 70.196
  19:44:51 STEP 1183 H=1183:00 +CPU= 43.621
  19:45:38 STEP 1184 H=1184:00 +CPU= 44.993
  19:46:51 STEP 1185 H=1185:00 +CPU= 68.727
  19:47:39 STEP 1186 H=1186:00 +CPU= 45.070
  19:48:25 STEP 1187 H=1187:00 +CPU= 43.677
  19:49:46 STEP 1188 H=1188:00 +CPU= 76.651
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001188
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001188
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001188
  19:50:32 STEP 1189 H=1189:00 +CPU= 43.541
  19:51:20 STEP 1190 H=1190:00 +CPU= 44.962
  19:52:32 STEP 1191 H=1191:00 +CPU= 68.628
  19:53:19 STEP 1192 H=1192:00 +CPU= 44.759
  19:54:05 STEP 1193 H=1193:00 +CPU= 43.512
  19:55:19 STEP 1194 H=1194:00 +CPU= 70.035
  19:56:06 STEP 1195 H=1195:00 +CPU= 43.576
  19:56:53 STEP 1196 H=1196:00 +CPU= 44.838
  19:58:05 STEP 1197 H=1197:00 +CPU= 68.637
  19:58:53 STEP 1198 H=1198:00 +CPU= 44.780
  19:59:40 STEP 1199 H=1199:00 +CPU= 45.052
  20:01:08 STEP 1200 H=1200:00 +CPU= 80.469
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001200
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001200
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001200
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001176
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001176
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001176
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001188
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001188
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001188
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_49.zip
Uploading the intermediate file: upload_file_49.zip
Uploading trickle at timestep: 4320000
  20:01:54 STEP 1201 H=1201:00 +CPU= 43.493
  20:02:42 STEP 1202 H=1202:00 +CPU= 44.914
  20:03:54 STEP 1203 H=1203:00 +CPU= 68.660
  20:04:42 STEP 1204 H=1204:00 +CPU= 44.850
  20:05:28 STEP 1205 H=1205:00 +CPU= 43.567
  20:06:42 STEP 1206 H=1206:00 +CPU= 69.997
  20:07:28 STEP 1207 H=1207:00 +CPU= 43.518
  20:08:15 STEP 1208 H=1208:00 +CPU= 44.872
  20:09:28 STEP 1209 H=1209:00 +CPU= 68.681
  20:10:15 STEP 1210 H=1210:00 +CPU= 44.929
  20:11:02 STEP 1211 H=1211:00 +CPU= 43.863
  20:12:23 STEP 1212 H=1212:00 +CPU= 76.674
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001212
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001212
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001212
  20:13:09 STEP 1213 H=1213:00 +CPU= 43.744
  20:13:56 STEP 1214 H=1214:00 +CPU= 44.773
  20:15:09 STEP 1215 H=1215:00 +CPU= 68.386
  20:15:56 STEP 1216 H=1216:00 +CPU= 44.786
  20:16:42 STEP 1217 H=1217:00 +CPU= 43.482
  20:17:56 STEP 1218 H=1218:00 +CPU= 70.109
  20:18:42 STEP 1219 H=1219:00 +CPU= 43.571
  20:19:30 STEP 1220 H=1220:00 +CPU= 45.019
  20:20:43 STEP 1221 H=1221:00 +CPU= 68.733
  20:21:30 STEP 1222 H=1222:00 +CPU= 44.841
  20:22:18 STEP 1223 H=1223:00 +CPU= 44.934
  20:23:46 STEP 1224 H=1224:00 +CPU= 80.891
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001224
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001224
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001224
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001200
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001200
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001200
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001212
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001212
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001212
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_50.zip
Uploading the intermediate file: upload_file_50.zip
  20:24:32 STEP 1225 H=1225:00 +CPU= 43.472
  20:25:19 STEP 1226 H=1226:00 +CPU= 44.901
  20:26:32 STEP 1227 H=1227:00 +CPU= 68.774
  20:27:20 STEP 1228 H=1228:00 +CPU= 44.838
  20:28:06 STEP 1229 H=1229:00 +CPU= 43.717
  20:29:20 STEP 1230 H=1230:00 +CPU= 70.376
  20:30:07 STEP 1231 H=1231:00 +CPU= 43.723
  20:30:54 STEP 1232 H=1232:00 +CPU= 45.029
  20:32:07 STEP 1233 H=1233:00 +CPU= 68.826
  20:32:57 STEP 1234 H=1234:00 +CPU= 44.598
  20:33:44 STEP 1235 H=1235:00 +CPU= 43.790
  20:35:05 STEP 1236 H=1236:00 +CPU= 77.021
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001236
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001236
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001236
  20:35:51 STEP 1237 H=1237:00 +CPU= 43.667
  20:36:39 STEP 1238 H=1238:00 +CPU= 44.895
  20:37:52 STEP 1239 H=1239:00 +CPU= 68.940
  20:38:39 STEP 1240 H=1240:00 +CPU= 44.879
  20:39:25 STEP 1241 H=1241:00 +CPU= 43.734
  20:40:40 STEP 1242 H=1242:00 +CPU= 70.127
  20:41:26 STEP 1243 H=1243:00 +CPU= 43.683
  20:42:13 STEP 1244 H=1244:00 +CPU= 44.881
  20:43:26 STEP 1245 H=1245:00 +CPU= 68.693
  20:44:14 STEP 1246 H=1246:00 +CPU= 44.797
  20:45:01 STEP 1247 H=1247:00 +CPU= 45.093
  20:46:29 STEP 1248 H=1248:00 +CPU= 80.843
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001248
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001248
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001248
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001224
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001224
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001224
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001236
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001236
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001236
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_51.zip
Uploading the intermediate file: upload_file_51.zip
  20:47:15 STEP 1249 H=1249:00 +CPU= 43.547
  20:48:03 STEP 1250 H=1250:00 +CPU= 44.738
  20:49:15 STEP 1251 H=1251:00 +CPU= 68.801
  20:50:03 STEP 1252 H=1252:00 +CPU= 44.838
  20:50:49 STEP 1253 H=1253:00 +CPU= 43.664
  20:52:04 STEP 1254 H=1254:00 +CPU= 70.259
  20:52:50 STEP 1255 H=1255:00 +CPU= 43.517
  20:53:37 STEP 1256 H=1256:00 +CPU= 45.001
  20:54:50 STEP 1257 H=1257:00 +CPU= 68.618
  20:55:37 STEP 1258 H=1258:00 +CPU= 44.987
  20:56:23 STEP 1259 H=1259:00 +CPU= 43.570
  20:57:45 STEP 1260 H=1260:00 +CPU= 76.945
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001260
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001260
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001260
  20:58:31 STEP 1261 H=1261:00 +CPU= 43.663
  20:59:19 STEP 1262 H=1262:00 +CPU= 45.067
  21:00:31 STEP 1263 H=1263:00 +CPU= 68.650
  21:01:19 STEP 1264 H=1264:00 +CPU= 44.914
  21:02:05 STEP 1265 H=1265:00 +CPU= 43.486
  21:03:19 STEP 1266 H=1266:00 +CPU= 70.274
  21:04:05 STEP 1267 H=1267:00 +CPU= 43.570
  21:04:52 STEP 1268 H=1268:00 +CPU= 44.840
  21:06:05 STEP 1269 H=1269:00 +CPU= 68.717
  21:06:52 STEP 1270 H=1270:00 +CPU= 44.837
  21:07:40 STEP 1271 H=1271:00 +CPU= 45.210
  21:09:08 STEP 1272 H=1272:00 +CPU= 80.738
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001272
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001272
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001272
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001248
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001248
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001248
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001260
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001260
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001260
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_52.zip
Uploading the intermediate file: upload_file_52.zip
  21:09:54 STEP 1273 H=1273:00 +CPU= 43.483
  21:10:42 STEP 1274 H=1274:00 +CPU= 44.926
  21:11:54 STEP 1275 H=1275:00 +CPU= 68.638
  21:12:42 STEP 1276 H=1276:00 +CPU= 44.852
  21:13:28 STEP 1277 H=1277:00 +CPU= 43.713
  21:14:43 STEP 1278 H=1278:00 +CPU= 70.444
  21:15:29 STEP 1279 H=1279:00 +CPU= 43.732
  21:16:16 STEP 1280 H=1280:00 +CPU= 44.796
  21:17:29 STEP 1281 H=1281:00 +CPU= 68.980
  21:18:17 STEP 1282 H=1282:00 +CPU= 44.937
  21:19:03 STEP 1283 H=1283:00 +CPU= 43.699
  21:20:24 STEP 1284 H=1284:00 +CPU= 76.772
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001284
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001284
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001284
  21:21:10 STEP 1285 H=1285:00 +CPU= 43.652
  21:21:58 STEP 1286 H=1286:00 +CPU= 45.065
  21:23:11 STEP 1287 H=1287:00 +CPU= 68.735
  21:23:58 STEP 1288 H=1288:00 +CPU= 44.958
  21:24:44 STEP 1289 H=1289:00 +CPU= 43.681
  21:25:58 STEP 1290 H=1290:00 +CPU= 69.926
  21:26:44 STEP 1291 H=1291:00 +CPU= 43.532
  21:27:32 STEP 1292 H=1292:00 +CPU= 44.945
  21:28:44 STEP 1293 H=1293:00 +CPU= 68.740
  21:29:32 STEP 1294 H=1294:00 +CPU= 44.793
  21:30:19 STEP 1295 H=1295:00 +CPU= 44.927
  21:31:46 STEP 1296 H=1296:00 +CPU= 80.869
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001296
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001296
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001296
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001272
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001272
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001272
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001284
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001284
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001284
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_53.zip
Uploading the intermediate file: upload_file_53.zip
  21:32:32 STEP 1297 H=1297:00 +CPU= 43.500
  21:33:20 STEP 1298 H=1298:00 +CPU= 44.834
  21:34:32 STEP 1299 H=1299:00 +CPU= 68.576
  21:35:19 STEP 1300 H=1300:00 +CPU= 44.791
  21:36:05 STEP 1301 H=1301:00 +CPU= 43.477
  21:37:19 STEP 1302 H=1302:00 +CPU= 70.218
  21:38:05 STEP 1303 H=1303:00 +CPU= 43.569
  21:38:53 STEP 1304 H=1304:00 +CPU= 45.096
  21:40:06 STEP 1305 H=1305:00 +CPU= 68.738
  21:40:53 STEP 1306 H=1306:00 +CPU= 44.762
  21:41:39 STEP 1307 H=1307:00 +CPU= 43.783
  21:43:00 STEP 1308 H=1308:00 +CPU= 76.926
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001308
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001308
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001308
  21:43:47 STEP 1309 H=1309:00 +CPU= 43.631
  21:44:34 STEP 1310 H=1310:00 +CPU= 45.088
  21:45:47 STEP 1311 H=1311:00 +CPU= 68.988
  21:46:35 STEP 1312 H=1312:00 +CPU= 44.937
  21:47:21 STEP 1313 H=1313:00 +CPU= 43.642
  21:48:35 STEP 1314 H=1314:00 +CPU= 70.215
  21:49:21 STEP 1315 H=1315:00 +CPU= 43.632
  21:50:09 STEP 1316 H=1316:00 +CPU= 44.740
  21:51:21 STEP 1317 H=1317:00 +CPU= 68.921
  21:52:09 STEP 1318 H=1318:00 +CPU= 44.759
  21:52:56 STEP 1319 H=1319:00 +CPU= 44.959
  21:54:23 STEP 1320 H=1320:00 +CPU= 80.711
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001320
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001320
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001320
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001296
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001296
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001296
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001308
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001308
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001308
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_54.zip
Uploading the intermediate file: upload_file_54.zip
  21:55:09 STEP 1321 H=1321:00 +CPU= 43.548
  21:55:56 STEP 1322 H=1322:00 +CPU= 44.798
  21:57:09 STEP 1323 H=1323:00 +CPU= 68.796
  21:57:56 STEP 1324 H=1324:00 +CPU= 44.784
  21:58:42 STEP 1325 H=1325:00 +CPU= 43.541
  21:59:56 STEP 1326 H=1326:00 +CPU= 70.238
  22:00:43 STEP 1327 H=1327:00 +CPU= 43.682
  22:01:30 STEP 1328 H=1328:00 +CPU= 44.874
  22:02:43 STEP 1329 H=1329:00 +CPU= 68.819
  22:03:30 STEP 1330 H=1330:00 +CPU= 44.955
  22:04:16 STEP 1331 H=1331:00 +CPU= 43.637
  22:05:38 STEP 1332 H=1332:00 +CPU= 77.281
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001332
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001332
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001332
  22:06:24 STEP 1333 H=1333:00 +CPU= 43.650
  22:07:12 STEP 1334 H=1334:00 +CPU= 44.971
  22:08:25 STEP 1335 H=1335:00 +CPU= 68.827
  22:09:12 STEP 1336 H=1336:00 +CPU= 44.871
  22:09:58 STEP 1337 H=1337:00 +CPU= 43.720
  22:11:12 STEP 1338 H=1338:00 +CPU= 70.345
  22:11:59 STEP 1339 H=1339:00 +CPU= 43.703
  22:12:46 STEP 1340 H=1340:00 +CPU= 44.698
  22:13:58 STEP 1341 H=1341:00 +CPU= 68.571
  22:14:45 STEP 1342 H=1342:00 +CPU= 44.872
  22:15:33 STEP 1343 H=1343:00 +CPU= 45.022
  22:17:00 STEP 1344 H=1344:00 +CPU= 80.689
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001344
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001344
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001344
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001320
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001320
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001320
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001332
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001332
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001332
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_55.zip
Uploading the intermediate file: upload_file_55.zip
  22:17:46 STEP 1345 H=1345:00 +CPU= 43.441
  22:18:33 STEP 1346 H=1346:00 +CPU= 44.841
  22:19:46 STEP 1347 H=1347:00 +CPU= 68.743
  22:20:33 STEP 1348 H=1348:00 +CPU= 44.700
  22:21:19 STEP 1349 H=1349:00 +CPU= 43.519
  22:22:33 STEP 1350 H=1350:00 +CPU= 70.183
  22:23:19 STEP 1351 H=1351:00 +CPU= 43.603
  22:24:07 STEP 1352 H=1352:00 +CPU= 44.903
  22:25:19 STEP 1353 H=1353:00 +CPU= 68.690
  22:26:07 STEP 1354 H=1354:00 +CPU= 44.846
  22:26:53 STEP 1355 H=1355:00 +CPU= 43.567
  22:28:14 STEP 1356 H=1356:00 +CPU= 76.785
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001356
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001356
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001356
  22:29:00 STEP 1357 H=1357:00 +CPU= 43.672
  22:29:48 STEP 1358 H=1358:00 +CPU= 44.940
  22:31:00 STEP 1359 H=1359:00 +CPU= 68.679
  22:31:48 STEP 1360 H=1360:00 +CPU= 44.908
  22:32:33 STEP 1361 H=1361:00 +CPU= 43.415
  22:33:47 STEP 1362 H=1362:00 +CPU= 70.133
  22:34:34 STEP 1363 H=1363:00 +CPU= 43.622
  22:35:21 STEP 1364 H=1364:00 +CPU= 44.864
  22:36:33 STEP 1365 H=1365:00 +CPU= 68.553
  22:37:21 STEP 1366 H=1366:00 +CPU= 44.832
  22:38:08 STEP 1367 H=1367:00 +CPU= 44.855
  22:39:34 STEP 1368 H=1368:00 +CPU= 80.545
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001368
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001368
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001368
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001344
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001344
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001344
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001356
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001356
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001356
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_56.zip
Uploading the intermediate file: upload_file_56.zip
  22:40:20 STEP 1369 H=1369:00 +CPU= 43.390
  22:41:08 STEP 1370 H=1370:00 +CPU= 44.819
  22:42:20 STEP 1371 H=1371:00 +CPU= 68.648
  22:43:08 STEP 1372 H=1372:00 +CPU= 44.870
  22:43:54 STEP 1373 H=1373:00 +CPU= 43.632
  22:45:08 STEP 1374 H=1374:00 +CPU= 70.328
  22:45:54 STEP 1375 H=1375:00 +CPU= 43.462
  22:46:41 STEP 1376 H=1376:00 +CPU= 44.989
  22:47:54 STEP 1377 H=1377:00 +CPU= 68.799
  22:48:42 STEP 1378 H=1378:00 +CPU= 44.738
  22:49:27 STEP 1379 H=1379:00 +CPU= 43.380
  22:50:48 STEP 1380 H=1380:00 +CPU= 76.797
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001380
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001380
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001380
  22:51:34 STEP 1381 H=1381:00 +CPU= 43.676
  22:52:22 STEP 1382 H=1382:00 +CPU= 44.712
  22:53:34 STEP 1383 H=1383:00 +CPU= 68.742
  22:54:21 STEP 1384 H=1384:00 +CPU= 44.747
  22:55:07 STEP 1385 H=1385:00 +CPU= 43.647
  22:56:21 STEP 1386 H=1386:00 +CPU= 70.150
  22:57:07 STEP 1387 H=1387:00 +CPU= 43.478
  22:57:54 STEP 1388 H=1388:00 +CPU= 44.887
  22:59:07 STEP 1389 H=1389:00 +CPU= 68.744
  22:59:54 STEP 1390 H=1390:00 +CPU= 44.793
  23:00:42 STEP 1391 H=1391:00 +CPU= 44.868
  23:02:08 STEP 1392 H=1392:00 +CPU= 80.671
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001392
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001392
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001392
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001368
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001368
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001368
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001380
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001380
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001380
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_57.zip
Uploading the intermediate file: upload_file_57.zip
  23:02:54 STEP 1393 H=1393:00 +CPU= 43.511
  23:03:41 STEP 1394 H=1394:00 +CPU= 44.767
  23:04:54 STEP 1395 H=1395:00 +CPU= 68.783
  23:05:41 STEP 1396 H=1396:00 +CPU= 44.779
  23:06:27 STEP 1397 H=1397:00 +CPU= 43.654
  23:07:41 STEP 1398 H=1398:00 +CPU= 70.108
  23:08:27 STEP 1399 H=1399:00 +CPU= 43.556
  23:09:14 STEP 1400 H=1400:00 +CPU= 44.894
  23:10:27 STEP 1401 H=1401:00 +CPU= 68.623
  23:11:14 STEP 1402 H=1402:00 +CPU= 44.810
  23:12:00 STEP 1403 H=1403:00 +CPU= 43.458
  23:13:21 STEP 1404 H=1404:00 +CPU= 76.775
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001404
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001404
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001404
  23:14:06 STEP 1405 H=1405:00 +CPU= 43.486
  23:14:54 STEP 1406 H=1406:00 +CPU= 44.819
  23:16:06 STEP 1407 H=1407:00 +CPU= 68.589
  23:16:53 STEP 1408 H=1408:00 +CPU= 44.830
  23:17:39 STEP 1409 H=1409:00 +CPU= 43.438
  23:18:53 STEP 1410 H=1410:00 +CPU= 70.227
  23:19:39 STEP 1411 H=1411:00 +CPU= 43.668
  23:20:26 STEP 1412 H=1412:00 +CPU= 44.756
  23:21:39 STEP 1413 H=1413:00 +CPU= 68.684
  23:22:26 STEP 1414 H=1414:00 +CPU= 44.840
  23:23:14 STEP 1415 H=1415:00 +CPU= 45.022
  23:24:40 STEP 1416 H=1416:00 +CPU= 80.652
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001416
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001416
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001416
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001392
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001392
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001392
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001404
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001404
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001404
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_58.zip
Uploading the intermediate file: upload_file_58.zip
  23:25:26 STEP 1417 H=1417:00 +CPU= 43.501
  23:26:13 STEP 1418 H=1418:00 +CPU= 44.864
  23:27:26 STEP 1419 H=1419:00 +CPU= 68.735
  23:28:13 STEP 1420 H=1420:00 +CPU= 44.765
  23:28:59 STEP 1421 H=1421:00 +CPU= 43.620
  23:30:13 STEP 1422 H=1422:00 +CPU= 70.164
  23:30:59 STEP 1423 H=1423:00 +CPU= 43.705
  23:31:46 STEP 1424 H=1424:00 +CPU= 44.863
  23:32:58 STEP 1425 H=1425:00 +CPU= 68.628
  23:33:46 STEP 1426 H=1426:00 +CPU= 44.960
  23:34:32 STEP 1427 H=1427:00 +CPU= 43.476
  23:35:52 STEP 1428 H=1428:00 +CPU= 76.678
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001428
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001428
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001428
  23:36:38 STEP 1429 H=1429:00 +CPU= 43.560
  23:37:26 STEP 1430 H=1430:00 +CPU= 44.808
  23:38:38 STEP 1431 H=1431:00 +CPU= 68.703
  23:39:25 STEP 1432 H=1432:00 +CPU= 44.909
  23:40:11 STEP 1433 H=1433:00 +CPU= 43.521
  23:41:25 STEP 1434 H=1434:00 +CPU= 70.276
  23:42:11 STEP 1435 H=1435:00 +CPU= 43.554
  23:42:58 STEP 1436 H=1436:00 +CPU= 44.899
  23:44:11 STEP 1437 H=1437:00 +CPU= 68.818
  23:44:58 STEP 1438 H=1438:00 +CPU= 44.711
  23:45:45 STEP 1439 H=1439:00 +CPU= 44.974
  23:47:12 STEP 1440 H=1440:00 +CPU= 80.776
Moving to projects directory: /home/luuser/slots/0/ICMGGhq0f+001440
Moving to projects directory: /home/luuser/slots/0/ICMSHhq0f+001440
Moving to projects directory: /home/luuser/slots/0/ICMUAhq0f+001440
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001416
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001416
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001416
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMGGhq0f+001428
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMSHhq0f+001428
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12182992/ICMUAhq0f+001428
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0348_1997050100_123_966_12182992_2_r1555048885_59.zip
Uploading the intermediate file: upload_file_59.zip
Uploading trickle at timestep: 5184000
  23:47:59 STEP 1441 H=1441:00 +CPU= 43.490
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:14:04 1537854 22295621 oifs_43r3_ps_0348_1997050100_123_966_12182992_2 5,184,000 77,070 0.0149
09 May 2023 16:13:55 1537854 22295621 oifs_43r3_ps_0348_1997050100_123_966_12182992_2 4,320,000 64,251 0.0149
09 May 2023 16:13:45 1537854 22295621 oifs_43r3_ps_0348_1997050100_123_966_12182992_2 3,456,000 51,425 0.0149
09 May 2023 16:13:35 1537854 22295621 oifs_43r3_ps_0348_1997050100_123_966_12182992_2 2,592,000 38,600 0.0149
09 May 2023 16:13:26 1537854 22295621 oifs_43r3_ps_0348_1997050100_123_966_12182992_2 1,728,000 25,762 0.0149
09 May 2023 16:13:19 1537854 22295621 oifs_43r3_ps_0348_1997050100_123_966_12182992_2 864,000 12,927 0.0150


©2024 climateprediction.net