climateprediction.net home page
Task 22265411

Task 22265411

Name oifs_43r3_ps_0806_1994050100_123_963_12180450_0
Workunit 12180450
Created 22 Dec 2022, 6:42:15 UTC
Sent 22 Dec 2022, 6:56:44 UTC
Report deadline 21 Jan 2023, 6:56:44 UTC
Received 25 Dec 2022, 19:16:46 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1537224
Run time 19 hours 42 min 15 sec
CPU time 17 hours 44 min 13 sec
Validate state Invalid
Credit 11,007.36
Device peak FLOPS 5.24 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.75 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,234.66 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
oving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001860
  11:28:27 STEP 1861 H=1861:00 +CPU= 21.167
  11:28:50 STEP 1862 H=1862:00 +CPU= 22.125
  11:29:24 STEP 1863 H=1863:00 +CPU= 33.849
  11:29:46 STEP 1864 H=1864:00 +CPU= 21.961
  11:30:08 STEP 1865 H=1865:00 +CPU= 21.721
  11:30:43 STEP 1866 H=1866:00 +CPU= 34.478
  11:31:05 STEP 1867 H=1867:00 +CPU= 21.923
  11:31:28 STEP 1868 H=1868:00 +CPU= 22.869
  11:32:03 STEP 1869 H=1869:00 +CPU= 34.433
  11:32:26 STEP 1870 H=1870:00 +CPU= 22.364
  11:32:49 STEP 1871 H=1871:00 +CPU= 22.522
  11:33:32 STEP 1872 H=1872:00 +CPU= 41.359
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001872
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001872
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001860
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_77.zip
Uploading the intermediate file: upload_file_77.zip
  11:33:54 STEP 1873 H=1873:00 +CPU= 22.272
  11:34:18 STEP 1874 H=1874:00 +CPU= 23.121
  11:34:52 STEP 1875 H=1875:00 +CPU= 34.079
  11:35:15 STEP 1876 H=1876:00 +CPU= 22.165
  11:35:36 STEP 1877 H=1877:00 +CPU= 21.449
  11:36:12 STEP 1878 H=1878:00 +CPU= 35.862
  11:36:35 STEP 1879 H=1879:00 +CPU= 21.945
  11:36:58 STEP 1880 H=1880:00 +CPU= 22.770
  11:37:32 STEP 1881 H=1881:00 +CPU= 34.510
  11:37:55 STEP 1882 H=1882:00 +CPU= 22.780
  11:38:18 STEP 1883 H=1883:00 +CPU= 22.266
  11:38:58 STEP 1884 H=1884:00 +CPU= 39.798
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001884
  11:39:21 STEP 1885 H=1885:00 +CPU= 22.604
  11:39:44 STEP 1886 H=1886:00 +CPU= 23.314
  11:40:20 STEP 1887 H=1887:00 +CPU= 34.723
  11:40:43 STEP 1888 H=1888:00 +CPU= 22.931
  11:41:05 STEP 1889 H=1889:00 +CPU= 22.267
  11:41:41 STEP 1890 H=1890:00 +CPU= 34.961
  11:42:03 STEP 1891 H=1891:00 +CPU= 22.475
  11:42:27 STEP 1892 H=1892:00 +CPU= 22.552
  11:43:01 STEP 1893 H=1893:00 +CPU= 34.057
  11:43:24 STEP 1894 H=1894:00 +CPU= 22.567
  11:43:46 STEP 1895 H=1895:00 +CPU= 22.069
  11:44:28 STEP 1896 H=1896:00 +CPU= 41.126
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001884
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_78.zip
Uploading the intermediate file: upload_file_78.zip
  11:44:52 STEP 1897 H=1897:00 +CPU= 21.910
  11:45:15 STEP 1898 H=1898:00 +CPU= 22.361
  11:45:48 STEP 1899 H=1899:00 +CPU= 33.199
  11:46:10 STEP 1900 H=1900:00 +CPU= 22.048
  11:46:32 STEP 1901 H=1901:00 +CPU= 21.348
  11:47:07 STEP 1902 H=1902:00 +CPU= 34.511
  11:47:28 STEP 1903 H=1903:00 +CPU= 21.663
  11:47:51 STEP 1904 H=1904:00 +CPU= 22.268
  11:48:25 STEP 1905 H=1905:00 +CPU= 34.280
  11:48:49 STEP 1906 H=1906:00 +CPU= 23.641
  11:49:13 STEP 1907 H=1907:00 +CPU= 23.495
  11:49:53 STEP 1908 H=1908:00 +CPU= 38.096
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001908
  11:50:16 STEP 1909 H=1909:00 +CPU= 22.223
  11:50:39 STEP 1910 H=1910:00 +CPU= 22.851
  11:51:14 STEP 1911 H=1911:00 +CPU= 34.649
  11:51:36 STEP 1912 H=1912:00 +CPU= 22.140
  11:51:58 STEP 1913 H=1913:00 +CPU= 21.098
  11:52:32 STEP 1914 H=1914:00 +CPU= 34.258
  11:52:54 STEP 1915 H=1915:00 +CPU= 21.526
  11:53:16 STEP 1916 H=1916:00 +CPU= 22.337
  11:53:50 STEP 1917 H=1917:00 +CPU= 33.895
  11:54:13 STEP 1918 H=1918:00 +CPU= 22.282
  11:54:35 STEP 1919 H=1919:00 +CPU= 22.075
  11:55:16 STEP 1920 H=1920:00 +CPU= 39.984
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001908
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_79.zip
Uploading the intermediate file: upload_file_79.zip
  11:55:38 STEP 1921 H=1921:00 +CPU= 21.511
  11:56:00 STEP 1922 H=1922:00 +CPU= 22.031
  11:56:34 STEP 1923 H=1923:00 +CPU= 33.606
  11:56:56 STEP 1924 H=1924:00 +CPU= 22.239
  11:57:18 STEP 1925 H=1925:00 +CPU= 21.932
  11:57:54 STEP 1926 H=1926:00 +CPU= 34.992
  11:58:16 STEP 1927 H=1927:00 +CPU= 22.435
  11:58:40 STEP 1928 H=1928:00 +CPU= 22.997
  11:59:14 STEP 1929 H=1929:00 +CPU= 34.559
  11:59:37 STEP 1930 H=1930:00 +CPU= 22.301
  11:59:59 STEP 1931 H=1931:00 +CPU= 21.712
  12:00:37 STEP 1932 H=1932:00 +CPU= 37.865
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001932
  12:00:59 STEP 1933 H=1933:00 +CPU= 21.760
  12:01:22 STEP 1934 H=1934:00 +CPU= 22.321
  12:01:57 STEP 1935 H=1935:00 +CPU= 33.904
  12:02:19 STEP 1936 H=1936:00 +CPU= 22.339
  12:02:41 STEP 1937 H=1937:00 +CPU= 21.461
  12:03:14 STEP 1938 H=1938:00 +CPU= 33.695
  12:03:36 STEP 1939 H=1939:00 +CPU= 21.913
  12:03:59 STEP 1940 H=1940:00 +CPU= 22.396
  12:04:33 STEP 1941 H=1941:00 +CPU= 33.399
  12:04:56 STEP 1942 H=1942:00 +CPU= 22.254
  12:05:18 STEP 1943 H=1943:00 +CPU= 22.326
  12:06:00 STEP 1944 H=1944:00 +CPU= 40.825
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001932
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_80.zip
Uploading the intermediate file: upload_file_80.zip
  12:06:23 STEP 1945 H=1945:00 +CPU= 22.015
Uploading trickle at timestep: 6998400
  12:06:46 STEP 1946 H=1946:00 +CPU= 22.424
  12:07:20 STEP 1947 H=1947:00 +CPU= 33.861
  12:07:42 STEP 1948 H=1948:00 +CPU= 22.634
  12:08:05 STEP 1949 H=1949:00 +CPU= 21.978
  12:08:40 STEP 1950 H=1950:00 +CPU= 34.550
  12:09:02 STEP 1951 H=1951:00 +CPU= 21.951
  12:09:25 STEP 1952 H=1952:00 +CPU= 22.614
  12:09:59 STEP 1953 H=1953:00 +CPU= 33.947
  12:10:22 STEP 1954 H=1954:00 +CPU= 22.546
  12:10:44 STEP 1955 H=1955:00 +CPU= 21.946
  12:11:22 STEP 1956 H=1956:00 +CPU= 37.567
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001956
  12:11:44 STEP 1957 H=1957:00 +CPU= 22.137
  12:12:07 STEP 1958 H=1958:00 +CPU= 23.083
  12:12:42 STEP 1959 H=1959:00 +CPU= 35.057
  12:13:07 STEP 1960 H=1960:00 +CPU= 23.867
  12:13:31 STEP 1961 H=1961:00 +CPU= 23.687
  12:14:09 STEP 1962 H=1962:00 +CPU= 37.798
  12:14:33 STEP 1963 H=1963:00 +CPU= 23.085
  12:14:56 STEP 1964 H=1964:00 +CPU= 23.399
  12:15:31 STEP 1965 H=1965:00 +CPU= 34.809
  12:15:54 STEP 1966 H=1966:00 +CPU= 22.968
  12:16:17 STEP 1967 H=1967:00 +CPU= 22.798
  12:16:58 STEP 1968 H=1968:00 +CPU= 40.241
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001956
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_81.zip
Uploading the intermediate file: upload_file_81.zip
  12:17:20 STEP 1969 H=1969:00 +CPU= 21.663
  12:17:43 STEP 1970 H=1970:00 +CPU= 22.717
  12:18:18 STEP 1971 H=1971:00 +CPU= 34.951
  12:18:41 STEP 1972 H=1972:00 +CPU= 22.374
  12:19:04 STEP 1973 H=1973:00 +CPU= 21.886
  12:19:39 STEP 1974 H=1974:00 +CPU= 34.961
  12:20:01 STEP 1975 H=1975:00 +CPU= 21.974
  12:20:24 STEP 1976 H=1976:00 +CPU= 21.981
  12:20:58 STEP 1977 H=1977:00 +CPU= 33.793
  12:21:21 STEP 1978 H=1978:00 +CPU= 22.723
  12:21:44 STEP 1979 H=1979:00 +CPU= 22.316
  12:22:23 STEP 1980 H=1980:00 +CPU= 39.367
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001980
  12:22:46 STEP 1981 H=1981:00 +CPU= 22.380
  12:23:10 STEP 1982 H=1982:00 +CPU= 22.891
  12:23:45 STEP 1983 H=1983:00 +CPU= 34.803
  12:24:07 STEP 1984 H=1984:00 +CPU= 22.295
  12:24:30 STEP 1985 H=1985:00 +CPU= 21.927
  12:25:05 STEP 1986 H=1986:00 +CPU= 35.061
  12:25:27 STEP 1987 H=1987:00 +CPU= 21.859
  12:25:51 STEP 1988 H=1988:00 +CPU= 23.398
  12:26:27 STEP 1989 H=1989:00 +CPU= 36.209
  12:26:51 STEP 1990 H=1990:00 +CPU= 23.288
  12:27:16 STEP 1991 H=1991:00 +CPU= 24.204
  12:28:01 STEP 1992 H=1992:00 +CPU= 42.293
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001980
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_82.zip
Uploading the intermediate file: upload_file_82.zip
  12:28:24 STEP 1993 H=1993:00 +CPU= 22.586
  12:28:48 STEP 1994 H=1994:00 +CPU= 23.081
  12:29:25 STEP 1995 H=1995:00 +CPU= 36.022
  12:29:49 STEP 1996 H=1996:00 +CPU= 23.743
  12:30:12 STEP 1997 H=1997:00 +CPU= 23.077
  12:30:50 STEP 1998 H=1998:00 +CPU= 36.848
  12:31:14 STEP 1999 H=1999:00 +CPU= 23.737
  12:31:39 STEP 2000 H=2000:00 +CPU= 24.572
  12:32:16 STEP 2001 H=2001:00 +CPU= 36.426
  12:32:38 STEP 2002 H=2002:00 +CPU= 22.196
  12:33:00 STEP 2003 H=2003:00 +CPU= 21.923
  12:33:40 STEP 2004 H=2004:00 +CPU= 39.655
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002004
  12:34:03 STEP 2005 H=2005:00 +CPU= 22.119
  12:34:26 STEP 2006 H=2006:00 +CPU= 22.723
  12:35:01 STEP 2007 H=2007:00 +CPU= 34.438
  12:35:25 STEP 2008 H=2008:00 +CPU= 23.376
  12:35:48 STEP 2009 H=2009:00 +CPU= 22.743
  12:36:24 STEP 2010 H=2010:00 +CPU= 35.783
  12:36:46 STEP 2011 H=2011:00 +CPU= 22.116
  12:37:09 STEP 2012 H=2012:00 +CPU= 22.960
  12:37:44 STEP 2013 H=2013:00 +CPU= 34.267
  12:38:09 STEP 2014 H=2014:00 +CPU= 22.623
  12:38:31 STEP 2015 H=2015:00 +CPU= 22.381
  12:39:14 STEP 2016 H=2016:00 +CPU= 40.884
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002004
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_83.zip
Uploading the intermediate file: upload_file_83.zip
  12:39:36 STEP 2017 H=2017:00 +CPU= 21.714
  12:39:59 STEP 2018 H=2018:00 +CPU= 22.943
  12:40:34 STEP 2019 H=2019:00 +CPU= 34.385
  12:40:57 STEP 2020 H=2020:00 +CPU= 22.946
  12:41:20 STEP 2021 H=2021:00 +CPU= 22.651
  12:41:57 STEP 2022 H=2022:00 +CPU= 36.631
  12:42:20 STEP 2023 H=2023:00 +CPU= 22.343
  12:42:42 STEP 2024 H=2024:00 +CPU= 22.430
  12:43:17 STEP 2025 H=2025:00 +CPU= 34.184
  12:43:40 STEP 2026 H=2026:00 +CPU= 22.004
  12:44:02 STEP 2027 H=2027:00 +CPU= 21.410
  12:44:40 STEP 2028 H=2028:00 +CPU= 37.457
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002028
  12:45:02 STEP 2029 H=2029:00 +CPU= 21.401
  12:45:25 STEP 2030 H=2030:00 +CPU= 22.938
  12:46:01 STEP 2031 H=2031:00 +CPU= 35.258
  12:46:24 STEP 2032 H=2032:00 +CPU= 22.816
  12:46:46 STEP 2033 H=2033:00 +CPU= 21.719
  12:47:22 STEP 2034 H=2034:00 +CPU= 35.154
  12:47:44 STEP 2035 H=2035:00 +CPU= 21.928
  12:48:07 STEP 2036 H=2036:00 +CPU= 22.440
  12:48:41 STEP 2037 H=2037:00 +CPU= 34.073
  12:49:04 STEP 2038 H=2038:00 +CPU= 22.233
  12:49:27 STEP 2039 H=2039:00 +CPU= 22.360
  12:50:09 STEP 2040 H=2040:00 +CPU= 40.723
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002028
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_84.zip
Uploading the intermediate file: upload_file_84.zip
  12:50:32 STEP 2041 H=2041:00 +CPU= 21.677
  12:50:55 STEP 2042 H=2042:00 +CPU= 22.249
  12:51:29 STEP 2043 H=2043:00 +CPU= 33.589
  12:51:51 STEP 2044 H=2044:00 +CPU= 22.108
  12:52:13 STEP 2045 H=2045:00 +CPU= 21.725
  12:52:48 STEP 2046 H=2046:00 +CPU= 34.813
  12:53:10 STEP 2047 H=2047:00 +CPU= 21.587
  12:53:33 STEP 2048 H=2048:00 +CPU= 22.583
  12:54:07 STEP 2049 H=2049:00 +CPU= 33.628
  12:54:29 STEP 2050 H=2050:00 +CPU= 22.047
  12:54:52 STEP 2051 H=2051:00 +CPU= 22.336
  12:55:31 STEP 2052 H=2052:00 +CPU= 38.441
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002052
  12:55:53 STEP 2053 H=2053:00 +CPU= 21.929
  12:56:15 STEP 2054 H=2054:00 +CPU= 22.285
  12:56:49 STEP 2055 H=2055:00 +CPU= 33.764
  12:57:12 STEP 2056 H=2056:00 +CPU= 22.323
  12:57:33 STEP 2057 H=2057:00 +CPU= 21.375
  12:58:08 STEP 2058 H=2058:00 +CPU= 34.169
  12:58:30 STEP 2059 H=2059:00 +CPU= 21.500
  12:58:52 STEP 2060 H=2060:00 +CPU= 22.074
  12:59:26 STEP 2061 H=2061:00 +CPU= 34.230
  12:59:49 STEP 2062 H=2062:00 +CPU= 21.962
  13:00:11 STEP 2063 H=2063:00 +CPU= 22.342
  13:00:52 STEP 2064 H=2064:00 +CPU= 39.469
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_85.zip
Uploading the intermediate file: upload_file_85.zip
  13:01:16 STEP 2065 H=2065:00 +CPU= 21.825
  13:01:40 STEP 2066 H=2066:00 +CPU= 23.375
  13:02:17 STEP 2067 H=2067:00 +CPU= 36.867
  13:02:42 STEP 2068 H=2068:00 +CPU= 23.818
  13:03:05 STEP 2069 H=2069:00 +CPU= 22.477
  13:03:42 STEP 2070 H=2070:00 +CPU= 36.202
  13:04:04 STEP 2071 H=2071:00 +CPU= 21.682
  13:04:27 STEP 2072 H=2072:00 +CPU= 22.781
  13:05:03 STEP 2073 H=2073:00 +CPU= 34.698
  13:05:26 STEP 2074 H=2074:00 +CPU= 22.937
  13:05:48 STEP 2075 H=2075:00 +CPU= 21.981
  13:06:28 STEP 2076 H=2076:00 +CPU= 38.565
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002076
  13:06:50 STEP 2077 H=2077:00 +CPU= 21.705
  13:07:12 STEP 2078 H=2078:00 +CPU= 22.343
  13:07:47 STEP 2079 H=2079:00 +CPU= 34.266
  13:08:10 STEP 2080 H=2080:00 +CPU= 22.270
  13:08:32 STEP 2081 H=2081:00 +CPU= 21.412
  13:09:06 STEP 2082 H=2082:00 +CPU= 34.167
  13:09:28 STEP 2083 H=2083:00 +CPU= 21.752
  13:09:50 STEP 2084 H=2084:00 +CPU= 21.190
  13:10:23 STEP 2085 H=2085:00 +CPU= 33.172
  13:10:46 STEP 2086 H=2086:00 +CPU= 22.734
  13:11:09 STEP 2087 H=2087:00 +CPU= 22.630
  13:11:51 STEP 2088 H=2088:00 +CPU= 40.700
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_86.zip
Uploading the intermediate file: upload_file_86.zip
  13:12:13 STEP 2089 H=2089:00 +CPU= 21.380
  13:12:35 STEP 2090 H=2090:00 +CPU= 22.275
  13:13:09 STEP 2091 H=2091:00 +CPU= 33.679
  13:13:33 STEP 2092 H=2092:00 +CPU= 22.738
  13:13:54 STEP 2093 H=2093:00 +CPU= 21.117
  13:14:28 STEP 2094 H=2094:00 +CPU= 34.034
  13:14:50 STEP 2095 H=2095:00 +CPU= 21.586
  13:15:13 STEP 2096 H=2096:00 +CPU= 22.630
  13:15:47 STEP 2097 H=2097:00 +CPU= 33.579
  13:16:10 STEP 2098 H=2098:00 +CPU= 22.372
  13:16:32 STEP 2099 H=2099:00 +CPU= 22.322
  13:17:11 STEP 2100 H=2100:00 +CPU= 38.199
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002100
  13:17:33 STEP 2101 H=2101:00 +CPU= 21.501
  13:17:55 STEP 2102 H=2102:00 +CPU= 22.357
  13:18:29 STEP 2103 H=2103:00 +CPU= 33.636
  13:18:52 STEP 2104 H=2104:00 +CPU= 22.383
  13:19:14 STEP 2105 H=2105:00 +CPU= 21.626
  13:19:48 STEP 2106 H=2106:00 +CPU= 33.999
  13:20:10 STEP 2107 H=2107:00 +CPU= 21.522
  13:20:32 STEP 2108 H=2108:00 +CPU= 22.225
  13:21:06 STEP 2109 H=2109:00 +CPU= 33.174
  13:21:29 STEP 2110 H=2110:00 +CPU= 21.966
  13:21:52 STEP 2111 H=2111:00 +CPU= 22.385
  13:22:33 STEP 2112 H=2112:00 +CPU= 39.806
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_87.zip
Uploading the intermediate file: upload_file_87.zip
  13:22:58 STEP 2113 H=2113:00 +CPU= 22.936
  13:23:25 STEP 2114 H=2114:00 +CPU= 25.787
  13:24:03 STEP 2115 H=2115:00 +CPU= 37.448
  13:24:26 STEP 2116 H=2116:00 +CPU= 22.456
  13:24:48 STEP 2117 H=2117:00 +CPU= 22.150
  13:25:23 STEP 2118 H=2118:00 +CPU= 34.961
  13:25:45 STEP 2119 H=2119:00 +CPU= 21.765
  13:26:08 STEP 2120 H=2120:00 +CPU= 22.430
  13:26:42 STEP 2121 H=2121:00 +CPU= 33.600
  13:27:04 STEP 2122 H=2122:00 +CPU= 22.385
  13:27:26 STEP 2123 H=2123:00 +CPU= 21.830
  13:28:04 STEP 2124 H=2124:00 +CPU= 37.331
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002124
  13:28:26 STEP 2125 H=2125:00 +CPU= 21.671
  13:28:49 STEP 2126 H=2126:00 +CPU= 22.308
  13:29:23 STEP 2127 H=2127:00 +CPU= 33.113
  13:29:45 STEP 2128 H=2128:00 +CPU= 22.381
  13:30:07 STEP 2129 H=2129:00 +CPU= 21.202
  13:30:42 STEP 2130 H=2130:00 +CPU= 34.432
  13:31:03 STEP 2131 H=2131:00 +CPU= 21.257
  13:31:25 STEP 2132 H=2132:00 +CPU= 21.820
  13:31:59 STEP 2133 H=2133:00 +CPU= 33.669
  13:32:22 STEP 2134 H=2134:00 +CPU= 22.425
  13:32:45 STEP 2135 H=2135:00 +CPU= 22.499
  13:33:25 STEP 2136 H=2136:00 +CPU= 39.327
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_88.zip
Uploading the intermediate file: upload_file_88.zip
  13:33:46 STEP 2137 H=2137:00 +CPU= 21.113
  13:34:08 STEP 2138 H=2138:00 +CPU= 21.480
  13:34:42 STEP 2139 H=2139:00 +CPU= 33.736
  13:35:03 STEP 2140 H=2140:00 +CPU= 21.439
  13:35:25 STEP 2141 H=2141:00 +CPU= 21.232
  13:35:59 STEP 2142 H=2142:00 +CPU= 34.309
  13:36:20 STEP 2143 H=2143:00 +CPU= 21.007
  13:36:42 STEP 2144 H=2144:00 +CPU= 21.228
  13:37:15 STEP 2145 H=2145:00 +CPU= 33.353
  13:37:37 STEP 2146 H=2146:00 +CPU= 21.940
  13:37:58 STEP 2147 H=2147:00 +CPU= 21.110
  13:38:35 STEP 2148 H=2148:00 +CPU= 36.470
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002148
  13:38:56 STEP 2149 H=2149:00 +CPU= 20.788
  13:39:18 STEP 2150 H=2150:00 +CPU= 21.549
  13:39:51 STEP 2151 H=2151:00 +CPU= 33.593
  13:40:14 STEP 2152 H=2152:00 +CPU= 22.027
  13:40:34 STEP 2153 H=2153:00 +CPU= 20.590
  13:41:09 STEP 2154 H=2154:00 +CPU= 34.003
  13:41:30 STEP 2155 H=2155:00 +CPU= 20.867
  13:41:51 STEP 2156 H=2156:00 +CPU= 21.577
  13:42:25 STEP 2157 H=2157:00 +CPU= 33.221
  13:42:46 STEP 2158 H=2158:00 +CPU= 21.412
  13:43:08 STEP 2159 H=2159:00 +CPU= 21.958
  13:43:49 STEP 2160 H=2160:00 +CPU= 39.813
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_89.zip
Uploading the intermediate file: upload_file_89.zip
  13:44:11 STEP 2161 H=2161:00 +CPU= 21.811
  13:44:33 STEP 2162 H=2162:00 +CPU= 21.935
  13:45:05 STEP 2163 H=2163:00 +CPU= 32.661
  13:45:27 STEP 2164 H=2164:00 +CPU= 21.136
  13:45:48 STEP 2165 H=2165:00 +CPU= 20.865
  13:46:22 STEP 2166 H=2166:00 +CPU= 33.944
  13:46:43 STEP 2167 H=2167:00 +CPU= 20.526
  13:47:04 STEP 2168 H=2168:00 +CPU= 21.449
  13:47:38 STEP 2169 H=2169:00 +CPU= 33.854
  13:48:00 STEP 2170 H=2170:00 +CPU= 21.949
  13:48:22 STEP 2171 H=2171:00 +CPU= 21.956
  13:49:00 STEP 2172 H=2172:00 +CPU= 37.466
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002172
  13:49:21 STEP 2173 H=2173:00 +CPU= 20.987
  13:49:43 STEP 2174 H=2174:00 +CPU= 21.717
  13:50:16 STEP 2175 H=2175:00 +CPU= 32.717
  13:50:38 STEP 2176 H=2176:00 +CPU= 22.096
  13:51:00 STEP 2177 H=2177:00 +CPU= 21.311
  13:51:33 STEP 2178 H=2178:00 +CPU= 33.390
  13:51:54 STEP 2179 H=2179:00 +CPU= 21.066
  13:52:16 STEP 2180 H=2180:00 +CPU= 21.446
  13:52:50 STEP 2181 H=2181:00 +CPU= 33.566
  13:53:12 STEP 2182 H=2182:00 +CPU= 22.056
  13:53:34 STEP 2183 H=2183:00 +CPU= 21.974
  13:54:14 STEP 2184 H=2184:00 +CPU= 39.956
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_90.zip
Uploading the intermediate file: upload_file_90.zip
  13:54:36 STEP 2185 H=2185:00 +CPU= 21.649
Uploading trickle at timestep: 7862400
  13:54:59 STEP 2186 H=2186:00 +CPU= 22.451
  13:55:33 STEP 2187 H=2187:00 +CPU= 34.161
  13:55:55 STEP 2188 H=2188:00 +CPU= 21.912
  13:56:16 STEP 2189 H=2189:00 +CPU= 20.793
  13:56:51 STEP 2190 H=2190:00 +CPU= 34.192
  13:57:12 STEP 2191 H=2191:00 +CPU= 21.256
  13:57:34 STEP 2192 H=2192:00 +CPU= 21.621
  13:58:07 STEP 2193 H=2193:00 +CPU= 33.294
  13:58:30 STEP 2194 H=2194:00 +CPU= 22.243
  13:58:51 STEP 2195 H=2195:00 +CPU= 21.133
  13:59:27 STEP 2196 H=2196:00 +CPU= 36.305
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002196
  13:59:48 STEP 2197 H=2197:00 +CPU= 20.930
  14:00:10 STEP 2198 H=2198:00 +CPU= 21.903
  14:00:44 STEP 2199 H=2199:00 +CPU= 33.430
  14:01:06 STEP 2200 H=2200:00 +CPU= 21.783
  14:01:27 STEP 2201 H=2201:00 +CPU= 21.002
  14:02:01 STEP 2202 H=2202:00 +CPU= 33.848
  14:02:22 STEP 2203 H=2203:00 +CPU= 20.902
  14:02:44 STEP 2204 H=2204:00 +CPU= 21.693
  14:03:18 STEP 2205 H=2205:00 +CPU= 33.456
  14:03:40 STEP 2206 H=2206:00 +CPU= 21.626
  14:04:01 STEP 2207 H=2207:00 +CPU= 21.477
  14:04:40 STEP 2208 H=2208:00 +CPU= 38.487
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_91.zip
Uploading the intermediate file: upload_file_91.zip
  14:05:01 STEP 2209 H=2209:00 +CPU= 20.876
  14:05:23 STEP 2210 H=2210:00 +CPU= 21.888
  14:05:57 STEP 2211 H=2211:00 +CPU= 33.459
  14:06:20 STEP 2212 H=2212:00 +CPU= 22.683
  14:06:42 STEP 2213 H=2213:00 +CPU= 22.646
  14:07:20 STEP 2214 H=2214:00 +CPU= 37.303
  14:07:45 STEP 2215 H=2215:00 +CPU= 24.861
  14:08:09 STEP 2216 H=2216:00 +CPU= 23.914
  14:08:44 STEP 2217 H=2217:00 +CPU= 34.618
  14:09:09 STEP 2218 H=2218:00 +CPU= 23.105
  14:09:32 STEP 2219 H=2219:00 +CPU= 22.587
  14:10:12 STEP 2220 H=2220:00 +CPU= 39.059
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002220
  14:10:33 STEP 2221 H=2221:00 +CPU= 21.244
  14:10:55 STEP 2222 H=2222:00 +CPU= 22.226
  14:11:30 STEP 2223 H=2223:00 +CPU= 34.338
  14:11:53 STEP 2224 H=2224:00 +CPU= 23.311
  14:12:16 STEP 2225 H=2225:00 +CPU= 22.118
  14:12:51 STEP 2226 H=2226:00 +CPU= 34.764
  14:13:13 STEP 2227 H=2227:00 +CPU= 21.803
  14:13:36 STEP 2228 H=2228:00 +CPU= 22.797
  14:14:09 STEP 2229 H=2229:00 +CPU= 33.693
  14:14:31 STEP 2230 H=2230:00 +CPU= 22.003
  14:14:54 STEP 2231 H=2231:00 +CPU= 22.347
  14:15:36 STEP 2232 H=2232:00 +CPU= 40.447
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_92.zip
Uploading the intermediate file: upload_file_92.zip
  14:15:59 STEP 2233 H=2233:00 +CPU= 21.923
  14:16:22 STEP 2234 H=2234:00 +CPU= 22.713
  14:16:56 STEP 2235 H=2235:00 +CPU= 34.152
  14:17:20 STEP 2236 H=2236:00 +CPU= 23.131
  14:17:42 STEP 2237 H=2237:00 +CPU= 21.938
  14:18:17 STEP 2238 H=2238:00 +CPU= 34.775
  14:18:39 STEP 2239 H=2239:00 +CPU= 21.626
  14:19:01 STEP 2240 H=2240:00 +CPU= 22.163
  14:19:37 STEP 2241 H=2241:00 +CPU= 34.756
  14:20:00 STEP 2242 H=2242:00 +CPU= 22.680
  14:20:23 STEP 2243 H=2243:00 +CPU= 22.521
  14:21:03 STEP 2244 H=2244:00 +CPU= 38.824
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002244
  14:21:25 STEP 2245 H=2245:00 +CPU= 21.760
  14:21:48 STEP 2246 H=2246:00 +CPU= 22.267
  14:22:23 STEP 2247 H=2247:00 +CPU= 34.467
  14:22:46 STEP 2248 H=2248:00 +CPU= 22.271
  14:23:08 STEP 2249 H=2249:00 +CPU= 21.783
  14:23:43 STEP 2250 H=2250:00 +CPU= 34.320
  14:24:06 STEP 2251 H=2251:00 +CPU= 22.160
  14:24:29 STEP 2252 H=2252:00 +CPU= 22.461
  14:25:04 STEP 2253 H=2253:00 +CPU= 35.225
  14:25:28 STEP 2254 H=2254:00 +CPU= 22.620
  14:25:50 STEP 2255 H=2255:00 +CPU= 22.540
  14:26:32 STEP 2256 H=2256:00 +CPU= 40.596
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_93.zip
Uploading the intermediate file: upload_file_93.zip
  14:26:55 STEP 2257 H=2257:00 +CPU= 22.159
  14:27:17 STEP 2258 H=2258:00 +CPU= 22.580
  14:27:54 STEP 2259 H=2259:00 +CPU= 33.907
  14:28:17 STEP 2260 H=2260:00 +CPU= 23.361
  14:28:41 STEP 2261 H=2261:00 +CPU= 22.753
  14:29:16 STEP 2262 H=2262:00 +CPU= 34.888
  14:29:38 STEP 2263 H=2263:00 +CPU= 21.758
  14:30:02 STEP 2264 H=2264:00 +CPU= 23.346
  14:30:37 STEP 2265 H=2265:00 +CPU= 34.207
  14:31:01 STEP 2266 H=2266:00 +CPU= 22.964
  14:31:24 STEP 2267 H=2267:00 +CPU= 22.829
  14:32:07 STEP 2268 H=2268:00 +CPU= 42.750
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002268
  14:32:30 STEP 2269 H=2269:00 +CPU= 22.209
  14:32:53 STEP 2270 H=2270:00 +CPU= 22.082
  14:33:28 STEP 2271 H=2271:00 +CPU= 34.400
  14:33:51 STEP 2272 H=2272:00 +CPU= 23.010
  14:34:14 STEP 2273 H=2273:00 +CPU= 22.495
  14:34:49 STEP 2274 H=2274:00 +CPU= 34.950
  14:35:10 STEP 2275 H=2275:00 +CPU= 20.559
  14:35:32 STEP 2276 H=2276:00 +CPU= 21.775
  14:36:07 STEP 2277 H=2277:00 +CPU= 34.329
  14:36:30 STEP 2278 H=2278:00 +CPU= 22.784
  14:36:54 STEP 2279 H=2279:00 +CPU= 23.009
  14:37:35 STEP 2280 H=2280:00 +CPU= 40.367
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_94.zip
Uploading the intermediate file: upload_file_94.zip
  14:37:58 STEP 2281 H=2281:00 +CPU= 21.646
  14:38:22 STEP 2282 H=2282:00 +CPU= 22.132
  14:38:55 STEP 2283 H=2283:00 +CPU= 33.324
  14:39:18 STEP 2284 H=2284:00 +CPU= 22.438
  14:39:40 STEP 2285 H=2285:00 +CPU= 21.391
  14:40:16 STEP 2286 H=2286:00 +CPU= 35.594
  14:40:39 STEP 2287 H=2287:00 +CPU= 22.517
  14:41:01 STEP 2288 H=2288:00 +CPU= 22.753
  14:41:36 STEP 2289 H=2289:00 +CPU= 34.335
  14:41:59 STEP 2290 H=2290:00 +CPU= 22.365
  14:42:22 STEP 2291 H=2291:00 +CPU= 22.457
  14:43:02 STEP 2292 H=2292:00 +CPU= 39.825
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002292
  14:43:26 STEP 2293 H=2293:00 +CPU= 22.958
  14:43:49 STEP 2294 H=2294:00 +CPU= 22.752
  14:44:27 STEP 2295 H=2295:00 +CPU= 37.419
  14:44:52 STEP 2296 H=2296:00 +CPU= 24.460
  14:45:14 STEP 2297 H=2297:00 +CPU= 21.764
  14:45:49 STEP 2298 H=2298:00 +CPU= 34.983
  14:46:12 STEP 2299 H=2299:00 +CPU= 22.131
  14:46:35 STEP 2300 H=2300:00 +CPU= 22.578
  14:47:09 STEP 2301 H=2301:00 +CPU= 34.653
  14:47:35 STEP 2302 H=2302:00 +CPU= 24.789
  14:48:00 STEP 2303 H=2303:00 +CPU= 24.592
  14:48:45 STEP 2304 H=2304:00 +CPU= 43.585
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_95.zip
Uploading the intermediate file: upload_file_95.zip
  14:49:08 STEP 2305 H=2305:00 +CPU= 22.368
  14:49:31 STEP 2306 H=2306:00 +CPU= 23.000
  14:50:07 STEP 2307 H=2307:00 +CPU= 34.982
  14:50:30 STEP 2308 H=2308:00 +CPU= 23.471
  14:50:54 STEP 2309 H=2309:00 +CPU= 23.132
  14:51:29 STEP 2310 H=2310:00 +CPU= 35.339
  14:51:51 STEP 2311 H=2311:00 +CPU= 21.421
  14:52:14 STEP 2312 H=2312:00 +CPU= 22.558
  14:52:49 STEP 2313 H=2313:00 +CPU= 34.800
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.035] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3996MB (heap), 4394MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.037] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.037] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.037] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.037] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.037] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3996 MB (maxheap), 4394 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.037] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.039] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                CLOUD_LAYER 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145307:1671976387:15932.040] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                 CLOUDSC 
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145308:1671976388:15933.040] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:egonmint:1:1:2439:2439] [20221225:145308:1671976388:15933.040] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000103411F  cloudsc_                 3094  cloudsc.F90
oifs_43r3_model.e  0000000001014D57  cloud_layer_              131  cloud_layer.F90
oifs_43r3_model.e  0000000000F1A790  callpar_                 1223  callpar.F90
oifs_43r3_model.e  0000000000E4940F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000E3610C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BFA3D3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1994050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0806
(argv4) batchid: 963
(argv5) wuid: 12180450
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/17
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/17/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/17/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_383a916b6cac07bbc3df124709b19107 to: /var/lib/boinc-client/slots/17/oifs_43r3_ps_0806_1994050100_123_963_12180450.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/17/oifs_43r3_ps_0806_1994050100_123_963_12180450.zip
ic_ancil_file: ic_ancil_12180450
ifsdata_file: ifsdata_12180450
climate_data_file: clim_data_12180450
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_b9f22764f8b40de03f102173c7ec219e to: /var/lib/boinc-client/slots/17/ic_ancil_12180450.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/17/ic_ancil_12180450.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_287f7a412ed1c5dc8d886916f8ff1b4f to: /var/lib/boinc-client/slots/17/ifsdata/ifsdata_12180450.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/17/ifsdata/ifsdata_12180450.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/17/159l_2/clim_data_12180450.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/17/159l_2/clim_data_12180450.zip
Checking for progress XML file: /var/lib/boinc-client/slots/17/progress_file_12180450.xml
Opened progress file ok : /var/lib/boinc-client/slots/17/progress_file_12180450.xml
-- Model is restarting --
Adjusting last_iter, 2314, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/17/progress_file_12180450.xml
last_cpu_time: 62972
upload_file_number: 96
last_iter: 2303
last_upload: 8294400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061
The child process has been launched with process id: 2349
Executing the command: /var/lib/boinc-client/slots/17/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:egonmint:1:1:2349:2349] [20221225:184234:1671990154:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  18:43:18 STEP 2304 H=2304:00 +CPU= 37.795
  18:43:41 STEP 2305 H=2305:00 +CPU= 21.763
  18:44:03 STEP 2306 H=2306:00 +CPU= 21.669
  18:44:37 STEP 2307 H=2307:00 +CPU= 33.543
  18:45:00 STEP 2308 H=2308:00 +CPU= 22.091
  18:45:22 STEP 2309 H=2309:00 +CPU= 21.631
  18:45:57 STEP 2310 H=2310:00 +CPU= 34.297
  18:46:19 STEP 2311 H=2311:00 +CPU= 21.518
  18:46:42 STEP 2312 H=2312:00 +CPU= 22.327
  18:47:17 STEP 2313 H=2313:00 +CPU= 34.033
  18:47:39 STEP 2314 H=2314:00 +CPU= 22.154
  18:48:01 STEP 2315 H=2315:00 +CPU= 21.679
  18:48:40 STEP 2316 H=2316:00 +CPU= 37.923
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002316
  18:49:02 STEP 2317 H=2317:00 +CPU= 21.851
  18:49:24 STEP 2318 H=2318:00 +CPU= 21.840
  18:49:58 STEP 2319 H=2319:00 +CPU= 33.851
  18:50:20 STEP 2320 H=2320:00 +CPU= 21.828
  18:50:42 STEP 2321 H=2321:00 +CPU= 21.357
  18:51:17 STEP 2322 H=2322:00 +CPU= 34.294
  18:51:38 STEP 2323 H=2323:00 +CPU= 21.593
  18:52:01 STEP 2324 H=2324:00 +CPU= 22.434
  18:52:35 STEP 2325 H=2325:00 +CPU= 33.573
  18:52:58 STEP 2326 H=2326:00 +CPU= 22.315
  18:53:20 STEP 2327 H=2327:00 +CPU= 21.514
  18:54:00 STEP 2328 H=2328:00 +CPU= 39.968
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/17/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180450/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0806_1994050100_123_963_12180450_0_r1074307061_96.zip
Uploading the intermediate file: upload_file_96.zip
  18:54:23 STEP 2329 H=2329:00 +CPU= 21.593
  18:54:45 STEP 2330 H=2330:00 +CPU= 22.144
  18:55:19 STEP 2331 H=2331:00 +CPU= 33.337
  18:55:40 STEP 2332 H=2332:00 +CPU= 21.000
  18:56:01 STEP 2333 H=2333:00 +CPU= 20.514
  18:56:35 STEP 2334 H=2334:00 +CPU= 34.267
  18:56:57 STEP 2335 H=2335:00 +CPU= 21.574
  18:57:19 STEP 2336 H=2336:00 +CPU= 22.204
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:54:57 1537224 22265411 oifs_43r3_ps_0806_1994050100_123_963_12180450_0 7,862,400 59,498 0.0076
09 May 2023 15:54:51 1537224 22265411 oifs_43r3_ps_0806_1994050100_123_963_12180450_0 6,998,400 53,104 0.0076
09 May 2023 15:50:54 1537224 22265411 oifs_43r3_ps_0806_1994050100_123_963_12180450_0 5,011,200 38,031 0.0076
09 May 2023 15:48:42 1537224 22265411 oifs_43r3_ps_0806_1994050100_123_963_12180450_0 2,851,200 21,855 0.0077
09 May 2023 15:48:26 1537224 22265411 oifs_43r3_ps_0806_1994050100_123_963_12180450_0 1,641,600 12,583 0.0077


©2024 climateprediction.net