climateprediction.net home page
Task 22278199

Task 22278199

Name oifs_43r3_ps_0814_2006050100_123_975_12192458_0
Workunit 12192458
Created 23 Dec 2022, 14:09:28 UTC
Sent 29 Dec 2022, 11:27:09 UTC
Report deadline 28 Jan 2023, 11:27:09 UTC
Received 9 Jan 2023, 1:17:35 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1491253
Run time 22 hours 16 min 46 sec
CPU time 20 hours 7 min 12 sec
Validate state Invalid
Credit 12,942.72
Device peak FLOPS 6.20 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,408.78 MB
Peak swap size 4,978.16 MB
Peak disk usage 7,827.62 MB
Stderr 
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
ar/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_85.zip
Uploading the intermediate file: upload_file_85.zip
  10:47:06 STEP 2065 H=2065:00 +CPU= 22.212
  10:47:32 STEP 2066 H=2066:00 +CPU= 22.900
  10:48:11 STEP 2067 H=2067:00 +CPU= 34.083
  10:48:37 STEP 2068 H=2068:00 +CPU= 22.779
  10:49:02 STEP 2069 H=2069:00 +CPU= 22.261
  10:49:42 STEP 2070 H=2070:00 +CPU= 34.853
  10:50:07 STEP 2071 H=2071:00 +CPU= 22.217
  10:50:33 STEP 2072 H=2072:00 +CPU= 22.849
  10:51:12 STEP 2073 H=2073:00 +CPU= 34.122
  10:51:38 STEP 2074 H=2074:00 +CPU= 22.841
  10:52:03 STEP 2075 H=2075:00 +CPU= 22.219
  10:52:46 STEP 2076 H=2076:00 +CPU= 37.868
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002076
  10:53:11 STEP 2077 H=2077:00 +CPU= 22.333
  10:53:37 STEP 2078 H=2078:00 +CPU= 23.000
  10:54:17 STEP 2079 H=2079:00 +CPU= 34.173
  10:54:43 STEP 2080 H=2080:00 +CPU= 22.912
  10:55:08 STEP 2081 H=2081:00 +CPU= 22.279
  10:55:48 STEP 2082 H=2082:00 +CPU= 34.849
  10:56:13 STEP 2083 H=2083:00 +CPU= 22.269
  10:56:39 STEP 2084 H=2084:00 +CPU= 22.943
  10:57:18 STEP 2085 H=2085:00 +CPU= 34.151
  10:57:44 STEP 2086 H=2086:00 +CPU= 22.809
  10:58:10 STEP 2087 H=2087:00 +CPU= 23.045
  10:58:56 STEP 2088 H=2088:00 +CPU= 40.507
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_86.zip
Uploading the intermediate file: upload_file_86.zip
  10:59:22 STEP 2089 H=2089:00 +CPU= 22.196
Uploading trickle at timestep: 7516800
  10:59:48 STEP 2090 H=2090:00 +CPU= 22.868
  11:00:27 STEP 2091 H=2091:00 +CPU= 34.128
  11:00:53 STEP 2092 H=2092:00 +CPU= 22.885
  11:01:19 STEP 2093 H=2093:00 +CPU= 22.189
  11:01:58 STEP 2094 H=2094:00 +CPU= 34.621
  11:02:24 STEP 2095 H=2095:00 +CPU= 22.322
  11:02:50 STEP 2096 H=2096:00 +CPU= 22.807
  11:03:29 STEP 2097 H=2097:00 +CPU= 34.078
  11:03:56 STEP 2098 H=2098:00 +CPU= 22.978
  11:04:21 STEP 2099 H=2099:00 +CPU= 22.282
  11:05:05 STEP 2100 H=2100:00 +CPU= 37.782
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002100
  11:05:30 STEP 2101 H=2101:00 +CPU= 22.245
  11:05:56 STEP 2102 H=2102:00 +CPU= 22.689
  11:06:35 STEP 2103 H=2103:00 +CPU= 33.981
  11:07:01 STEP 2104 H=2104:00 +CPU= 22.849
  11:07:26 STEP 2105 H=2105:00 +CPU= 22.285
  11:08:05 STEP 2106 H=2106:00 +CPU= 34.795
  11:08:32 STEP 2107 H=2107:00 +CPU= 22.280
  11:08:58 STEP 2108 H=2108:00 +CPU= 22.829
  11:09:37 STEP 2109 H=2109:00 +CPU= 34.065
  11:10:02 STEP 2110 H=2110:00 +CPU= 22.667
  11:10:27 STEP 2111 H=2111:00 +CPU= 22.654
  11:11:11 STEP 2112 H=2112:00 +CPU= 39.909
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_87.zip
Uploading the intermediate file: upload_file_87.zip
  11:11:35 STEP 2113 H=2113:00 +CPU= 22.029
  11:12:00 STEP 2114 H=2114:00 +CPU= 22.632
  11:12:37 STEP 2115 H=2115:00 +CPU= 33.608
  11:13:02 STEP 2116 H=2116:00 +CPU= 22.567
  11:13:26 STEP 2117 H=2117:00 +CPU= 21.981
  11:14:04 STEP 2118 H=2118:00 +CPU= 34.359
  11:14:28 STEP 2119 H=2119:00 +CPU= 21.967
  11:14:53 STEP 2120 H=2120:00 +CPU= 22.538
  11:15:30 STEP 2121 H=2121:00 +CPU= 33.612
  11:15:55 STEP 2122 H=2122:00 +CPU= 22.603
  11:16:19 STEP 2123 H=2123:00 +CPU= 22.000
  11:16:59 STEP 2124 H=2124:00 +CPU= 37.372
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002124
  11:17:24 STEP 2125 H=2125:00 +CPU= 22.008
  11:17:48 STEP 2126 H=2126:00 +CPU= 22.626
  11:18:25 STEP 2127 H=2127:00 +CPU= 33.500
  11:18:50 STEP 2128 H=2128:00 +CPU= 22.584
  11:19:14 STEP 2129 H=2129:00 +CPU= 21.804
  11:19:51 STEP 2130 H=2130:00 +CPU= 34.233
  11:20:15 STEP 2131 H=2131:00 +CPU= 21.870
  11:20:40 STEP 2132 H=2132:00 +CPU= 22.446
  11:21:17 STEP 2133 H=2133:00 +CPU= 33.442
  11:21:41 STEP 2134 H=2134:00 +CPU= 22.316
  11:22:06 STEP 2135 H=2135:00 +CPU= 22.606
  11:22:50 STEP 2136 H=2136:00 +CPU= 39.828
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_88.zip
Uploading the intermediate file: upload_file_88.zip
  11:23:15 STEP 2137 H=2137:00 +CPU= 22.144
  11:23:41 STEP 2138 H=2138:00 +CPU= 22.768
  11:24:20 STEP 2139 H=2139:00 +CPU= 33.941
  11:24:46 STEP 2140 H=2140:00 +CPU= 22.615
  11:25:12 STEP 2141 H=2141:00 +CPU= 22.121
  11:25:51 STEP 2142 H=2142:00 +CPU= 34.661
  11:26:16 STEP 2143 H=2143:00 +CPU= 22.045
  11:26:42 STEP 2144 H=2144:00 +CPU= 22.733
  11:27:20 STEP 2145 H=2145:00 +CPU= 33.810
  11:27:45 STEP 2146 H=2146:00 +CPU= 22.537
  11:28:09 STEP 2147 H=2147:00 +CPU= 21.897
  11:28:50 STEP 2148 H=2148:00 +CPU= 37.275
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002148
  11:29:14 STEP 2149 H=2149:00 +CPU= 21.946
  11:29:40 STEP 2150 H=2150:00 +CPU= 22.964
  11:30:19 STEP 2151 H=2151:00 +CPU= 34.168
  11:30:45 STEP 2152 H=2152:00 +CPU= 23.045
  11:31:10 STEP 2153 H=2153:00 +CPU= 22.341
  11:31:50 STEP 2154 H=2154:00 +CPU= 34.852
  11:32:15 STEP 2155 H=2155:00 +CPU= 22.050
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  19:13:56 STEP 2156 H=2156:00 +CPU= 26.896
  19:14:41 STEP 2157 H=2157:00 +CPU= 34.590
  19:15:06 STEP 2158 H=2158:00 +CPU= 22.487
  19:15:31 STEP 2159 H=2159:00 +CPU= 22.492
  19:16:16 STEP 2160 H=2160:00 +CPU= 39.598
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_89.zip
Uploading the intermediate file: upload_file_89.zip
  19:16:40 STEP 2161 H=2161:00 +CPU= 21.820
  19:17:05 STEP 2162 H=2162:00 +CPU= 22.520
  19:17:42 STEP 2163 H=2163:00 +CPU= 33.373
  19:18:07 STEP 2164 H=2164:00 +CPU= 22.520
  19:18:30 STEP 2165 H=2165:00 +CPU= 21.835
  19:19:09 STEP 2166 H=2166:00 +CPU= 34.313
  19:19:33 STEP 2167 H=2167:00 +CPU= 22.187
  19:19:59 STEP 2168 H=2168:00 +CPU= 22.641
  19:20:36 STEP 2169 H=2169:00 +CPU= 33.834
  19:21:02 STEP 2170 H=2170:00 +CPU= 22.792
  19:21:26 STEP 2171 H=2171:00 +CPU= 21.887
  19:22:08 STEP 2172 H=2172:00 +CPU= 37.506
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002172
  19:22:32 STEP 2173 H=2173:00 +CPU= 22.063
  19:22:58 STEP 2174 H=2174:00 +CPU= 22.720
  19:23:35 STEP 2175 H=2175:00 +CPU= 33.723
  19:24:01 STEP 2176 H=2176:00 +CPU= 22.761
  19:24:25 STEP 2177 H=2177:00 +CPU= 21.991
  19:25:04 STEP 2178 H=2178:00 +CPU= 34.349
  19:25:28 STEP 2179 H=2179:00 +CPU= 22.105
  19:25:54 STEP 2180 H=2180:00 +CPU= 22.654
  19:26:31 STEP 2181 H=2181:00 +CPU= 33.641
  19:26:57 STEP 2182 H=2182:00 +CPU= 22.736
  19:27:22 STEP 2183 H=2183:00 +CPU= 22.739
  19:28:06 STEP 2184 H=2184:00 +CPU= 39.817
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_90.zip
Uploading the intermediate file: upload_file_90.zip
  19:28:31 STEP 2185 H=2185:00 +CPU= 22.057
  19:28:56 STEP 2186 H=2186:00 +CPU= 22.618
  19:29:35 STEP 2187 H=2187:00 +CPU= 33.919
  19:30:02 STEP 2188 H=2188:00 +CPU= 22.855
  19:30:27 STEP 2189 H=2189:00 +CPU= 22.158
  19:31:05 STEP 2190 H=2190:00 +CPU= 34.460
  19:31:30 STEP 2191 H=2191:00 +CPU= 22.309
  19:31:55 STEP 2192 H=2192:00 +CPU= 22.818
  19:32:33 STEP 2193 H=2193:00 +CPU= 33.919
  19:32:59 STEP 2194 H=2194:00 +CPU= 22.818
  19:33:24 STEP 2195 H=2195:00 +CPU= 22.243
  19:34:06 STEP 2196 H=2196:00 +CPU= 37.783
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002196
  19:34:30 STEP 2197 H=2197:00 +CPU= 22.205
  19:34:56 STEP 2198 H=2198:00 +CPU= 22.795
  19:35:34 STEP 2199 H=2199:00 +CPU= 34.029
  19:35:59 STEP 2200 H=2200:00 +CPU= 22.904
  19:36:24 STEP 2201 H=2201:00 +CPU= 22.258
  19:37:03 STEP 2202 H=2202:00 +CPU= 34.706
  19:37:27 STEP 2203 H=2203:00 +CPU= 22.302
  19:37:53 STEP 2204 H=2204:00 +CPU= 22.913
  19:38:31 STEP 2205 H=2205:00 +CPU= 33.991
  19:38:56 STEP 2206 H=2206:00 +CPU= 22.863
  19:39:22 STEP 2207 H=2207:00 +CPU= 23.066
  19:40:10 STEP 2208 H=2208:00 +CPU= 40.731
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_91.zip
Uploading the intermediate file: upload_file_91.zip
  19:40:35 STEP 2209 H=2209:00 +CPU= 22.342
  19:41:01 STEP 2210 H=2210:00 +CPU= 22.911
  19:41:39 STEP 2211 H=2211:00 +CPU= 34.252
  19:42:05 STEP 2212 H=2212:00 +CPU= 23.022
  19:42:30 STEP 2213 H=2213:00 +CPU= 22.497
  19:43:09 STEP 2214 H=2214:00 +CPU= 34.956
  19:43:34 STEP 2215 H=2215:00 +CPU= 22.479
  19:44:00 STEP 2216 H=2216:00 +CPU= 23.142
  19:44:38 STEP 2217 H=2217:00 +CPU= 34.381
  19:45:04 STEP 2218 H=2218:00 +CPU= 23.186
  19:45:29 STEP 2219 H=2219:00 +CPU= 22.502
  19:46:12 STEP 2220 H=2220:00 +CPU= 38.398
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002220
  19:46:37 STEP 2221 H=2221:00 +CPU= 22.544
  19:47:03 STEP 2222 H=2222:00 +CPU= 23.078
  19:47:42 STEP 2223 H=2223:00 +CPU= 34.606
  19:48:08 STEP 2224 H=2224:00 +CPU= 23.134
  19:48:32 STEP 2225 H=2225:00 +CPU= 22.426
  19:49:11 STEP 2226 H=2226:00 +CPU= 34.768
  19:49:36 STEP 2227 H=2227:00 +CPU= 22.413
  19:50:02 STEP 2228 H=2228:00 +CPU= 22.992
  19:50:40 STEP 2229 H=2229:00 +CPU= 34.223
  19:51:05 STEP 2230 H=2230:00 +CPU= 23.002
  19:51:31 STEP 2231 H=2231:00 +CPU= 23.144
  19:52:16 STEP 2232 H=2232:00 +CPU= 40.364
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_92.zip
Uploading the intermediate file: upload_file_92.zip
  19:52:41 STEP 2233 H=2233:00 +CPU= 22.483
  19:53:07 STEP 2234 H=2234:00 +CPU= 23.155
  19:53:45 STEP 2235 H=2235:00 +CPU= 34.259
  19:54:10 STEP 2236 H=2236:00 +CPU= 22.886
  19:54:35 STEP 2237 H=2237:00 +CPU= 22.293
  19:55:14 STEP 2238 H=2238:00 +CPU= 34.929
  19:55:39 STEP 2239 H=2239:00 +CPU= 22.411
  19:56:05 STEP 2240 H=2240:00 +CPU= 23.065
  19:56:43 STEP 2241 H=2241:00 +CPU= 34.277
  19:57:09 STEP 2242 H=2242:00 +CPU= 23.149
  19:57:36 STEP 2243 H=2243:00 +CPU= 22.695
  19:58:20 STEP 2244 H=2244:00 +CPU= 38.592
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002244
  19:58:45 STEP 2245 H=2245:00 +CPU= 22.615
  19:59:11 STEP 2246 H=2246:00 +CPU= 23.211
  19:59:50 STEP 2247 H=2247:00 +CPU= 34.421
  20:00:17 STEP 2248 H=2248:00 +CPU= 23.231
  20:00:43 STEP 2249 H=2249:00 +CPU= 22.557
  20:01:24 STEP 2250 H=2250:00 +CPU= 35.583
  20:01:52 STEP 2251 H=2251:00 +CPU= 22.851
  20:02:21 STEP 2252 H=2252:00 +CPU= 23.388
  20:03:01 STEP 2253 H=2253:00 +CPU= 35.030
  20:03:27 STEP 2254 H=2254:00 +CPU= 22.994
  20:03:53 STEP 2255 H=2255:00 +CPU= 23.311
  20:04:39 STEP 2256 H=2256:00 +CPU= 40.966
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_93.zip
Uploading the intermediate file: upload_file_93.zip
  20:05:04 STEP 2257 H=2257:00 +CPU= 22.540
  20:05:30 STEP 2258 H=2258:00 +CPU= 23.147
  20:06:09 STEP 2259 H=2259:00 +CPU= 34.792
  20:06:35 STEP 2260 H=2260:00 +CPU= 23.564
  20:07:01 STEP 2261 H=2261:00 +CPU= 22.753
  20:07:40 STEP 2262 H=2262:00 +CPU= 35.535
  20:08:05 STEP 2263 H=2263:00 +CPU= 22.682
  20:08:31 STEP 2264 H=2264:00 +CPU= 23.554
  20:09:10 STEP 2265 H=2265:00 +CPU= 34.728
  20:09:37 STEP 2266 H=2266:00 +CPU= 23.617
  20:10:02 STEP 2267 H=2267:00 +CPU= 22.833
  20:10:47 STEP 2268 H=2268:00 +CPU= 39.328
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002268
  20:11:13 STEP 2269 H=2269:00 +CPU= 23.155
  20:11:41 STEP 2270 H=2270:00 +CPU= 23.851
  20:12:22 STEP 2271 H=2271:00 +CPU= 35.500
  20:12:48 STEP 2272 H=2272:00 +CPU= 23.835
  20:13:14 STEP 2273 H=2273:00 +CPU= 23.424
  20:13:55 STEP 2274 H=2274:00 +CPU= 36.078
  20:14:21 STEP 2275 H=2275:00 +CPU= 23.231
  20:14:47 STEP 2276 H=2276:00 +CPU= 23.863
  20:15:27 STEP 2277 H=2277:00 +CPU= 35.500
  20:15:53 STEP 2278 H=2278:00 +CPU= 23.933
  20:16:20 STEP 2279 H=2279:00 +CPU= 24.101
  20:17:09 STEP 2280 H=2280:00 +CPU= 42.071
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_94.zip
Uploading the intermediate file: upload_file_94.zip
  20:17:35 STEP 2281 H=2281:00 +CPU= 23.193
  20:18:02 STEP 2282 H=2282:00 +CPU= 23.851
  20:18:42 STEP 2283 H=2283:00 +CPU= 35.344
  20:19:08 STEP 2284 H=2284:00 +CPU= 23.731
  20:19:33 STEP 2285 H=2285:00 +CPU= 22.809
  20:20:14 STEP 2286 H=2286:00 +CPU= 35.909
  20:20:39 STEP 2287 H=2287:00 +CPU= 22.966
  20:21:06 STEP 2288 H=2288:00 +CPU= 23.675
  20:21:45 STEP 2289 H=2289:00 +CPU= 35.120
  20:22:11 STEP 2290 H=2290:00 +CPU= 23.499
  20:22:37 STEP 2291 H=2291:00 +CPU= 22.948
  20:23:23 STEP 2292 H=2292:00 +CPU= 39.520
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002292
  20:23:51 STEP 2293 H=2293:00 +CPU= 23.400
  20:24:18 STEP 2294 H=2294:00 +CPU= 23.355
  20:24:58 STEP 2295 H=2295:00 +CPU= 34.907
  20:25:25 STEP 2296 H=2296:00 +CPU= 23.566
  20:25:51 STEP 2297 H=2297:00 +CPU= 22.832
  20:26:31 STEP 2298 H=2298:00 +CPU= 35.634
  20:26:56 STEP 2299 H=2299:00 +CPU= 22.870
  20:27:22 STEP 2300 H=2300:00 +CPU= 23.515
  20:28:01 STEP 2301 H=2301:00 +CPU= 35.253
  20:28:28 STEP 2302 H=2302:00 +CPU= 23.682
  20:28:55 STEP 2303 H=2303:00 +CPU= 23.725
  20:29:42 STEP 2304 H=2304:00 +CPU= 41.388
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_95.zip
Uploading the intermediate file: upload_file_95.zip
  20:30:07 STEP 2305 H=2305:00 +CPU= 22.866
  20:30:34 STEP 2306 H=2306:00 +CPU= 23.484
  20:31:12 STEP 2307 H=2307:00 +CPU= 34.908
  20:31:38 STEP 2308 H=2308:00 +CPU= 23.409
  20:32:04 STEP 2309 H=2309:00 +CPU= 22.952
  20:32:44 STEP 2310 H=2310:00 +CPU= 35.877
  20:33:10 STEP 2311 H=2311:00 +CPU= 22.938
  20:33:36 STEP 2312 H=2312:00 +CPU= 23.628
  20:34:14 STEP 2313 H=2313:00 +CPU= 35.013
  20:34:40 STEP 2314 H=2314:00 +CPU= 23.611
  20:35:06 STEP 2315 H=2315:00 +CPU= 22.969
  20:35:50 STEP 2316 H=2316:00 +CPU= 39.173
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002316
  20:36:16 STEP 2317 H=2317:00 +CPU= 22.934
  20:36:42 STEP 2318 H=2318:00 +CPU= 23.639
  20:37:21 STEP 2319 H=2319:00 +CPU= 35.057
  20:37:47 STEP 2320 H=2320:00 +CPU= 23.636
  20:38:12 STEP 2321 H=2321:00 +CPU= 23.013
  20:38:52 STEP 2322 H=2322:00 +CPU= 35.758
  20:39:17 STEP 2323 H=2323:00 +CPU= 22.893
  20:39:43 STEP 2324 H=2324:00 +CPU= 23.592
  20:40:22 STEP 2325 H=2325:00 +CPU= 35.073
  20:40:48 STEP 2326 H=2326:00 +CPU= 23.624
  20:41:14 STEP 2327 H=2327:00 +CPU= 23.728
  20:42:03 STEP 2328 H=2328:00 +CPU= 42.318
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_96.zip
Uploading the intermediate file: upload_file_96.zip
Uploading trickle at timestep: 8380800
  20:42:31 STEP 2329 H=2329:00 +CPU= 23.696
  20:42:58 STEP 2330 H=2330:00 +CPU= 24.327
  20:43:40 STEP 2331 H=2331:00 +CPU= 36.159
  20:44:08 STEP 2332 H=2332:00 +CPU= 24.564
  20:44:36 STEP 2333 H=2333:00 +CPU= 24.011
  20:45:18 STEP 2334 H=2334:00 +CPU= 36.900
  20:45:44 STEP 2335 H=2335:00 +CPU= 23.029
  20:46:11 STEP 2336 H=2336:00 +CPU= 23.653
  20:46:51 STEP 2337 H=2337:00 +CPU= 35.323
  20:47:17 STEP 2338 H=2338:00 +CPU= 23.727
  20:47:44 STEP 2339 H=2339:00 +CPU= 23.225
  20:48:27 STEP 2340 H=2340:00 +CPU= 38.999
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002340
  20:48:52 STEP 2341 H=2341:00 +CPU= 23.043
  20:49:18 STEP 2342 H=2342:00 +CPU= 23.676
  20:49:57 STEP 2343 H=2343:00 +CPU= 34.906
  20:50:23 STEP 2344 H=2344:00 +CPU= 23.521
  20:50:49 STEP 2345 H=2345:00 +CPU= 22.984
  20:51:29 STEP 2346 H=2346:00 +CPU= 35.572
  20:51:54 STEP 2347 H=2347:00 +CPU= 22.786
  20:52:20 STEP 2348 H=2348:00 +CPU= 23.349
  20:52:58 STEP 2349 H=2349:00 +CPU= 34.752
  20:53:24 STEP 2350 H=2350:00 +CPU= 23.328
  20:53:50 STEP 2351 H=2351:00 +CPU= 23.553
  20:54:36 STEP 2352 H=2352:00 +CPU= 41.076
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_97.zip
Uploading the intermediate file: upload_file_97.zip
  20:55:01 STEP 2353 H=2353:00 +CPU= 22.752
  20:55:27 STEP 2354 H=2354:00 +CPU= 23.439
  20:56:06 STEP 2355 H=2355:00 +CPU= 34.924
  20:56:32 STEP 2356 H=2356:00 +CPU= 23.434
  20:56:57 STEP 2357 H=2357:00 +CPU= 22.797
  20:57:36 STEP 2358 H=2358:00 +CPU= 35.510
  20:58:02 STEP 2359 H=2359:00 +CPU= 22.889
  20:58:28 STEP 2360 H=2360:00 +CPU= 23.447
  20:59:06 STEP 2361 H=2361:00 +CPU= 34.207
  20:59:31 STEP 2362 H=2362:00 +CPU= 22.814
  20:59:56 STEP 2363 H=2363:00 +CPU= 22.073
  21:00:38 STEP 2364 H=2364:00 +CPU= 37.777
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002364
  21:01:03 STEP 2365 H=2365:00 +CPU= 22.405
  21:01:30 STEP 2366 H=2366:00 +CPU= 23.253
  21:02:09 STEP 2367 H=2367:00 +CPU= 34.560
  21:02:36 STEP 2368 H=2368:00 +CPU= 23.280
  21:03:02 STEP 2369 H=2369:00 +CPU= 22.642
  21:03:41 STEP 2370 H=2370:00 +CPU= 35.346
  21:04:07 STEP 2371 H=2371:00 +CPU= 22.515
  21:04:33 STEP 2372 H=2372:00 +CPU= 23.124
  21:05:12 STEP 2373 H=2373:00 +CPU= 34.600
  21:05:39 STEP 2374 H=2374:00 +CPU= 23.408
  21:06:05 STEP 2375 H=2375:00 +CPU= 23.529
  21:06:52 STEP 2376 H=2376:00 +CPU= 41.238
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002364
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_98.zip
Uploading the intermediate file: upload_file_98.zip
  21:07:18 STEP 2377 H=2377:00 +CPU= 22.781
  21:07:44 STEP 2378 H=2378:00 +CPU= 23.304
  21:08:23 STEP 2379 H=2379:00 +CPU= 34.591
  21:08:50 STEP 2380 H=2380:00 +CPU= 23.324
  21:09:16 STEP 2381 H=2381:00 +CPU= 22.798
  21:09:56 STEP 2382 H=2382:00 +CPU= 35.349
  21:10:21 STEP 2383 H=2383:00 +CPU= 22.724
  21:10:48 STEP 2384 H=2384:00 +CPU= 23.404
  21:11:27 STEP 2385 H=2385:00 +CPU= 34.799
  21:11:54 STEP 2386 H=2386:00 +CPU= 23.349
  21:12:20 STEP 2387 H=2387:00 +CPU= 22.815
  21:13:03 STEP 2388 H=2388:00 +CPU= 38.590
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002388
  21:13:29 STEP 2389 H=2389:00 +CPU= 22.753
  21:13:55 STEP 2390 H=2390:00 +CPU= 23.247
  21:14:35 STEP 2391 H=2391:00 +CPU= 34.723
  21:15:01 STEP 2392 H=2392:00 +CPU= 23.370
  21:15:26 STEP 2393 H=2393:00 +CPU= 22.697
  21:16:06 STEP 2394 H=2394:00 +CPU= 35.108
  21:16:31 STEP 2395 H=2395:00 +CPU= 22.487
  21:16:57 STEP 2396 H=2396:00 +CPU= 23.150
  21:17:35 STEP 2397 H=2397:00 +CPU= 34.430
  21:18:01 STEP 2398 H=2398:00 +CPU= 23.152
  21:18:27 STEP 2399 H=2399:00 +CPU= 23.321
  21:19:13 STEP 2400 H=2400:00 +CPU= 40.615
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_99.zip
Uploading the intermediate file: upload_file_99.zip
  21:19:38 STEP 2401 H=2401:00 +CPU= 22.528
  21:20:04 STEP 2402 H=2402:00 +CPU= 23.077
  21:20:43 STEP 2403 H=2403:00 +CPU= 34.391
  21:21:09 STEP 2404 H=2404:00 +CPU= 23.198
  21:21:34 STEP 2405 H=2405:00 +CPU= 22.610
  21:22:13 STEP 2406 H=2406:00 +CPU= 35.213
  21:22:39 STEP 2407 H=2407:00 +CPU= 22.536
  21:23:05 STEP 2408 H=2408:00 +CPU= 23.045
  21:23:43 STEP 2409 H=2409:00 +CPU= 34.457
  21:24:09 STEP 2410 H=2410:00 +CPU= 23.023
  21:24:35 STEP 2411 H=2411:00 +CPU= 22.676
  21:25:17 STEP 2412 H=2412:00 +CPU= 38.215
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002412
  21:25:43 STEP 2413 H=2413:00 +CPU= 22.534
  21:26:09 STEP 2414 H=2414:00 +CPU= 23.224
  21:26:48 STEP 2415 H=2415:00 +CPU= 34.458
  21:27:13 STEP 2416 H=2416:00 +CPU= 23.111
  21:27:39 STEP 2417 H=2417:00 +CPU= 22.567
  21:28:18 STEP 2418 H=2418:00 +CPU= 35.016
  21:28:43 STEP 2419 H=2419:00 +CPU= 22.493
  21:29:09 STEP 2420 H=2420:00 +CPU= 22.966
  21:29:47 STEP 2421 H=2421:00 +CPU= 34.261
  21:30:13 STEP 2422 H=2422:00 +CPU= 22.921
  21:30:39 STEP 2423 H=2423:00 +CPU= 23.131
  21:31:24 STEP 2424 H=2424:00 +CPU= 40.597
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_100.zip
Uploading the intermediate file: upload_file_100.zip
  21:31:50 STEP 2425 H=2425:00 +CPU= 22.340
  21:32:16 STEP 2426 H=2426:00 +CPU= 23.070
  21:32:54 STEP 2427 H=2427:00 +CPU= 34.187
  21:33:20 STEP 2428 H=2428:00 +CPU= 23.087
  21:33:45 STEP 2429 H=2429:00 +CPU= 22.474
  21:34:24 STEP 2430 H=2430:00 +CPU= 34.802
  21:34:49 STEP 2431 H=2431:00 +CPU= 22.528
  21:35:15 STEP 2432 H=2432:00 +CPU= 22.954
  21:35:53 STEP 2433 H=2433:00 +CPU= 34.316
  21:36:19 STEP 2434 H=2434:00 +CPU= 23.047
  21:36:44 STEP 2435 H=2435:00 +CPU= 22.478
  21:37:28 STEP 2436 H=2436:00 +CPU= 38.202
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002436
  21:37:53 STEP 2437 H=2437:00 +CPU= 22.303
  21:38:19 STEP 2438 H=2438:00 +CPU= 22.993
  21:39:00 STEP 2439 H=2439:00 +CPU= 34.668
  21:39:27 STEP 2440 H=2440:00 +CPU= 22.938
  21:39:53 STEP 2441 H=2441:00 +CPU= 22.239
  21:40:35 STEP 2442 H=2442:00 +CPU= 34.921
  21:41:02 STEP 2443 H=2443:00 +CPU= 22.297
  21:41:29 STEP 2444 H=2444:00 +CPU= 23.038
  21:42:10 STEP 2445 H=2445:00 +CPU= 34.186
  21:42:37 STEP 2446 H=2446:00 +CPU= 22.951
  21:43:04 STEP 2447 H=2447:00 +CPU= 23.056
  21:43:53 STEP 2448 H=2448:00 +CPU= 40.543
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_101.zip
Uploading the intermediate file: upload_file_101.zip
  21:44:20 STEP 2449 H=2449:00 +CPU= 22.345
  21:44:47 STEP 2450 H=2450:00 +CPU= 22.959
  21:45:27 STEP 2451 H=2451:00 +CPU= 34.336
  21:45:55 STEP 2452 H=2452:00 +CPU= 23.057
  21:46:21 STEP 2453 H=2453:00 +CPU= 22.530
  21:47:03 STEP 2454 H=2454:00 +CPU= 34.980
  21:47:30 STEP 2455 H=2455:00 +CPU= 22.429
  21:47:57 STEP 2456 H=2456:00 +CPU= 23.040
  21:48:38 STEP 2457 H=2457:00 +CPU= 34.403
  21:49:05 STEP 2458 H=2458:00 +CPU= 22.970
  21:49:32 STEP 2459 H=2459:00 +CPU= 22.406
  21:50:17 STEP 2460 H=2460:00 +CPU= 38.107
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002460
  21:50:44 STEP 2461 H=2461:00 +CPU= 22.412
  21:51:12 STEP 2462 H=2462:00 +CPU= 22.998
  21:51:52 STEP 2463 H=2463:00 +CPU= 34.229
  21:52:19 STEP 2464 H=2464:00 +CPU= 22.862
  21:52:46 STEP 2465 H=2465:00 +CPU= 22.336
  21:53:27 STEP 2466 H=2466:00 +CPU= 34.939
  21:53:53 STEP 2467 H=2467:00 +CPU= 22.293
  21:54:20 STEP 2468 H=2468:00 +CPU= 22.962
  21:55:01 STEP 2469 H=2469:00 +CPU= 34.301
  21:55:28 STEP 2470 H=2470:00 +CPU= 22.845
  21:55:56 STEP 2471 H=2471:00 +CPU= 23.085
  21:56:44 STEP 2472 H=2472:00 +CPU= 40.438
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_102.zip
Uploading the intermediate file: upload_file_102.zip
  21:57:10 STEP 2473 H=2473:00 +CPU= 22.150
  21:57:37 STEP 2474 H=2474:00 +CPU= 22.863
  21:58:18 STEP 2475 H=2475:00 +CPU= 34.059
  21:58:45 STEP 2476 H=2476:00 +CPU= 22.638
  21:59:11 STEP 2477 H=2477:00 +CPU= 22.071
  21:59:52 STEP 2478 H=2478:00 +CPU= 34.667
  22:00:18 STEP 2479 H=2479:00 +CPU= 22.225
  22:00:46 STEP 2480 H=2480:00 +CPU= 22.902
  22:01:27 STEP 2481 H=2481:00 +CPU= 34.042
  22:01:54 STEP 2482 H=2482:00 +CPU= 22.741
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:02:33 STEP 2483 H=2483:00 +CPU= 22.176
  22:03:19 STEP 2484 H=2484:00 +CPU= 37.787
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002484
  22:03:45 STEP 2485 H=2485:00 +CPU= 22.044
  22:04:12 STEP 2486 H=2486:00 +CPU= 22.802
  22:04:53 STEP 2487 H=2487:00 +CPU= 34.026
  22:05:20 STEP 2488 H=2488:00 +CPU= 22.662
  22:05:46 STEP 2489 H=2489:00 +CPU= 22.212
  22:06:28 STEP 2490 H=2490:00 +CPU= 34.619
  22:06:54 STEP 2491 H=2491:00 +CPU= 22.076
  22:07:21 STEP 2492 H=2492:00 +CPU= 22.629
  22:08:01 STEP 2493 H=2493:00 +CPU= 33.732
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:11:52 STEP 2494 H=2494:00 +CPU= 22.898
  22:12:19 STEP 2495 H=2495:00 +CPU= 22.659
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:14:21 STEP 2496 H=2496:00 +CPU= 40.395
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_103.zip
Uploading the intermediate file: upload_file_103.zip
  22:14:47 STEP 2497 H=2497:00 +CPU= 21.977
  22:15:15 STEP 2498 H=2498:00 +CPU= 22.737
  22:15:56 STEP 2499 H=2499:00 +CPU= 34.069
  22:16:23 STEP 2500 H=2500:00 +CPU= 22.826
  22:16:50 STEP 2501 H=2501:00 +CPU= 22.213
  22:17:32 STEP 2502 H=2502:00 +CPU= 34.628
  22:17:58 STEP 2503 H=2503:00 +CPU= 22.002
  22:18:25 STEP 2504 H=2504:00 +CPU= 22.623
  22:19:06 STEP 2505 H=2505:00 +CPU= 34.063
  22:19:33 STEP 2506 H=2506:00 +CPU= 22.470
  22:20:00 STEP 2507 H=2507:00 +CPU= 22.142
  22:20:44 STEP 2508 H=2508:00 +CPU= 37.348
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002508
  22:21:10 STEP 2509 H=2509:00 +CPU= 21.922
  22:21:36 STEP 2510 H=2510:00 +CPU= 22.623
  22:22:16 STEP 2511 H=2511:00 +CPU= 33.751
  22:22:43 STEP 2512 H=2512:00 +CPU= 22.688
  22:23:10 STEP 2513 H=2513:00 +CPU= 22.182
  22:23:50 STEP 2514 H=2514:00 +CPU= 34.265
  22:24:15 STEP 2515 H=2515:00 +CPU= 21.948
  22:24:42 STEP 2516 H=2516:00 +CPU= 22.798
  22:25:20 STEP 2517 H=2517:00 +CPU= 33.996
  22:25:45 STEP 2518 H=2518:00 +CPU= 22.733
  22:26:11 STEP 2519 H=2519:00 +CPU= 22.996
  22:26:56 STEP 2520 H=2520:00 +CPU= 40.030
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_104.zip
Uploading the intermediate file: upload_file_104.zip
  22:27:21 STEP 2521 H=2521:00 +CPU= 22.326
  22:27:48 STEP 2522 H=2522:00 +CPU= 23.124
  22:28:31 STEP 2523 H=2523:00 +CPU= 34.472
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:30:20 STEP 2524 H=2524:00 +CPU= 23.005
  22:30:46 STEP 2525 H=2525:00 +CPU= 22.320
  22:31:28 STEP 2526 H=2526:00 +CPU= 34.659
  22:31:54 STEP 2527 H=2527:00 +CPU= 22.115
  22:32:22 STEP 2528 H=2528:00 +CPU= 22.854
  22:33:01 STEP 2529 H=2529:00 +CPU= 34.275
  22:33:29 STEP 2530 H=2530:00 +CPU= 22.869
  22:33:56 STEP 2531 H=2531:00 +CPU= 22.153
  22:34:41 STEP 2532 H=2532:00 +CPU= 37.838
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002532
  22:35:07 STEP 2533 H=2533:00 +CPU= 22.208
  22:35:34 STEP 2534 H=2534:00 +CPU= 22.798
  22:36:14 STEP 2535 H=2535:00 +CPU= 33.608
  22:36:40 STEP 2536 H=2536:00 +CPU= 22.712
  22:37:07 STEP 2537 H=2537:00 +CPU= 21.844
  22:37:47 STEP 2538 H=2538:00 +CPU= 34.963
  22:38:13 STEP 2539 H=2539:00 +CPU= 22.410
  22:38:39 STEP 2540 H=2540:00 +CPU= 23.028
  22:39:18 STEP 2541 H=2541:00 +CPU= 34.225
  22:39:45 STEP 2542 H=2542:00 +CPU= 23.056
  22:40:11 STEP 2543 H=2543:00 +CPU= 23.111
  22:40:58 STEP 2544 H=2544:00 +CPU= 40.487
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_105.zip
Uploading the intermediate file: upload_file_105.zip
  22:41:23 STEP 2545 H=2545:00 +CPU= 22.414
  22:41:50 STEP 2546 H=2546:00 +CPU= 23.012
  22:42:29 STEP 2547 H=2547:00 +CPU= 34.224
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:43:22 STEP 2548 H=2548:00 +CPU= 29.785
  09:43:52 STEP 2549 H=2549:00 +CPU= 24.366
  09:44:33 STEP 2550 H=2550:00 +CPU= 36.758
  09:45:00 STEP 2551 H=2551:00 +CPU= 23.930
  09:45:28 STEP 2552 H=2552:00 +CPU= 24.270
  09:46:09 STEP 2553 H=2553:00 +CPU= 36.363
  09:46:36 STEP 2554 H=2554:00 +CPU= 24.483
  09:47:03 STEP 2555 H=2555:00 +CPU= 23.889
  09:47:49 STEP 2556 H=2556:00 +CPU= 40.398
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002556
  09:48:16 STEP 2557 H=2557:00 +CPU= 23.684
  09:48:44 STEP 2558 H=2558:00 +CPU= 24.649
  09:49:25 STEP 2559 H=2559:00 +CPU= 36.073
  09:49:52 STEP 2560 H=2560:00 +CPU= 24.604
  09:50:19 STEP 2561 H=2561:00 +CPU= 23.731
  09:51:01 STEP 2562 H=2562:00 +CPU= 37.106
  09:51:28 STEP 2563 H=2563:00 +CPU= 23.947
  09:51:56 STEP 2564 H=2564:00 +CPU= 24.669
  09:52:36 STEP 2565 H=2565:00 +CPU= 36.149
  09:53:04 STEP 2566 H=2566:00 +CPU= 24.752
  09:53:32 STEP 2567 H=2567:00 +CPU= 24.722
  09:54:23 STEP 2568 H=2568:00 +CPU= 43.232
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994_106.zip
Uploading the intermediate file: upload_file_106.zip
Uploading trickle at timestep: 9244800
  09:54:50 STEP 2569 H=2569:00 +CPU= 23.950
  09:55:18 STEP 2570 H=2570:00 +CPU= 24.763
  09:55:59 STEP 2571 H=2571:00 +CPU= 36.437
  09:56:27 STEP 2572 H=2572:00 +CPU= 24.627
  09:56:54 STEP 2573 H=2573:00 +CPU= 24.036
  09:57:38 STEP 2574 H=2574:00 +CPU= 37.626
  09:58:08 STEP 2575 H=2575:00 +CPU= 24.387
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2006050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0814
(argv4) batchid: 975
(argv5) wuid: 12192458
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12192458
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_7fcb9e2461b393758364227bbfe7f835 to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0814_2006050100_123_975_12192458.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0814_2006050100_123_975_12192458.zip
ic_ancil_file: ic_ancil_12192458
ifsdata_file: ifsdata_12192458
climate_data_file: clim_data_12192458
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_2abb4b25874c83ab7bcf30e79c7898ac to: /var/lib/boinc-client/slots/0/ic_ancil_12192458.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12192458.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_23fbcc13644b434e98b097577249cc26 to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12192458.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12192458.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12192458.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12192458.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12192458.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12192458.xml
-- Model is restarting --
Adjusting last_iter, 2575, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12192458.xml
last_cpu_time: 71574
upload_file_number: 107
last_iter: 2567
last_upload: 9244800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0814_2006050100_123_975_12192458_0_r867721994
The child process has been launched with process id: 3425433
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Lian:1:1:3425433:3425433] [20230109:105745:1673222265:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  10:58:52 STEP 2568 H=2568:00 +CPU= 53.261
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:59:36 STEP 2569 H=2569:00 +CPU= 27.809
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:00:20 STEP 2570 H=2570:00 +CPU= 28.514
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:01:20 STEP 2571 H=2571:00 +CPU= 42.335
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:02:06 STEP 2572 H=2572:00 +CPU= 29.527
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:02:52 STEP 2573 H=2573:00 +CPU= 29.809
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:03:56 STEP 2574 H=2574:00 +CPU= 44.144
  11:04:29 STEP 2575 H=2575:00 +CPU= 28.638
  11:05:02 STEP 2576 H=2576:00 +CPU= 28.661
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:06:03 STEP 2577 H=2577:00 +CPU= 44.028
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:06:54 STEP 2578 H=2578:00 +CPU= 30.497
  11:07:27 STEP 2579 H=2579:00 +CPU= 29.351
  11:08:25 STEP 2580 H=2580:00 +CPU= 50.792
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+002580
  11:08:57 STEP 2581 H=2581:00 +CPU= 28.804
  11:09:32 STEP 2582 H=2582:00 +CPU= 30.404
  11:10:20 STEP 2583 H=2583:00 +CPU= 43.074
  11:10:55 STEP 2584 H=2584:00 +CPU= 31.258
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:11:52 STEP 2585 H=2585:00 +CPU= 29.572
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:12:57 STEP 2586 H=2586:00 +CPU= 45.681
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:13:43 STEP 2587 H=2587:00 +CPU= 29.481
  11:14:18 STEP 2588 H=2588:00 +CPU= 31.329
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:15:20 STEP 2589 H=2589:00 +CPU= 44.160
  11:15:56 STEP 2590 H=2590:00 +CPU= 31.529
  11:16:30 STEP 2591 H=2591:00 +CPU= 30.242
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:13:09 1491253 22278199 oifs_43r3_ps_0814_2006050100_123_975_12192458_0 9,244,800 71,381 0.0077
09 May 2023 16:12:36 1491253 22278199 oifs_43r3_ps_0814_2006050100_123_975_12192458_0 8,380,800 64,827 0.0077
09 May 2023 16:12:13 1491253 22278199 oifs_43r3_ps_0814_2006050100_123_975_12192458_0 7,516,800 58,314 0.0078
09 May 2023 16:11:32 1491253 22278199 oifs_43r3_ps_0814_2006050100_123_975_12192458_0 6,134,400 47,803 0.0078
09 May 2023 16:11:18 1491253 22278199 oifs_43r3_ps_0814_2006050100_123_975_12192458_0 4,492,800 34,603 0.0077
09 May 2023 16:02:42 1491253 22278199 oifs_43r3_ps_0814_2006050100_123_975_12192458_0 2,937,600 22,512 0.0077
09 May 2023 16:02:20 1491253 22278199 oifs_43r3_ps_0814_2006050100_123_975_12192458_0 864,000 6,611 0.0077


©2024 climateprediction.net