climateprediction.net home page
Task 22281909

Task 22281909

Name oifs_43r3_ps_0313_2010050100_123_979_12195957_0
Workunit 12195957
Created 23 Dec 2022, 23:20:55 UTC
Sent 5 Jan 2023, 4:23:27 UTC
Report deadline 4 Feb 2023, 4:23:27 UTC
Received 5 Jan 2023, 16:38:40 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1414915
Run time 12 hours 4 min 49 sec
CPU time 12 hours 3 min 21 sec
Validate state Invalid
Credit 7,257.60
Device peak FLOPS 4.73 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.68 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,637.96 MB
Stderr
<core_client_version>7.6.31</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded
</message>
<stderr_txt>
ojects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_92.zip
Uploading the intermediate file: upload_file_92.zip
  15:30:21 STEP 2233 H=2233:00 +CPU= 14.092
  15:30:34 STEP 2234 H=2234:00 +CPU= 13.160
  15:30:54 STEP 2235 H=2235:00 +CPU= 20.140
  15:31:07 STEP 2236 H=2236:00 +CPU= 13.652
  15:31:20 STEP 2237 H=2237:00 +CPU= 12.820
  15:31:41 STEP 2238 H=2238:00 +CPU= 20.860
  15:31:54 STEP 2239 H=2239:00 +CPU= 12.848
  15:32:08 STEP 2240 H=2240:00 +CPU= 13.632
  15:32:28 STEP 2241 H=2241:00 +CPU= 20.244
  15:32:42 STEP 2242 H=2242:00 +CPU= 13.748
  15:32:55 STEP 2243 H=2243:00 +CPU= 13.184
  15:33:17 STEP 2244 H=2244:00 +CPU= 22.380
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002244
  15:33:31 STEP 2245 H=2245:00 +CPU= 13.316
  15:33:44 STEP 2246 H=2246:00 +CPU= 13.164
  15:34:04 STEP 2247 H=2247:00 +CPU= 20.604
  15:34:17 STEP 2248 H=2248:00 +CPU= 13.180
  15:34:31 STEP 2249 H=2249:00 +CPU= 13.104
  15:34:51 STEP 2250 H=2250:00 +CPU= 20.548
  15:35:04 STEP 2251 H=2251:00 +CPU= 12.856
  15:35:18 STEP 2252 H=2252:00 +CPU= 13.960
  15:35:38 STEP 2253 H=2253:00 +CPU= 20.068
  15:35:52 STEP 2254 H=2254:00 +CPU= 13.736
  15:36:05 STEP 2255 H=2255:00 +CPU= 13.200
  15:36:31 STEP 2256 H=2256:00 +CPU= 24.916
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_93.zip
Uploading the intermediate file: upload_file_93.zip
  15:36:44 STEP 2257 H=2257:00 +CPU= 12.768
  15:36:58 STEP 2258 H=2258:00 +CPU= 13.588
  15:37:18 STEP 2259 H=2259:00 +CPU= 20.580
  15:37:32 STEP 2260 H=2260:00 +CPU= 14.048
  15:37:46 STEP 2261 H=2261:00 +CPU= 13.224
  15:38:06 STEP 2262 H=2262:00 +CPU= 20.436
  15:38:19 STEP 2263 H=2263:00 +CPU= 13.464
  15:38:33 STEP 2264 H=2264:00 +CPU= 13.632
  15:38:54 STEP 2265 H=2265:00 +CPU= 20.440
  15:39:07 STEP 2266 H=2266:00 +CPU= 13.256
  15:39:20 STEP 2267 H=2267:00 +CPU= 12.860
  15:39:42 STEP 2268 H=2268:00 +CPU= 22.572
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002268
  15:39:55 STEP 2269 H=2269:00 +CPU= 13.156
  15:40:10 STEP 2270 H=2270:00 +CPU= 13.616
  15:40:30 STEP 2271 H=2271:00 +CPU= 20.168
  15:40:44 STEP 2272 H=2272:00 +CPU= 14.240
  15:40:57 STEP 2273 H=2273:00 +CPU= 13.308
  15:41:18 STEP 2274 H=2274:00 +CPU= 21.036
  15:41:31 STEP 2275 H=2275:00 +CPU= 12.804
  15:41:44 STEP 2276 H=2276:00 +CPU= 13.096
  15:42:05 STEP 2277 H=2277:00 +CPU= 20.472
  15:42:18 STEP 2278 H=2278:00 +CPU= 13.432
  15:42:32 STEP 2279 H=2279:00 +CPU= 13.492
  15:42:55 STEP 2280 H=2280:00 +CPU= 23.620
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_94.zip
Uploading the intermediate file: upload_file_94.zip
  15:43:08 STEP 2281 H=2281:00 +CPU= 13.092
  15:43:22 STEP 2282 H=2282:00 +CPU= 13.568
  15:43:42 STEP 2283 H=2283:00 +CPU= 20.332
  15:43:56 STEP 2284 H=2284:00 +CPU= 13.216
  15:44:08 STEP 2285 H=2285:00 +CPU= 12.820
  15:44:29 STEP 2286 H=2286:00 +CPU= 20.960
  15:44:43 STEP 2287 H=2287:00 +CPU= 13.556
  15:44:56 STEP 2288 H=2288:00 +CPU= 13.472
  15:45:17 STEP 2289 H=2289:00 +CPU= 20.504
  15:45:31 STEP 2290 H=2290:00 +CPU= 13.680
  15:45:44 STEP 2291 H=2291:00 +CPU= 13.284
  15:46:07 STEP 2292 H=2292:00 +CPU= 22.976
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002292
  15:46:20 STEP 2293 H=2293:00 +CPU= 12.840
  15:46:33 STEP 2294 H=2294:00 +CPU= 13.172
  15:46:53 STEP 2295 H=2295:00 +CPU= 20.468
  15:47:07 STEP 2296 H=2296:00 +CPU= 13.268
  15:47:20 STEP 2297 H=2297:00 +CPU= 13.184
  15:47:40 STEP 2298 H=2298:00 +CPU= 20.440
  15:47:53 STEP 2299 H=2299:00 +CPU= 13.036
  15:48:07 STEP 2300 H=2300:00 +CPU= 13.612
  15:48:27 STEP 2301 H=2301:00 +CPU= 20.516
  15:48:41 STEP 2302 H=2302:00 +CPU= 13.328
  15:48:54 STEP 2303 H=2303:00 +CPU= 13.160
  15:49:19 STEP 2304 H=2304:00 +CPU= 24.152
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_95.zip
Uploading the intermediate file: upload_file_95.zip
  15:49:31 STEP 2305 H=2305:00 +CPU= 12.792
  15:49:45 STEP 2306 H=2306:00 +CPU= 13.788
  15:50:05 STEP 2307 H=2307:00 +CPU= 20.188
  15:50:19 STEP 2308 H=2308:00 +CPU= 13.896
  15:50:32 STEP 2309 H=2309:00 +CPU= 13.112
  15:50:53 STEP 2310 H=2310:00 +CPU= 20.912
  15:51:07 STEP 2311 H=2311:00 +CPU= 13.308
  15:51:20 STEP 2312 H=2312:00 +CPU= 13.200
  15:51:40 STEP 2313 H=2313:00 +CPU= 20.336
  15:51:53 STEP 2314 H=2314:00 +CPU= 13.204
  15:52:06 STEP 2315 H=2315:00 +CPU= 12.912
  15:52:29 STEP 2316 H=2316:00 +CPU= 22.496
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002316
  15:52:42 STEP 2317 H=2317:00 +CPU= 13.204
  15:52:55 STEP 2318 H=2318:00 +CPU= 13.360
  15:53:15 STEP 2319 H=2319:00 +CPU= 19.972
  15:53:29 STEP 2320 H=2320:00 +CPU= 13.796
  15:53:42 STEP 2321 H=2321:00 +CPU= 12.768
  15:54:03 STEP 2322 H=2322:00 +CPU= 21.012
  15:54:16 STEP 2323 H=2323:00 +CPU= 13.020
  15:54:29 STEP 2324 H=2324:00 +CPU= 13.164
  15:54:49 STEP 2325 H=2325:00 +CPU= 20.312
  15:55:03 STEP 2326 H=2326:00 +CPU= 13.232
  15:55:17 STEP 2327 H=2327:00 +CPU= 14.112
  15:55:41 STEP 2328 H=2328:00 +CPU= 24.560
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_96.zip
Uploading the intermediate file: upload_file_96.zip
  15:55:55 STEP 2329 H=2329:00 +CPU= 13.332
  15:56:08 STEP 2330 H=2330:00 +CPU= 13.156
  15:56:28 STEP 2331 H=2331:00 +CPU= 19.976
  15:56:42 STEP 2332 H=2332:00 +CPU= 14.044
  15:56:55 STEP 2333 H=2333:00 +CPU= 13.016
  15:57:16 STEP 2334 H=2334:00 +CPU= 20.628
  15:57:28 STEP 2335 H=2335:00 +CPU= 12.792
  15:57:42 STEP 2336 H=2336:00 +CPU= 13.116
  15:58:02 STEP 2337 H=2337:00 +CPU= 20.368
  15:58:16 STEP 2338 H=2338:00 +CPU= 13.744
  15:58:28 STEP 2339 H=2339:00 +CPU= 12.792
  15:58:50 STEP 2340 H=2340:00 +CPU= 22.032
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002340
  15:59:04 STEP 2341 H=2341:00 +CPU= 13.400
  15:59:17 STEP 2342 H=2342:00 +CPU= 13.484
  15:59:38 STEP 2343 H=2343:00 +CPU= 20.216
  15:59:51 STEP 2344 H=2344:00 +CPU= 13.172
  16:00:04 STEP 2345 H=2345:00 +CPU= 13.260
  16:00:25 STEP 2346 H=2346:00 +CPU= 21.108
  16:00:38 STEP 2347 H=2347:00 +CPU= 13.292
  16:00:51 STEP 2348 H=2348:00 +CPU= 13.084
  16:01:11 STEP 2349 H=2349:00 +CPU= 19.876
  16:01:25 STEP 2350 H=2350:00 +CPU= 13.672
  16:01:38 STEP 2351 H=2351:00 +CPU= 13.384
  16:02:04 STEP 2352 H=2352:00 +CPU= 24.128
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_97.zip
Uploading the intermediate file: upload_file_97.zip
  16:02:17 STEP 2353 H=2353:00 +CPU= 12.836
  16:02:30 STEP 2354 H=2354:00 +CPU= 13.428
  16:02:51 STEP 2355 H=2355:00 +CPU= 20.428
  16:03:04 STEP 2356 H=2356:00 +CPU= 13.740
  16:03:17 STEP 2357 H=2357:00 +CPU= 12.816
  16:03:38 STEP 2358 H=2358:00 +CPU= 20.480
  16:03:51 STEP 2359 H=2359:00 +CPU= 13.444
  16:04:04 STEP 2360 H=2360:00 +CPU= 13.172
  16:04:25 STEP 2361 H=2361:00 +CPU= 20.448
  16:04:38 STEP 2362 H=2362:00 +CPU= 13.228
  16:04:51 STEP 2363 H=2363:00 +CPU= 12.872
  16:05:13 STEP 2364 H=2364:00 +CPU= 22.580
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002364
  16:05:27 STEP 2365 H=2365:00 +CPU= 13.484
  16:05:40 STEP 2366 H=2366:00 +CPU= 13.576
  16:06:00 STEP 2367 H=2367:00 +CPU= 20.024
  16:06:14 STEP 2368 H=2368:00 +CPU= 13.780
  16:06:27 STEP 2369 H=2369:00 +CPU= 12.812
  16:06:48 STEP 2370 H=2370:00 +CPU= 20.908
  16:07:01 STEP 2371 H=2371:00 +CPU= 12.836
  16:07:14 STEP 2372 H=2372:00 +CPU= 13.428
  16:07:35 STEP 2373 H=2373:00 +CPU= 20.356
  16:07:48 STEP 2374 H=2374:00 +CPU= 13.144
  16:08:02 STEP 2375 H=2375:00 +CPU= 13.864
  16:08:25 STEP 2376 H=2376:00 +CPU= 23.800
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002364
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_98.zip
Uploading the intermediate file: upload_file_98.zip
  16:08:39 STEP 2377 H=2377:00 +CPU= 13.244
  16:08:52 STEP 2378 H=2378:00 +CPU= 13.304
  16:09:14 STEP 2379 H=2379:00 +CPU= 21.644
  16:09:28 STEP 2380 H=2380:00 +CPU= 14.096
  16:09:42 STEP 2381 H=2381:00 +CPU= 13.744
  16:10:03 STEP 2382 H=2382:00 +CPU= 20.808
  16:10:16 STEP 2383 H=2383:00 +CPU= 13.656
  16:10:30 STEP 2384 H=2384:00 +CPU= 13.736
  16:10:50 STEP 2385 H=2385:00 +CPU= 20.064
  16:11:04 STEP 2386 H=2386:00 +CPU= 13.828
  16:11:17 STEP 2387 H=2387:00 +CPU= 12.832
  16:11:39 STEP 2388 H=2388:00 +CPU= 22.504
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002388
  16:11:52 STEP 2389 H=2389:00 +CPU= 12.836
  16:12:05 STEP 2390 H=2390:00 +CPU= 13.260
  16:12:26 STEP 2391 H=2391:00 +CPU= 20.324
  16:12:39 STEP 2392 H=2392:00 +CPU= 13.268
  16:12:52 STEP 2393 H=2393:00 +CPU= 13.320
  16:13:13 STEP 2394 H=2394:00 +CPU= 20.344
  16:13:26 STEP 2395 H=2395:00 +CPU= 13.736
  16:13:40 STEP 2396 H=2396:00 +CPU= 13.364
  16:14:00 STEP 2397 H=2397:00 +CPU= 20.536
  16:14:14 STEP 2398 H=2398:00 +CPU= 13.652
  16:14:27 STEP 2399 H=2399:00 +CPU= 13.268
  16:14:51 STEP 2400 H=2400:00 +CPU= 24.000
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_99.zip
Uploading the intermediate file: upload_file_99.zip
  16:15:04 STEP 2401 H=2401:00 +CPU= 12.996
  16:15:18 STEP 2402 H=2402:00 +CPU= 13.928
  16:15:38 STEP 2403 H=2403:00 +CPU= 20.224
  16:15:52 STEP 2404 H=2404:00 +CPU= 13.744
  16:16:05 STEP 2405 H=2405:00 +CPU= 12.812
  16:16:26 STEP 2406 H=2406:00 +CPU= 20.544
  16:16:39 STEP 2407 H=2407:00 +CPU= 13.544
  16:16:53 STEP 2408 H=2408:00 +CPU= 13.532
  16:17:13 STEP 2409 H=2409:00 +CPU= 20.544
  16:17:27 STEP 2410 H=2410:00 +CPU= 13.352
  16:17:40 STEP 2411 H=2411:00 +CPU= 13.116
  16:18:02 STEP 2412 H=2412:00 +CPU= 22.464
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002412
  16:18:16 STEP 2413 H=2413:00 +CPU= 13.400
  16:18:29 STEP 2414 H=2414:00 +CPU= 13.628
  16:18:49 STEP 2415 H=2415:00 +CPU= 20.268
  16:19:04 STEP 2416 H=2416:00 +CPU= 14.024
  16:19:17 STEP 2417 H=2417:00 +CPU= 13.060
  16:19:38 STEP 2418 H=2418:00 +CPU= 21.336
  16:19:51 STEP 2419 H=2419:00 +CPU= 13.360
  16:20:05 STEP 2420 H=2420:00 +CPU= 14.024
  16:20:26 STEP 2421 H=2421:00 +CPU= 20.984
  16:20:40 STEP 2422 H=2422:00 +CPU= 13.564
  16:20:53 STEP 2423 H=2423:00 +CPU= 13.648
  16:21:17 STEP 2424 H=2424:00 +CPU= 23.844
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_100.zip
Uploading the intermediate file: upload_file_100.zip
  16:21:31 STEP 2425 H=2425:00 +CPU= 13.356
  16:21:44 STEP 2426 H=2426:00 +CPU= 13.628
  16:22:05 STEP 2427 H=2427:00 +CPU= 20.400
  16:22:18 STEP 2428 H=2428:00 +CPU= 13.668
  16:22:31 STEP 2429 H=2429:00 +CPU= 12.684
  16:22:52 STEP 2430 H=2430:00 +CPU= 20.832
  16:23:05 STEP 2431 H=2431:00 +CPU= 12.880
  16:23:19 STEP 2432 H=2432:00 +CPU= 13.724
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2010050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0313
(argv4) batchid: 979
(argv5) wuid: 12195957
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_61c03b9882ab0cf46e1181303e8c3674 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0313_2010050100_123_979_12195957.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0313_2010050100_123_979_12195957.zip
ic_ancil_file: ic_ancil_12195957
ifsdata_file: ifsdata_12195957
climate_data_file: clim_data_12195957
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_8c13abe53356dd9969ea810face8e63e to: /var/lib/boinc-client/slots/1/ic_ancil_12195957.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12195957.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8cfbcc8158ce08cb96f716c71de5dcde to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12195957.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12195957.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12195957.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12195957.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12195957.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12195957.xml
-- Model is restarting --
Adjusting last_iter, 2432, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12195957.xml
last_cpu_time: 39209
upload_file_number: 101
last_iter: 2423
last_upload: 8726400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726
The child process has been launched with process id: 32513
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:32513:32513] [20230105:162425:1672932265:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  16:24:51 STEP 2424 H=2424:00 +CPU= 23.812
  16:25:03 STEP 2425 H=2425:00 +CPU= 12.540
  16:25:17 STEP 2426 H=2426:00 +CPU= 13.844
  16:25:38 STEP 2427 H=2427:00 +CPU= 21.100
  16:25:52 STEP 2428 H=2428:00 +CPU= 13.544
  16:26:05 STEP 2429 H=2429:00 +CPU= 13.240
  16:26:26 STEP 2430 H=2430:00 +CPU= 20.400
  16:26:39 STEP 2431 H=2431:00 +CPU= 12.964
  16:26:52 STEP 2432 H=2432:00 +CPU= 13.684
  16:27:13 STEP 2433 H=2433:00 +CPU= 20.300
  16:27:26 STEP 2434 H=2434:00 +CPU= 13.852
  16:27:39 STEP 2435 H=2435:00 +CPU= 12.812
  16:28:02 STEP 2436 H=2436:00 +CPU= 22.584
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002436
  16:28:15 STEP 2437 H=2437:00 +CPU= 12.864
  16:28:29 STEP 2438 H=2438:00 +CPU= 14.252
  16:28:49 STEP 2439 H=2439:00 +CPU= 20.388
  16:29:03 STEP 2440 H=2440:00 +CPU= 13.532
  16:29:16 STEP 2441 H=2441:00 +CPU= 13.348
  16:29:37 STEP 2442 H=2442:00 +CPU= 20.384
  16:29:50 STEP 2443 H=2443:00 +CPU= 13.308
  16:30:04 STEP 2444 H=2444:00 +CPU= 13.724
  16:30:24 STEP 2445 H=2445:00 +CPU= 20.740
  16:30:38 STEP 2446 H=2446:00 +CPU= 13.716
  16:30:51 STEP 2447 H=2447:00 +CPU= 13.288
  16:31:17 STEP 2448 H=2448:00 +CPU= 25.356
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_101.zip
Uploading the intermediate file: upload_file_101.zip
  16:31:31 STEP 2449 H=2449:00 +CPU= 13.844
  16:31:46 STEP 2450 H=2450:00 +CPU= 15.060
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:32:37 STEP 2451 H=2451:00 +CPU= 25.892
  16:32:52 STEP 2452 H=2452:00 +CPU= 14.664
  16:33:08 STEP 2453 H=2453:00 +CPU= 16.392
  16:33:37 STEP 2454 H=2454:00 +CPU= 29.404
  16:33:56 STEP 2455 H=2455:00 +CPU= 18.016
  16:34:11 STEP 2456 H=2456:00 +CPU= 15.008
  16:34:31 STEP 2457 H=2457:00 +CPU= 20.080
  16:34:45 STEP 2458 H=2458:00 +CPU= 13.928
  16:34:59 STEP 2459 H=2459:00 +CPU= 14.064
  16:35:22 STEP 2460 H=2460:00 +CPU= 23.180
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002460
  16:35:35 STEP 2461 H=2461:00 +CPU= 12.832
  16:35:48 STEP 2462 H=2462:00 +CPU= 13.180
  16:36:09 STEP 2463 H=2463:00 +CPU= 20.760
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:36:38 STEP 2464 H=2464:00 +CPU= 18.288
  16:36:56 STEP 2465 H=2465:00 +CPU= 18.400
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:37:41 STEP 2466 H=2466:00 +CPU= 26.016
  16:37:55 STEP 2467 H=2467:00 +CPU= 13.456
  16:38:08 STEP 2468 H=2468:00 +CPU= 13.380
  16:38:29 STEP 2469 H=2469:00 +CPU= 21.164
  16:38:43 STEP 2470 H=2470:00 +CPU= 13.072
  16:38:56 STEP 2471 H=2471:00 +CPU= 13.296
  16:39:20 STEP 2472 H=2472:00 +CPU= 24.028
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_102.zip
Uploading the intermediate file: upload_file_102.zip
  16:39:34 STEP 2473 H=2473:00 +CPU= 13.760
  16:39:47 STEP 2474 H=2474:00 +CPU= 13.608
  16:40:08 STEP 2475 H=2475:00 +CPU= 20.364
  16:40:21 STEP 2476 H=2476:00 +CPU= 13.868
  16:40:34 STEP 2477 H=2477:00 +CPU= 12.828
  16:40:55 STEP 2478 H=2478:00 +CPU= 21.048
  16:41:08 STEP 2479 H=2479:00 +CPU= 12.812
  16:41:22 STEP 2480 H=2480:00 +CPU= 13.432
  16:41:42 STEP 2481 H=2481:00 +CPU= 20.376
  16:41:55 STEP 2482 H=2482:00 +CPU= 13.184
  16:42:08 STEP 2483 H=2483:00 +CPU= 13.312
  16:42:31 STEP 2484 H=2484:00 +CPU= 22.380
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002484
  16:42:44 STEP 2485 H=2485:00 +CPU= 13.332
  16:42:58 STEP 2486 H=2486:00 +CPU= 13.364
  16:43:18 STEP 2487 H=2487:00 +CPU= 20.676
  16:43:32 STEP 2488 H=2488:00 +CPU= 13.228
  16:43:45 STEP 2489 H=2489:00 +CPU= 13.132
  16:44:05 STEP 2490 H=2490:00 +CPU= 20.744
  16:44:18 STEP 2491 H=2491:00 +CPU= 12.752
  16:44:32 STEP 2492 H=2492:00 +CPU= 13.592
  16:44:52 STEP 2493 H=2493:00 +CPU= 20.608
  16:45:06 STEP 2494 H=2494:00 +CPU= 14.032
  16:45:20 STEP 2495 H=2495:00 +CPU= 13.728
  16:45:45 STEP 2496 H=2496:00 +CPU= 24.084
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_103.zip
Uploading the intermediate file: upload_file_103.zip
  16:45:58 STEP 2497 H=2497:00 +CPU= 12.784
  16:46:12 STEP 2498 H=2498:00 +CPU= 13.760
  16:46:32 STEP 2499 H=2499:00 +CPU= 20.000
  16:46:45 STEP 2500 H=2500:00 +CPU= 13.252
  16:46:58 STEP 2501 H=2501:00 +CPU= 13.284
  16:47:19 STEP 2502 H=2502:00 +CPU= 20.668
  16:47:32 STEP 2503 H=2503:00 +CPU= 13.088
  16:47:45 STEP 2504 H=2504:00 +CPU= 13.184
  16:48:07 STEP 2505 H=2505:00 +CPU= 20.672
  16:48:20 STEP 2506 H=2506:00 +CPU= 13.356
  16:48:34 STEP 2507 H=2507:00 +CPU= 13.420
  16:48:57 STEP 2508 H=2508:00 +CPU= 22.356
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002508
  16:49:10 STEP 2509 H=2509:00 +CPU= 13.048
  16:49:24 STEP 2510 H=2510:00 +CPU= 13.472
  16:49:44 STEP 2511 H=2511:00 +CPU= 19.900
  16:49:57 STEP 2512 H=2512:00 +CPU= 13.760
  16:50:11 STEP 2513 H=2513:00 +CPU= 13.344
  16:50:32 STEP 2514 H=2514:00 +CPU= 21.276
  16:50:45 STEP 2515 H=2515:00 +CPU= 12.776
  16:50:58 STEP 2516 H=2516:00 +CPU= 13.564
  16:51:19 STEP 2517 H=2517:00 +CPU= 20.804
  16:51:33 STEP 2518 H=2518:00 +CPU= 13.348
  16:51:46 STEP 2519 H=2519:00 +CPU= 13.660
  16:52:10 STEP 2520 H=2520:00 +CPU= 23.564
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_104.zip
Uploading the intermediate file: upload_file_104.zip
  16:52:23 STEP 2521 H=2521:00 +CPU= 13.052
  16:52:37 STEP 2522 H=2522:00 +CPU= 13.656
  16:52:57 STEP 2523 H=2523:00 +CPU= 20.420
  16:53:10 STEP 2524 H=2524:00 +CPU= 13.188
  16:53:23 STEP 2525 H=2525:00 +CPU= 12.788
  16:53:44 STEP 2526 H=2526:00 +CPU= 20.860
  16:53:57 STEP 2527 H=2527:00 +CPU= 13.284
  16:54:11 STEP 2528 H=2528:00 +CPU= 13.400
  16:54:31 STEP 2529 H=2529:00 +CPU= 20.340
  16:54:45 STEP 2530 H=2530:00 +CPU= 14.092
  16:54:59 STEP 2531 H=2531:00 +CPU= 13.348
  16:55:22 STEP 2532 H=2532:00 +CPU= 23.160
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002532
  16:55:35 STEP 2533 H=2533:00 +CPU= 12.868
  16:55:48 STEP 2534 H=2534:00 +CPU= 13.284
  16:56:08 STEP 2535 H=2535:00 +CPU= 20.420
  16:56:22 STEP 2536 H=2536:00 +CPU= 13.756
  16:56:35 STEP 2537 H=2537:00 +CPU= 12.840
  16:56:55 STEP 2538 H=2538:00 +CPU= 20.432
  16:57:08 STEP 2539 H=2539:00 +CPU= 13.132
  16:57:22 STEP 2540 H=2540:00 +CPU= 13.576
  16:57:43 STEP 2541 H=2541:00 +CPU= 20.772
  16:57:56 STEP 2542 H=2542:00 +CPU= 13.156
  16:58:09 STEP 2543 H=2543:00 +CPU= 13.188
  16:58:34 STEP 2544 H=2544:00 +CPU= 24.080
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_105.zip
Uploading the intermediate file: upload_file_105.zip
  16:58:46 STEP 2545 H=2545:00 +CPU= 12.776
  16:59:00 STEP 2546 H=2546:00 +CPU= 13.872
  16:59:20 STEP 2547 H=2547:00 +CPU= 19.996
  16:59:34 STEP 2548 H=2548:00 +CPU= 13.344
  16:59:47 STEP 2549 H=2549:00 +CPU= 13.256
  17:00:08 STEP 2550 H=2550:00 +CPU= 21.688
  17:00:22 STEP 2551 H=2551:00 +CPU= 13.812
  17:00:36 STEP 2552 H=2552:00 +CPU= 13.228
  17:00:56 STEP 2553 H=2553:00 +CPU= 20.452
  17:01:09 STEP 2554 H=2554:00 +CPU= 13.284
  17:01:22 STEP 2555 H=2555:00 +CPU= 13.012
  17:01:45 STEP 2556 H=2556:00 +CPU= 22.556
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002556
  17:01:58 STEP 2557 H=2557:00 +CPU= 13.240
  17:02:11 STEP 2558 H=2558:00 +CPU= 13.320
  17:02:32 STEP 2559 H=2559:00 +CPU= 20.976
  17:02:47 STEP 2560 H=2560:00 +CPU= 14.644
  17:03:00 STEP 2561 H=2561:00 +CPU= 13.212
  17:03:22 STEP 2562 H=2562:00 +CPU= 21.872
  17:03:35 STEP 2563 H=2563:00 +CPU= 12.840
  17:03:48 STEP 2564 H=2564:00 +CPU= 13.192
  17:04:09 STEP 2565 H=2565:00 +CPU= 20.484
  17:04:22 STEP 2566 H=2566:00 +CPU= 13.228
  17:04:36 STEP 2567 H=2567:00 +CPU= 13.864
  17:04:59 STEP 2568 H=2568:00 +CPU= 23.556
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_106.zip
Uploading the intermediate file: upload_file_106.zip
  17:05:13 STEP 2569 H=2569:00 +CPU= 13.408
  17:05:26 STEP 2570 H=2570:00 +CPU= 13.316
  17:05:46 STEP 2571 H=2571:00 +CPU= 20.332
  17:05:59 STEP 2572 H=2572:00 +CPU= 13.128
  17:06:12 STEP 2573 H=2573:00 +CPU= 12.812
  17:06:33 STEP 2574 H=2574:00 +CPU= 20.800
  17:06:46 STEP 2575 H=2575:00 +CPU= 12.764
  17:07:00 STEP 2576 H=2576:00 +CPU= 13.792
  17:07:20 STEP 2577 H=2577:00 +CPU= 20.256
  17:07:33 STEP 2578 H=2578:00 +CPU= 13.400
  17:07:46 STEP 2579 H=2579:00 +CPU= 13.204
  17:08:08 STEP 2580 H=2580:00 +CPU= 22.000
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002580
  17:08:22 STEP 2581 H=2581:00 +CPU= 13.600
  17:08:36 STEP 2582 H=2582:00 +CPU= 13.528
  17:08:56 STEP 2583 H=2583:00 +CPU= 20.272
  17:09:09 STEP 2584 H=2584:00 +CPU= 13.152
  17:09:22 STEP 2585 H=2585:00 +CPU= 12.760
  17:09:43 STEP 2586 H=2586:00 +CPU= 21.052
  17:09:56 STEP 2587 H=2587:00 +CPU= 13.156
  17:10:10 STEP 2588 H=2588:00 +CPU= 13.844
  17:10:30 STEP 2589 H=2589:00 +CPU= 20.216
  17:10:43 STEP 2590 H=2590:00 +CPU= 13.136
  17:10:57 STEP 2591 H=2591:00 +CPU= 14.060
  17:11:21 STEP 2592 H=2592:00 +CPU= 24.052
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_107.zip
Uploading the intermediate file: upload_file_107.zip
  17:11:34 STEP 2593 H=2593:00 +CPU= 12.804
  17:11:47 STEP 2594 H=2594:00 +CPU= 13.224
  17:12:08 STEP 2595 H=2595:00 +CPU= 20.972
  17:12:22 STEP 2596 H=2596:00 +CPU= 13.584
  17:12:35 STEP 2597 H=2597:00 +CPU= 12.944
  17:12:55 STEP 2598 H=2598:00 +CPU= 20.324
  17:13:08 STEP 2599 H=2599:00 +CPU= 12.856
  17:13:22 STEP 2600 H=2600:00 +CPU= 13.928
  17:13:42 STEP 2601 H=2601:00 +CPU= 20.444
  17:13:56 STEP 2602 H=2602:00 +CPU= 13.264
  17:14:09 STEP 2603 H=2603:00 +CPU= 12.888
  17:14:32 STEP 2604 H=2604:00 +CPU= 22.704
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002604
  17:14:45 STEP 2605 H=2605:00 +CPU= 13.360
  17:14:58 STEP 2606 H=2606:00 +CPU= 13.188
  17:15:18 STEP 2607 H=2607:00 +CPU= 20.256
  17:15:32 STEP 2608 H=2608:00 +CPU= 13.328
  17:15:45 STEP 2609 H=2609:00 +CPU= 13.144
  17:16:06 STEP 2610 H=2610:00 +CPU= 20.832
  17:16:18 STEP 2611 H=2611:00 +CPU= 12.692
  17:16:32 STEP 2612 H=2612:00 +CPU= 13.320
  17:16:52 STEP 2613 H=2613:00 +CPU= 20.540
  17:17:06 STEP 2614 H=2614:00 +CPU= 13.380
  17:17:19 STEP 2615 H=2615:00 +CPU= 13.716
  17:17:43 STEP 2616 H=2616:00 +CPU= 23.592
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_108.zip
Uploading the intermediate file: upload_file_108.zip
  17:17:56 STEP 2617 H=2617:00 +CPU= 13.012
  17:18:10 STEP 2618 H=2618:00 +CPU= 13.680
  17:18:30 STEP 2619 H=2619:00 +CPU= 20.500
  17:18:43 STEP 2620 H=2620:00 +CPU= 13.148
  17:18:56 STEP 2621 H=2621:00 +CPU= 12.712
  17:19:17 STEP 2622 H=2622:00 +CPU= 20.852
  17:19:30 STEP 2623 H=2623:00 +CPU= 12.836
  17:19:44 STEP 2624 H=2624:00 +CPU= 13.788
  17:20:04 STEP 2625 H=2625:00 +CPU= 20.832
  17:20:18 STEP 2626 H=2626:00 +CPU= 13.684
  17:20:31 STEP 2627 H=2627:00 +CPU= 13.280
  17:20:54 STEP 2628 H=2628:00 +CPU= 22.232
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002628
  17:21:07 STEP 2629 H=2629:00 +CPU= 13.416
  17:21:20 STEP 2630 H=2630:00 +CPU= 13.500
  17:21:41 STEP 2631 H=2631:00 +CPU= 20.472
  17:21:54 STEP 2632 H=2632:00 +CPU= 13.216
  17:22:07 STEP 2633 H=2633:00 +CPU= 12.880
  17:22:28 STEP 2634 H=2634:00 +CPU= 20.844
  17:22:41 STEP 2635 H=2635:00 +CPU= 13.080
  17:22:55 STEP 2636 H=2636:00 +CPU= 13.580
  17:23:15 STEP 2637 H=2637:00 +CPU= 19.992
  17:23:28 STEP 2638 H=2638:00 +CPU= 13.124
  17:23:41 STEP 2639 H=2639:00 +CPU= 13.820
  17:24:06 STEP 2640 H=2640:00 +CPU= 24.684
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_109.zip
Uploading the intermediate file: upload_file_109.zip
  17:24:19 STEP 2641 H=2641:00 +CPU= 12.776
  17:24:32 STEP 2642 H=2642:00 +CPU= 13.292
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2010050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0313
(argv4) batchid: 979
(argv5) wuid: 12195957
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_61c03b9882ab0cf46e1181303e8c3674 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0313_2010050100_123_979_12195957.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0313_2010050100_123_979_12195957.zip
ic_ancil_file: ic_ancil_12195957
ifsdata_file: ifsdata_12195957
climate_data_file: clim_data_12195957
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_8c13abe53356dd9969ea810face8e63e to: /var/lib/boinc-client/slots/1/ic_ancil_12195957.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12195957.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8cfbcc8158ce08cb96f716c71de5dcde to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12195957.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12195957.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12195957.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12195957.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12195957.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12195957.xml
-- Model is restarting --
Adjusting last_iter, 2643, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12195957.xml
last_cpu_time: 42765
upload_file_number: 110
last_iter: 2639
last_upload: 9504000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726
The child process has been launched with process id: 17235
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:atlas:1:1:17235:17235] [20230105:172550:1672935950:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  17:26:16 STEP 2640 H=2640:00 +CPU= 23.616
  17:26:28 STEP 2641 H=2641:00 +CPU= 12.808
  17:26:42 STEP 2642 H=2642:00 +CPU= 13.768
  17:27:03 STEP 2643 H=2643:00 +CPU= 20.428
  17:27:17 STEP 2644 H=2644:00 +CPU= 13.972
  17:27:30 STEP 2645 H=2645:00 +CPU= 13.140
  17:27:51 STEP 2646 H=2646:00 +CPU= 21.128
  17:28:05 STEP 2647 H=2647:00 +CPU= 13.728
  17:28:18 STEP 2648 H=2648:00 +CPU= 13.328
  17:28:39 STEP 2649 H=2649:00 +CPU= 20.704
  17:28:52 STEP 2650 H=2650:00 +CPU= 13.596
  17:29:06 STEP 2651 H=2651:00 +CPU= 13.540
  17:29:29 STEP 2652 H=2652:00 +CPU= 23.444
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002652
  17:29:42 STEP 2653 H=2653:00 +CPU= 13.264
  17:29:56 STEP 2654 H=2654:00 +CPU= 13.832
  17:30:17 STEP 2655 H=2655:00 +CPU= 20.788
  17:30:31 STEP 2656 H=2656:00 +CPU= 14.196
  17:30:44 STEP 2657 H=2657:00 +CPU= 13.116
  17:31:06 STEP 2658 H=2658:00 +CPU= 21.880
  17:31:20 STEP 2659 H=2659:00 +CPU= 13.520
  17:31:34 STEP 2660 H=2660:00 +CPU= 13.956
  17:31:54 STEP 2661 H=2661:00 +CPU= 20.424
  17:32:08 STEP 2662 H=2662:00 +CPU= 13.412
  17:32:21 STEP 2663 H=2663:00 +CPU= 13.600
  17:32:46 STEP 2664 H=2664:00 +CPU= 25.076
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12195957/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0313_2010050100_123_979_12195957_0_r1322695726_110.zip
Uploading the intermediate file: upload_file_110.zip
  17:33:00 STEP 2665 H=2665:00 +CPU= 13.288
  17:33:13 STEP 2666 H=2666:00 +CPU= 13.504
  17:33:34 STEP 2667 H=2667:00 +CPU= 20.396
  17:33:47 STEP 2668 H=2668:00 +CPU= 13.392
  17:34:00 STEP 2669 H=2669:00 +CPU= 12.920
  17:34:21 STEP 2670 H=2670:00 +CPU= 21.088
  17:34:34 STEP 2671 H=2671:00 +CPU= 12.580
  17:34:47 STEP 2672 H=2672:00 +CPU= 13.168
  17:35:07 STEP 2673 H=2673:00 +CPU= 20.548
  17:35:21 STEP 2674 H=2674:00 +CPU= 14.008
  17:35:35 STEP 2675 H=2675:00 +CPU= 13.384
  17:35:57 STEP 2676 H=2676:00 +CPU= 22.252
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002676
  17:36:10 STEP 2677 H=2677:00 +CPU= 13.092
  17:36:23 STEP 2678 H=2678:00 +CPU= 13.336
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:08:55 1414915 22281909 oifs_43r3_ps_0313_2010050100_123_979_12195957_0 5,184,000 22,612 0.0044
09 May 2023 16:08:51 1414915 22281909 oifs_43r3_ps_0313_2010050100_123_979_12195957_0 4,320,000 18,848 0.0044
09 May 2023 16:08:47 1414915 22281909 oifs_43r3_ps_0313_2010050100_123_979_12195957_0 3,456,000 15,089 0.0044
09 May 2023 16:08:44 1414915 22281909 oifs_43r3_ps_0313_2010050100_123_979_12195957_0 2,592,000 11,320 0.0044
09 May 2023 16:08:40 1414915 22281909 oifs_43r3_ps_0313_2010050100_123_979_12195957_0 1,728,000 7,550 0.0044
09 May 2023 16:08:37 1414915 22281909 oifs_43r3_ps_0313_2010050100_123_979_12195957_0 864,000 3,797 0.0044


©2024 climateprediction.net