climateprediction.net home page
Task 22261802

Task 22261802

Name oifs_43r3_ps_0385_1991050100_123_960_12177029_0
Workunit 12177029
Created 21 Dec 2022, 21:40:25 UTC
Sent 4 Jan 2023, 7:15:44 UTC
Report deadline 3 Feb 2023, 7:15:44 UTC
Received 4 Jan 2023, 19:10:54 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1499708
Run time 10 hours 43 min 11 sec
CPU time 10 hours 10 min 37 sec
Validate state Invalid
Credit 8,951.04
Device peak FLOPS 7.09 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,333.91 MB
Peak swap size 4,978.08 MB
Peak disk usage 2,807.87 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
EP 1331 H=1331:00 +CPU= 15.146
  15:57:55 STEP 1332 H=1332:00 +CPU= 27.128
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001332
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001332
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001332
  15:58:12 STEP 1333 H=1333:00 +CPU= 15.659
  15:58:30 STEP 1334 H=1334:00 +CPU= 16.569
  15:58:56 STEP 1335 H=1335:00 +CPU= 24.886
  15:59:13 STEP 1336 H=1336:00 +CPU= 16.711
  15:59:31 STEP 1337 H=1337:00 +CPU= 16.279
  15:59:59 STEP 1338 H=1338:00 +CPU= 26.083
  16:00:16 STEP 1339 H=1339:00 +CPU= 16.142
  16:00:32 STEP 1340 H=1340:00 +CPU= 15.825
  16:00:57 STEP 1341 H=1341:00 +CPU= 24.328
  16:01:14 STEP 1342 H=1342:00 +CPU= 16.198
  16:01:31 STEP 1343 H=1343:00 +CPU= 16.077
  16:02:04 STEP 1344 H=1344:00 +CPU= 29.020
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001344
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001344
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001332
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_55.zip
Uploading the intermediate file: upload_file_55.zip
  16:02:20 STEP 1345 H=1345:00 +CPU= 15.475
  16:02:37 STEP 1346 H=1346:00 +CPU= 16.353
  16:03:02 STEP 1347 H=1347:00 +CPU= 24.527
  16:03:18 STEP 1348 H=1348:00 +CPU= 15.502
  16:03:33 STEP 1349 H=1349:00 +CPU= 15.344
  16:03:59 STEP 1350 H=1350:00 +CPU= 24.977
  16:04:16 STEP 1351 H=1351:00 +CPU= 15.921
  16:04:34 STEP 1352 H=1352:00 +CPU= 16.139
  16:05:01 STEP 1353 H=1353:00 +CPU= 25.804
  16:05:19 STEP 1354 H=1354:00 +CPU= 16.576
  16:05:35 STEP 1355 H=1355:00 +CPU= 15.796
  16:06:05 STEP 1356 H=1356:00 +CPU= 28.389
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001356
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001356
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001356
  16:06:23 STEP 1357 H=1357:00 +CPU= 16.378
  16:06:40 STEP 1358 H=1358:00 +CPU= 16.702
  16:07:06 STEP 1359 H=1359:00 +CPU= 24.922
  16:07:22 STEP 1360 H=1360:00 +CPU= 15.958
  16:07:38 STEP 1361 H=1361:00 +CPU= 15.662
  16:08:05 STEP 1362 H=1362:00 +CPU= 25.402
  16:08:21 STEP 1363 H=1363:00 +CPU= 15.860
  16:08:38 STEP 1364 H=1364:00 +CPU= 16.531
  16:09:03 STEP 1365 H=1365:00 +CPU= 24.441
  16:09:20 STEP 1366 H=1366:00 +CPU= 15.971
  16:09:37 STEP 1367 H=1367:00 +CPU= 16.415
  16:10:10 STEP 1368 H=1368:00 +CPU= 28.601
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001368
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001368
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001356
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_56.zip
Uploading the intermediate file: upload_file_56.zip
  16:10:26 STEP 1369 H=1369:00 +CPU= 15.624
  16:10:43 STEP 1370 H=1370:00 +CPU= 16.354
  16:11:11 STEP 1371 H=1371:00 +CPU= 25.393
  16:11:28 STEP 1372 H=1372:00 +CPU= 16.631
  16:11:45 STEP 1373 H=1373:00 +CPU= 16.221
  16:12:11 STEP 1374 H=1374:00 +CPU= 25.560
  16:12:28 STEP 1375 H=1375:00 +CPU= 16.197
  16:12:45 STEP 1376 H=1376:00 +CPU= 16.511
  16:13:12 STEP 1377 H=1377:00 +CPU= 25.525
  16:13:29 STEP 1378 H=1378:00 +CPU= 16.303
  16:13:46 STEP 1379 H=1379:00 +CPU= 15.741
  16:14:14 STEP 1380 H=1380:00 +CPU= 27.601
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001380
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001380
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001380
  16:14:31 STEP 1381 H=1381:00 +CPU= 15.541
  16:14:47 STEP 1382 H=1382:00 +CPU= 15.891
  16:15:12 STEP 1383 H=1383:00 +CPU= 24.226
  16:15:28 STEP 1384 H=1384:00 +CPU= 15.849
  16:15:43 STEP 1385 H=1385:00 +CPU= 15.143
  16:16:09 STEP 1386 H=1386:00 +CPU= 25.424
  16:16:25 STEP 1387 H=1387:00 +CPU= 15.063
  16:16:42 STEP 1388 H=1388:00 +CPU= 16.049
  16:17:09 STEP 1389 H=1389:00 +CPU= 24.975
  16:17:26 STEP 1390 H=1390:00 +CPU= 16.491
  16:17:44 STEP 1391 H=1391:00 +CPU= 16.437
  16:18:18 STEP 1392 H=1392:00 +CPU= 30.080
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001392
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001392
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001380
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_57.zip
Uploading the intermediate file: upload_file_57.zip
  16:18:34 STEP 1393 H=1393:00 +CPU= 15.505
  16:18:51 STEP 1394 H=1394:00 +CPU= 16.797
  16:19:18 STEP 1395 H=1395:00 +CPU= 25.559
  16:19:35 STEP 1396 H=1396:00 +CPU= 15.926
  16:19:51 STEP 1397 H=1397:00 +CPU= 15.736
  16:20:16 STEP 1398 H=1398:00 +CPU= 24.962
  16:20:32 STEP 1399 H=1399:00 +CPU= 15.385
  16:20:49 STEP 1400 H=1400:00 +CPU= 15.984
  16:21:14 STEP 1401 H=1401:00 +CPU= 24.679
  16:21:32 STEP 1402 H=1402:00 +CPU= 16.434
  16:21:49 STEP 1403 H=1403:00 +CPU= 15.760
  16:22:19 STEP 1404 H=1404:00 +CPU= 28.749
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001404
  16:22:36 STEP 1405 H=1405:00 +CPU= 15.898
  16:22:53 STEP 1406 H=1406:00 +CPU= 16.254
  16:23:21 STEP 1407 H=1407:00 +CPU= 26.296
  16:23:38 STEP 1408 H=1408:00 +CPU= 16.225
  16:23:54 STEP 1409 H=1409:00 +CPU= 15.797
  16:24:21 STEP 1410 H=1410:00 +CPU= 25.704
  16:24:38 STEP 1411 H=1411:00 +CPU= 15.631
  16:24:54 STEP 1412 H=1412:00 +CPU= 15.672
  16:25:20 STEP 1413 H=1413:00 +CPU= 25.050
  16:25:36 STEP 1414 H=1414:00 +CPU= 16.103
  16:25:53 STEP 1415 H=1415:00 +CPU= 15.970
  16:26:25 STEP 1416 H=1416:00 +CPU= 29.557
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_58.zip
Uploading the intermediate file: upload_file_58.zip
  16:26:40 STEP 1417 H=1417:00 +CPU= 15.349
  16:26:57 STEP 1418 H=1418:00 +CPU= 15.867
  16:27:21 STEP 1419 H=1419:00 +CPU= 24.260
  16:27:38 STEP 1420 H=1420:00 +CPU= 15.864
  16:27:54 STEP 1421 H=1421:00 +CPU= 15.796
  16:28:23 STEP 1422 H=1422:00 +CPU= 26.290
  16:28:40 STEP 1423 H=1423:00 +CPU= 16.268
  16:28:58 STEP 1424 H=1424:00 +CPU= 17.029
  16:29:25 STEP 1425 H=1425:00 +CPU= 25.483
  16:29:42 STEP 1426 H=1426:00 +CPU= 16.489
  16:29:59 STEP 1427 H=1427:00 +CPU= 16.353
  16:30:29 STEP 1428 H=1428:00 +CPU= 28.685
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001428
  16:30:45 STEP 1429 H=1429:00 +CPU= 15.745
  16:31:02 STEP 1430 H=1430:00 +CPU= 16.016
  16:31:27 STEP 1431 H=1431:00 +CPU= 24.861
  16:31:44 STEP 1432 H=1432:00 +CPU= 16.089
  16:32:00 STEP 1433 H=1433:00 +CPU= 15.536
  16:32:26 STEP 1434 H=1434:00 +CPU= 25.298
  16:32:43 STEP 1435 H=1435:00 +CPU= 15.738
  16:33:00 STEP 1436 H=1436:00 +CPU= 16.111
  16:33:26 STEP 1437 H=1437:00 +CPU= 24.772
  16:33:42 STEP 1438 H=1438:00 +CPU= 16.068
  16:33:59 STEP 1439 H=1439:00 +CPU= 16.515
  16:34:31 STEP 1440 H=1440:00 +CPU= 29.450
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_59.zip
Uploading the intermediate file: upload_file_59.zip
  16:34:47 STEP 1441 H=1441:00 +CPU= 15.876
  16:35:04 STEP 1442 H=1442:00 +CPU= 15.999
  16:35:29 STEP 1443 H=1443:00 +CPU= 24.600
  16:35:46 STEP 1444 H=1444:00 +CPU= 15.983
  16:36:02 STEP 1445 H=1445:00 +CPU= 15.531
  16:36:28 STEP 1446 H=1446:00 +CPU= 24.996
  16:36:44 STEP 1447 H=1447:00 +CPU= 15.839
  16:37:01 STEP 1448 H=1448:00 +CPU= 16.135
  16:37:27 STEP 1449 H=1449:00 +CPU= 25.569
  16:37:44 STEP 1450 H=1450:00 +CPU= 16.129
  16:37:59 STEP 1451 H=1451:00 +CPU= 15.432
  16:38:27 STEP 1452 H=1452:00 +CPU= 27.328
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001452
  16:38:43 STEP 1453 H=1453:00 +CPU= 15.346
  16:39:00 STEP 1454 H=1454:00 +CPU= 15.887
  16:39:27 STEP 1455 H=1455:00 +CPU= 25.100
  16:39:44 STEP 1456 H=1456:00 +CPU= 16.171
  16:40:00 STEP 1457 H=1457:00 +CPU= 15.938
  16:40:29 STEP 1458 H=1458:00 +CPU= 26.661
  16:40:45 STEP 1459 H=1459:00 +CPU= 15.930
  16:41:02 STEP 1460 H=1460:00 +CPU= 16.348
  16:41:29 STEP 1461 H=1461:00 +CPU= 25.641
  16:41:46 STEP 1462 H=1462:00 +CPU= 16.162
  16:42:03 STEP 1463 H=1463:00 +CPU= 16.046
  16:42:39 STEP 1464 H=1464:00 +CPU= 29.276
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001464
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001464
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001452
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_60.zip
Uploading the intermediate file: upload_file_60.zip
  16:42:56 STEP 1465 H=1465:00 +CPU= 15.881
  16:43:12 STEP 1466 H=1466:00 +CPU= 16.363
  16:43:38 STEP 1467 H=1467:00 +CPU= 25.134
  16:43:54 STEP 1468 H=1468:00 +CPU= 15.975
  16:44:10 STEP 1469 H=1469:00 +CPU= 15.436
  16:44:36 STEP 1470 H=1470:00 +CPU= 25.098
  16:44:53 STEP 1471 H=1471:00 +CPU= 15.938
  16:45:10 STEP 1472 H=1472:00 +CPU= 16.667
  16:45:35 STEP 1473 H=1473:00 +CPU= 24.709
  16:45:51 STEP 1474 H=1474:00 +CPU= 15.382
  16:46:07 STEP 1475 H=1475:00 +CPU= 15.425
  16:46:34 STEP 1476 H=1476:00 +CPU= 27.031
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001476
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001476
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001476
  16:46:50 STEP 1477 H=1477:00 +CPU= 15.241
  16:47:07 STEP 1478 H=1478:00 +CPU= 16.498
  16:47:36 STEP 1479 H=1479:00 +CPU= 25.837
  16:47:52 STEP 1480 H=1480:00 +CPU= 16.419
  16:48:09 STEP 1481 H=1481:00 +CPU= 16.301
  16:48:37 STEP 1482 H=1482:00 +CPU= 26.161
  16:48:52 STEP 1483 H=1483:00 +CPU= 15.560
  16:49:12 STEP 1484 H=1484:00 +CPU= 16.986
  16:49:38 STEP 1485 H=1485:00 +CPU= 25.379
  16:49:55 STEP 1486 H=1486:00 +CPU= 16.238
  16:50:12 STEP 1487 H=1487:00 +CPU= 16.391
  16:50:47 STEP 1488 H=1488:00 +CPU= 29.369
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001488
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001488
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001476
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_61.zip
Uploading the intermediate file: upload_file_61.zip
  16:51:03 STEP 1489 H=1489:00 +CPU= 15.790
  16:51:20 STEP 1490 H=1490:00 +CPU= 16.016
  16:51:46 STEP 1491 H=1491:00 +CPU= 24.901
  16:52:03 STEP 1492 H=1492:00 +CPU= 16.851
  16:52:19 STEP 1493 H=1493:00 +CPU= 15.651
  16:52:46 STEP 1494 H=1494:00 +CPU= 25.726
  16:53:02 STEP 1495 H=1495:00 +CPU= 15.590
  16:53:19 STEP 1496 H=1496:00 +CPU= 15.987
  16:53:45 STEP 1497 H=1497:00 +CPU= 25.087
  16:54:02 STEP 1498 H=1498:00 +CPU= 16.343
  16:54:17 STEP 1499 H=1499:00 +CPU= 15.286
  16:54:46 STEP 1500 H=1500:00 +CPU= 27.968
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001500
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001500
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001500
  16:55:03 STEP 1501 H=1501:00 +CPU= 15.790
  16:55:19 STEP 1502 H=1502:00 +CPU= 15.794
  16:55:45 STEP 1503 H=1503:00 +CPU= 25.365
  16:56:01 STEP 1504 H=1504:00 +CPU= 15.709
  16:56:16 STEP 1505 H=1505:00 +CPU= 15.167
  16:56:42 STEP 1506 H=1506:00 +CPU= 25.565
  16:56:59 STEP 1507 H=1507:00 +CPU= 15.841
  16:57:17 STEP 1508 H=1508:00 +CPU= 16.422
  16:57:45 STEP 1509 H=1509:00 +CPU= 25.664
  16:58:03 STEP 1510 H=1510:00 +CPU= 16.991
  16:58:20 STEP 1511 H=1511:00 +CPU= 16.559
  16:58:53 STEP 1512 H=1512:00 +CPU= 29.907
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001512
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001512
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001500
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_62.zip
Uploading the intermediate file: upload_file_62.zip
  16:59:10 STEP 1513 H=1513:00 +CPU= 16.289
  16:59:28 STEP 1514 H=1514:00 +CPU= 16.796
  16:59:55 STEP 1515 H=1515:00 +CPU= 25.257
  17:00:11 STEP 1516 H=1516:00 +CPU= 16.434
  17:00:28 STEP 1517 H=1517:00 +CPU= 15.766
  17:00:54 STEP 1518 H=1518:00 +CPU= 25.611
  17:01:11 STEP 1519 H=1519:00 +CPU= 15.925
  17:01:27 STEP 1520 H=1520:00 +CPU= 16.030
  17:01:52 STEP 1521 H=1521:00 +CPU= 24.599
  17:02:09 STEP 1522 H=1522:00 +CPU= 16.138
  17:02:25 STEP 1523 H=1523:00 +CPU= 15.435
  17:02:53 STEP 1524 H=1524:00 +CPU= 27.627
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001524
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001524
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001524
  17:03:11 STEP 1525 H=1525:00 +CPU= 16.145
  17:03:29 STEP 1526 H=1526:00 +CPU= 16.633
  17:03:55 STEP 1527 H=1527:00 +CPU= 25.177
  17:04:13 STEP 1528 H=1528:00 +CPU= 16.771
  17:04:30 STEP 1529 H=1529:00 +CPU= 16.501
  17:04:58 STEP 1530 H=1530:00 +CPU= 25.959
  17:05:16 STEP 1531 H=1531:00 +CPU= 16.487
  17:05:33 STEP 1532 H=1532:00 +CPU= 16.686
  17:06:00 STEP 1533 H=1533:00 +CPU= 25.400
  17:06:16 STEP 1534 H=1534:00 +CPU= 16.158
  17:06:33 STEP 1535 H=1535:00 +CPU= 15.923
  17:07:07 STEP 1536 H=1536:00 +CPU= 29.615
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001536
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001536
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001524
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001524
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001524
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_63.zip
Uploading the intermediate file: upload_file_63.zip
  17:07:23 STEP 1537 H=1537:00 +CPU= 15.137
Uploading trickle at timestep: 5529600
  17:07:39 STEP 1538 H=1538:00 +CPU= 16.319
  17:08:04 STEP 1539 H=1539:00 +CPU= 24.188
  17:08:20 STEP 1540 H=1540:00 +CPU= 15.833
  17:08:37 STEP 1541 H=1541:00 +CPU= 15.921
  17:09:06 STEP 1542 H=1542:00 +CPU= 26.305
  17:09:23 STEP 1543 H=1543:00 +CPU= 16.162
  17:09:40 STEP 1544 H=1544:00 +CPU= 16.851
  17:10:07 STEP 1545 H=1545:00 +CPU= 25.171
  17:10:25 STEP 1546 H=1546:00 +CPU= 17.000
  17:10:42 STEP 1547 H=1547:00 +CPU= 16.611
  17:11:12 STEP 1548 H=1548:00 +CPU= 28.199
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001548
  17:11:30 STEP 1549 H=1549:00 +CPU= 16.122
  17:11:47 STEP 1550 H=1550:00 +CPU= 16.432
  17:12:12 STEP 1551 H=1551:00 +CPU= 25.015
  17:12:29 STEP 1552 H=1552:00 +CPU= 15.740
  17:12:45 STEP 1553 H=1553:00 +CPU= 15.522
  17:13:12 STEP 1554 H=1554:00 +CPU= 25.261
  17:13:31 STEP 1555 H=1555:00 +CPU= 16.168
  17:13:47 STEP 1556 H=1556:00 +CPU= 15.869
  17:14:12 STEP 1557 H=1557:00 +CPU= 24.804
  17:14:29 STEP 1558 H=1558:00 +CPU= 15.761
  17:14:45 STEP 1559 H=1559:00 +CPU= 16.156
  17:15:16 STEP 1560 H=1560:00 +CPU= 29.001
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001560
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001560
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001548
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_64.zip
Uploading the intermediate file: upload_file_64.zip
  17:15:31 STEP 1561 H=1561:00 +CPU= 15.432
  17:15:49 STEP 1562 H=1562:00 +CPU= 16.413
  17:16:17 STEP 1563 H=1563:00 +CPU= 25.702
  17:16:34 STEP 1564 H=1564:00 +CPU= 16.800
  17:16:51 STEP 1565 H=1565:00 +CPU= 16.062
  17:17:18 STEP 1566 H=1566:00 +CPU= 25.591
  17:17:35 STEP 1567 H=1567:00 +CPU= 15.806
  17:17:53 STEP 1568 H=1568:00 +CPU= 17.146
  17:18:20 STEP 1569 H=1569:00 +CPU= 25.927
  17:18:37 STEP 1570 H=1570:00 +CPU= 16.273
  17:18:53 STEP 1571 H=1571:00 +CPU= 15.810
  17:19:22 STEP 1572 H=1572:00 +CPU= 27.746
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001572
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001572
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001572
  17:19:38 STEP 1573 H=1573:00 +CPU= 15.640
  17:19:54 STEP 1574 H=1574:00 +CPU= 16.121
  17:20:20 STEP 1575 H=1575:00 +CPU= 24.617
  17:20:37 STEP 1576 H=1576:00 +CPU= 16.198
  17:20:53 STEP 1577 H=1577:00 +CPU= 15.491
  17:21:19 STEP 1578 H=1578:00 +CPU= 25.394
  17:21:36 STEP 1579 H=1579:00 +CPU= 15.800
  17:21:52 STEP 1580 H=1580:00 +CPU= 16.084
  17:22:18 STEP 1581 H=1581:00 +CPU= 24.911
  17:22:34 STEP 1582 H=1582:00 +CPU= 16.060
  17:22:50 STEP 1583 H=1583:00 +CPU= 15.647
  17:23:26 STEP 1584 H=1584:00 +CPU= 29.795
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001584
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001584
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001572
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_65.zip
Uploading the intermediate file: upload_file_65.zip
  17:23:42 STEP 1585 H=1585:00 +CPU= 15.403
  17:23:58 STEP 1586 H=1586:00 +CPU= 16.072
  17:24:24 STEP 1587 H=1587:00 +CPU= 24.774
  17:24:40 STEP 1588 H=1588:00 +CPU= 15.936
  17:24:57 STEP 1589 H=1589:00 +CPU= 15.587
  17:25:23 STEP 1590 H=1590:00 +CPU= 25.232
  17:25:39 STEP 1591 H=1591:00 +CPU= 15.324
  17:25:55 STEP 1592 H=1592:00 +CPU= 16.239
  17:26:21 STEP 1593 H=1593:00 +CPU= 24.834
  17:26:38 STEP 1594 H=1594:00 +CPU= 16.133
  17:26:54 STEP 1595 H=1595:00 +CPU= 15.431
  17:27:23 STEP 1596 H=1596:00 +CPU= 27.962
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001596
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001596
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001596
  17:27:39 STEP 1597 H=1597:00 +CPU= 15.747
  17:27:56 STEP 1598 H=1598:00 +CPU= 16.606
  17:28:22 STEP 1599 H=1599:00 +CPU= 24.969
  17:28:39 STEP 1600 H=1600:00 +CPU= 16.177
  17:28:55 STEP 1601 H=1601:00 +CPU= 15.802
  17:29:21 STEP 1602 H=1602:00 +CPU= 25.315
  17:29:38 STEP 1603 H=1603:00 +CPU= 15.170
  17:29:55 STEP 1604 H=1604:00 +CPU= 16.033
  17:30:21 STEP 1605 H=1605:00 +CPU= 24.942
  17:30:38 STEP 1606 H=1606:00 +CPU= 16.325
  17:30:55 STEP 1607 H=1607:00 +CPU= 16.597
  17:31:26 STEP 1608 H=1608:00 +CPU= 29.105
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001608
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001608
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001596
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_66.zip
Uploading the intermediate file: upload_file_66.zip
  17:31:42 STEP 1609 H=1609:00 +CPU= 15.695
  17:31:59 STEP 1610 H=1610:00 +CPU= 16.060
  17:32:24 STEP 1611 H=1611:00 +CPU= 24.930
  17:32:40 STEP 1612 H=1612:00 +CPU= 15.798
  17:32:57 STEP 1613 H=1613:00 +CPU= 15.905
  17:33:23 STEP 1614 H=1614:00 +CPU= 25.067
  17:33:39 STEP 1615 H=1615:00 +CPU= 15.774
  17:33:56 STEP 1616 H=1616:00 +CPU= 16.088
  17:34:22 STEP 1617 H=1617:00 +CPU= 25.125
  17:34:38 STEP 1618 H=1618:00 +CPU= 16.084
  17:34:55 STEP 1619 H=1619:00 +CPU= 15.820
  17:35:22 STEP 1620 H=1620:00 +CPU= 26.901
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001620
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001620
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001620
  17:35:38 STEP 1621 H=1621:00 +CPU= 15.328
  17:35:55 STEP 1622 H=1622:00 +CPU= 16.470
  17:36:23 STEP 1623 H=1623:00 +CPU= 25.241
  17:36:40 STEP 1624 H=1624:00 +CPU= 16.480
  17:36:57 STEP 1625 H=1625:00 +CPU= 16.299
  17:37:25 STEP 1626 H=1626:00 +CPU= 25.589
  17:37:44 STEP 1627 H=1627:00 +CPU= 16.221
  17:38:02 STEP 1628 H=1628:00 +CPU= 16.816
  17:38:29 STEP 1629 H=1629:00 +CPU= 25.532
  17:38:46 STEP 1630 H=1630:00 +CPU= 16.312
  17:39:03 STEP 1631 H=1631:00 +CPU= 16.293
  17:39:37 STEP 1632 H=1632:00 +CPU= 29.246
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001632
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001632
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001620
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001620
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001620
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_67.zip
Uploading the intermediate file: upload_file_67.zip
  17:39:52 STEP 1633 H=1633:00 +CPU= 15.443
  17:40:10 STEP 1634 H=1634:00 +CPU= 16.706
  17:40:35 STEP 1635 H=1635:00 +CPU= 24.520
  17:40:52 STEP 1636 H=1636:00 +CPU= 16.102
  17:41:08 STEP 1637 H=1637:00 +CPU= 15.796
  17:41:35 STEP 1638 H=1638:00 +CPU= 25.707
  17:41:51 STEP 1639 H=1639:00 +CPU= 15.560
  17:42:08 STEP 1640 H=1640:00 +CPU= 16.251
  17:42:33 STEP 1641 H=1641:00 +CPU= 24.836
  17:42:50 STEP 1642 H=1642:00 +CPU= 15.974
  17:43:06 STEP 1643 H=1643:00 +CPU= 15.904
  17:43:35 STEP 1644 H=1644:00 +CPU= 27.521
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001644
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001644
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001644
  17:43:51 STEP 1645 H=1645:00 +CPU= 16.331
  17:44:08 STEP 1646 H=1646:00 +CPU= 16.168
  17:44:33 STEP 1647 H=1647:00 +CPU= 24.893
  17:44:50 STEP 1648 H=1648:00 +CPU= 16.230
  17:45:06 STEP 1649 H=1649:00 +CPU= 15.515
  17:45:33 STEP 1650 H=1650:00 +CPU= 25.587
  17:45:51 STEP 1651 H=1651:00 +CPU= 16.039
  17:46:08 STEP 1652 H=1652:00 +CPU= 16.611
  17:46:34 STEP 1653 H=1653:00 +CPU= 25.262
  17:46:52 STEP 1654 H=1654:00 +CPU= 16.420
  17:47:09 STEP 1655 H=1655:00 +CPU= 16.608
  17:47:45 STEP 1656 H=1656:00 +CPU= 30.470
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001656
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001656
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001644
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_68.zip
Uploading the intermediate file: upload_file_68.zip
  17:48:01 STEP 1657 H=1657:00 +CPU= 16.094
  17:48:19 STEP 1658 H=1658:00 +CPU= 16.865
  17:48:46 STEP 1659 H=1659:00 +CPU= 25.341
  17:49:03 STEP 1660 H=1660:00 +CPU= 16.173
  17:49:19 STEP 1661 H=1661:00 +CPU= 16.074
  17:49:45 STEP 1662 H=1662:00 +CPU= 25.381
  17:50:02 STEP 1663 H=1663:00 +CPU= 15.851
  17:50:19 STEP 1664 H=1664:00 +CPU= 16.319
  17:50:45 STEP 1665 H=1665:00 +CPU= 24.970
  17:51:01 STEP 1666 H=1666:00 +CPU= 16.279
  17:51:18 STEP 1667 H=1667:00 +CPU= 15.651
  17:51:46 STEP 1668 H=1668:00 +CPU= 27.556
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001668
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001668
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001668
  17:52:02 STEP 1669 H=1669:00 +CPU= 15.785
  17:52:19 STEP 1670 H=1670:00 +CPU= 16.022
  17:52:45 STEP 1671 H=1671:00 +CPU= 25.194
  17:53:02 STEP 1672 H=1672:00 +CPU= 16.427
  17:53:19 STEP 1673 H=1673:00 +CPU= 16.109
  17:53:45 STEP 1674 H=1674:00 +CPU= 25.175
  17:54:01 STEP 1675 H=1675:00 +CPU= 15.738
  17:54:18 STEP 1676 H=1676:00 +CPU= 16.092
  17:54:43 STEP 1677 H=1677:00 +CPU= 24.874
  17:55:01 STEP 1678 H=1678:00 +CPU= 16.346
  17:55:17 STEP 1679 H=1679:00 +CPU= 16.424
  17:55:48 STEP 1680 H=1680:00 +CPU= 29.299
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001680
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001680
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001668
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_69.zip
Uploading the intermediate file: upload_file_69.zip
  17:56:05 STEP 1681 H=1681:00 +CPU= 15.634
  17:56:21 STEP 1682 H=1682:00 +CPU= 16.202
  17:56:47 STEP 1683 H=1683:00 +CPU= 24.462
  17:57:04 STEP 1684 H=1684:00 +CPU= 16.502
  17:57:20 STEP 1685 H=1685:00 +CPU= 15.707
  17:57:46 STEP 1686 H=1686:00 +CPU= 25.125
  17:58:02 STEP 1687 H=1687:00 +CPU= 15.845
  17:58:19 STEP 1688 H=1688:00 +CPU= 16.418
  17:58:45 STEP 1689 H=1689:00 +CPU= 24.651
  17:59:02 STEP 1690 H=1690:00 +CPU= 16.216
  17:59:18 STEP 1691 H=1691:00 +CPU= 15.808
  17:59:47 STEP 1692 H=1692:00 +CPU= 27.661
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001692
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001692
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001692
  18:00:03 STEP 1693 H=1693:00 +CPU= 15.844
  18:00:20 STEP 1694 H=1694:00 +CPU= 16.128
  18:00:45 STEP 1695 H=1695:00 +CPU= 24.686
  18:01:02 STEP 1696 H=1696:00 +CPU= 16.272
  18:01:19 STEP 1697 H=1697:00 +CPU= 15.877
  18:01:45 STEP 1698 H=1698:00 +CPU= 24.989
  18:02:01 STEP 1699 H=1699:00 +CPU= 15.873
  18:02:18 STEP 1700 H=1700:00 +CPU= 16.203
  18:02:44 STEP 1701 H=1701:00 +CPU= 25.168
  18:03:00 STEP 1702 H=1702:00 +CPU= 15.975
  18:03:17 STEP 1703 H=1703:00 +CPU= 16.293
  18:03:51 STEP 1704 H=1704:00 +CPU= 29.276
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001704
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001704
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001692
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001692
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001692
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_70.zip
Uploading the intermediate file: upload_file_70.zip
  18:04:07 STEP 1705 H=1705:00 +CPU= 15.773
  18:04:24 STEP 1706 H=1706:00 +CPU= 16.208
  18:04:49 STEP 1707 H=1707:00 +CPU= 24.323
  18:05:05 STEP 1708 H=1708:00 +CPU= 16.047
  18:05:21 STEP 1709 H=1709:00 +CPU= 15.601
  18:05:47 STEP 1710 H=1710:00 +CPU= 25.243
  18:06:04 STEP 1711 H=1711:00 +CPU= 15.921
  18:06:20 STEP 1712 H=1712:00 +CPU= 16.353
  18:06:45 STEP 1713 H=1713:00 +CPU= 24.364
  18:07:02 STEP 1714 H=1714:00 +CPU= 16.026
  18:07:18 STEP 1715 H=1715:00 +CPU= 15.905
  18:07:46 STEP 1716 H=1716:00 +CPU= 27.718
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001716
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001716
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001716
  18:08:03 STEP 1717 H=1717:00 +CPU= 15.548
  18:08:20 STEP 1718 H=1718:00 +CPU= 16.670
  18:08:47 STEP 1719 H=1719:00 +CPU= 25.207
  18:09:04 STEP 1720 H=1720:00 +CPU= 16.972
  18:09:21 STEP 1721 H=1721:00 +CPU= 16.018
  18:09:49 STEP 1722 H=1722:00 +CPU= 26.218
  18:10:05 STEP 1723 H=1723:00 +CPU= 15.438
  18:10:23 STEP 1724 H=1724:00 +CPU= 17.121
  18:10:49 STEP 1725 H=1725:00 +CPU= 25.097
  18:11:07 STEP 1726 H=1726:00 +CPU= 16.398
  18:11:23 STEP 1727 H=1727:00 +CPU= 16.262
  18:11:59 STEP 1728 H=1728:00 +CPU= 29.109
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001728
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001728
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001716
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001716
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001716
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_71.zip
Uploading the intermediate file: upload_file_71.zip
  18:12:15 STEP 1729 H=1729:00 +CPU= 15.916
  18:12:32 STEP 1730 H=1730:00 +CPU= 16.403
  18:12:58 STEP 1731 H=1731:00 +CPU= 25.184
  18:13:15 STEP 1732 H=1732:00 +CPU= 16.631
  18:13:31 STEP 1733 H=1733:00 +CPU= 15.211
  18:13:57 STEP 1734 H=1734:00 +CPU= 25.127
  18:14:13 STEP 1735 H=1735:00 +CPU= 15.770
  18:14:30 STEP 1736 H=1736:00 +CPU= 15.899
  18:14:56 STEP 1737 H=1737:00 +CPU= 25.172
  18:15:12 STEP 1738 H=1738:00 +CPU= 15.961
  18:15:28 STEP 1739 H=1739:00 +CPU= 15.610
  18:15:56 STEP 1740 H=1740:00 +CPU= 27.163
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001740
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001740
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001740
  18:16:12 STEP 1741 H=1741:00 +CPU= 15.570
  18:16:29 STEP 1742 H=1742:00 +CPU= 16.174
  18:16:55 STEP 1743 H=1743:00 +CPU= 24.699
  18:17:12 STEP 1744 H=1744:00 +CPU= 16.220
  18:17:28 STEP 1745 H=1745:00 +CPU= 15.700
  18:17:54 STEP 1746 H=1746:00 +CPU= 25.110
  18:18:10 STEP 1747 H=1747:00 +CPU= 15.836
  18:18:27 STEP 1748 H=1748:00 +CPU= 16.241
  18:18:53 STEP 1749 H=1749:00 +CPU= 24.785
  18:19:10 STEP 1750 H=1750:00 +CPU= 16.222
  18:19:27 STEP 1751 H=1751:00 +CPU= 16.142
  18:20:03 STEP 1752 H=1752:00 +CPU= 29.322
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001752
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001752
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001740
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_72.zip
Uploading the intermediate file: upload_file_72.zip
  18:20:19 STEP 1753 H=1753:00 +CPU= 15.814
  18:20:36 STEP 1754 H=1754:00 +CPU= 16.156
  18:21:02 STEP 1755 H=1755:00 +CPU= 24.952
  18:21:19 STEP 1756 H=1756:00 +CPU= 16.220
  18:21:35 STEP 1757 H=1757:00 +CPU= 15.860
  18:22:01 STEP 1758 H=1758:00 +CPU= 25.514
  18:22:18 STEP 1759 H=1759:00 +CPU= 15.879
  18:22:35 STEP 1760 H=1760:00 +CPU= 16.305
  18:23:00 STEP 1761 H=1761:00 +CPU= 24.906
  18:23:17 STEP 1762 H=1762:00 +CPU= 16.359
  18:23:34 STEP 1763 H=1763:00 +CPU= 15.845
  18:24:03 STEP 1764 H=1764:00 +CPU= 28.137
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001764
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001764
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001764
  18:24:19 STEP 1765 H=1765:00 +CPU= 15.565
  18:24:35 STEP 1766 H=1766:00 +CPU= 15.895
  18:25:00 STEP 1767 H=1767:00 +CPU= 24.091
  18:25:16 STEP 1768 H=1768:00 +CPU= 15.645
  18:25:32 STEP 1769 H=1769:00 +CPU= 15.601
  18:25:59 STEP 1770 H=1770:00 +CPU= 24.749
  18:26:16 STEP 1771 H=1771:00 +CPU= 16.241
  18:26:34 STEP 1772 H=1772:00 +CPU= 17.189
  18:27:00 STEP 1773 H=1773:00 +CPU= 25.262
  18:27:19 STEP 1774 H=1774:00 +CPU= 16.867
  18:27:37 STEP 1775 H=1775:00 +CPU= 16.684
  18:28:10 STEP 1776 H=1776:00 +CPU= 29.988
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001776
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001776
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001764
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_73.zip
Uploading the intermediate file: upload_file_73.zip
  18:28:27 STEP 1777 H=1777:00 +CPU= 16.221
Uploading trickle at timestep: 6393600
  18:28:45 STEP 1778 H=1778:00 +CPU= 16.667
  18:29:11 STEP 1779 H=1779:00 +CPU= 24.527
  18:29:27 STEP 1780 H=1780:00 +CPU= 16.086
  18:29:44 STEP 1781 H=1781:00 +CPU= 15.495
  18:30:09 STEP 1782 H=1782:00 +CPU= 25.093
  18:30:26 STEP 1783 H=1783:00 +CPU= 15.872
  18:30:42 STEP 1784 H=1784:00 +CPU= 16.094
  18:31:08 STEP 1785 H=1785:00 +CPU= 24.769
  18:31:24 STEP 1786 H=1786:00 +CPU= 16.131
  18:31:41 STEP 1787 H=1787:00 +CPU= 16.540
  18:32:12 STEP 1788 H=1788:00 +CPU= 28.168
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001788
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001788
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001788
  18:32:29 STEP 1789 H=1789:00 +CPU= 16.179
  18:32:46 STEP 1790 H=1790:00 +CPU= 16.494
  18:33:14 STEP 1791 H=1791:00 +CPU= 25.334
  18:33:31 STEP 1792 H=1792:00 +CPU= 16.299
  18:33:49 STEP 1793 H=1793:00 +CPU= 15.991
  18:34:19 STEP 1794 H=1794:00 +CPU= 25.542
  18:34:35 STEP 1795 H=1795:00 +CPU= 15.606
  18:34:52 STEP 1796 H=1796:00 +CPU= 16.351
  18:35:18 STEP 1797 H=1797:00 +CPU= 24.883
  18:35:35 STEP 1798 H=1798:00 +CPU= 16.405
  18:35:52 STEP 1799 H=1799:00 +CPU= 16.262
  18:36:27 STEP 1800 H=1800:00 +CPU= 29.151
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001800
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001800
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001788
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_74.zip
Uploading the intermediate file: upload_file_74.zip
  18:36:43 STEP 1801 H=1801:00 +CPU= 15.850
  18:37:00 STEP 1802 H=1802:00 +CPU= 16.029
  18:37:26 STEP 1803 H=1803:00 +CPU= 24.987
  18:37:42 STEP 1804 H=1804:00 +CPU= 16.298
  18:37:59 STEP 1805 H=1805:00 +CPU= 15.605
  18:38:25 STEP 1806 H=1806:00 +CPU= 25.302
  18:38:41 STEP 1807 H=1807:00 +CPU= 15.756
  18:38:58 STEP 1808 H=1808:00 +CPU= 16.114
  18:39:23 STEP 1809 H=1809:00 +CPU= 24.703
  18:39:40 STEP 1810 H=1810:00 +CPU= 16.185
  18:39:56 STEP 1811 H=1811:00 +CPU= 15.796
  18:40:25 STEP 1812 H=1812:00 +CPU= 27.423
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001812
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001812
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001812
  18:40:41 STEP 1813 H=1813:00 +CPU= 15.867
  18:40:58 STEP 1814 H=1814:00 +CPU= 16.481
  18:41:24 STEP 1815 H=1815:00 +CPU= 25.025
  18:41:41 STEP 1816 H=1816:00 +CPU= 15.854
  18:41:57 STEP 1817 H=1817:00 +CPU= 15.435
  18:42:22 STEP 1818 H=1818:00 +CPU= 24.913
  18:42:39 STEP 1819 H=1819:00 +CPU= 15.681
  18:42:55 STEP 1820 H=1820:00 +CPU= 16.008
  18:43:21 STEP 1821 H=1821:00 +CPU= 24.677
  18:43:37 STEP 1822 H=1822:00 +CPU= 16.167
  18:43:54 STEP 1823 H=1823:00 +CPU= 15.834
  18:44:24 STEP 1824 H=1824:00 +CPU= 28.708
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001824
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001824
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001812
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001812
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001812
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_75.zip
Uploading the intermediate file: upload_file_75.zip
  18:44:40 STEP 1825 H=1825:00 +CPU= 15.418
  18:44:57 STEP 1826 H=1826:00 +CPU= 16.423
  18:45:22 STEP 1827 H=1827:00 +CPU= 24.610
  18:45:40 STEP 1828 H=1828:00 +CPU= 16.544
  18:45:56 STEP 1829 H=1829:00 +CPU= 15.837
  18:46:22 STEP 1830 H=1830:00 +CPU= 25.360
  18:46:39 STEP 1831 H=1831:00 +CPU= 16.120
  18:46:56 STEP 1832 H=1832:00 +CPU= 16.437
  18:47:22 STEP 1833 H=1833:00 +CPU= 25.161
  18:47:38 STEP 1834 H=1834:00 +CPU= 16.076
  18:47:54 STEP 1835 H=1835:00 +CPU= 15.377
  18:48:22 STEP 1836 H=1836:00 +CPU= 27.329
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001836
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001836
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001836
  18:48:38 STEP 1837 H=1837:00 +CPU= 15.522
  18:48:55 STEP 1838 H=1838:00 +CPU= 16.125
  18:49:21 STEP 1839 H=1839:00 +CPU= 24.952
  18:49:37 STEP 1840 H=1840:00 +CPU= 15.996
  18:49:53 STEP 1841 H=1841:00 +CPU= 15.408
  18:50:19 STEP 1842 H=1842:00 +CPU= 25.283
  18:50:36 STEP 1843 H=1843:00 +CPU= 15.711
  18:50:52 STEP 1844 H=1844:00 +CPU= 15.880
  18:51:18 STEP 1845 H=1845:00 +CPU= 24.580
  18:51:34 STEP 1846 H=1846:00 +CPU= 16.066
  18:51:50 STEP 1847 H=1847:00 +CPU= 16.003
  18:52:20 STEP 1848 H=1848:00 +CPU= 28.658
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001848
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001848
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001836
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_76.zip
Uploading the intermediate file: upload_file_76.zip
  18:52:36 STEP 1849 H=1849:00 +CPU= 15.649
  18:52:54 STEP 1850 H=1850:00 +CPU= 16.384
  18:53:21 STEP 1851 H=1851:00 +CPU= 25.043
  18:53:39 STEP 1852 H=1852:00 +CPU= 16.681
  18:53:56 STEP 1853 H=1853:00 +CPU= 16.546
  18:54:23 STEP 1854 H=1854:00 +CPU= 26.195
  18:54:41 STEP 1855 H=1855:00 +CPU= 16.317
  18:54:58 STEP 1856 H=1856:00 +CPU= 16.663
  18:55:25 STEP 1857 H=1857:00 +CPU= 25.310
  18:55:42 STEP 1858 H=1858:00 +CPU= 16.249
  18:55:58 STEP 1859 H=1859:00 +CPU= 15.676
  18:56:27 STEP 1860 H=1860:00 +CPU= 27.332
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001860
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001860
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001860
  18:56:43 STEP 1861 H=1861:00 +CPU= 15.988
  18:57:00 STEP 1862 H=1862:00 +CPU= 16.117
  18:57:25 STEP 1863 H=1863:00 +CPU= 24.678
  18:57:42 STEP 1864 H=1864:00 +CPU= 16.349
  18:58:02 STEP 1865 H=1865:00 +CPU= 15.924
  18:58:28 STEP 1866 H=1866:00 +CPU= 25.535
  18:58:44 STEP 1867 H=1867:00 +CPU= 15.288
  18:59:00 STEP 1868 H=1868:00 +CPU= 16.081
  18:59:25 STEP 1869 H=1869:00 +CPU= 24.385
  18:59:42 STEP 1870 H=1870:00 +CPU= 16.112
  18:59:59 STEP 1871 H=1871:00 +CPU= 16.141
  19:00:32 STEP 1872 H=1872:00 +CPU= 29.018
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001872
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001872
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMGGhq0f+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMSHhq0f+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029/ICMUAhq0f+001860
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916_77.zip
Uploading the intermediate file: upload_file_77.zip
  19:00:48 STEP 1873 H=1873:00 +CPU= 16.148
  19:01:05 STEP 1874 H=1874:00 +CPU= 16.410
  19:01:33 STEP 1875 H=1875:00 +CPU= 25.795
  19:01:51 STEP 1876 H=1876:00 +CPU= 17.034
  19:02:08 STEP 1877 H=1877:00 +CPU= 16.653
  19:02:34 STEP 1878 H=1878:00 +CPU= 25.338
  19:02:49 STEP 1879 H=1879:00 +CPU= 14.790
  19:03:05 STEP 1880 H=1880:00 +CPU= 15.190
  19:03:28 STEP 1881 H=1881:00 +CPU= 23.358
  19:03:45 STEP 1882 H=1882:00 +CPU= 16.148
  19:04:02 STEP 1883 H=1883:00 +CPU= 15.882
  19:04:36 STEP 1884 H=1884:00 +CPU= 28.432
Moving to projects directory: /var/lib/boinc/slots/20/ICMGGhq0f+001884
Moving to projects directory: /var/lib/boinc/slots/20/ICMSHhq0f+001884
Moving to projects directory: /var/lib/boinc/slots/20/ICMUAhq0f+001884
  19:04:54 STEP 1885 H=1885:00 +CPU= 16.481
  19:05:11 STEP 1886 H=1886:00 +CPU= 16.583
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1991050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0385
(argv4) batchid: 960
(argv5) wuid: 12177029
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/20
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12177029
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/20/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/20/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_88adc8ac52ba4be28cfdb0f177651837 to: /var/lib/boinc/slots/20/oifs_43r3_ps_0385_1991050100_123_960_12177029.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/20/oifs_43r3_ps_0385_1991050100_123_960_12177029.zip
ic_ancil_file: ic_ancil_12177029
ifsdata_file: ifsdata_12177029
climate_data_file: clim_data_12177029
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_c918c0794a94b11f9c16614a1d5bed9f to: /var/lib/boinc/slots/20/ic_ancil_12177029.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/20/ic_ancil_12177029.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6926b28ec4ce30b90bc0a44ff779538c to: /var/lib/boinc/slots/20/ifsdata/ifsdata_12177029.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/20/ifsdata/ifsdata_12177029.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/20/159l_2/clim_data_12177029.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/20/159l_2/clim_data_12177029.zip
Checking for progress XML file: /var/lib/boinc/slots/20/progress_file_12177029.xml
Opened progress file ok : /var/lib/boinc/slots/20/progress_file_12177029.xml
-- Model is restarting --
Adjusting last_iter, 1887, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/20/progress_file_12177029.xml
last_cpu_time: 36571
upload_file_number: 78
last_iter: 1871
last_upload: 6739200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0385_1991050100_123_960_12177029_0_r639058916
The child process has been launched with process id: 28859
Executing the command: /var/lib/boinc/slots/20/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190635:1672855595:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  19:07:14 STEP 1872 H=1872:00 +CPU= 28.332
  19:07:34 STEP 1873 H=1873:00 +CPU= 16.710
  19:07:55 STEP 1874 H=1874:00 +CPU= 17.012
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190805:1672855685:89.832] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4150MB (heap), 4277MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190805:1672855685:89.877] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190805:1672855685:89.877] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190805:1672855685:89.877] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0xf1b400000588], nsigs = 1
[EC_DRHOOK:etam-komp:1:1:28859:28859] [20230104:190805:1672855685:89.877] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:08:00 1499708 22261802 oifs_43r3_ps_0385_1991050100_123_960_12177029_0 6,393,600 34,444 0.0054
09 May 2023 16:07:58 1499708 22261802 oifs_43r3_ps_0385_1991050100_123_960_12177029_0 5,529,600 29,802 0.0054
09 May 2023 16:07:53 1499708 22261802 oifs_43r3_ps_0385_1991050100_123_960_12177029_0 4,665,600 25,140 0.0054
09 May 2023 16:07:50 1499708 22261802 oifs_43r3_ps_0385_1991050100_123_960_12177029_0 3,801,600 20,481 0.0054
09 May 2023 16:07:49 1499708 22261802 oifs_43r3_ps_0385_1991050100_123_960_12177029_0 2,937,600 15,841 0.0054
09 May 2023 16:07:45 1499708 22261802 oifs_43r3_ps_0385_1991050100_123_960_12177029_0 2,073,600 11,201 0.0054
09 May 2023 16:07:42 1499708 22261802 oifs_43r3_ps_0385_1991050100_123_960_12177029_0 864,000 4,627 0.0054


©2024 climateprediction.net