climateprediction.net home page
Task 22285864

Task 22285864

Name oifs_43r3_ps_0026_2014050100_123_983_12199670_0
Workunit 12199670
Created 24 Dec 2022, 9:06:07 UTC
Sent 5 Jan 2023, 8:26:39 UTC
Report deadline 4 Feb 2023, 8:26:39 UTC
Received 8 Jan 2023, 8:13:08 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1537140
Run time 21 hours 25 min 23 sec
CPU time 9 hours 51 min 56 sec
Validate state Invalid
Credit 7,741.44
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,393.49 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,855.40 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
30 H=1530:00 +CPU= 21.392
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:50:36 STEP 1531 H=1531:00 +CPU= 13.062
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:50:53 STEP 1532 H=1532:00 +CPU= 13.579
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:51:20 STEP 1533 H=1533:00 +CPU= 20.605
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:51:36 STEP 1534 H=1534:00 +CPU= 13.721
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:51:53 STEP 1535 H=1535:00 +CPU= 13.465
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:52:23 STEP 1536 H=1536:00 +CPU= 24.161
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2014050100_123_983_12199670_0_r752357528_63.zip
Uploading the intermediate file: upload_file_63.zip
  22:52:39 STEP 1537 H=1537:00 +CPU= 13.351
Uploading trickle at timestep: 5529600
  22:52:53 STEP 1538 H=1538:00 +CPU= 14.221
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:53:20 STEP 1539 H=1539:00 +CPU= 20.567
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:53:35 STEP 1540 H=1540:00 +CPU= 13.496
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:53:52 STEP 1541 H=1541:00 +CPU= 13.107
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:54:19 STEP 1542 H=1542:00 +CPU= 20.927
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:54:34 STEP 1543 H=1543:00 +CPU= 13.009
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:54:52 STEP 1544 H=1544:00 +CPU= 13.870
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:55:19 STEP 1545 H=1545:00 +CPU= 20.561
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:55:34 STEP 1546 H=1546:00 +CPU= 13.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:55:52 STEP 1547 H=1547:00 +CPU= 13.333
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:56:21 STEP 1548 H=1548:00 +CPU= 22.833
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001548
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:56:36 STEP 1549 H=1549:00 +CPU= 13.327
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:56:53 STEP 1550 H=1550:00 +CPU= 13.494
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:57:20 STEP 1551 H=1551:00 +CPU= 20.719
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:57:36 STEP 1552 H=1552:00 +CPU= 13.532
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:57:52 STEP 1553 H=1553:00 +CPU= 13.930
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:58:19 STEP 1554 H=1554:00 +CPU= 20.855
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:58:34 STEP 1555 H=1555:00 +CPU= 13.308
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:58:50 STEP 1556 H=1556:00 +CPU= 14.080
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:59:14 STEP 1557 H=1557:00 +CPU= 20.410
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:59:32 STEP 1558 H=1558:00 +CPU= 13.644
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:59:50 STEP 1559 H=1559:00 +CPU= 13.921
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:00:20 STEP 1560 H=1560:00 +CPU= 24.397
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001548
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2014050100_123_983_12199670_0_r752357528_64.zip
Uploading the intermediate file: upload_file_64.zip
  23:00:36 STEP 1561 H=1561:00 +CPU= 13.146
  23:00:50 STEP 1562 H=1562:00 +CPU= 14.440
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:01:15 STEP 1563 H=1563:00 +CPU= 20.895
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:01:33 STEP 1564 H=1564:00 +CPU= 13.598
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:01:50 STEP 1565 H=1565:00 +CPU= 13.117
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:02:15 STEP 1566 H=1566:00 +CPU= 20.862
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:02:32 STEP 1567 H=1567:00 +CPU= 13.310
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:02:50 STEP 1568 H=1568:00 +CPU= 13.591
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:03:15 STEP 1569 H=1569:00 +CPU= 21.016
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:03:32 STEP 1570 H=1570:00 +CPU= 13.466
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:03:49 STEP 1571 H=1571:00 +CPU= 13.254
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:04:16 STEP 1572 H=1572:00 +CPU= 22.590
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001572
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:04:33 STEP 1573 H=1573:00 +CPU= 13.156
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:04:51 STEP 1574 H=1574:00 +CPU= 14.215
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:05:19 STEP 1575 H=1575:00 +CPU= 21.178
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:05:34 STEP 1576 H=1576:00 +CPU= 13.161
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:05:51 STEP 1577 H=1577:00 +CPU= 13.173
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:06:16 STEP 1578 H=1578:00 +CPU= 20.528
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:06:33 STEP 1579 H=1579:00 +CPU= 13.150
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:06:50 STEP 1580 H=1580:00 +CPU= 13.458
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:07:13 STEP 1581 H=1581:00 +CPU= 20.521
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:07:31 STEP 1582 H=1582:00 +CPU= 13.521
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:07:49 STEP 1583 H=1583:00 +CPU= 14.088
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:08:20 STEP 1584 H=1584:00 +CPU= 24.531
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001572
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2014050100_123_983_12199670_0_r752357528_65.zip
Uploading the intermediate file: upload_file_65.zip
  23:08:33 STEP 1585 H=1585:00 +CPU= 13.161
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:08:49 STEP 1586 H=1586:00 +CPU= 14.170
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:09:14 STEP 1587 H=1587:00 +CPU= 20.244
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:09:31 STEP 1588 H=1588:00 +CPU= 13.556
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:09:48 STEP 1589 H=1589:00 +CPU= 12.928
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:10:13 STEP 1590 H=1590:00 +CPU= 20.735
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:10:30 STEP 1591 H=1591:00 +CPU= 12.949
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:10:45 STEP 1592 H=1592:00 +CPU= 13.277
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:11:12 STEP 1593 H=1593:00 +CPU= 20.385
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:11:29 STEP 1594 H=1594:00 +CPU= 13.394
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:11:44 STEP 1595 H=1595:00 +CPU= 12.961
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:12:12 STEP 1596 H=1596:00 +CPU= 22.382
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001596
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:12:30 STEP 1597 H=1597:00 +CPU= 12.974
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:12:45 STEP 1598 H=1598:00 +CPU= 13.327
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:13:11 STEP 1599 H=1599:00 +CPU= 20.381
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:13:29 STEP 1600 H=1600:00 +CPU= 13.378
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:13:44 STEP 1601 H=1601:00 +CPU= 13.129
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:14:11 STEP 1602 H=1602:00 +CPU= 20.717
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:14:28 STEP 1603 H=1603:00 +CPU= 13.043
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:14:43 STEP 1604 H=1604:00 +CPU= 13.408
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:15:10 STEP 1605 H=1605:00 +CPU= 20.630
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:15:25 STEP 1606 H=1606:00 +CPU= 13.397
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:15:43 STEP 1607 H=1607:00 +CPU= 13.381
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:16:13 STEP 1608 H=1608:00 +CPU= 24.741
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001596
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2014050100_123_983_12199670_0_r752357528_66.zip
Uploading the intermediate file: upload_file_66.zip
  23:16:29 STEP 1609 H=1609:00 +CPU= 13.351
  23:16:43 STEP 1610 H=1610:00 +CPU= 14.163
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:17:10 STEP 1611 H=1611:00 +CPU= 20.559
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:17:25 STEP 1612 H=1612:00 +CPU= 13.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:17:42 STEP 1613 H=1613:00 +CPU= 13.148
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:18:09 STEP 1614 H=1614:00 +CPU= 20.641
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:18:24 STEP 1615 H=1615:00 +CPU= 12.684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:18:41 STEP 1616 H=1616:00 +CPU= 13.595
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:19:08 STEP 1617 H=1617:00 +CPU= 20.250
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:19:23 STEP 1618 H=1618:00 +CPU= 13.495
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:19:40 STEP 1619 H=1619:00 +CPU= 13.236
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:20:09 STEP 1620 H=1620:00 +CPU= 22.794
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001620
  23:20:24 STEP 1621 H=1621:00 +CPU= 12.912
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:20:42 STEP 1622 H=1622:00 +CPU= 13.491
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:21:08 STEP 1623 H=1623:00 +CPU= 20.103
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:21:23 STEP 1624 H=1624:00 +CPU= 13.315
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:21:40 STEP 1625 H=1625:00 +CPU= 13.142
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:22:05 STEP 1626 H=1626:00 +CPU= 20.450
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:22:22 STEP 1627 H=1627:00 +CPU= 13.131
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:22:39 STEP 1628 H=1628:00 +CPU= 13.423
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:23:04 STEP 1629 H=1629:00 +CPU= 20.294
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:23:21 STEP 1630 H=1630:00 +CPU= 13.409
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:23:38 STEP 1631 H=1631:00 +CPU= 13.244
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:24:09 STEP 1632 H=1632:00 +CPU= 24.141
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2014050100_123_983_12199670_0_r752357528_67.zip
Uploading the intermediate file: upload_file_67.zip
  23:24:22 STEP 1633 H=1633:00 +CPU= 13.375
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:24:38 STEP 1634 H=1634:00 +CPU= 13.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:25:02 STEP 1635 H=1635:00 +CPU= 20.358
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:25:20 STEP 1636 H=1636:00 +CPU= 13.536
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:25:35 STEP 1637 H=1637:00 +CPU= 12.865
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:26:00 STEP 1638 H=1638:00 +CPU= 21.317
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:26:15 STEP 1639 H=1639:00 +CPU= 12.915
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:26:33 STEP 1640 H=1640:00 +CPU= 13.335
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:26:59 STEP 1641 H=1641:00 +CPU= 20.205
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:27:14 STEP 1642 H=1642:00 +CPU= 13.386
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:27:31 STEP 1643 H=1643:00 +CPU= 12.989
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:28:00 STEP 1644 H=1644:00 +CPU= 22.653
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001644
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:28:15 STEP 1645 H=1645:00 +CPU= 13.075
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:28:33 STEP 1646 H=1646:00 +CPU= 13.620
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:29:00 STEP 1647 H=1647:00 +CPU= 21.687
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:29:19 STEP 1648 H=1648:00 +CPU= 14.369
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:29:34 STEP 1649 H=1649:00 +CPU= 13.504
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  23:30:04 STEP 1650 H=1650:00 +CPU= 23.597
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.179] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4749MB (heap), 4236MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.179] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.179] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.179] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.179] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.192] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 4749 MB (maxheap), 4236 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.193] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.213] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.213] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.213] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            CALL_SL_HEAP 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             CALL_SL 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              >OMP-CALL_SL     1        (1004) 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               LAPINEA 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233009:1673130609:20589.214] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                LARMES 
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233010:1673130610:20590.214] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:5382:5382] [20230107:233010:1673130610:20590.214] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000DE95E6  larmes_                   491  larmes.F90
oifs_43r3_model.e  0000000000DE50FD  lapinea_                  270  lapinea.F90
oifs_43r3_model.e  0000000000D7A4EC  call_sl_                  300  call_sl.F90
oifs_43r3_model.e  0000000000D784FD  call_sl_heap_              94  call_sl_heap.F90
oifs_43r3_model.e  0000000000BF9958  gp_model_                 533  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2014050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0026
(argv4) batchid: 983
(argv5) wuid: 12199670
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_d9c2e6acc6bfe9386c1d6b2f2a542290 to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0026_2014050100_123_983_12199670.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0026_2014050100_123_983_12199670.zip
ic_ancil_file: ic_ancil_12199670
ifsdata_file: ifsdata_12199670
climate_data_file: clim_data_12199670
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_91f8f7eedbda150fb202938ad4cde3b2 to: /var/lib/boinc-client/slots/2/ic_ancil_12199670.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12199670.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_17cf4604c5d93a1aa80e6c72e9dde805 to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12199670.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12199670.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12199670.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12199670.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12199670.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12199670.xml
-- Model is restarting --
Adjusting last_iter, 1650, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12199670.xml
last_cpu_time: 34793
upload_file_number: 68
last_iter: 1631
last_upload: 5875200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0026_2014050100_123_983_12199670_0_r752357528
The child process has been launched with process id: 1734
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:olaf-LIFEBOOK-U939:1:1:1734:1734] [20230108:085422:1673164462:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:54:53 STEP 1632 H=1632:00 +CPU= 22.991
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:55:08 STEP 1633 H=1633:00 +CPU= 12.639
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:55:25 STEP 1634 H=1634:00 +CPU= 13.017
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:55:48 STEP 1635 H=1635:00 +CPU= 19.203
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:56:05 STEP 1636 H=1636:00 +CPU= 12.766
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:56:22 STEP 1637 H=1637:00 +CPU= 12.512
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:56:46 STEP 1638 H=1638:00 +CPU= 19.818
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:57:02 STEP 1639 H=1639:00 +CPU= 12.466
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:57:17 STEP 1640 H=1640:00 +CPU= 12.983
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:57:43 STEP 1641 H=1641:00 +CPU= 19.555
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:57:58 STEP 1642 H=1642:00 +CPU= 12.973
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:58:14 STEP 1643 H=1643:00 +CPU= 12.696
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:58:38 STEP 1644 H=1644:00 +CPU= 22.073
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001644
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:58:55 STEP 1645 H=1645:00 +CPU= 12.789
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:59:13 STEP 1646 H=1646:00 +CPU= 13.356
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:59:37 STEP 1647 H=1647:00 +CPU= 20.476
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  08:59:55 STEP 1648 H=1648:00 +CPU= 13.599
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:00:12 STEP 1649 H=1649:00 +CPU= 13.151
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:00:37 STEP 1650 H=1650:00 +CPU= 20.699
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:00:54 STEP 1651 H=1651:00 +CPU= 13.224
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:01:11 STEP 1652 H=1652:00 +CPU= 13.403
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:01:36 STEP 1653 H=1653:00 +CPU= 20.399
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:01:53 STEP 1654 H=1654:00 +CPU= 13.504
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:02:09 STEP 1655 H=1655:00 +CPU= 13.404
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:02:39 STEP 1656 H=1656:00 +CPU= 24.301
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMGGhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMSHhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12199670/ICMUAhq0f+001644
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0026_2014050100_123_983_12199670_0_r752357528_68.zip
Uploading the intermediate file: upload_file_68.zip
  09:02:56 STEP 1657 H=1657:00 +CPU= 13.175
  09:03:11 STEP 1658 H=1658:00 +CPU= 14.274
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:03:36 STEP 1659 H=1659:00 +CPU= 20.924
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:03:53 STEP 1660 H=1660:00 +CPU= 13.664
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:04:08 STEP 1661 H=1661:00 +CPU= 13.120
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:04:35 STEP 1662 H=1662:00 +CPU= 20.910
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:04:53 STEP 1663 H=1663:00 +CPU= 13.354
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:05:11 STEP 1664 H=1664:00 +CPU= 14.525
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:05:36 STEP 1665 H=1665:00 +CPU= 20.445
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:05:53 STEP 1666 H=1666:00 +CPU= 13.675
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:06:08 STEP 1667 H=1667:00 +CPU= 13.012
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:06:37 STEP 1668 H=1668:00 +CPU= 22.630
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001668
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:06:54 STEP 1669 H=1669:00 +CPU= 13.131
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:07:12 STEP 1670 H=1670:00 +CPU= 13.498
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:07:36 STEP 1671 H=1671:00 +CPU= 20.204
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:07:54 STEP 1672 H=1672:00 +CPU= 13.605
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:08:09 STEP 1673 H=1673:00 +CPU= 13.087
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:08:36 STEP 1674 H=1674:00 +CPU= 20.979
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:08:53 STEP 1675 H=1675:00 +CPU= 13.021
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  09:09:08 STEP 1676 H=1676:00 +CPU= 13.369
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Abort request received from the BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:12:07 1537140 22285864 oifs_43r3_ps_0026_2014050100_123_983_12199670_0 5,529,600 32,970 0.0060
09 May 2023 16:12:04 1537140 22285864 oifs_43r3_ps_0026_2014050100_123_983_12199670_0 4,665,600 29,145 0.0062
09 May 2023 16:12:04 1537140 22285864 oifs_43r3_ps_0026_2014050100_123_983_12199670_0 4,665,600 29,145 0.0062
09 May 2023 16:12:00 1537140 22285864 oifs_43r3_ps_0026_2014050100_123_983_12199670_0 3,801,600 25,339 0.0067
09 May 2023 16:11:57 1537140 22285864 oifs_43r3_ps_0026_2014050100_123_983_12199670_0 2,937,600 21,582 0.0073


©2024 climateprediction.net