<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
509 H= 509:00 +CPU= 23.622
11:27:52 STEP 510 H= 510:00 +CPU= 38.095
11:28:16 STEP 511 H= 511:00 +CPU= 22.859
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:45:03 STEP 512 H= 512:00 +CPU= 25.230
11:45:41 STEP 513 H= 513:00 +CPU= 37.261
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:47:26 STEP 514 H= 514:00 +CPU= 24.114
11:47:48 STEP 515 H= 515:00 +CPU= 21.487
11:48:24 STEP 516 H= 516:00 +CPU= 35.814
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000516
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:49:57 STEP 517 H= 517:00 +CPU= 21.076
11:50:18 STEP 518 H= 518:00 +CPU= 21.191
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:52:01 STEP 519 H= 519:00 +CPU= 32.874
11:52:22 STEP 520 H= 520:00 +CPU= 20.789
11:52:42 STEP 521 H= 521:00 +CPU= 20.018
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:54:48 STEP 522 H= 522:00 +CPU= 33.728
11:55:09 STEP 523 H= 523:00 +CPU= 20.384
11:55:29 STEP 524 H= 524:00 +CPU= 20.618
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:56:53 STEP 525 H= 525:00 +CPU= 32.630
11:57:14 STEP 526 H= 526:00 +CPU= 20.650
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:58:56 STEP 527 H= 527:00 +CPU= 21.349
11:59:35 STEP 528 H= 528:00 +CPU= 38.269
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599_21.zip
Uploading the intermediate file: upload_file_21.zip
11:59:55 STEP 529 H= 529:00 +CPU= 20.137
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:01:28 STEP 530 H= 530:00 +CPU= 21.530
12:02:00 STEP 531 H= 531:00 +CPU= 31.836
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:03:32 STEP 532 H= 532:00 +CPU= 21.282
12:03:54 STEP 533 H= 533:00 +CPU= 20.995
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:05:37 STEP 534 H= 534:00 +CPU= 33.591
12:05:58 STEP 535 H= 535:00 +CPU= 20.714
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:07:20 STEP 536 H= 536:00 +CPU= 21.978
12:07:53 STEP 537 H= 537:00 +CPU= 32.061
12:08:13 STEP 538 H= 538:00 +CPU= 20.675
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:09:35 STEP 539 H= 539:00 +CPU= 21.227
12:10:10 STEP 540 H= 540:00 +CPU= 35.535
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000540
12:11:54 STEP 541 H= 541:00 +CPU= 21.437
12:12:15 STEP 542 H= 542:00 +CPU= 20.934
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:13:48 STEP 543 H= 543:00 +CPU= 32.853
12:14:09 STEP 544 H= 544:00 +CPU= 20.948
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:15:42 STEP 545 H= 545:00 +CPU= 19.956
12:16:16 STEP 546 H= 546:00 +CPU= 33.985
12:16:36 STEP 547 H= 547:00 +CPU= 20.353
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:18:19 STEP 548 H= 548:00 +CPU= 21.438
12:18:52 STEP 549 H= 549:00 +CPU= 33.121
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:20:55 STEP 550 H= 550:00 +CPU= 21.573
12:21:17 STEP 551 H= 551:00 +CPU= 21.940
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:23:37 STEP 552 H= 552:00 +CPU= 38.455
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599_22.zip
Uploading the intermediate file: upload_file_22.zip
12:23:58 STEP 553 H= 553:00 +CPU= 21.250
12:24:19 STEP 554 H= 554:00 +CPU= 20.387
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:26:23 STEP 555 H= 555:00 +CPU= 32.564
12:26:44 STEP 556 H= 556:00 +CPU= 21.624
12:27:06 STEP 557 H= 557:00 +CPU= 21.351
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:33:06 STEP 558 H= 558:00 +CPU= 38.082
12:33:30 STEP 559 H= 559:00 +CPU= 23.493
12:33:55 STEP 560 H= 560:00 +CPU= 23.569
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:52:33 STEP 561 H= 561:00 +CPU= 34.792
12:52:59 STEP 562 H= 562:00 +CPU= 25.551
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:00:26 STEP 563 H= 563:00 +CPU= 24.050
13:01:08 STEP 564 H= 564:00 +CPU= 41.079
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000564
13:10:16 STEP 565 H= 565:00 +CPU= 24.611
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:20:42 STEP 566 H= 566:00 +CPU= 23.473
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0082
(argv4) batchid: 958
(argv5) wuid: 12174726
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1512e6638f62a85410f51b32cb768e0f to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0082_1989050100_123_958_12174726.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0082_1989050100_123_958_12174726.zip
ic_ancil_file: ic_ancil_12174726
ifsdata_file: ifsdata_12174726
climate_data_file: clim_data_12174726
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_942945b3c8aa464aa307e17333375999 to: /var/lib/boinc-client/slots/0/ic_ancil_12174726.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12174726.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_dd21013ae231f94a2c0483afeef64e8d to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12174726.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12174726.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12174726.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12174726.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12174726.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12174726.xml
-- Model is restarting --
Adjusting last_iter, 567, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12174726.xml
last_cpu_time: 16405
upload_file_number: 23
last_iter: 551
last_upload: 1987200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599
The child process has been launched with process id: 3237040
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237040:3237040] [20221222:133417:1671683657:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
13:34:59 STEP 552 H= 552:00 +CPU= 37.603
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:36:52 STEP 553 H= 553:00 +CPU= 20.301
13:37:12 STEP 554 H= 554:00 +CPU= 20.500
13:37:45 STEP 555 H= 555:00 +CPU= 32.034
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:39:26 STEP 556 H= 556:00 +CPU= 20.815
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:40:57 STEP 557 H= 557:00 +CPU= 20.520
13:41:31 STEP 558 H= 558:00 +CPU= 33.457
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:43:12 STEP 559 H= 559:00 +CPU= 20.125
13:43:33 STEP 560 H= 560:00 +CPU= 20.964
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:45:17 STEP 561 H= 561:00 +CPU= 32.806
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:46:50 STEP 562 H= 562:00 +CPU= 21.534
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:48:22 STEP 563 H= 563:00 +CPU= 20.614
13:48:59 STEP 564 H= 564:00 +CPU= 37.011
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000564
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:50:40 STEP 565 H= 565:00 +CPU= 19.645
13:51:02 STEP 566 H= 566:00 +CPU= 22.143
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:53:29 STEP 567 H= 567:00 +CPU= 33.109
13:53:49 STEP 568 H= 568:00 +CPU= 20.578
13:54:10 STEP 569 H= 569:00 +CPU= 19.939
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:56:03 STEP 570 H= 570:00 +CPU= 33.135
13:56:23 STEP 571 H= 571:00 +CPU= 19.707
13:56:44 STEP 572 H= 572:00 +CPU= 20.930
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:58:58 STEP 573 H= 573:00 +CPU= 34.037
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:00:30 STEP 574 H= 574:00 +CPU= 21.059
14:00:52 STEP 575 H= 575:00 +CPU= 21.880
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:03:14 STEP 576 H= 576:00 +CPU= 39.393
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599_23.zip
Uploading the intermediate file: upload_file_23.zip
14:04:37 STEP 577 H= 577:00 +CPU= 21.095
14:04:59 STEP 578 H= 578:00 +CPU= 21.303
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:07:03 STEP 579 H= 579:00 +CPU= 33.234
14:07:25 STEP 580 H= 580:00 +CPU= 21.562
14:07:45 STEP 581 H= 581:00 +CPU= 20.031
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:10:32 STEP 582 H= 582:00 +CPU= 34.155
14:10:52 STEP 583 H= 583:00 +CPU= 19.995
14:11:13 STEP 584 H= 584:00 +CPU= 21.039
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:13:08 STEP 585 H= 585:00 +CPU= 33.112
14:13:29 STEP 586 H= 586:00 +CPU= 20.613
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:15:21 STEP 587 H= 587:00 +CPU= 21.108
14:15:57 STEP 588 H= 588:00 +CPU= 35.818
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000588
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:17:49 STEP 589 H= 589:00 +CPU= 20.547
14:18:10 STEP 590 H= 590:00 +CPU= 21.264
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:20:04 STEP 591 H= 591:00 +CPU= 32.722
14:20:25 STEP 592 H= 592:00 +CPU= 20.597
14:20:45 STEP 593 H= 593:00 +CPU= 19.945
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:22:50 STEP 594 H= 594:00 +CPU= 34.431
14:23:10 STEP 595 H= 595:00 +CPU= 19.823
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:25:23 STEP 596 H= 596:00 +CPU= 21.073
14:25:58 STEP 597 H= 597:00 +CPU= 34.169
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:27:41 STEP 598 H= 598:00 +CPU= 21.559
14:28:02 STEP 599 H= 599:00 +CPU= 20.882
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:30:22 STEP 600 H= 600:00 +CPU= 39.284
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599_24.zip
Uploading the intermediate file: upload_file_24.zip
14:30:43 STEP 601 H= 601:00 +CPU= 20.368
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:32:55 STEP 602 H= 602:00 +CPU= 21.175
14:33:29 STEP 603 H= 603:00 +CPU= 32.998
14:33:50 STEP 604 H= 604:00 +CPU= 20.977
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:35:31 STEP 605 H= 605:00 +CPU= 20.602
14:36:04 STEP 606 H= 606:00 +CPU= 33.079
14:36:24 STEP 607 H= 607:00 +CPU= 20.377
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:37:59 STEP 608 H= 608:00 +CPU= 22.454
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:39:45 STEP 609 H= 609:00 +CPU= 33.720
14:40:06 STEP 610 H= 610:00 +CPU= 20.994
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:42:19 STEP 611 H= 611:00 +CPU= 21.075
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:44:58 STEP 612 H= 612:00 +CPU= 36.884
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000612
14:45:19 STEP 613 H= 613:00 +CPU= 20.727
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:48:11 STEP 614 H= 614:00 +CPU= 21.426
14:48:45 STEP 615 H= 615:00 +CPU= 34.251
14:49:06 STEP 616 H= 616:00 +CPU= 20.919
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:50:49 STEP 617 H= 617:00 +CPU= 21.053
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:52:44 STEP 618 H= 618:00 +CPU= 33.605
14:53:05 STEP 619 H= 619:00 +CPU= 20.937
14:53:26 STEP 620 H= 620:00 +CPU= 21.277
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:55:20 STEP 621 H= 621:00 +CPU= 33.346
14:55:40 STEP 622 H= 622:00 +CPU= 20.288
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:57:54 STEP 623 H= 623:00 +CPU= 21.798
14:58:34 STEP 624 H= 624:00 +CPU= 39.388
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599_25.zip
Uploading the intermediate file: upload_file_25.zip
15:00:18 STEP 625 H= 625:00 +CPU= 21.052
15:00:39 STEP 626 H= 626:00 +CPU= 21.250
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:02:34 STEP 627 H= 627:00 +CPU= 32.620
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:03:48 STEP 628 H= 628:00 +CPU= 21.704
15:04:08 STEP 629 H= 629:00 +CPU= 19.742
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:06:24 STEP 630 H= 630:00 +CPU= 34.225
15:06:44 STEP 631 H= 631:00 +CPU= 20.071
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:08:27 STEP 632 H= 632:00 +CPU= 21.427
15:09:00 STEP 633 H= 633:00 +CPU= 32.476
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:10:42 STEP 634 H= 634:00 +CPU= 21.069
15:11:03 STEP 635 H= 635:00 +CPU= 21.219
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:13:01 STEP 636 H= 636:00 +CPU= 36.403
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000636
15:13:21 STEP 637 H= 637:00 +CPU= 20.336
15:13:42 STEP 638 H= 638:00 +CPU= 20.763
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:15:46 STEP 639 H= 639:00 +CPU= 32.655
15:16:07 STEP 640 H= 640:00 +CPU= 21.097
15:16:28 STEP 641 H= 641:00 +CPU= 20.458
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:18:23 STEP 642 H= 642:00 +CPU= 33.704
15:18:43 STEP 643 H= 643:00 +CPU= 20.618
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:20:25 STEP 644 H= 644:00 +CPU= 20.913
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:22:29 STEP 645 H= 645:00 +CPU= 33.539
15:22:51 STEP 646 H= 646:00 +CPU= 21.165
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:24:42 STEP 647 H= 647:00 +CPU= 20.990
15:25:20 STEP 648 H= 648:00 +CPU= 38.540
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599_26.zip
Uploading the intermediate file: upload_file_26.zip
15:25:40 STEP 649 H= 649:00 +CPU= 19.762
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:27:26 STEP 650 H= 650:00 +CPU= 21.803
15:27:58 STEP 651 H= 651:00 +CPU= 31.658
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:29:50 STEP 652 H= 652:00 +CPU= 20.823
15:30:11 STEP 653 H= 653:00 +CPU= 20.907
15:30:44 STEP 654 H= 654:00 +CPU= 32.712
15:31:05 STEP 655 H= 655:00 +CPU= 20.892
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:33:07 STEP 656 H= 656:00 +CPU= 21.156
15:33:40 STEP 657 H= 657:00 +CPU= 31.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:35:33 STEP 658 H= 658:00 +CPU= 21.047
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0082
(argv4) batchid: 958
(argv5) wuid: 12174726
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1512e6638f62a85410f51b32cb768e0f to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0082_1989050100_123_958_12174726.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0082_1989050100_123_958_12174726.zip
ic_ancil_file: ic_ancil_12174726
ifsdata_file: ifsdata_12174726
climate_data_file: clim_data_12174726
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_942945b3c8aa464aa307e17333375999 to: /var/lib/boinc-client/slots/0/ic_ancil_12174726.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12174726.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_dd21013ae231f94a2c0483afeef64e8d to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12174726.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12174726.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12174726.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12174726.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12174726.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12174726.xml
-- Model is restarting --
Adjusting last_iter, 658, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12174726.xml
last_cpu_time: 19122
upload_file_number: 27
last_iter: 647
last_upload: 2332800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599
The child process has been launched with process id: 3256984
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3256984:3256984] [20221222:153809:1671691089:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
15:38:49 STEP 648 H= 648:00 +CPU= 36.534
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:40:30 STEP 649 H= 649:00 +CPU= 20.605
15:40:52 STEP 650 H= 650:00 +CPU= 21.515
15:41:24 STEP 651 H= 651:00 +CPU= 32.192
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:43:37 STEP 652 H= 652:00 +CPU= 21.492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:45:00 STEP 653 H= 653:00 +CPU= 20.653
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:46:35 STEP 654 H= 654:00 +CPU= 33.937
15:46:56 STEP 655 H= 655:00 +CPU= 21.134
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:48:39 STEP 656 H= 656:00 +CPU= 21.359
15:49:11 STEP 657 H= 657:00 +CPU= 32.494
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:50:55 STEP 658 H= 658:00 +CPU= 21.957
15:51:15 STEP 659 H= 659:00 +CPU= 20.515
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:53:13 STEP 660 H= 660:00 +CPU= 36.205
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000660
15:53:34 STEP 661 H= 661:00 +CPU= 20.897
15:53:55 STEP 662 H= 662:00 +CPU= 20.971
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:01:41 STEP 663 H= 663:00 +CPU= 34.161
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:12:37 STEP 664 H= 664:00 +CPU= 23.090
16:12:59 STEP 665 H= 665:00 +CPU= 22.250
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:17:26 STEP 666 H= 666:00 +CPU= 35.791
16:17:50 STEP 667 H= 667:00 +CPU= 23.364
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:23:34 STEP 668 H= 668:00 +CPU= 21.923
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:28:51 STEP 669 H= 669:00 +CPU= 35.327
16:29:14 STEP 670 H= 670:00 +CPU= 22.741
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:36:38 STEP 671 H= 671:00 +CPU= 22.868
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:42:51 STEP 672 H= 672:00 +CPU= 40.310
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599_27.zip
Uploading the intermediate file: upload_file_27.zip
16:43:13 STEP 673 H= 673:00 +CPU= 21.961
16:43:35 STEP 674 H= 674:00 +CPU= 21.534
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:47:51 STEP 675 H= 675:00 +CPU= 34.078
16:48:13 STEP 676 H= 676:00 +CPU= 22.073
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:51:15 STEP 677 H= 677:00 +CPU= 21.034
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:55:22 STEP 678 H= 678:00 +CPU= 35.174
16:55:44 STEP 679 H= 679:00 +CPU= 21.364
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:00:49 STEP 680 H= 680:00 +CPU= 22.817
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:06:16 STEP 681 H= 681:00 +CPU= 34.991
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:12:39 STEP 682 H= 682:00 +CPU= 22.127
17:13:02 STEP 683 H= 683:00 +CPU= 22.338
17:13:40 STEP 684 H= 684:00 +CPU= 37.661
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000684
17:17:15 STEP 685 H= 685:00 +CPU= 22.488
17:17:38 STEP 686 H= 686:00 +CPU= 22.407
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:29:25 STEP 687 H= 687:00 +CPU= 35.017
17:29:48 STEP 688 H= 688:00 +CPU= 22.661
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:33:12 STEP 689 H= 689:00 +CPU= 22.619
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:36:19 STEP 690 H= 690:00 +CPU= 35.537
17:36:41 STEP 691 H= 691:00 +CPU= 21.745
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:40:47 STEP 692 H= 692:00 +CPU= 23.261
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0082
(argv4) batchid: 958
(argv5) wuid: 12174726
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1512e6638f62a85410f51b32cb768e0f to: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0082_1989050100_123_958_12174726.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_ps_0082_1989050100_123_958_12174726.zip
ic_ancil_file: ic_ancil_12174726
ifsdata_file: ifsdata_12174726
climate_data_file: clim_data_12174726
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_942945b3c8aa464aa307e17333375999 to: /var/lib/boinc-client/slots/0/ic_ancil_12174726.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12174726.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_dd21013ae231f94a2c0483afeef64e8d to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12174726.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12174726.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12174726.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12174726.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12174726.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12174726.xml
-- Model is restarting --
Adjusting last_iter, 693, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12174726.xml
last_cpu_time: 20311
upload_file_number: 28
last_iter: 671
last_upload: 2419200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599
The child process has been launched with process id: 3276539
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276539:3276539] [20221222:174705:1671698825:0.007] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:51:26 STEP 672 H= 672:00 +CPU= 41.062
17:51:49 STEP 673 H= 673:00 +CPU= 22.593
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:56:04 STEP 674 H= 674:00 +CPU= 23.004
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:59:01 STEP 675 H= 675:00 +CPU= 35.310
17:59:25 STEP 676 H= 676:00 +CPU= 23.382
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:02:49 STEP 677 H= 677:00 +CPU= 22.737
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:07:28 STEP 678 H= 678:00 +CPU= 36.064
18:07:51 STEP 679 H= 679:00 +CPU= 23.162
18:08:14 STEP 680 H= 680:00 +CPU= 22.508
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:11:52 STEP 681 H= 681:00 +CPU= 35.246
18:12:15 STEP 682 H= 682:00 +CPU= 22.460
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:18:09 STEP 683 H= 683:00 +CPU= 22.370
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:21:49 STEP 684 H= 684:00 +CPU= 38.656
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000684
18:22:13 STEP 685 H= 685:00 +CPU= 23.337
18:22:35 STEP 686 H= 686:00 +CPU= 22.253
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:25:53 STEP 687 H= 687:00 +CPU= 35.452
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:29:57 STEP 688 H= 688:00 +CPU= 22.453
18:30:21 STEP 689 H= 689:00 +CPU= 23.074
18:30:56 STEP 690 H= 690:00 +CPU= 35.382
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:37:10 STEP 691 H= 691:00 +CPU= 21.312
18:37:34 STEP 692 H= 692:00 +CPU= 23.161
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:45:21 STEP 693 H= 693:00 +CPU= 34.852
18:45:44 STEP 694 H= 694:00 +CPU= 22.971
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:48:58 STEP 695 H= 695:00 +CPU= 22.410
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:54:11 STEP 696 H= 696:00 +CPU= 40.663
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174726/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0082_1989050100_123_958_12174726_0_r1210900599_28.zip
Uploading the intermediate file: upload_file_28.zip
18:54:34 STEP 697 H= 697:00 +CPU= 22.191
18:54:57 STEP 698 H= 698:00 +CPU= 22.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:59:42 STEP 699 H= 699:00 +CPU= 34.637
19:00:05 STEP 700 H= 700:00 +CPU= 22.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:04:07 STEP 701 H= 701:00 +CPU= 21.394
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:07:43 STEP 702 H= 702:00 +CPU= 35.115
19:08:06 STEP 703 H= 703:00 +CPU= 21.909
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:12:19 STEP 704 H= 704:00 +CPU= 22.048
19:12:54 STEP 705 H= 705:00 +CPU= 34.537
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:16:19 STEP 706 H= 706:00 +CPU= 22.961
19:16:41 STEP 707 H= 707:00 +CPU= 22.061
19:17:19 STEP 708 H= 708:00 +CPU= 37.755
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAhq0f+000708
19:20:33 STEP 709 H= 709:00 +CPU= 22.395
19:20:57 STEP 710 H= 710:00 +CPU= 23.760
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
