<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3214827:3214827] [20221222:112705:1671676025:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
11:27:52 STEP 504 H= 504:00 +CPU= 41.752
11:28:16 STEP 505 H= 505:00 +CPU= 23.106
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:45:03 STEP 506 H= 506:00 +CPU= 24.850
11:45:41 STEP 507 H= 507:00 +CPU= 37.180
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:47:27 STEP 508 H= 508:00 +CPU= 24.321
11:47:48 STEP 509 H= 509:00 +CPU= 21.182
11:48:21 STEP 510 H= 510:00 +CPU= 32.772
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:49:55 STEP 511 H= 511:00 +CPU= 21.051
11:50:16 STEP 512 H= 512:00 +CPU= 21.021
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:52:00 STEP 513 H= 513:00 +CPU= 32.895
11:52:21 STEP 514 H= 514:00 +CPU= 21.421
11:52:42 STEP 515 H= 515:00 +CPU= 20.326
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:54:51 STEP 516 H= 516:00 +CPU= 36.755
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000516
11:55:11 STEP 517 H= 517:00 +CPU= 20.504
11:55:33 STEP 518 H= 518:00 +CPU= 21.009
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:56:55 STEP 519 H= 519:00 +CPU= 32.657
11:57:16 STEP 520 H= 520:00 +CPU= 20.681
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:58:59 STEP 521 H= 521:00 +CPU= 21.118
11:59:32 STEP 522 H= 522:00 +CPU= 32.799
11:59:52 STEP 523 H= 523:00 +CPU= 20.427
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:01:24 STEP 524 H= 524:00 +CPU= 21.167
12:01:57 STEP 525 H= 525:00 +CPU= 32.229
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:03:28 STEP 526 H= 526:00 +CPU= 21.248
12:03:51 STEP 527 H= 527:00 +CPU= 22.380
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:05:41 STEP 528 H= 528:00 +CPU= 38.861
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275_21.zip
Uploading the intermediate file: upload_file_21.zip
12:06:01 STEP 529 H= 529:00 +CPU= 19.825
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:07:19 STEP 530 H= 530:00 +CPU= 20.862
12:07:51 STEP 531 H= 531:00 +CPU= 32.198
12:08:12 STEP 532 H= 532:00 +CPU= 20.861
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:09:34 STEP 533 H= 533:00 +CPU= 21.026
12:10:07 STEP 534 H= 534:00 +CPU= 32.857
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:11:50 STEP 535 H= 535:00 +CPU= 20.954
12:12:11 STEP 536 H= 536:00 +CPU= 21.475
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:13:45 STEP 537 H= 537:00 +CPU= 32.601
12:14:06 STEP 538 H= 538:00 +CPU= 21.265
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:15:39 STEP 539 H= 539:00 +CPU= 20.138
12:16:16 STEP 540 H= 540:00 +CPU= 36.726
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000540
12:16:36 STEP 541 H= 541:00 +CPU= 20.200
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:18:17 STEP 542 H= 542:00 +CPU= 21.015
12:18:50 STEP 543 H= 543:00 +CPU= 33.105
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:20:52 STEP 544 H= 544:00 +CPU= 20.917
12:21:14 STEP 545 H= 545:00 +CPU= 21.136
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:23:29 STEP 546 H= 546:00 +CPU= 32.837
12:23:51 STEP 547 H= 547:00 +CPU= 21.490
12:24:12 STEP 548 H= 548:00 +CPU= 20.748
12:24:44 STEP 549 H= 549:00 +CPU= 31.914
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:26:36 STEP 550 H= 550:00 +CPU= 21.573
12:26:58 STEP 551 H= 551:00 +CPU= 21.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:33:04 STEP 552 H= 552:00 +CPU= 43.751
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275_22.zip
Uploading the intermediate file: upload_file_22.zip
12:33:27 STEP 553 H= 553:00 +CPU= 22.880
12:33:52 STEP 554 H= 554:00 +CPU= 24.024
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
12:52:30 STEP 555 H= 555:00 +CPU= 34.801
12:52:56 STEP 556 H= 556:00 +CPU= 24.946
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:00:22 STEP 557 H= 557:00 +CPU= 24.625
13:01:00 STEP 558 H= 558:00 +CPU= 37.183
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:10:07 STEP 559 H= 559:00 +CPU= 24.114
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:20:34 STEP 560 H= 560:00 +CPU= 24.507
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:25:11 STEP 561 H= 561:00 +CPU= 34.413
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:34:17 STEP 562 H= 562:00 +CPU= 24.566
13:34:39 STEP 563 H= 563:00 +CPU= 21.357
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0048
(argv4) batchid: 958
(argv5) wuid: 12174692
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f1446101fc20e8a5a7f2314b7b0a37dc to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0048_1989050100_123_958_12174692.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0048_1989050100_123_958_12174692.zip
ic_ancil_file: ic_ancil_12174692
ifsdata_file: ifsdata_12174692
climate_data_file: clim_data_12174692
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_942945b3c8aa464aa307e17333375999 to: /var/lib/boinc-client/slots/1/ic_ancil_12174692.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12174692.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0fdaf7e63fe61014060c27bfa5f34c3a to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12174692.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12174692.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12174692.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12174692.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12174692.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12174692.xml
-- Model is restarting --
Adjusting last_iter, 564, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12174692.xml
last_cpu_time: 16486
upload_file_number: 23
last_iter: 551
last_upload: 1987200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275
The child process has been launched with process id: 3237372
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3237372:3237372] [20221222:133638:1671683798:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
13:37:19 STEP 552 H= 552:00 +CPU= 36.205
13:37:39 STEP 553 H= 553:00 +CPU= 19.990
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:39:21 STEP 554 H= 554:00 +CPU= 21.139
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:41:07 STEP 555 H= 555:00 +CPU= 33.208
13:41:28 STEP 556 H= 556:00 +CPU= 21.244
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:43:10 STEP 557 H= 557:00 +CPU= 20.587
13:43:43 STEP 558 H= 558:00 +CPU= 32.981
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:45:16 STEP 559 H= 559:00 +CPU= 20.937
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:46:47 STEP 560 H= 560:00 +CPU= 20.815
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:48:30 STEP 561 H= 561:00 +CPU= 33.243
13:48:52 STEP 562 H= 562:00 +CPU= 21.476
13:49:12 STEP 563 H= 563:00 +CPU= 20.268
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:52:12 STEP 564 H= 564:00 +CPU= 35.792
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000564
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:53:25 STEP 565 H= 565:00 +CPU= 21.874
13:53:45 STEP 566 H= 566:00 +CPU= 20.647
13:54:18 STEP 567 H= 567:00 +CPU= 32.155
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:56:00 STEP 568 H= 568:00 +CPU= 21.016
13:56:20 STEP 569 H= 569:00 +CPU= 20.124
13:56:53 STEP 570 H= 570:00 +CPU= 33.085
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:58:56 STEP 571 H= 571:00 +CPU= 21.287
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:00:27 STEP 572 H= 572:00 +CPU= 21.150
14:01:01 STEP 573 H= 573:00 +CPU= 33.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:03:05 STEP 574 H= 574:00 +CPU= 21.993
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:04:28 STEP 575 H= 575:00 +CPU= 20.882
14:05:07 STEP 576 H= 576:00 +CPU= 38.918
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275_23.zip
Uploading the intermediate file: upload_file_23.zip
14:05:28 STEP 577 H= 577:00 +CPU= 20.127
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:06:59 STEP 578 H= 578:00 +CPU= 20.327
14:07:32 STEP 579 H= 579:00 +CPU= 32.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:09:03 STEP 580 H= 580:00 +CPU= 20.966
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:10:26 STEP 581 H= 581:00 +CPU= 21.068
14:10:59 STEP 582 H= 582:00 +CPU= 32.965
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:12:40 STEP 583 H= 583:00 +CPU= 20.312
14:13:02 STEP 584 H= 584:00 +CPU= 21.984
14:13:35 STEP 585 H= 585:00 +CPU= 32.251
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:15:27 STEP 586 H= 586:00 +CPU= 21.388
14:15:48 STEP 587 H= 587:00 +CPU= 20.617
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:17:55 STEP 588 H= 588:00 +CPU= 35.470
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000588
14:18:15 STEP 589 H= 589:00 +CPU= 20.392
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:19:58 STEP 590 H= 590:00 +CPU= 21.335
14:20:30 STEP 591 H= 591:00 +CPU= 32.434
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:22:22 STEP 592 H= 592:00 +CPU= 20.713
14:22:45 STEP 593 H= 593:00 +CPU= 21.974
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:25:10 STEP 594 H= 594:00 +CPU= 33.102
14:25:30 STEP 595 H= 595:00 +CPU= 20.429
14:25:53 STEP 596 H= 596:00 +CPU= 22.412
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:27:48 STEP 597 H= 597:00 +CPU= 33.398
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:29:51 STEP 598 H= 598:00 +CPU= 20.811
14:30:13 STEP 599 H= 599:00 +CPU= 21.899
14:30:52 STEP 600 H= 600:00 +CPU= 38.419
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275_24.zip
Uploading the intermediate file: upload_file_24.zip
14:33:04 STEP 601 H= 601:00 +CPU= 21.094
14:33:25 STEP 602 H= 602:00 +CPU= 21.049
14:33:58 STEP 603 H= 603:00 +CPU= 32.610
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:35:41 STEP 604 H= 604:00 +CPU= 21.417
14:36:01 STEP 605 H= 605:00 +CPU= 20.220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:37:46 STEP 606 H= 606:00 +CPU= 33.476
14:38:08 STEP 607 H= 607:00 +CPU= 21.406
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:39:41 STEP 608 H= 608:00 +CPU= 21.282
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:42:05 STEP 609 H= 609:00 +CPU= 33.115
14:42:27 STEP 610 H= 610:00 +CPU= 21.436
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:44:50 STEP 611 H= 611:00 +CPU= 21.079
14:45:27 STEP 612 H= 612:00 +CPU= 36.202
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000612
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:48:19 STEP 613 H= 613:00 +CPU= 21.891
14:48:41 STEP 614 H= 614:00 +CPU= 21.776
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:50:35 STEP 615 H= 615:00 +CPU= 32.967
14:50:58 STEP 616 H= 616:00 +CPU= 22.011
14:51:19 STEP 617 H= 617:00 +CPU= 20.745
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:53:12 STEP 618 H= 618:00 +CPU= 33.588
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:54:54 STEP 619 H= 619:00 +CPU= 20.216
14:55:16 STEP 620 H= 620:00 +CPU= 21.979
14:55:49 STEP 621 H= 621:00 +CPU= 32.641
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:58:02 STEP 622 H= 622:00 +CPU= 21.761
14:58:25 STEP 623 H= 623:00 +CPU= 22.213
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:00:25 STEP 624 H= 624:00 +CPU= 39.026
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275_25.zip
Uploading the intermediate file: upload_file_25.zip
15:00:45 STEP 625 H= 625:00 +CPU= 20.207
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:02:28 STEP 626 H= 626:00 +CPU= 20.761
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:03:53 STEP 627 H= 627:00 +CPU= 33.219
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:05:13 STEP 628 H= 628:00 +CPU= 20.579
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:06:17 STEP 629 H= 629:00 +CPU= 21.250
15:06:50 STEP 630 H= 630:00 +CPU= 32.849
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:08:33 STEP 631 H= 631:00 +CPU= 20.648
15:08:54 STEP 632 H= 632:00 +CPU= 21.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:10:47 STEP 633 H= 633:00 +CPU= 33.065
15:11:08 STEP 634 H= 634:00 +CPU= 21.081
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:12:51 STEP 635 H= 635:00 +CPU= 20.872
15:13:27 STEP 636 H= 636:00 +CPU= 35.741
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000636
15:13:47 STEP 637 H= 637:00 +CPU= 20.037
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:15:40 STEP 638 H= 638:00 +CPU= 21.331
15:16:13 STEP 639 H= 639:00 +CPU= 32.462
15:16:34 STEP 640 H= 640:00 +CPU= 21.038
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:18:16 STEP 641 H= 641:00 +CPU= 21.063
15:18:49 STEP 642 H= 642:00 +CPU= 33.012
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:20:31 STEP 643 H= 643:00 +CPU= 20.794
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:22:24 STEP 644 H= 644:00 +CPU= 21.360
15:22:57 STEP 645 H= 645:00 +CPU= 32.918
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:24:50 STEP 646 H= 646:00 +CPU= 21.694
15:25:11 STEP 647 H= 647:00 +CPU= 20.826
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:27:21 STEP 648 H= 648:00 +CPU= 38.325
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275_26.zip
Uploading the intermediate file: upload_file_26.zip
15:27:41 STEP 649 H= 649:00 +CPU= 20.218
15:28:02 STEP 650 H= 650:00 +CPU= 20.687
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:30:06 STEP 651 H= 651:00 +CPU= 32.817
15:30:27 STEP 652 H= 652:00 +CPU= 20.526
15:30:47 STEP 653 H= 653:00 +CPU= 20.052
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:33:01 STEP 654 H= 654:00 +CPU= 32.982
15:33:21 STEP 655 H= 655:00 +CPU= 19.956
15:33:42 STEP 656 H= 656:00 +CPU= 20.222
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:35:46 STEP 657 H= 657:00 +CPU= 32.449
15:36:07 STEP 658 H= 658:00 +CPU= 20.778
15:36:27 STEP 659 H= 659:00 +CPU= 20.161
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0048
(argv4) batchid: 958
(argv5) wuid: 12174692
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f1446101fc20e8a5a7f2314b7b0a37dc to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0048_1989050100_123_958_12174692.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0048_1989050100_123_958_12174692.zip
ic_ancil_file: ic_ancil_12174692
ifsdata_file: ifsdata_12174692
climate_data_file: clim_data_12174692
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_942945b3c8aa464aa307e17333375999 to: /var/lib/boinc-client/slots/1/ic_ancil_12174692.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12174692.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0fdaf7e63fe61014060c27bfa5f34c3a to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12174692.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12174692.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12174692.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12174692.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12174692.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12174692.xml
-- Model is restarting --
Adjusting last_iter, 660, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12174692.xml
last_cpu_time: 19223
upload_file_number: 27
last_iter: 647
last_upload: 2332800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275
The child process has been launched with process id: 3257344
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3257344:3257344] [20221222:154020:1671691220:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
15:41:01 STEP 648 H= 648:00 +CPU= 37.574
15:41:21 STEP 649 H= 649:00 +CPU= 20.220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:43:33 STEP 650 H= 650:00 +CPU= 21.237
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:45:08 STEP 651 H= 651:00 +CPU= 33.329
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:46:30 STEP 652 H= 652:00 +CPU= 20.871
15:46:52 STEP 653 H= 653:00 +CPU= 21.306
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:48:46 STEP 654 H= 654:00 +CPU= 33.472
15:49:07 STEP 655 H= 655:00 +CPU= 20.346
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:50:49 STEP 656 H= 656:00 +CPU= 20.915
15:51:22 STEP 657 H= 657:00 +CPU= 33.234
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
15:53:04 STEP 658 H= 658:00 +CPU= 20.862
15:53:26 STEP 659 H= 659:00 +CPU= 21.912
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:01:15 STEP 660 H= 660:00 +CPU= 36.714
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000660
16:01:37 STEP 661 H= 661:00 +CPU= 21.823
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:12:32 STEP 662 H= 662:00 +CPU= 22.334
16:13:07 STEP 663 H= 663:00 +CPU= 34.819
16:13:30 STEP 664 H= 664:00 +CPU= 22.404
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:17:44 STEP 665 H= 665:00 +CPU= 23.213
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:23:41 STEP 666 H= 666:00 +CPU= 35.345
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:28:45 STEP 667 H= 667:00 +CPU= 22.976
16:29:09 STEP 668 H= 668:00 +CPU= 23.994
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:36:47 STEP 669 H= 669:00 +CPU= 34.366
16:37:09 STEP 670 H= 670:00 +CPU= 22.544
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:43:04 STEP 671 H= 671:00 +CPU= 22.134
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:47:26 STEP 672 H= 672:00 +CPU= 40.627
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275_27.zip
Uploading the intermediate file: upload_file_27.zip
16:47:48 STEP 673 H= 673:00 +CPU= 21.602
16:48:10 STEP 674 H= 674:00 +CPU= 21.758
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:54:58 STEP 675 H= 675:00 +CPU= 34.946
16:55:20 STEP 676 H= 676:00 +CPU= 22.423
16:55:42 STEP 677 H= 677:00 +CPU= 21.568
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:01:00 STEP 678 H= 678:00 +CPU= 36.363
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:06:13 STEP 679 H= 679:00 +CPU= 21.691
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:12:38 STEP 680 H= 680:00 +CPU= 22.224
17:13:13 STEP 681 H= 681:00 +CPU= 34.661
17:13:36 STEP 682 H= 682:00 +CPU= 22.556
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:17:10 STEP 683 H= 683:00 +CPU= 22.561
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:23:30 STEP 684 H= 684:00 +CPU= 38.285
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:29:25 STEP 685 H= 685:00 +CPU= 22.370
17:29:48 STEP 686 H= 686:00 +CPU= 22.888
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:33:27 STEP 687 H= 687:00 +CPU= 35.900
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:36:21 STEP 688 H= 688:00 +CPU= 22.526
17:36:42 STEP 689 H= 689:00 +CPU= 21.417
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1989050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0048
(argv4) batchid: 958
(argv5) wuid: 12174692
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f1446101fc20e8a5a7f2314b7b0a37dc to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0048_1989050100_123_958_12174692.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0048_1989050100_123_958_12174692.zip
ic_ancil_file: ic_ancil_12174692
ifsdata_file: ifsdata_12174692
climate_data_file: clim_data_12174692
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_942945b3c8aa464aa307e17333375999 to: /var/lib/boinc-client/slots/1/ic_ancil_12174692.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12174692.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0fdaf7e63fe61014060c27bfa5f34c3a to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12174692.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12174692.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12174692.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12174692.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12174692.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12174692.xml
-- Model is restarting --
Adjusting last_iter, 690, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12174692.xml
last_cpu_time: 20353
upload_file_number: 28
last_iter: 671
last_upload: 2419200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275
The child process has been launched with process id: 3276540
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:3276540:3276540] [20221222:174705:1671698825:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:51:25 STEP 672 H= 672:00 +CPU= 40.208
17:51:48 STEP 673 H= 673:00 +CPU= 22.182
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:56:03 STEP 674 H= 674:00 +CPU= 22.464
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:58:59 STEP 675 H= 675:00 +CPU= 34.620
17:59:22 STEP 676 H= 676:00 +CPU= 23.396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:02:47 STEP 677 H= 677:00 +CPU= 22.887
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:07:24 STEP 678 H= 678:00 +CPU= 34.616
18:07:48 STEP 679 H= 679:00 +CPU= 23.387
18:08:10 STEP 680 H= 680:00 +CPU= 22.181
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:11:48 STEP 681 H= 681:00 +CPU= 35.255
18:12:10 STEP 682 H= 682:00 +CPU= 21.479
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:18:03 STEP 683 H= 683:00 +CPU= 22.293
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:21:44 STEP 684 H= 684:00 +CPU= 38.491
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000684
18:22:07 STEP 685 H= 685:00 +CPU= 23.089
18:22:30 STEP 686 H= 686:00 +CPU= 22.382
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:25:46 STEP 687 H= 687:00 +CPU= 34.660
18:26:10 STEP 688 H= 688:00 +CPU= 23.028
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:30:15 STEP 689 H= 689:00 +CPU= 22.813
18:30:51 STEP 690 H= 690:00 +CPU= 36.026
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:37:06 STEP 691 H= 691:00 +CPU= 22.285
18:37:29 STEP 692 H= 692:00 +CPU= 22.818
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:45:17 STEP 693 H= 693:00 +CPU= 35.304
18:45:40 STEP 694 H= 694:00 +CPU= 22.707
18:46:03 STEP 695 H= 695:00 +CPU= 22.082
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:54:08 STEP 696 H= 696:00 +CPU= 40.709
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12174692/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0048_1989050100_123_958_12174692_0_r1864250275_28.zip
Uploading the intermediate file: upload_file_28.zip
18:54:30 STEP 697 H= 697:00 +CPU= 21.752
18:54:53 STEP 698 H= 698:00 +CPU= 22.564
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:59:39 STEP 699 H= 699:00 +CPU= 34.210
19:00:02 STEP 700 H= 700:00 +CPU= 22.594
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:04:05 STEP 701 H= 701:00 +CPU= 21.446
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:07:41 STEP 702 H= 702:00 +CPU= 34.098
19:08:03 STEP 703 H= 703:00 +CPU= 21.775
19:08:25 STEP 704 H= 704:00 +CPU= 22.055
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:12:52 STEP 705 H= 705:00 +CPU= 34.188
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
19:16:16 STEP 706 H= 706:00 +CPU= 22.687
19:16:39 STEP 707 H= 707:00 +CPU= 22.058
19:17:17 STEP 708 H= 708:00 +CPU= 38.108
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000708
19:20:31 STEP 709 H= 709:00 +CPU= 22.613
19:20:55 STEP 710 H= 710:00 +CPU= 23.603
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
