climateprediction.net home page
Task 22286986

Task 22286986

Name oifs_43r3_ps_0084_2015050100_123_984_12200728_0
Workunit 12200728
Created 24 Dec 2022, 11:51:41 UTC
Sent 4 Jan 2023, 10:38:49 UTC
Report deadline 3 Feb 2023, 10:38:49 UTC
Received 11 Jan 2023, 12:57:37 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1454366
Run time 21 hours 48 min 31 sec
CPU time 20 hours 24 min 46 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 5.04 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,417.47 MB
Peak swap size 4,978.08 MB
Peak disk usage 6,997.59 MB
Stderr
<core_client_version>7.16.16</core_client_version>
<![CDATA[
<stderr_txt>
s
Resuming the child process
  17:44:54 STEP 2738 H=2738:00 +CPU= 20.191
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:45:29 STEP 2739 H=2739:00 +CPU= 31.033
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:45:52 STEP 2740 H=2740:00 +CPU= 20.325
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:46:13 STEP 2741 H=2741:00 +CPU= 19.657
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:46:49 STEP 2742 H=2742:00 +CPU= 31.686
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:47:11 STEP 2743 H=2743:00 +CPU= 19.673
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:47:33 STEP 2744 H=2744:00 +CPU= 20.307
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:48:09 STEP 2745 H=2745:00 +CPU= 31.105
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:48:31 STEP 2746 H=2746:00 +CPU= 20.210
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:48:53 STEP 2747 H=2747:00 +CPU= 19.776
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:49:31 STEP 2748 H=2748:00 +CPU= 34.215
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002748
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:49:53 STEP 2749 H=2749:00 +CPU= 19.659
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:50:15 STEP 2750 H=2750:00 +CPU= 20.265
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:50:51 STEP 2751 H=2751:00 +CPU= 31.071
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:51:13 STEP 2752 H=2752:00 +CPU= 20.281
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:51:35 STEP 2753 H=2753:00 +CPU= 19.738
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:52:11 STEP 2754 H=2754:00 +CPU= 31.680
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:52:32 STEP 2755 H=2755:00 +CPU= 19.631
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:52:55 STEP 2756 H=2756:00 +CPU= 20.307
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:53:30 STEP 2757 H=2757:00 +CPU= 31.050
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:53:52 STEP 2758 H=2758:00 +CPU= 20.149
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:54:15 STEP 2759 H=2759:00 +CPU= 20.309
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:54:55 STEP 2760 H=2760:00 +CPU= 36.186
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0084_2015050100_123_984_12200728_0_r1256276306_114.zip
Uploading the intermediate file: upload_file_114.zip
  17:55:15 STEP 2761 H=2761:00 +CPU= 19.599
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:55:37 STEP 2762 H=2762:00 +CPU= 20.274
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:56:12 STEP 2763 H=2763:00 +CPU= 30.991
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:56:35 STEP 2764 H=2764:00 +CPU= 20.293
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:56:56 STEP 2765 H=2765:00 +CPU= 19.744
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:57:32 STEP 2766 H=2766:00 +CPU= 31.622
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:57:54 STEP 2767 H=2767:00 +CPU= 19.748
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:58:16 STEP 2768 H=2768:00 +CPU= 20.336
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:58:52 STEP 2769 H=2769:00 +CPU= 30.999
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:59:14 STEP 2770 H=2770:00 +CPU= 20.361
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:59:36 STEP 2771 H=2771:00 +CPU= 19.773
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:00:14 STEP 2772 H=2772:00 +CPU= 34.073
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002772
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:00:36 STEP 2773 H=2773:00 +CPU= 19.793
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:00:58 STEP 2774 H=2774:00 +CPU= 20.333
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:01:34 STEP 2775 H=2775:00 +CPU= 30.959
  18:01:54 STEP 2776 H=2776:00 +CPU= 20.214
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:02:16 STEP 2777 H=2777:00 +CPU= 19.668
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:02:49 STEP 2778 H=2778:00 +CPU= 31.632
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:03:11 STEP 2779 H=2779:00 +CPU= 19.674
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:03:33 STEP 2780 H=2780:00 +CPU= 20.204
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:04:09 STEP 2781 H=2781:00 +CPU= 30.990
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:04:31 STEP 2782 H=2782:00 +CPU= 20.232
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:04:53 STEP 2783 H=2783:00 +CPU= 20.236
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:05:35 STEP 2784 H=2784:00 +CPU= 36.100
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0084_2015050100_123_984_12200728_0_r1256276306_115.zip
Uploading the intermediate file: upload_file_115.zip
  18:05:54 STEP 2785 H=2785:00 +CPU= 19.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:06:17 STEP 2786 H=2786:00 +CPU= 20.220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:06:52 STEP 2787 H=2787:00 +CPU= 30.955
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:07:14 STEP 2788 H=2788:00 +CPU= 20.308
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:07:36 STEP 2789 H=2789:00 +CPU= 19.702
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:08:12 STEP 2790 H=2790:00 +CPU= 31.637
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:08:34 STEP 2791 H=2791:00 +CPU= 19.702
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:08:56 STEP 2792 H=2792:00 +CPU= 20.173
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:09:31 STEP 2793 H=2793:00 +CPU= 31.058
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:09:53 STEP 2794 H=2794:00 +CPU= 20.352
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:10:15 STEP 2795 H=2795:00 +CPU= 19.656
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:10:54 STEP 2796 H=2796:00 +CPU= 34.311
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002796
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:11:15 STEP 2797 H=2797:00 +CPU= 19.753
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:11:38 STEP 2798 H=2798:00 +CPU= 20.214
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:12:13 STEP 2799 H=2799:00 +CPU= 31.059
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:12:35 STEP 2800 H=2800:00 +CPU= 20.249
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:12:57 STEP 2801 H=2801:00 +CPU= 19.745
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:13:33 STEP 2802 H=2802:00 +CPU= 31.686
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:13:55 STEP 2803 H=2803:00 +CPU= 19.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:14:17 STEP 2804 H=2804:00 +CPU= 20.331
Suspend request received from the BOINC client, suspending the child process
Quit request received from the BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2015050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0084
(argv4) batchid: 984
(argv5) wuid: 12200728
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_494a3268bcf8a5a69f89b67e672b4891 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0084_2015050100_123_984_12200728.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0084_2015050100_123_984_12200728.zip
ic_ancil_file: ic_ancil_12200728
ifsdata_file: ifsdata_12200728
climate_data_file: clim_data_12200728
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_c3a7a1d21fa2cbee2d96b97baf803571 to: /var/lib/boinc-client/slots/1/ic_ancil_12200728.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12200728.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1e5969ab2d39709704891aba0d93dd7c to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12200728.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12200728.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12200728.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12200728.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12200728.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12200728.xml
-- Model is restarting --
Adjusting last_iter, 2805, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12200728.xml
last_cpu_time: 69398
upload_file_number: 116
last_iter: 2783
last_upload: 10022400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0084_2015050100_123_984_12200728_0_r1256276306
The child process has been launched with process id: 558733
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:558733:558733] [20230110:002620:1673303180:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:27:04 STEP 2784 H=2784:00 +CPU= 35.140
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:27:25 STEP 2785 H=2785:00 +CPU= 19.705
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:27:50 STEP 2786 H=2786:00 +CPU= 20.269
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:28:23 STEP 2787 H=2787:00 +CPU= 30.994
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:28:46 STEP 2788 H=2788:00 +CPU= 20.277
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:29:09 STEP 2789 H=2789:00 +CPU= 19.715
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:29:43 STEP 2790 H=2790:00 +CPU= 31.546
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:30:05 STEP 2791 H=2791:00 +CPU= 19.749
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:30:30 STEP 2792 H=2792:00 +CPU= 20.301
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:31:03 STEP 2793 H=2793:00 +CPU= 31.009
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:31:25 STEP 2794 H=2794:00 +CPU= 20.298
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:31:49 STEP 2795 H=2795:00 +CPU= 19.714
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:32:26 STEP 2796 H=2796:00 +CPU= 34.189
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002796
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:32:50 STEP 2797 H=2797:00 +CPU= 19.707
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:33:12 STEP 2798 H=2798:00 +CPU= 20.271
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:33:45 STEP 2799 H=2799:00 +CPU= 31.053
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:34:10 STEP 2800 H=2800:00 +CPU= 20.256
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:34:31 STEP 2801 H=2801:00 +CPU= 19.539
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:35:05 STEP 2802 H=2802:00 +CPU= 31.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:35:29 STEP 2803 H=2803:00 +CPU= 19.697
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:35:52 STEP 2804 H=2804:00 +CPU= 20.085
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:36:25 STEP 2805 H=2805:00 +CPU= 31.013
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:36:49 STEP 2806 H=2806:00 +CPU= 20.203
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:37:12 STEP 2807 H=2807:00 +CPU= 20.254
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:37:52 STEP 2808 H=2808:00 +CPU= 36.151
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0084_2015050100_123_984_12200728_0_r1256276306_116.zip
Uploading the intermediate file: upload_file_116.zip
  00:38:12 STEP 2809 H=2809:00 +CPU= 19.627
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:38:34 STEP 2810 H=2810:00 +CPU= 20.309
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:39:10 STEP 2811 H=2811:00 +CPU= 31.049
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:39:32 STEP 2812 H=2812:00 +CPU= 20.211
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:39:54 STEP 2813 H=2813:00 +CPU= 19.715
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:40:30 STEP 2814 H=2814:00 +CPU= 31.781
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:40:52 STEP 2815 H=2815:00 +CPU= 19.748
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:41:15 STEP 2816 H=2816:00 +CPU= 20.384
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:41:50 STEP 2817 H=2817:00 +CPU= 31.111
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:42:12 STEP 2818 H=2818:00 +CPU= 20.295
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:42:34 STEP 2819 H=2819:00 +CPU= 19.805
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:43:13 STEP 2820 H=2820:00 +CPU= 34.275
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002820
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:43:35 STEP 2821 H=2821:00 +CPU= 19.827
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:43:58 STEP 2822 H=2822:00 +CPU= 20.397
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:44:33 STEP 2823 H=2823:00 +CPU= 31.624
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:44:57 STEP 2824 H=2824:00 +CPU= 21.574
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:45:20 STEP 2825 H=2825:00 +CPU= 20.988
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:45:58 STEP 2826 H=2826:00 +CPU= 33.229
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:46:20 STEP 2827 H=2827:00 +CPU= 19.621
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:46:43 STEP 2828 H=2828:00 +CPU= 20.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:47:19 STEP 2829 H=2829:00 +CPU= 31.478
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:47:41 STEP 2830 H=2830:00 +CPU= 20.310
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:48:04 STEP 2831 H=2831:00 +CPU= 20.404
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:48:45 STEP 2832 H=2832:00 +CPU= 36.232
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0084_2015050100_123_984_12200728_0_r1256276306_117.zip
Uploading the intermediate file: upload_file_117.zip
  00:49:07 STEP 2833 H=2833:00 +CPU= 19.685
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:49:29 STEP 2834 H=2834:00 +CPU= 20.309
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:50:03 STEP 2835 H=2835:00 +CPU= 31.095
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:50:25 STEP 2836 H=2836:00 +CPU= 20.374
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:50:49 STEP 2837 H=2837:00 +CPU= 19.737
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:51:23 STEP 2838 H=2838:00 +CPU= 31.695
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:51:45 STEP 2839 H=2839:00 +CPU= 19.832
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:52:10 STEP 2840 H=2840:00 +CPU= 20.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:52:43 STEP 2841 H=2841:00 +CPU= 31.050
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:53:06 STEP 2842 H=2842:00 +CPU= 20.319
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:53:29 STEP 2843 H=2843:00 +CPU= 19.723
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:54:06 STEP 2844 H=2844:00 +CPU= 34.209
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002844
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:54:29 STEP 2845 H=2845:00 +CPU= 19.656
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:54:52 STEP 2846 H=2846:00 +CPU= 20.289
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:55:25 STEP 2847 H=2847:00 +CPU= 30.950
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:55:49 STEP 2848 H=2848:00 +CPU= 20.100
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:56:11 STEP 2849 H=2849:00 +CPU= 19.704
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:56:45 STEP 2850 H=2850:00 +CPU= 31.603
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:57:06 STEP 2851 H=2851:00 +CPU= 19.612
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:57:31 STEP 2852 H=2852:00 +CPU= 20.109
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:58:04 STEP 2853 H=2853:00 +CPU= 31.043
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:58:26 STEP 2854 H=2854:00 +CPU= 20.309
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:58:51 STEP 2855 H=2855:00 +CPU= 20.241
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:59:31 STEP 2856 H=2856:00 +CPU= 36.167
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0084_2015050100_123_984_12200728_0_r1256276306_118.zip
Uploading the intermediate file: upload_file_118.zip
  00:59:51 STEP 2857 H=2857:00 +CPU= 19.617
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:00:13 STEP 2858 H=2858:00 +CPU= 20.237
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:00:46 STEP 2859 H=2859:00 +CPU= 31.033
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:01:11 STEP 2860 H=2860:00 +CPU= 20.164
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:01:33 STEP 2861 H=2861:00 +CPU= 19.601
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:02:09 STEP 2862 H=2862:00 +CPU= 31.734
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:02:31 STEP 2863 H=2863:00 +CPU= 19.700
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:02:53 STEP 2864 H=2864:00 +CPU= 20.240
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:03:26 STEP 2865 H=2865:00 +CPU= 31.073
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:03:51 STEP 2866 H=2866:00 +CPU= 20.269
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:04:13 STEP 2867 H=2867:00 +CPU= 19.770
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:04:51 STEP 2868 H=2868:00 +CPU= 34.239
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002868
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:05:13 STEP 2869 H=2869:00 +CPU= 19.742
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:05:35 STEP 2870 H=2870:00 +CPU= 20.304
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:06:11 STEP 2871 H=2871:00 +CPU= 31.101
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:06:33 STEP 2872 H=2872:00 +CPU= 20.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:06:55 STEP 2873 H=2873:00 +CPU= 19.767
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:07:30 STEP 2874 H=2874:00 +CPU= 31.548
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:07:52 STEP 2875 H=2875:00 +CPU= 19.635
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:08:15 STEP 2876 H=2876:00 +CPU= 20.307
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:08:50 STEP 2877 H=2877:00 +CPU= 31.023
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:09:12 STEP 2878 H=2878:00 +CPU= 20.086
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:09:34 STEP 2879 H=2879:00 +CPU= 20.368
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:10:15 STEP 2880 H=2880:00 +CPU= 36.110
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0084_2015050100_123_984_12200728_0_r1256276306_119.zip
Uploading the intermediate file: upload_file_119.zip
  01:10:34 STEP 2881 H=2881:00 +CPU= 19.645
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:10:57 STEP 2882 H=2882:00 +CPU= 20.280
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:11:32 STEP 2883 H=2883:00 +CPU= 31.001
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:11:54 STEP 2884 H=2884:00 +CPU= 20.270
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:12:16 STEP 2885 H=2885:00 +CPU= 19.680
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:12:52 STEP 2886 H=2886:00 +CPU= 31.658
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:13:14 STEP 2887 H=2887:00 +CPU= 19.732
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:13:36 STEP 2888 H=2888:00 +CPU= 20.267
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:14:11 STEP 2889 H=2889:00 +CPU= 30.936
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:14:34 STEP 2890 H=2890:00 +CPU= 20.312
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:14:55 STEP 2891 H=2891:00 +CPU= 19.734
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:15:34 STEP 2892 H=2892:00 +CPU= 34.076
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002892
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:15:55 STEP 2893 H=2893:00 +CPU= 19.655
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:16:18 STEP 2894 H=2894:00 +CPU= 20.329
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:16:53 STEP 2895 H=2895:00 +CPU= 30.983
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:17:15 STEP 2896 H=2896:00 +CPU= 20.331
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:17:37 STEP 2897 H=2897:00 +CPU= 19.750
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:18:13 STEP 2898 H=2898:00 +CPU= 31.683
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:18:35 STEP 2899 H=2899:00 +CPU= 19.738
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:18:57 STEP 2900 H=2900:00 +CPU= 20.247
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:19:32 STEP 2901 H=2901:00 +CPU= 30.910
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:19:55 STEP 2902 H=2902:00 +CPU= 20.256
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:20:17 STEP 2903 H=2903:00 +CPU= 20.311
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:20:58 STEP 2904 H=2904:00 +CPU= 36.179
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0084_2015050100_123_984_12200728_0_r1256276306_120.zip
Uploading the intermediate file: upload_file_120.zip
  01:21:17 STEP 2905 H=2905:00 +CPU= 19.659
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:21:40 STEP 2906 H=2906:00 +CPU= 20.235
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:22:15 STEP 2907 H=2907:00 +CPU= 31.023
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:22:37 STEP 2908 H=2908:00 +CPU= 20.282
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:22:59 STEP 2909 H=2909:00 +CPU= 19.689
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:23:35 STEP 2910 H=2910:00 +CPU= 31.670
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:23:57 STEP 2911 H=2911:00 +CPU= 19.736
  01:24:17 STEP 2912 H=2912:00 +CPU= 20.278
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:24:52 STEP 2913 H=2913:00 +CPU= 31.037
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:25:15 STEP 2914 H=2914:00 +CPU= 20.352
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:25:36 STEP 2915 H=2915:00 +CPU= 19.622
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:26:15 STEP 2916 H=2916:00 +CPU= 34.226
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002916
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:26:37 STEP 2917 H=2917:00 +CPU= 19.735
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:26:59 STEP 2918 H=2918:00 +CPU= 20.309
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:27:34 STEP 2919 H=2919:00 +CPU= 31.047
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:27:56 STEP 2920 H=2920:00 +CPU= 20.257
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:28:18 STEP 2921 H=2921:00 +CPU= 19.737
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:28:54 STEP 2922 H=2922:00 +CPU= 31.618
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:29:16 STEP 2923 H=2923:00 +CPU= 19.617
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:29:38 STEP 2924 H=2924:00 +CPU= 20.289
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:30:13 STEP 2925 H=2925:00 +CPU= 30.968
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:30:35 STEP 2926 H=2926:00 +CPU= 20.144
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:30:58 STEP 2927 H=2927:00 +CPU= 20.378
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:31:38 STEP 2928 H=2928:00 +CPU= 36.140
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0084_2015050100_123_984_12200728_0_r1256276306_121.zip
Uploading the intermediate file: upload_file_121.zip
  01:31:58 STEP 2929 H=2929:00 +CPU= 19.516
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:32:20 STEP 2930 H=2930:00 +CPU= 20.226
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:32:56 STEP 2931 H=2931:00 +CPU= 31.005
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:33:18 STEP 2932 H=2932:00 +CPU= 20.133
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:33:40 STEP 2933 H=2933:00 +CPU= 19.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:34:15 STEP 2934 H=2934:00 +CPU= 31.588
  01:34:35 STEP 2935 H=2935:00 +CPU= 19.538
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:34:57 STEP 2936 H=2936:00 +CPU= 20.320
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:35:32 STEP 2937 H=2937:00 +CPU= 31.007
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:35:55 STEP 2938 H=2938:00 +CPU= 20.333
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:36:17 STEP 2939 H=2939:00 +CPU= 19.762
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:36:55 STEP 2940 H=2940:00 +CPU= 34.190
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002940
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:37:17 STEP 2941 H=2941:00 +CPU= 19.753
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:37:39 STEP 2942 H=2942:00 +CPU= 20.282
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:38:14 STEP 2943 H=2943:00 +CPU= 30.965
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:38:37 STEP 2944 H=2944:00 +CPU= 20.294
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:38:59 STEP 2945 H=2945:00 +CPU= 19.695
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:39:34 STEP 2946 H=2946:00 +CPU= 31.545
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:39:56 STEP 2947 H=2947:00 +CPU= 19.712
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:40:18 STEP 2948 H=2948:00 +CPU= 20.255
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:40:54 STEP 2949 H=2949:00 +CPU= 30.936
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:41:16 STEP 2950 H=2950:00 +CPU= 20.255
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:41:38 STEP 2951 H=2951:00 +CPU= 20.294
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:42:17 STEP 2952 H=2952:00 +CPU= 34.975
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200728/ICMUAhq0f+002952
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0084_2015050100_123_984_12200728_0_r1256276306_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
01:45:40 (558720): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:14:08 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 10,623,600 73,486 0.0069
09 May 2023 16:13:45 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 9,244,800 63,192 0.0068
09 May 2023 16:13:40 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 8,380,800 57,433 0.0069
09 May 2023 16:13:36 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 7,516,800 51,644 0.0069
09 May 2023 16:13:31 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 6,652,800 45,854 0.0069
09 May 2023 16:13:27 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 5,788,800 40,065 0.0069
09 May 2023 16:13:19 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 4,665,600 32,004 0.0069
09 May 2023 16:13:15 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 3,801,600 26,186 0.0069
09 May 2023 16:13:11 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 2,937,600 20,381 0.0069
09 May 2023 16:12:29 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 1,814,400 12,531 0.0069
09 May 2023 16:10:16 1454366 22286986 oifs_43r3_ps_0084_2015050100_123_984_12200728_0 864,000 5,726 0.0066


©2024 climateprediction.net