climateprediction.net home page
Task 22313185

Task 22313185

Name oifs_43r3_bl_a4pw_2016092300_15_991_12212903_0
Workunit 12212903
Created 13 Feb 2023, 23:15:50 UTC
Sent 14 Feb 2023, 6:27:04 UTC
Report deadline 15 Apr 2023, 6:27:04 UTC
Received 15 Feb 2023, 3:59:42 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1524158
Run time 10 hours 42 min 37 sec
CPU time 10 hours 37 min 57 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 4.40 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,075.25 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,361.24 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
:39 STEP 1027 H= 256:45 +CPU= 26.482
  10:25:06 STEP 1028 H= 257:00 +CPU= 27.160
  10:25:33 STEP 1029 H= 257:15 +CPU= 26.523
  10:26:00 STEP 1030 H= 257:30 +CPU= 26.517
  10:26:26 STEP 1031 H= 257:45 +CPU= 26.488
  10:26:59 STEP 1032 H= 258:00 +CPU= 33.114
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001032
  10:27:26 STEP 1033 H= 258:15 +CPU= 26.675
  10:27:53 STEP 1034 H= 258:30 +CPU= 26.509
  10:28:19 STEP 1035 H= 258:45 +CPU= 26.680
  10:28:47 STEP 1036 H= 259:00 +CPU= 27.223
  10:29:13 STEP 1037 H= 259:15 +CPU= 26.558
  10:29:40 STEP 1038 H= 259:30 +CPU= 26.741
  10:30:07 STEP 1039 H= 259:45 +CPU= 26.774
  10:30:34 STEP 1040 H= 260:00 +CPU= 27.372
  10:31:01 STEP 1041 H= 260:15 +CPU= 26.819
  10:31:28 STEP 1042 H= 260:30 +CPU= 26.777
  10:31:55 STEP 1043 H= 260:45 +CPU= 26.798
  10:32:28 STEP 1044 H= 261:00 +CPU= 33.024
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001044
  10:32:55 STEP 1045 H= 261:15 +CPU= 26.673
  10:33:22 STEP 1046 H= 261:30 +CPU= 26.709
  10:33:49 STEP 1047 H= 261:45 +CPU= 26.819
  10:34:16 STEP 1048 H= 262:00 +CPU= 27.413
  10:34:43 STEP 1049 H= 262:15 +CPU= 26.804
  10:35:10 STEP 1050 H= 262:30 +CPU= 26.698
  10:35:36 STEP 1051 H= 262:45 +CPU= 26.704
  10:36:04 STEP 1052 H= 263:00 +CPU= 27.274
  10:36:31 STEP 1053 H= 263:15 +CPU= 26.781
  10:36:57 STEP 1054 H= 263:30 +CPU= 26.721
  10:37:24 STEP 1055 H= 263:45 +CPU= 26.718
  10:38:01 STEP 1056 H= 264:00 +CPU= 36.184
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4pw_2016092300_15_991_12212903_0_r30404364_10.zip
Uploading the intermediate file: upload_file_10.zip
  10:38:28 STEP 1057 H= 264:15 +CPU= 26.983
  10:38:55 STEP 1058 H= 264:30 +CPU= 26.912
  10:39:22 STEP 1059 H= 264:45 +CPU= 27.311
  10:39:50 STEP 1060 H= 265:00 +CPU= 27.470
  10:40:17 STEP 1061 H= 265:15 +CPU= 26.958
  10:40:44 STEP 1062 H= 265:30 +CPU= 26.866
  10:41:11 STEP 1063 H= 265:45 +CPU= 27.295
  10:41:39 STEP 1064 H= 266:00 +CPU= 27.817
  10:42:06 STEP 1065 H= 266:15 +CPU= 26.917
  10:42:33 STEP 1066 H= 266:30 +CPU= 26.862
  10:43:00 STEP 1067 H= 266:45 +CPU= 26.841
  10:43:33 STEP 1068 H= 267:00 +CPU= 33.312
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001068
  10:44:00 STEP 1069 H= 267:15 +CPU= 26.607
  10:44:27 STEP 1070 H= 267:30 +CPU= 26.622
  10:44:53 STEP 1071 H= 267:45 +CPU= 26.601
  10:45:21 STEP 1072 H= 268:00 +CPU= 27.241
  10:45:48 STEP 1073 H= 268:15 +CPU= 26.877
  10:46:14 STEP 1074 H= 268:30 +CPU= 26.556
  10:46:41 STEP 1075 H= 268:45 +CPU= 26.828
  10:47:09 STEP 1076 H= 269:00 +CPU= 27.373
  10:47:36 STEP 1077 H= 269:15 +CPU= 26.796
  10:48:02 STEP 1078 H= 269:30 +CPU= 26.700
  10:48:29 STEP 1079 H= 269:45 +CPU= 26.713
  10:49:02 STEP 1080 H= 270:00 +CPU= 33.080
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001080
  10:49:29 STEP 1081 H= 270:15 +CPU= 26.688
  10:49:56 STEP 1082 H= 270:30 +CPU= 26.641
  10:50:23 STEP 1083 H= 270:45 +CPU= 26.720
  10:50:50 STEP 1084 H= 271:00 +CPU= 27.317
  10:51:17 STEP 1085 H= 271:15 +CPU= 26.778
  10:51:44 STEP 1086 H= 271:30 +CPU= 26.795
  10:52:10 STEP 1087 H= 271:45 +CPU= 26.674
  10:52:38 STEP 1088 H= 272:00 +CPU= 27.329
  10:53:05 STEP 1089 H= 272:15 +CPU= 26.751
  10:53:32 STEP 1090 H= 272:30 +CPU= 26.843
  10:53:58 STEP 1091 H= 272:45 +CPU= 26.787
  10:54:32 STEP 1092 H= 273:00 +CPU= 33.102
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001092
  10:54:58 STEP 1093 H= 273:15 +CPU= 26.769
  10:55:25 STEP 1094 H= 273:30 +CPU= 26.794
  10:55:52 STEP 1095 H= 273:45 +CPU= 26.795
  10:56:20 STEP 1096 H= 274:00 +CPU= 27.380
  10:56:46 STEP 1097 H= 274:15 +CPU= 26.789
  10:57:14 STEP 1098 H= 274:30 +CPU= 26.983
  10:57:40 STEP 1099 H= 274:45 +CPU= 26.600
  10:58:08 STEP 1100 H= 275:00 +CPU= 27.223
  10:58:34 STEP 1101 H= 275:15 +CPU= 26.625
  10:59:01 STEP 1102 H= 275:30 +CPU= 26.626
  10:59:28 STEP 1103 H= 275:45 +CPU= 26.609
  11:00:04 STEP 1104 H= 276:00 +CPU= 36.092
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001104
  11:00:31 STEP 1105 H= 276:15 +CPU= 26.651
  11:00:57 STEP 1106 H= 276:30 +CPU= 26.662
  11:01:24 STEP 1107 H= 276:45 +CPU= 26.530
  11:01:51 STEP 1108 H= 277:00 +CPU= 27.177
  11:02:18 STEP 1109 H= 277:15 +CPU= 26.745
  11:02:44 STEP 1110 H= 277:30 +CPU= 26.470
  11:03:11 STEP 1111 H= 277:45 +CPU= 26.801
  11:03:39 STEP 1112 H= 278:00 +CPU= 27.395
  11:04:06 STEP 1113 H= 278:15 +CPU= 26.854
  11:04:33 STEP 1114 H= 278:30 +CPU= 26.836
  11:05:00 STEP 1115 H= 278:45 +CPU= 26.808
  11:05:33 STEP 1116 H= 279:00 +CPU= 33.018
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001116
  11:06:00 STEP 1117 H= 279:15 +CPU= 26.766
  11:06:26 STEP 1118 H= 279:30 +CPU= 26.808
  11:06:53 STEP 1119 H= 279:45 +CPU= 26.742
  11:07:21 STEP 1120 H= 280:00 +CPU= 27.285
  11:07:47 STEP 1121 H= 280:15 +CPU= 26.755
  11:08:14 STEP 1122 H= 280:30 +CPU= 26.765
  11:08:41 STEP 1123 H= 280:45 +CPU= 26.704
  11:09:08 STEP 1124 H= 281:00 +CPU= 27.051
  11:09:35 STEP 1125 H= 281:15 +CPU= 26.663
  11:10:02 STEP 1126 H= 281:30 +CPU= 26.637
  11:10:28 STEP 1127 H= 281:45 +CPU= 26.736
  11:11:01 STEP 1128 H= 282:00 +CPU= 32.959
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001128
  11:11:28 STEP 1129 H= 282:15 +CPU= 26.674
  11:11:55 STEP 1130 H= 282:30 +CPU= 26.679
  11:12:22 STEP 1131 H= 282:45 +CPU= 26.729
  11:12:49 STEP 1132 H= 283:00 +CPU= 27.279
  11:13:16 STEP 1133 H= 283:15 +CPU= 26.738
  11:13:43 STEP 1134 H= 283:30 +CPU= 26.758
  11:14:09 STEP 1135 H= 283:45 +CPU= 26.690
  11:14:37 STEP 1136 H= 284:00 +CPU= 27.347
  11:15:04 STEP 1137 H= 284:15 +CPU= 26.861
  11:15:31 STEP 1138 H= 284:30 +CPU= 26.691
  11:15:57 STEP 1139 H= 284:45 +CPU= 26.662
  11:16:31 STEP 1140 H= 285:00 +CPU= 33.174
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001140
  11:16:57 STEP 1141 H= 285:15 +CPU= 26.704
  11:17:24 STEP 1142 H= 285:30 +CPU= 26.639
  11:17:51 STEP 1143 H= 285:45 +CPU= 26.699
  11:18:18 STEP 1144 H= 286:00 +CPU= 27.273
  11:18:45 STEP 1145 H= 286:15 +CPU= 26.600
  11:19:12 STEP 1146 H= 286:30 +CPU= 26.712
  11:19:38 STEP 1147 H= 286:45 +CPU= 26.724
  11:20:06 STEP 1148 H= 287:00 +CPU= 27.438
  11:20:33 STEP 1149 H= 287:15 +CPU= 26.938
  11:21:00 STEP 1150 H= 287:30 +CPU= 26.767
  11:21:27 STEP 1151 H= 287:45 +CPU= 26.776
  11:22:03 STEP 1152 H= 288:00 +CPU= 36.225
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4pw_2016092300_15_991_12212903_0_r30404364_11.zip
Uploading the intermediate file: upload_file_11.zip
  11:22:30 STEP 1153 H= 288:15 +CPU= 27.005
  11:22:57 STEP 1154 H= 288:30 +CPU= 27.239
  11:23:25 STEP 1155 H= 288:45 +CPU= 27.103
  11:23:52 STEP 1156 H= 289:00 +CPU= 27.412
  11:24:19 STEP 1157 H= 289:15 +CPU= 26.833
  11:24:46 STEP 1158 H= 289:30 +CPU= 26.820
  11:25:13 STEP 1159 H= 289:45 +CPU= 26.782
  11:25:40 STEP 1160 H= 290:00 +CPU= 27.379
  11:26:07 STEP 1161 H= 290:15 +CPU= 26.892
  11:26:34 STEP 1162 H= 290:30 +CPU= 26.782
  11:27:01 STEP 1163 H= 290:45 +CPU= 27.217
  11:27:36 STEP 1164 H= 291:00 +CPU= 34.112
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001164
  11:28:03 STEP 1165 H= 291:15 +CPU= 26.930
  11:28:29 STEP 1166 H= 291:30 +CPU= 26.596
  11:28:56 STEP 1167 H= 291:45 +CPU= 26.666
  11:29:23 STEP 1168 H= 292:00 +CPU= 27.277
  11:29:50 STEP 1169 H= 292:15 +CPU= 26.748
  11:30:17 STEP 1170 H= 292:30 +CPU= 26.666
  11:30:44 STEP 1171 H= 292:45 +CPU= 26.693
  11:31:11 STEP 1172 H= 293:00 +CPU= 27.253
  11:31:38 STEP 1173 H= 293:15 +CPU= 26.541
  11:32:05 STEP 1174 H= 293:30 +CPU= 26.817
  11:32:31 STEP 1175 H= 293:45 +CPU= 26.777
  11:33:05 STEP 1176 H= 294:00 +CPU= 33.205
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001176
  11:33:31 STEP 1177 H= 294:15 +CPU= 26.607
  11:33:58 STEP 1178 H= 294:30 +CPU= 26.743
  11:34:25 STEP 1179 H= 294:45 +CPU= 26.598
  11:34:52 STEP 1180 H= 295:00 +CPU= 27.299
  11:35:19 STEP 1181 H= 295:15 +CPU= 26.694
  11:35:46 STEP 1182 H= 295:30 +CPU= 26.805
  11:36:13 STEP 1183 H= 295:45 +CPU= 26.686
  11:36:40 STEP 1184 H= 296:00 +CPU= 27.411
  11:37:07 STEP 1185 H= 296:15 +CPU= 26.886
  11:37:34 STEP 1186 H= 296:30 +CPU= 26.813
  11:38:01 STEP 1187 H= 296:45 +CPU= 26.929
  11:38:34 STEP 1188 H= 297:00 +CPU= 33.098
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001188
  11:39:01 STEP 1189 H= 297:15 +CPU= 26.845
  11:39:28 STEP 1190 H= 297:30 +CPU= 26.858
  11:39:55 STEP 1191 H= 297:45 +CPU= 26.852
  11:40:22 STEP 1192 H= 298:00 +CPU= 27.327
  11:40:49 STEP 1193 H= 298:15 +CPU= 26.771
  11:41:16 STEP 1194 H= 298:30 +CPU= 26.679
  11:41:43 STEP 1195 H= 298:45 +CPU= 26.685
  11:42:10 STEP 1196 H= 299:00 +CPU= 27.306
  11:42:37 STEP 1197 H= 299:15 +CPU= 26.855
  11:43:04 STEP 1198 H= 299:30 +CPU= 26.542
  11:43:30 STEP 1199 H= 299:45 +CPU= 26.643
  11:44:07 STEP 1200 H= 300:00 +CPU= 36.275
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001200
  11:44:33 STEP 1201 H= 300:15 +CPU= 26.648
  11:45:00 STEP 1202 H= 300:30 +CPU= 26.767
  11:45:27 STEP 1203 H= 300:45 +CPU= 26.707
  11:45:54 STEP 1204 H= 301:00 +CPU= 27.216
  11:46:21 STEP 1205 H= 301:15 +CPU= 26.767
  11:46:48 STEP 1206 H= 301:30 +CPU= 26.743
  11:47:15 STEP 1207 H= 301:45 +CPU= 26.766
  11:47:42 STEP 1208 H= 302:00 +CPU= 27.370
  11:48:09 STEP 1209 H= 302:15 +CPU= 26.603
  11:48:36 STEP 1210 H= 302:30 +CPU= 26.765
  11:49:03 STEP 1211 H= 302:45 +CPU= 26.898
  11:49:36 STEP 1212 H= 303:00 +CPU= 33.212
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001212
  11:50:03 STEP 1213 H= 303:15 +CPU= 26.691
  11:50:30 STEP 1214 H= 303:30 +CPU= 26.696
  11:50:57 STEP 1215 H= 303:45 +CPU= 26.799
  11:51:24 STEP 1216 H= 304:00 +CPU= 27.347
  11:51:51 STEP 1217 H= 304:15 +CPU= 26.627
  11:52:17 STEP 1218 H= 304:30 +CPU= 26.746
  11:52:44 STEP 1219 H= 304:45 +CPU= 26.774
  11:53:12 STEP 1220 H= 305:00 +CPU= 27.348
  11:53:39 STEP 1221 H= 305:15 +CPU= 26.717
  11:54:05 STEP 1222 H= 305:30 +CPU= 26.857
  11:54:32 STEP 1223 H= 305:45 +CPU= 26.782
  11:55:06 STEP 1224 H= 306:00 +CPU= 33.310
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001224
  11:55:33 STEP 1225 H= 306:15 +CPU= 26.911
  11:56:00 STEP 1226 H= 306:30 +CPU= 26.888
  11:56:27 STEP 1227 H= 306:45 +CPU= 26.835
  11:56:54 STEP 1228 H= 307:00 +CPU= 27.408
  11:57:21 STEP 1229 H= 307:15 +CPU= 26.842
  11:57:48 STEP 1230 H= 307:30 +CPU= 26.746
  11:58:15 STEP 1231 H= 307:45 +CPU= 26.680
  11:58:42 STEP 1232 H= 308:00 +CPU= 27.271
  11:59:09 STEP 1233 H= 308:15 +CPU= 26.708
  11:59:36 STEP 1234 H= 308:30 +CPU= 26.778
  12:00:02 STEP 1235 H= 308:45 +CPU= 26.592
  12:00:35 STEP 1236 H= 309:00 +CPU= 33.048
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001236
  12:01:02 STEP 1237 H= 309:15 +CPU= 26.744
  12:01:29 STEP 1238 H= 309:30 +CPU= 26.547
  12:01:55 STEP 1239 H= 309:45 +CPU= 26.583
  12:02:23 STEP 1240 H= 310:00 +CPU= 27.296
  12:02:50 STEP 1241 H= 310:15 +CPU= 26.727
  12:03:16 STEP 1242 H= 310:30 +CPU= 26.796
  12:03:43 STEP 1243 H= 310:45 +CPU= 26.828
  12:04:11 STEP 1244 H= 311:00 +CPU= 27.489
  12:04:38 STEP 1245 H= 311:15 +CPU= 26.816
  12:05:05 STEP 1246 H= 311:30 +CPU= 26.797
  12:05:32 STEP 1247 H= 311:45 +CPU= 26.940
  12:06:08 STEP 1248 H= 312:00 +CPU= 36.572
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4pw_2016092300_15_991_12212903_0_r30404364_12.zip
Uploading the intermediate file: upload_file_12.zip
  12:06:35 STEP 1249 H= 312:15 +CPU= 27.005
  12:07:03 STEP 1250 H= 312:30 +CPU= 27.162
  12:07:30 STEP 1251 H= 312:45 +CPU= 27.257
  12:07:58 STEP 1252 H= 313:00 +CPU= 27.385
  12:08:24 STEP 1253 H= 313:15 +CPU= 26.818
  12:08:51 STEP 1254 H= 313:30 +CPU= 26.812
  12:09:18 STEP 1255 H= 313:45 +CPU= 26.837
  12:09:46 STEP 1256 H= 314:00 +CPU= 27.422
  12:10:12 STEP 1257 H= 314:15 +CPU= 26.737
  12:10:39 STEP 1258 H= 314:30 +CPU= 26.785
  12:11:06 STEP 1259 H= 314:45 +CPU= 26.840
  12:11:40 STEP 1260 H= 315:00 +CPU= 33.481
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001260
  12:12:07 STEP 1261 H= 315:15 +CPU= 26.732
  12:12:34 STEP 1262 H= 315:30 +CPU= 27.077
  12:13:01 STEP 1263 H= 315:45 +CPU= 26.871
  12:13:29 STEP 1264 H= 316:00 +CPU= 27.677
  12:13:56 STEP 1265 H= 316:15 +CPU= 26.926
  12:14:23 STEP 1266 H= 316:30 +CPU= 27.223
  12:14:50 STEP 1267 H= 316:45 +CPU= 27.020
  12:15:17 STEP 1268 H= 317:00 +CPU= 27.145
  12:15:44 STEP 1269 H= 317:15 +CPU= 26.666
  12:16:11 STEP 1270 H= 317:30 +CPU= 26.712
  12:16:38 STEP 1271 H= 317:45 +CPU= 26.709
  12:17:11 STEP 1272 H= 318:00 +CPU= 32.999
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001272
  12:17:37 STEP 1273 H= 318:15 +CPU= 26.678
  12:18:04 STEP 1274 H= 318:30 +CPU= 26.736
  12:18:31 STEP 1275 H= 318:45 +CPU= 26.592
  12:18:58 STEP 1276 H= 319:00 +CPU= 27.087
  12:19:25 STEP 1277 H= 319:15 +CPU= 26.731
  12:19:52 STEP 1278 H= 319:30 +CPU= 26.813
  12:20:19 STEP 1279 H= 319:45 +CPU= 26.825
  12:20:46 STEP 1280 H= 320:00 +CPU= 27.391
  12:21:13 STEP 1281 H= 320:15 +CPU= 26.768
  12:21:40 STEP 1282 H= 320:30 +CPU= 26.815
  12:22:07 STEP 1283 H= 320:45 +CPU= 26.811
  12:22:40 STEP 1284 H= 321:00 +CPU= 33.224
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001284
  12:23:07 STEP 1285 H= 321:15 +CPU= 26.867
  12:23:34 STEP 1286 H= 321:30 +CPU= 26.910
  12:24:01 STEP 1287 H= 321:45 +CPU= 26.917
  12:24:28 STEP 1288 H= 322:00 +CPU= 27.445
  12:24:55 STEP 1289 H= 322:15 +CPU= 26.848
  12:25:22 STEP 1290 H= 322:30 +CPU= 26.864
  12:25:49 STEP 1291 H= 322:45 +CPU= 26.749
  12:26:17 STEP 1292 H= 323:00 +CPU= 27.347
  12:26:43 STEP 1293 H= 323:15 +CPU= 26.856
  12:27:10 STEP 1294 H= 323:30 +CPU= 26.853
  12:27:37 STEP 1295 H= 323:45 +CPU= 26.876
  12:28:14 STEP 1296 H= 324:00 +CPU= 36.786
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001296
  12:28:41 STEP 1297 H= 324:15 +CPU= 26.738
  12:29:08 STEP 1298 H= 324:30 +CPU= 26.837
  12:29:35 STEP 1299 H= 324:45 +CPU= 26.976
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4362MB (heap), 5076MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f0c0 (old at (nil))
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3843] 4362 MB (maxheap), 5076 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.285] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123000:1676395800:34419.286] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123001:1676395801:34420.286] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:digdug:1:1:4066:4066] [20230214:123001:1676395801:34420.286] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000012D289A  vdfmain_                  876  vdfmain.F90
oifs_43r3_model.e  00000000012BA8D9  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000012B0762  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000001163FC3  callpar_                  869  callpar.F90
oifs_43r3_model.e  00000000010B682F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  00000000010A915C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000E680C3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4pw
(argv4) batchid: 991
(argv5) wuid: 12212903
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_b6ae5de26485d2562d016cbf20bb59d0 to: /var/lib/boinc-client/slots/0/oifs_43r3_bl_a4pw_2016092300_15_991_12212903.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_bl_a4pw_2016092300_15_991_12212903.zip
ic_ancil_file: ic_ancil_12212903
ifsdata_file: ifsdata_12212903
climate_data_file: clim_data_12212903
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/0/ic_ancil_12212903.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12212903.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_5eadd818a3f774120ad6353b53e8f2dd to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12212903.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12212903.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12212903.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12212903.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12212903.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12212903.xml
-- Model is restarting --
Adjusting last_iter, 1300, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12212903.xml
last_cpu_time: 34317
upload_file_number: 13
last_iter: 1297
last_upload: 1123200
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a4pw_2016092300_15_991_12212903_0_r30404364
The child process has been launched with process id: 1921
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:digdug:1:1:1921:1921] [20230214:204832:1676425712:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  20:49:47 STEP 1296 H= 324:00 +CPU= 27.585
  20:50:14 STEP 1297 H= 324:15 +CPU= 26.503
  20:50:40 STEP 1298 H= 324:30 +CPU= 26.551
  20:51:07 STEP 1299 H= 324:45 +CPU= 26.592
  20:51:34 STEP 1300 H= 325:00 +CPU= 27.165
  20:52:01 STEP 1301 H= 325:15 +CPU= 26.690
  20:52:28 STEP 1302 H= 325:30 +CPU= 26.678
  20:52:54 STEP 1303 H= 325:45 +CPU= 26.684
  20:53:22 STEP 1304 H= 326:00 +CPU= 27.198
  20:53:48 STEP 1305 H= 326:15 +CPU= 26.601
  20:54:15 STEP 1306 H= 326:30 +CPU= 26.590
  20:54:42 STEP 1307 H= 326:45 +CPU= 26.619
  20:55:15 STEP 1308 H= 327:00 +CPU= 33.063
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001308
  20:55:41 STEP 1309 H= 327:15 +CPU= 26.517
  20:56:08 STEP 1310 H= 327:30 +CPU= 26.512
  20:56:35 STEP 1311 H= 327:45 +CPU= 26.566
  20:57:02 STEP 1312 H= 328:00 +CPU= 27.091
  20:57:28 STEP 1313 H= 328:15 +CPU= 26.335
  20:57:55 STEP 1314 H= 328:30 +CPU= 26.439
  20:58:21 STEP 1315 H= 328:45 +CPU= 26.513
  20:58:48 STEP 1316 H= 329:00 +CPU= 27.042
  20:59:15 STEP 1317 H= 329:15 +CPU= 26.449
  20:59:42 STEP 1318 H= 329:30 +CPU= 26.509
  21:00:08 STEP 1319 H= 329:45 +CPU= 26.459
  21:00:41 STEP 1320 H= 330:00 +CPU= 32.989
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001320
  21:01:08 STEP 1321 H= 330:15 +CPU= 26.623
  21:01:34 STEP 1322 H= 330:30 +CPU= 26.558
  21:02:01 STEP 1323 H= 330:45 +CPU= 26.597
  21:02:28 STEP 1324 H= 331:00 +CPU= 27.183
  21:02:55 STEP 1325 H= 331:15 +CPU= 26.605
  21:03:22 STEP 1326 H= 331:30 +CPU= 26.431
  21:03:48 STEP 1327 H= 331:45 +CPU= 26.466
  21:04:15 STEP 1328 H= 332:00 +CPU= 27.033
  21:04:42 STEP 1329 H= 332:15 +CPU= 26.541
  21:05:08 STEP 1330 H= 332:30 +CPU= 26.395
  21:05:35 STEP 1331 H= 332:45 +CPU= 26.564
  21:06:08 STEP 1332 H= 333:00 +CPU= 33.170
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001332
  21:06:35 STEP 1333 H= 333:15 +CPU= 26.497
  21:07:01 STEP 1334 H= 333:30 +CPU= 26.404
  21:07:28 STEP 1335 H= 333:45 +CPU= 26.670
  21:07:55 STEP 1336 H= 334:00 +CPU= 27.275
  21:08:22 STEP 1337 H= 334:15 +CPU= 26.713
  21:08:49 STEP 1338 H= 334:30 +CPU= 26.714
  21:09:16 STEP 1339 H= 334:45 +CPU= 26.711
  21:09:43 STEP 1340 H= 335:00 +CPU= 27.271
  21:10:10 STEP 1341 H= 335:15 +CPU= 26.702
  21:10:36 STEP 1342 H= 335:30 +CPU= 26.685
  21:11:03 STEP 1343 H= 335:45 +CPU= 26.590
  21:11:39 STEP 1344 H= 336:00 +CPU= 36.045
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4pw_2016092300_15_991_12212903_0_r30404364_13.zip
Uploading the intermediate file: upload_file_13.zip
  21:12:06 STEP 1345 H= 336:15 +CPU= 26.907
  21:12:33 STEP 1346 H= 336:30 +CPU= 27.117
  21:13:00 STEP 1347 H= 336:45 +CPU= 26.824
  21:13:28 STEP 1348 H= 337:00 +CPU= 27.117
  21:13:54 STEP 1349 H= 337:15 +CPU= 26.472
  21:14:21 STEP 1350 H= 337:30 +CPU= 26.628
  21:14:47 STEP 1351 H= 337:45 +CPU= 26.403
  21:15:15 STEP 1352 H= 338:00 +CPU= 27.122
  21:15:41 STEP 1353 H= 338:15 +CPU= 26.486
  21:16:08 STEP 1354 H= 338:30 +CPU= 26.525
  21:16:34 STEP 1355 H= 338:45 +CPU= 26.577
  21:17:07 STEP 1356 H= 339:00 +CPU= 32.876
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001356
  21:17:34 STEP 1357 H= 339:15 +CPU= 26.480
  21:18:00 STEP 1358 H= 339:30 +CPU= 26.592
  21:18:27 STEP 1359 H= 339:45 +CPU= 26.733
  21:18:54 STEP 1360 H= 340:00 +CPU= 27.155
  21:19:21 STEP 1361 H= 340:15 +CPU= 26.509
  21:19:48 STEP 1362 H= 340:30 +CPU= 26.510
  21:20:14 STEP 1363 H= 340:45 +CPU= 26.546
  21:20:42 STEP 1364 H= 341:00 +CPU= 27.210
  21:21:08 STEP 1365 H= 341:15 +CPU= 26.447
  21:21:35 STEP 1366 H= 341:30 +CPU= 26.570
  21:22:02 STEP 1367 H= 341:45 +CPU= 26.755
  21:22:35 STEP 1368 H= 342:00 +CPU= 33.207
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001368
  21:23:01 STEP 1369 H= 342:15 +CPU= 26.527
  21:23:28 STEP 1370 H= 342:30 +CPU= 26.657
  21:23:55 STEP 1371 H= 342:45 +CPU= 26.646
  21:24:22 STEP 1372 H= 343:00 +CPU= 27.233
  21:24:49 STEP 1373 H= 343:15 +CPU= 26.732
  21:25:16 STEP 1374 H= 343:30 +CPU= 26.743
  21:25:43 STEP 1375 H= 343:45 +CPU= 26.726
  21:26:10 STEP 1376 H= 344:00 +CPU= 27.230
  21:26:37 STEP 1377 H= 344:15 +CPU= 26.601
  21:27:03 STEP 1378 H= 344:30 +CPU= 26.575
  21:27:30 STEP 1379 H= 344:45 +CPU= 26.559
  21:28:03 STEP 1380 H= 345:00 +CPU= 32.983
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001380
  21:28:30 STEP 1381 H= 345:15 +CPU= 26.823
  21:28:56 STEP 1382 H= 345:30 +CPU= 26.531
  21:29:23 STEP 1383 H= 345:45 +CPU= 26.539
  21:29:50 STEP 1384 H= 346:00 +CPU= 27.104
  21:30:17 STEP 1385 H= 346:15 +CPU= 26.477
  21:30:43 STEP 1386 H= 346:30 +CPU= 26.609
  21:31:10 STEP 1387 H= 346:45 +CPU= 26.552
  21:31:37 STEP 1388 H= 347:00 +CPU= 27.188
  21:32:04 STEP 1389 H= 347:15 +CPU= 26.630
  21:32:31 STEP 1390 H= 347:30 +CPU= 26.596
  21:32:57 STEP 1391 H= 347:45 +CPU= 26.663
  21:33:33 STEP 1392 H= 348:00 +CPU= 35.961
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001392
  21:34:00 STEP 1393 H= 348:15 +CPU= 26.637
  21:34:27 STEP 1394 H= 348:30 +CPU= 26.527
  21:34:54 STEP 1395 H= 348:45 +CPU= 26.980
  21:35:21 STEP 1396 H= 349:00 +CPU= 27.335
  21:35:49 STEP 1397 H= 349:15 +CPU= 27.265
  21:36:15 STEP 1398 H= 349:30 +CPU= 26.671
  21:36:42 STEP 1399 H= 349:45 +CPU= 26.958
  21:37:10 STEP 1400 H= 350:00 +CPU= 27.366
  21:37:37 STEP 1401 H= 350:15 +CPU= 27.061
  21:38:04 STEP 1402 H= 350:30 +CPU= 26.601
  21:38:30 STEP 1403 H= 350:45 +CPU= 26.602
  21:39:04 STEP 1404 H= 351:00 +CPU= 33.208
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001404
  21:39:30 STEP 1405 H= 351:15 +CPU= 26.511
  21:39:57 STEP 1406 H= 351:30 +CPU= 26.610
  21:40:24 STEP 1407 H= 351:45 +CPU= 26.572
  21:40:51 STEP 1408 H= 352:00 +CPU= 27.189
  21:41:18 STEP 1409 H= 352:15 +CPU= 26.641
  21:41:44 STEP 1410 H= 352:30 +CPU= 26.810
  21:42:11 STEP 1411 H= 352:45 +CPU= 26.594
  21:42:38 STEP 1412 H= 353:00 +CPU= 27.178
  21:43:05 STEP 1413 H= 353:15 +CPU= 26.569
  21:43:32 STEP 1414 H= 353:30 +CPU= 26.596
  21:43:58 STEP 1415 H= 353:45 +CPU= 26.604
  21:44:31 STEP 1416 H= 354:00 +CPU= 32.877
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001416
  21:44:58 STEP 1417 H= 354:15 +CPU= 26.605
  21:45:25 STEP 1418 H= 354:30 +CPU= 26.605
  21:45:51 STEP 1419 H= 354:45 +CPU= 26.422
  21:46:18 STEP 1420 H= 355:00 +CPU= 27.019
  21:46:45 STEP 1421 H= 355:15 +CPU= 26.623
  21:47:11 STEP 1422 H= 355:30 +CPU= 26.518
  21:47:38 STEP 1423 H= 355:45 +CPU= 26.580
  21:48:05 STEP 1424 H= 356:00 +CPU= 27.156
  21:48:32 STEP 1425 H= 356:15 +CPU= 26.584
  21:48:59 STEP 1426 H= 356:30 +CPU= 26.583
  21:49:25 STEP 1427 H= 356:45 +CPU= 26.636
  21:49:58 STEP 1428 H= 357:00 +CPU= 33.075
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001428
  21:50:25 STEP 1429 H= 357:15 +CPU= 26.587
  21:50:52 STEP 1430 H= 357:30 +CPU= 26.522
  21:51:18 STEP 1431 H= 357:45 +CPU= 26.415
  21:51:45 STEP 1432 H= 358:00 +CPU= 27.118
  21:52:12 STEP 1433 H= 358:15 +CPU= 26.595
  21:52:39 STEP 1434 H= 358:30 +CPU= 26.696
  21:53:05 STEP 1435 H= 358:45 +CPU= 26.562
  21:53:33 STEP 1436 H= 359:00 +CPU= 27.151
  21:53:59 STEP 1437 H= 359:15 +CPU= 26.708
  21:54:26 STEP 1438 H= 359:30 +CPU= 26.781
  21:54:53 STEP 1439 H= 359:45 +CPU= 26.795
  21:55:28 STEP 1440 H= 360:00 +CPU= 34.175
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.1137030925E-20      I       0.1824074578E-05      I
ICloud fractionI      0.0000000000E+00      I       0.9746098695E-02      I
I  Relat. Hum. I      0.2364196473E+00      I       0.2746365391E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1772715844E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    307030145 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS767575.4 Mbytes/s, TOTAL TIME=    0.0097( 4.1%)
 21:55:28 STEP 1440 H= 360:00 +CPU= 34.175
 END CNT3
 NSTEP =  1440 CNT0   000000000  25.109  25.109  25.215   0.996               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN         4254/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     78    0       34      1044068     4254 ************    51.2    45.0     6.2    45.6    13.6
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 4015714.1 4015714.1    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 3968297.3 3968297.3     98.82      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  157    4052.2    4052.2     15.84      0.00
   9 SPCM     - SPECTRAL COMP.                     144     786.9     786.9      2.82      0.00
  10 SCAN2M - PHYSICS                              145   14106.3   14106.3     50.94      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 12     745.3     745.3      0.22      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  7      31.0      31.0      0.01      0.00
  17 GRIDFPOS IN CNT4                               12      24.3      24.3      0.01      0.00
  27 RESTART FILES - WRITING                         3    2981.2    2981.2      0.22      0.00
  28 RESTART FILES - READING                         1   46686.7   46686.7      1.16      0.00
  29 SU4FPOS IN CNT4                                12       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                12    5669.5    5669.5      1.69      0.00
  31 POSDDH IN STEPO                                 1       1.0       1.0      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    145    5068.1    5068.1     18.30      0.00
  39 SU0YOMB                                         1     305.1     305.1      0.01      0.00
  51 SCAN2M   - SL COMM. PART 1                    145      28.8      28.8      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              144     415.4     415.4      1.49      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              144      77.6      77.6      0.28      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  7       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      316     365.2     365.2      2.87      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       300     213.3     213.3      1.59      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        300      76.3      76.3      0.57      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       316     126.1     126.1      0.99      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      39.6      39.6      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            316      44.4      44.4      0.35      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            300      25.7      25.7      0.19      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            316      68.7      68.7      0.54      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            300      22.4      22.4      0.17      0.00
 400 GSTATS                                      17691       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 16413       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     4015.7 CPU TIME    3960.6 VECTOR TIME     3960.6


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      326.6
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/slots/0/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/0/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12212903/ICMSHh7zg+001428
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4pw_2016092300_15_991_12212903_0_r30404364_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
21:58:56 (1857): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:51:31 1524158 22313185 oifs_43r3_bl_a4pw_2016092300_15_991_12212903_0 1,295,100 38,277 0.0296
09 May 2023 16:51:16 1524158 22313185 oifs_43r3_bl_a4pw_2016092300_15_991_12212903_0 864,000 24,992 0.0289


©2024 climateprediction.net