climateprediction.net home page
Task 22311452

Task 22311452

Name oifs_43r3_bl_a3f0_2016092300_15_991_12211215_0
Workunit 12211215
Created 13 Feb 2023, 20:09:40 UTC
Sent 14 Feb 2023, 9:23:30 UTC
Report deadline 15 Apr 2023, 9:23:30 UTC
Received 15 Feb 2023, 14:44:32 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1518460
Run time 13 hours 12 min 56 sec
CPU time 13 hours 11 min 41 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.96 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,062.75 MB
Peak swap size 5,930.78 MB
Peak disk usage 2,131.35 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
ction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3f0_2016092300_15_991_12211215_0_r887727082_3.zip
Uploading the intermediate file: upload_file_3.zip
  08:25:14 STEP  385 H=  96:15 +CPU= 58.453
  08:26:12 STEP  386 H=  96:30 +CPU= 57.852
  08:27:09 STEP  387 H=  96:45 +CPU= 56.478
  08:28:07 STEP  388 H=  97:00 +CPU= 58.461
  08:29:06 STEP  389 H=  97:15 +CPU= 59.174
  08:29:59 STEP  390 H=  97:30 +CPU= 52.747
  08:30:51 STEP  391 H=  97:45 +CPU= 51.982
  08:31:49 STEP  392 H=  98:00 +CPU= 57.417
  08:32:46 STEP  393 H=  98:15 +CPU= 57.309
  08:33:45 STEP  394 H=  98:30 +CPU= 59.492
  08:34:42 STEP  395 H=  98:45 +CPU= 56.881
  08:35:51 STEP  396 H=  99:00 +CPU= 68.907
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000396
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000396
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000396
  08:36:47 STEP  397 H=  99:15 +CPU= 55.719
  08:37:44 STEP  398 H=  99:30 +CPU= 57.138
  08:38:42 STEP  399 H=  99:45 +CPU= 57.367
  08:39:38 STEP  400 H= 100:00 +CPU= 56.371
  08:40:31 STEP  401 H= 100:15 +CPU= 52.970
  08:41:23 STEP  402 H= 100:30 +CPU= 52.170
  08:42:17 STEP  403 H= 100:45 +CPU= 53.613
  08:43:17 STEP  404 H= 101:00 +CPU= 60.774
  08:44:13 STEP  405 H= 101:15 +CPU= 55.697
  08:45:11 STEP  406 H= 101:30 +CPU= 58.172
  08:46:10 STEP  407 H= 101:45 +CPU= 59.002
  08:47:19 STEP  408 H= 102:00 +CPU= 68.270
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000408
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000408
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000408
  08:48:18 STEP  409 H= 102:15 +CPU= 59.626
  08:49:14 STEP  410 H= 102:30 +CPU= 56.085
  08:50:14 STEP  411 H= 102:45 +CPU= 59.850
  08:51:14 STEP  412 H= 103:00 +CPU= 59.736
  08:52:12 STEP  413 H= 103:15 +CPU= 58.425
  08:53:11 STEP  414 H= 103:30 +CPU= 58.771
  08:54:05 STEP  415 H= 103:45 +CPU= 53.577
  08:55:04 STEP  416 H= 104:00 +CPU= 59.269
  08:56:01 STEP  417 H= 104:15 +CPU= 57.253
  08:56:58 STEP  418 H= 104:30 +CPU= 56.294
  08:57:55 STEP  419 H= 104:45 +CPU= 57.204
  08:59:01 STEP  420 H= 105:00 +CPU= 66.147
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000420
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000420
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000420
  08:59:54 STEP  421 H= 105:15 +CPU= 53.432
  09:00:50 STEP  422 H= 105:30 +CPU= 55.524
  09:01:51 STEP  423 H= 105:45 +CPU= 60.926
  09:02:46 STEP  424 H= 106:00 +CPU= 54.800
  09:03:39 STEP  425 H= 106:15 +CPU= 53.204
  09:04:35 STEP  426 H= 106:30 +CPU= 55.724
  09:05:29 STEP  427 H= 106:45 +CPU= 54.253
  09:06:27 STEP  428 H= 107:00 +CPU= 58.549
  09:07:25 STEP  429 H= 107:15 +CPU= 57.268
  09:08:24 STEP  430 H= 107:30 +CPU= 59.072
  09:09:22 STEP  431 H= 107:45 +CPU= 58.011
  09:10:40 STEP  432 H= 108:00 +CPU= 78.492
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000432
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000432
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000432
  09:11:39 STEP  433 H= 108:15 +CPU= 58.392
  09:12:38 STEP  434 H= 108:30 +CPU= 58.759
  09:13:36 STEP  435 H= 108:45 +CPU= 58.609
  09:14:34 STEP  436 H= 109:00 +CPU= 57.799
  09:15:32 STEP  437 H= 109:15 +CPU= 58.302
  09:16:31 STEP  438 H= 109:30 +CPU= 59.036
  09:17:30 STEP  439 H= 109:45 +CPU= 59.121
  09:18:30 STEP  440 H= 110:00 +CPU= 59.302
  09:19:26 STEP  441 H= 110:15 +CPU= 56.330
  09:20:20 STEP  442 H= 110:30 +CPU= 53.864
  09:21:22 STEP  443 H= 110:45 +CPU= 61.760
  09:22:31 STEP  444 H= 111:00 +CPU= 68.830
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000444
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000444
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000444
  09:23:27 STEP  445 H= 111:15 +CPU= 55.965
  09:24:27 STEP  446 H= 111:30 +CPU= 60.229
  09:25:25 STEP  447 H= 111:45 +CPU= 58.368
  09:26:24 STEP  448 H= 112:00 +CPU= 58.785
  09:27:20 STEP  449 H= 112:15 +CPU= 56.061
  09:28:19 STEP  450 H= 112:30 +CPU= 59.367
  09:29:17 STEP  451 H= 112:45 +CPU= 57.369
  09:30:11 STEP  452 H= 113:00 +CPU= 54.044
  09:31:04 STEP  453 H= 113:15 +CPU= 52.974
  09:31:56 STEP  454 H= 113:30 +CPU= 52.518
  09:32:55 STEP  455 H= 113:45 +CPU= 58.911
  09:34:05 STEP  456 H= 114:00 +CPU= 70.012
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000456
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000456
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000456
  09:35:03 STEP  457 H= 114:15 +CPU= 58.176
  09:35:59 STEP  458 H= 114:30 +CPU= 55.948
  09:36:53 STEP  459 H= 114:45 +CPU= 53.598
  09:37:47 STEP  460 H= 115:00 +CPU= 54.122
  09:38:46 STEP  461 H= 115:15 +CPU= 58.673
  09:39:46 STEP  462 H= 115:30 +CPU= 59.746
  09:40:43 STEP  463 H= 115:45 +CPU= 57.761
  09:41:41 STEP  464 H= 116:00 +CPU= 57.755
  09:42:37 STEP  465 H= 116:15 +CPU= 56.053
  09:43:35 STEP  466 H= 116:30 +CPU= 57.741
  09:44:35 STEP  467 H= 116:45 +CPU= 59.583
  09:45:48 STEP  468 H= 117:00 +CPU= 73.512
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000468
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000468
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000468
  09:46:47 STEP  469 H= 117:15 +CPU= 58.943
  09:47:45 STEP  470 H= 117:30 +CPU= 57.692
  09:48:42 STEP  471 H= 117:45 +CPU= 57.560
  09:49:42 STEP  472 H= 118:00 +CPU= 59.300
  09:50:42 STEP  473 H= 118:15 +CPU= 60.121
  09:51:41 STEP  474 H= 118:30 +CPU= 58.763
  09:52:38 STEP  475 H= 118:45 +CPU= 57.466
  09:53:35 STEP  476 H= 119:00 +CPU= 57.381
  09:54:34 STEP  477 H= 119:15 +CPU= 58.673
  09:55:33 STEP  478 H= 119:30 +CPU= 59.129
  09:56:31 STEP  479 H= 119:45 +CPU= 57.414
  09:57:49 STEP  480 H= 120:00 +CPU= 78.702
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000480
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000480
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3f0_2016092300_15_991_12211215_0_r887727082_4.zip
Uploading the intermediate file: upload_file_4.zip
  09:58:51 STEP  481 H= 120:15 +CPU= 61.719
  09:59:46 STEP  482 H= 120:30 +CPU= 54.810
  10:00:40 STEP  483 H= 120:45 +CPU= 54.394
  10:01:38 STEP  484 H= 121:00 +CPU= 57.307
  10:02:30 STEP  485 H= 121:15 +CPU= 51.979
  10:03:23 STEP  486 H= 121:30 +CPU= 53.352
  10:04:16 STEP  487 H= 121:45 +CPU= 53.263
  10:05:12 STEP  488 H= 122:00 +CPU= 55.945
  10:06:11 STEP  489 H= 122:15 +CPU= 58.698
  10:07:08 STEP  490 H= 122:30 +CPU= 56.934
  10:08:06 STEP  491 H= 122:45 +CPU= 58.282
  10:09:12 STEP  492 H= 123:00 +CPU= 65.733
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000492
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000492
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000492
  10:10:05 STEP  493 H= 123:15 +CPU= 53.232
  10:11:06 STEP  494 H= 123:30 +CPU= 60.641
  10:11:59 STEP  495 H= 123:45 +CPU= 53.735
  10:12:53 STEP  496 H= 124:00 +CPU= 53.809
  10:13:46 STEP  497 H= 124:15 +CPU= 52.811
  10:14:40 STEP  498 H= 124:30 +CPU= 53.820
  10:15:34 STEP  499 H= 124:45 +CPU= 53.700
  10:16:32 STEP  500 H= 125:00 +CPU= 58.199
  10:17:32 STEP  501 H= 125:15 +CPU= 60.491
  10:18:33 STEP  502 H= 125:30 +CPU= 60.809
  10:19:31 STEP  503 H= 125:45 +CPU= 57.942
  10:20:40 STEP  504 H= 126:00 +CPU= 69.007
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000504
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000504
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000504
  10:21:38 STEP  505 H= 126:15 +CPU= 58.112
  10:22:36 STEP  506 H= 126:30 +CPU= 57.458
  10:23:35 STEP  507 H= 126:45 +CPU= 58.999
  10:24:33 STEP  508 H= 127:00 +CPU= 57.732
  10:25:29 STEP  509 H= 127:15 +CPU= 56.107
  10:26:29 STEP  510 H= 127:30 +CPU= 60.686
  10:27:24 STEP  511 H= 127:45 +CPU= 54.323
  10:28:24 STEP  512 H= 128:00 +CPU= 59.986
  10:29:17 STEP  513 H= 128:15 +CPU= 52.893
  10:30:10 STEP  514 H= 128:30 +CPU= 52.997
  10:31:02 STEP  515 H= 128:45 +CPU= 52.374
  10:32:05 STEP  516 H= 129:00 +CPU= 63.428
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000516
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000516
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000516
  10:32:58 STEP  517 H= 129:15 +CPU= 52.967
  10:33:50 STEP  518 H= 129:30 +CPU= 51.793
  10:34:44 STEP  519 H= 129:45 +CPU= 53.884
  10:35:37 STEP  520 H= 130:00 +CPU= 53.142
  10:36:32 STEP  521 H= 130:15 +CPU= 54.301
  10:37:27 STEP  522 H= 130:30 +CPU= 55.596
  10:38:27 STEP  523 H= 130:45 +CPU= 59.560
  10:39:25 STEP  524 H= 131:00 +CPU= 58.624
  10:40:23 STEP  525 H= 131:15 +CPU= 57.534
  10:41:20 STEP  526 H= 131:30 +CPU= 56.823
  10:42:19 STEP  527 H= 131:45 +CPU= 58.968
  10:43:38 STEP  528 H= 132:00 +CPU= 79.269
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000528
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000528
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000528
  10:44:35 STEP  529 H= 132:15 +CPU= 56.925
  10:45:32 STEP  530 H= 132:30 +CPU= 57.117
  10:46:29 STEP  531 H= 132:45 +CPU= 57.249
  10:47:27 STEP  532 H= 133:00 +CPU= 57.912
  10:48:24 STEP  533 H= 133:15 +CPU= 56.709
  10:49:24 STEP  534 H= 133:30 +CPU= 59.939
  10:50:25 STEP  535 H= 133:45 +CPU= 61.561
  10:51:24 STEP  536 H= 134:00 +CPU= 58.793
  10:52:17 STEP  537 H= 134:15 +CPU= 52.831
  10:53:12 STEP  538 H= 134:30 +CPU= 54.682
  10:54:06 STEP  539 H= 134:45 +CPU= 53.798
  10:55:13 STEP  540 H= 135:00 +CPU= 67.784
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000540
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000540
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000540
  10:56:13 STEP  541 H= 135:15 +CPU= 59.157
  10:57:09 STEP  542 H= 135:30 +CPU= 56.733
  10:58:08 STEP  543 H= 135:45 +CPU= 59.083
  10:59:03 STEP  544 H= 136:00 +CPU= 54.965
  10:59:56 STEP  545 H= 136:15 +CPU= 52.252
  11:00:49 STEP  546 H= 136:30 +CPU= 52.896
  11:01:45 STEP  547 H= 136:45 +CPU= 56.178
  11:02:41 STEP  548 H= 137:00 +CPU= 55.944
  11:03:35 STEP  549 H= 137:15 +CPU= 54.246
  11:04:30 STEP  550 H= 137:30 +CPU= 54.969
  11:05:27 STEP  551 H= 137:45 +CPU= 56.496
  11:06:38 STEP  552 H= 138:00 +CPU= 70.930
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000552
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000552
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000552
  11:07:38 STEP  553 H= 138:15 +CPU= 60.661
  11:08:36 STEP  554 H= 138:30 +CPU= 57.327
  11:09:29 STEP  555 H= 138:45 +CPU= 53.379
  11:10:25 STEP  556 H= 139:00 +CPU= 55.797
  11:11:23 STEP  557 H= 139:15 +CPU= 58.605
  11:12:24 STEP  558 H= 139:30 +CPU= 60.216
  11:13:20 STEP  559 H= 139:45 +CPU= 56.870
  11:14:16 STEP  560 H= 140:00 +CPU= 55.608
  11:15:14 STEP  561 H= 140:15 +CPU= 58.384
  11:16:13 STEP  562 H= 140:30 +CPU= 58.342
  11:17:03 STEP  563 H= 140:45 +CPU= 50.224
  11:18:05 STEP  564 H= 141:00 +CPU= 62.122
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000564
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000564
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000564
  11:19:03 STEP  565 H= 141:15 +CPU= 57.310
  11:20:02 STEP  566 H= 141:30 +CPU= 59.641
  11:21:01 STEP  567 H= 141:45 +CPU= 58.699
  11:21:59 STEP  568 H= 142:00 +CPU= 58.338
  11:22:56 STEP  569 H= 142:15 +CPU= 56.243
  11:23:49 STEP  570 H= 142:30 +CPU= 53.733
  11:24:40 STEP  571 H= 142:45 +CPU= 50.270
  11:25:33 STEP  572 H= 143:00 +CPU= 53.414
  11:26:33 STEP  573 H= 143:15 +CPU= 59.499
  11:27:33 STEP  574 H= 143:30 +CPU= 60.359
  11:28:32 STEP  575 H= 143:45 +CPU= 59.075
  11:29:52 STEP  576 H= 144:00 +CPU= 80.401
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000576
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000576
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3f0_2016092300_15_991_12211215_0_r887727082_5.zip
Uploading the intermediate file: upload_file_5.zip
  11:30:52 STEP  577 H= 144:15 +CPU= 59.034
  11:31:52 STEP  578 H= 144:30 +CPU= 60.848
  11:32:52 STEP  579 H= 144:45 +CPU= 59.435
  11:33:54 STEP  580 H= 145:00 +CPU= 61.683
  11:34:52 STEP  581 H= 145:15 +CPU= 58.281
  11:35:52 STEP  582 H= 145:30 +CPU= 59.934
  11:36:52 STEP  583 H= 145:45 +CPU= 60.232
  11:37:45 STEP  584 H= 146:00 +CPU= 53.243
  11:38:45 STEP  585 H= 146:15 +CPU= 59.435
  11:39:45 STEP  586 H= 146:30 +CPU= 60.658
  11:40:44 STEP  587 H= 146:45 +CPU= 59.035
  11:41:57 STEP  588 H= 147:00 +CPU= 72.173
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000588
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000588
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000588
  11:42:58 STEP  589 H= 147:15 +CPU= 61.295
  11:43:59 STEP  590 H= 147:30 +CPU= 60.874
  11:44:59 STEP  591 H= 147:45 +CPU= 60.689
  11:45:57 STEP  592 H= 148:00 +CPU= 57.516
  11:46:56 STEP  593 H= 148:15 +CPU= 59.050
  11:47:54 STEP  594 H= 148:30 +CPU= 57.428
  11:48:53 STEP  595 H= 148:45 +CPU= 59.502
  11:49:50 STEP  596 H= 149:00 +CPU= 56.618
  11:50:49 STEP  597 H= 149:15 +CPU= 59.340
  11:51:48 STEP  598 H= 149:30 +CPU= 59.364
  11:52:49 STEP  599 H= 149:45 +CPU= 60.713
  11:54:03 STEP  600 H= 150:00 +CPU= 74.262
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000600
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000600
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000600
  11:55:03 STEP  601 H= 150:15 +CPU= 59.994
  11:56:04 STEP  602 H= 150:30 +CPU= 60.641
  11:57:05 STEP  603 H= 150:45 +CPU= 60.590
  11:58:02 STEP  604 H= 151:00 +CPU= 57.630
  11:58:56 STEP  605 H= 151:15 +CPU= 54.209
  11:59:50 STEP  606 H= 151:30 +CPU= 53.518
  12:00:44 STEP  607 H= 151:45 +CPU= 53.876
  12:01:38 STEP  608 H= 152:00 +CPU= 54.131
  12:02:34 STEP  609 H= 152:15 +CPU= 56.202
  12:03:28 STEP  610 H= 152:30 +CPU= 53.716
  12:04:23 STEP  611 H= 152:45 +CPU= 54.745
  12:05:29 STEP  612 H= 153:00 +CPU= 65.883
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000612
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000612
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000612
  12:06:27 STEP  613 H= 153:15 +CPU= 58.710
  12:07:24 STEP  614 H= 153:30 +CPU= 56.766
  12:08:21 STEP  615 H= 153:45 +CPU= 56.743
  12:09:17 STEP  616 H= 154:00 +CPU= 55.984
  12:10:13 STEP  617 H= 154:15 +CPU= 56.369
  12:11:14 STEP  618 H= 154:30 +CPU= 60.782
  12:12:16 STEP  619 H= 154:45 +CPU= 61.581
  12:13:18 STEP  620 H= 155:00 +CPU= 62.870
  12:14:18 STEP  621 H= 155:15 +CPU= 59.502
  12:15:18 STEP  622 H= 155:30 +CPU= 60.017
  12:16:18 STEP  623 H= 155:45 +CPU= 60.383
  12:17:41 STEP  624 H= 156:00 +CPU= 82.739
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000624
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000624
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000624
  12:18:40 STEP  625 H= 156:15 +CPU= 58.813
  12:19:36 STEP  626 H= 156:30 +CPU= 56.175
  12:20:34 STEP  627 H= 156:45 +CPU= 57.384
  12:21:34 STEP  628 H= 157:00 +CPU= 60.313
  12:22:33 STEP  629 H= 157:15 +CPU= 59.275
  12:23:35 STEP  630 H= 157:30 +CPU= 62.071
  12:24:35 STEP  631 H= 157:45 +CPU= 59.703
  12:25:29 STEP  632 H= 158:00 +CPU= 53.864
  12:26:26 STEP  633 H= 158:15 +CPU= 57.229
  12:27:26 STEP  634 H= 158:30 +CPU= 59.833
  12:28:27 STEP  635 H= 158:45 +CPU= 60.704
  12:29:38 STEP  636 H= 159:00 +CPU= 71.168
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000636
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000636
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000636
  12:30:36 STEP  637 H= 159:15 +CPU= 58.485
  12:31:33 STEP  638 H= 159:30 +CPU= 56.860
  12:32:30 STEP  639 H= 159:45 +CPU= 57.284
  12:33:30 STEP  640 H= 160:00 +CPU= 59.832
  12:34:29 STEP  641 H= 160:15 +CPU= 58.746
  12:35:32 STEP  642 H= 160:30 +CPU= 63.384
  12:36:29 STEP  643 H= 160:45 +CPU= 57.012
  12:37:27 STEP  644 H= 161:00 +CPU= 57.064
  12:38:20 STEP  645 H= 161:15 +CPU= 53.235
  12:39:17 STEP  646 H= 161:30 +CPU= 56.855
  12:40:14 STEP  647 H= 161:45 +CPU= 57.274
  12:41:22 STEP  648 H= 162:00 +CPU= 67.770
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000648
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000648
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000648
  12:42:20 STEP  649 H= 162:15 +CPU= 58.105
  12:43:16 STEP  650 H= 162:30 +CPU= 56.375
  12:44:13 STEP  651 H= 162:45 +CPU= 56.576
  12:45:07 STEP  652 H= 163:00 +CPU= 54.223
  12:46:01 STEP  653 H= 163:15 +CPU= 53.932
  12:47:00 STEP  654 H= 163:30 +CPU= 59.151
  12:47:58 STEP  655 H= 163:45 +CPU= 57.543
  12:48:56 STEP  656 H= 164:00 +CPU= 58.041
  12:49:54 STEP  657 H= 164:15 +CPU= 58.425
  12:50:57 STEP  658 H= 164:30 +CPU= 63.192
  12:51:54 STEP  659 H= 164:45 +CPU= 56.209
  12:53:08 STEP  660 H= 165:00 +CPU= 74.045
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000660
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000660
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000660
  12:54:12 STEP  661 H= 165:15 +CPU= 64.585
  12:55:09 STEP  662 H= 165:30 +CPU= 56.862
  12:56:06 STEP  663 H= 165:45 +CPU= 56.479
  12:57:03 STEP  664 H= 166:00 +CPU= 57.324
  12:58:00 STEP  665 H= 166:15 +CPU= 57.438
  12:58:55 STEP  666 H= 166:30 +CPU= 54.267
  12:59:50 STEP  667 H= 166:45 +CPU= 55.144
  13:00:45 STEP  668 H= 167:00 +CPU= 55.082
  13:01:40 STEP  669 H= 167:15 +CPU= 55.606
  13:02:41 STEP  670 H= 167:30 +CPU= 60.269
  13:03:38 STEP  671 H= 167:45 +CPU= 57.443
  13:04:57 STEP  672 H= 168:00 +CPU= 78.657
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000672
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000672
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3f0_2016092300_15_991_12211215_0_r887727082_6.zip
Uploading the intermediate file: upload_file_6.zip
  13:05:53 STEP  673 H= 168:15 +CPU= 56.471
  13:06:54 STEP  674 H= 168:30 +CPU= 60.809
  13:07:56 STEP  675 H= 168:45 +CPU= 62.188
  13:09:00 STEP  676 H= 169:00 +CPU= 63.403
  13:10:02 STEP  677 H= 169:15 +CPU= 62.297
  13:11:02 STEP  678 H= 169:30 +CPU= 59.938
  13:12:03 STEP  679 H= 169:45 +CPU= 61.093
  13:13:05 STEP  680 H= 170:00 +CPU= 61.649
  13:14:06 STEP  681 H= 170:15 +CPU= 61.500
  13:15:07 STEP  682 H= 170:30 +CPU= 60.302
  13:16:08 STEP  683 H= 170:45 +CPU= 61.071
  13:17:15 STEP  684 H= 171:00 +CPU= 67.408
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000684
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000684
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000684
  13:18:13 STEP  685 H= 171:15 +CPU= 57.635
  13:19:12 STEP  686 H= 171:30 +CPU= 59.732
  13:20:09 STEP  687 H= 171:45 +CPU= 56.771
  13:21:11 STEP  688 H= 172:00 +CPU= 61.946
  13:22:11 STEP  689 H= 172:15 +CPU= 59.932
  13:23:13 STEP  690 H= 172:30 +CPU= 62.125
  13:24:16 STEP  691 H= 172:45 +CPU= 62.388
  13:25:15 STEP  692 H= 173:00 +CPU= 58.762
  13:26:08 STEP  693 H= 173:15 +CPU= 53.324
  13:27:05 STEP  694 H= 173:30 +CPU= 57.497
  13:28:05 STEP  695 H= 173:45 +CPU= 59.147
  13:29:11 STEP  696 H= 174:00 +CPU= 66.202
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000696
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000696
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000696
  13:30:12 STEP  697 H= 174:15 +CPU= 61.720
  13:31:15 STEP  698 H= 174:30 +CPU= 62.613
  13:32:17 STEP  699 H= 174:45 +CPU= 62.203
  13:33:20 STEP  700 H= 175:00 +CPU= 62.980
  13:34:21 STEP  701 H= 175:15 +CPU= 60.431
  13:35:16 STEP  702 H= 175:30 +CPU= 55.564
  13:36:13 STEP  703 H= 175:45 +CPU= 56.846
  13:37:11 STEP  704 H= 176:00 +CPU= 58.040
  13:38:10 STEP  705 H= 176:15 +CPU= 58.871
  13:39:01 STEP  706 H= 176:30 +CPU= 51.107
  13:39:57 STEP  707 H= 176:45 +CPU= 56.255
  13:41:02 STEP  708 H= 177:00 +CPU= 64.960
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000708
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000708
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000708
  13:41:57 STEP  709 H= 177:15 +CPU= 54.535
  13:42:52 STEP  710 H= 177:30 +CPU= 55.415
  13:43:48 STEP  711 H= 177:45 +CPU= 55.742
  13:44:49 STEP  712 H= 178:00 +CPU= 61.176
  13:45:50 STEP  713 H= 178:15 +CPU= 60.866
  13:46:47 STEP  714 H= 178:30 +CPU= 57.234
  13:47:48 STEP  715 H= 178:45 +CPU= 60.226
  13:48:49 STEP  716 H= 179:00 +CPU= 60.930
  13:49:49 STEP  717 H= 179:15 +CPU= 60.369
  13:50:50 STEP  718 H= 179:30 +CPU= 61.307
  13:51:54 STEP  719 H= 179:45 +CPU= 63.327
  13:53:12 STEP  720 H= 180:00 +CPU= 78.562
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000720
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000720
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000720
  13:54:08 STEP  721 H= 180:15 +CPU= 56.009
  13:55:11 STEP  722 H= 180:30 +CPU= 62.256
  13:56:13 STEP  723 H= 180:45 +CPU= 62.556
  13:57:12 STEP  724 H= 181:00 +CPU= 58.506
  13:58:06 STEP  725 H= 181:15 +CPU= 54.014
  13:59:00 STEP  726 H= 181:30 +CPU= 54.629
  13:59:56 STEP  727 H= 181:45 +CPU= 55.761
  14:00:53 STEP  728 H= 182:00 +CPU= 57.135
  14:01:52 STEP  729 H= 182:15 +CPU= 58.511
  14:02:53 STEP  730 H= 182:30 +CPU= 60.858
  14:03:52 STEP  731 H= 182:45 +CPU= 59.577
  14:05:00 STEP  732 H= 183:00 +CPU= 67.398
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000732
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000732
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000732
  14:05:55 STEP  733 H= 183:15 +CPU= 55.235
  14:06:52 STEP  734 H= 183:30 +CPU= 57.338
  14:07:50 STEP  735 H= 183:45 +CPU= 58.086
  14:08:53 STEP  736 H= 184:00 +CPU= 62.494
  14:09:48 STEP  737 H= 184:15 +CPU= 55.458
  14:10:41 STEP  738 H= 184:30 +CPU= 53.241
  14:11:35 STEP  739 H= 184:45 +CPU= 53.384
  14:12:29 STEP  740 H= 185:00 +CPU= 54.125
  14:13:23 STEP  741 H= 185:15 +CPU= 53.959
  14:14:19 STEP  742 H= 185:30 +CPU= 55.788
  14:15:16 STEP  743 H= 185:45 +CPU= 57.376
  14:16:26 STEP  744 H= 186:00 +CPU= 69.542
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000744
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000744
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000744
  14:17:26 STEP  745 H= 186:15 +CPU= 60.450
  14:18:26 STEP  746 H= 186:30 +CPU= 60.085
  14:19:23 STEP  747 H= 186:45 +CPU= 57.154
  14:20:21 STEP  748 H= 187:00 +CPU= 57.273
  14:21:21 STEP  749 H= 187:15 +CPU= 60.133
  14:22:16 STEP  750 H= 187:30 +CPU= 54.612
  14:23:08 STEP  751 H= 187:45 +CPU= 52.343
  14:24:05 STEP  752 H= 188:00 +CPU= 56.878
  14:25:03 STEP  753 H= 188:15 +CPU= 58.137
  14:26:02 STEP  754 H= 188:30 +CPU= 58.817
  14:26:56 STEP  755 H= 188:45 +CPU= 54.143
  14:28:02 STEP  756 H= 189:00 +CPU= 66.270
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000756
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000756
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000756
  14:28:57 STEP  757 H= 189:15 +CPU= 54.674
  14:29:51 STEP  758 H= 189:30 +CPU= 54.547
  14:30:47 STEP  759 H= 189:45 +CPU= 55.434
  14:31:45 STEP  760 H= 190:00 +CPU= 58.402
  14:32:46 STEP  761 H= 190:15 +CPU= 61.113
  14:33:45 STEP  762 H= 190:30 +CPU= 58.632
  14:34:42 STEP  763 H= 190:45 +CPU= 57.375
  14:35:39 STEP  764 H= 191:00 +CPU= 56.796
  14:36:34 STEP  765 H= 191:15 +CPU= 54.577
  14:37:28 STEP  766 H= 191:30 +CPU= 53.767
  14:38:20 STEP  767 H= 191:45 +CPU= 52.835
  14:39:37 STEP  768 H= 192:00 +CPU= 76.145
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMGGh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMSHh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12211215/ICMUAh7zg+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a3f0_2016092300_15_991_12211215_0_r887727082_7.zip
Uploading the intermediate file: upload_file_7.zip
  14:40:31 STEP  769 H= 192:15 +CPU= 53.787
  14:41:23 STEP  770 H= 192:30 +CPU= 52.159
  14:42:14 STEP  771 H= 192:45 +CPU= 51.137
  14:43:07 STEP  772 H= 193:00 +CPU= 53.013
  14:44:00 STEP  773 H= 193:15 +CPU= 53.306
  14:44:56 STEP  774 H= 193:30 +CPU= 55.864
  14:45:55 STEP  775 H= 193:45 +CPU= 58.741
  14:46:52 STEP  776 H= 194:00 +CPU= 56.648
  14:47:50 STEP  777 H= 194:15 +CPU= 58.363
  14:48:50 STEP  778 H= 194:30 +CPU= 60.042
  14:49:48 STEP  779 H= 194:45 +CPU= 58.404
  14:50:54 STEP  780 H= 195:00 +CPU= 65.680
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000780
  14:51:49 STEP  781 H= 195:15 +CPU= 54.790
  14:52:47 STEP  782 H= 195:30 +CPU= 57.715
  14:53:45 STEP  783 H= 195:45 +CPU= 58.396
  14:54:42 STEP  784 H= 196:00 +CPU= 57.497
  14:55:37 STEP  785 H= 196:15 +CPU= 54.955
  14:56:33 STEP  786 H= 196:30 +CPU= 55.296
  14:57:29 STEP  787 H= 196:45 +CPU= 56.034
  14:58:28 STEP  788 H= 197:00 +CPU= 58.797
  14:59:24 STEP  789 H= 197:15 +CPU= 56.668
  15:00:18 STEP  790 H= 197:30 +CPU= 54.217
  15:01:18 STEP  791 H= 197:45 +CPU= 59.630
  15:02:18 STEP  792 H= 198:00 +CPU= 59.953
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000792
  15:03:12 STEP  793 H= 198:15 +CPU= 54.193
  15:04:11 STEP  794 H= 198:30 +CPU= 59.138
  15:05:06 STEP  795 H= 198:45 +CPU= 54.423
  15:06:03 STEP  796 H= 199:00 +CPU= 57.025
  15:06:52 STEP  797 H= 199:15 +CPU= 49.428
  15:07:47 STEP  798 H= 199:30 +CPU= 54.458
  15:08:41 STEP  799 H= 199:45 +CPU= 54.155
  15:09:29 STEP  800 H= 200:00 +CPU= 48.481
  15:10:20 STEP  801 H= 200:15 +CPU= 50.216
  15:11:13 STEP  802 H= 200:30 +CPU= 53.007
  15:12:09 STEP  803 H= 200:45 +CPU= 56.380
  15:13:10 STEP  804 H= 201:00 +CPU= 60.560
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMGGh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMSHh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/57/ICMUAh7zg+000804
  15:14:01 STEP  805 H= 201:15 +CPU= 51.806
  15:14:54 STEP  806 H= 201:30 +CPU= 52.286
  15:15:48 STEP  807 H= 201:45 +CPU= 53.835
  15:16:42 STEP  808 H= 202:00 +CPU= 54.357
  15:17:37 STEP  809 H= 202:15 +CPU= 55.428
  15:18:32 STEP  810 H= 202:30 +CPU= 54.559
  15:19:25 STEP  811 H= 202:45 +CPU= 53.167
  15:20:18 STEP  812 H= 203:00 +CPU= 52.455
  15:21:09 STEP  813 H= 203:15 +CPU= 51.723
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.642] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4362MB (heap), 5067MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.643] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.643] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f0c0 (old at (nil))
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.643] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc4a8f], nsigs = 1
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.643] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.643] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3843] 4362 MB (maxheap), 5067 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.643] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152138:1676470898:47510.644] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/57/oifs_43r3_model.exe' : sigcontextptr=0x7ffd9b23edd0
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with plain hex-dump
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [0]: [0x838803] : 0x838803
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [1]: [0x838e09] : 0x838e09
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [2]: [0x1d8bb20] : 0x1d8bb20
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [3]: [0x1cc4a8f] : 0x1cc4a8f
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [4]: [0x12d283b] : 0x12d283b
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [5]: [0x12ba8d9] : 0x12ba8d9
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [6]: [0x12b0762] : 0x12b0762
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [7]: [0x1163fc3] : 0x1163fc3
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [8]: [0x10b682f] : 0x10b682f
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [9]: [0x10a915c] : 0x10a915c
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [10]: [0xe680c3] : 0xe680c3
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [11]: [0x15c1e62] : 0x15c1e62
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [12]: [0x15ac40e] : 0x15ac40e
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [13]: [0x6620b4] : 0x6620b4
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [14]: [0x4a0e7c] : 0x4a0e7c
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [15]: [0x49dc29] : 0x49dc29
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [16]: [0x49ce72] : 0x49ce72
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [17]: [0x49cd68] : 0x49cd68
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [18]: [0x4078ad] : 0x4078ad
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [19]: [0x40276f] : 0x40276f
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [20]: [0x40270d] : 0x40270d
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [21]: [0x1d8287a] : 0x1d8287a
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [22]: [0x1d840d7] : 0x1d840d7
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [23]: [0x4025e5] : 0x4025e5
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.644] [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/57/oifs_43r3_model.exe' : sigcontextptr=0x7ffd9b23ece0
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with plain hex-dump
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [0]: [0x838803] : 0x838803
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [1]: [0x838e09] : 0x838e09
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [2]: [0x820d70] : 0x820d70
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [3]: [0x1d8bb20] : 0x1d8bb20
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [4]: [0x1cc4a8f] : 0x1cc4a8f
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [5]: [0x12d283b] : 0x12d283b
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [6]: [0x12ba8d9] : 0x12ba8d9
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [7]: [0x12b0762] : 0x12b0762
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [8]: [0x1163fc3] : 0x1163fc3
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [9]: [0x10b682f] : 0x10b682f
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [10]: [0x10a915c] : 0x10a915c
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [11]: [0xe680c3] : 0xe680c3
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [12]: [0x15c1e62] : 0x15c1e62
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [13]: [0x15ac40e] : 0x15ac40e
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [14]: [0x6620b4] : 0x6620b4
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [15]: [0x4a0e7c] : 0x4a0e7c
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [16]: [0x49dc29] : 0x49dc29
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [17]: [0x49ce72] : 0x49ce72
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [18]: [0x49cd68] : 0x49cd68
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [19]: [0x4078ad] : 0x4078ad
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [20]: [0x40276f] : 0x40276f
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [21]: [0x40270d] : 0x40270d
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [22]: [0x1d8287a] : 0x1d8287a
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [23]: [0x1d840d7] : 0x1d840d7
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:340] [24]: [0x4025e5] : 0x4025e5
  [LinuxTraceBack@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.654] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:Dinger:1:1:192592:192592] [20230215:152139:1676470899:47511.654] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC4A8F  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000012D283B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  00000000012BA8D9  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000012B0762  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000001163FC3  callpar_                  869  callpar.F90
oifs_43r3_model.e  00000000010B682F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  00000000010A915C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000E680C3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.840293516904891E-05 0.000000000000000E+00 0.180736038545598E-01
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.845296391338260E-05 0.000000000000000E+00 0.631615919991013E-02
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.196132540362789E-05 0.000000000000000E+00 0.133933329979389E-01
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.955698622155647E-06 0.000000000000000E+00 0.956200192427336E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.140933078245288E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =   804 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   804 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000804 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000804
 MPI-TASK:   1 -    324141674 BYTES IN      6 RECORDS TRANSFERRED IN    0.0010 SECONDS324141.7 Mbytes/s, TOTAL TIME=    0.0171( 5.8%)
 NSTEP =   804 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000804 MODE=a
 IO-STREAM CLOSED - ./ICMSHh7zg+000804
 MPI-TASK:   1 -    707765250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0255 SECONDS 27755.5 Mbytes/s, TOTAL TIME=    0.3381( 7.5%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000804 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000804
 MPI-TASK:   1 -   5186266775 BYTES IN    125 RECORDS TRANSFERRED IN    0.0160 SECONDS324141.7 Mbytes/s, TOTAL TIME=    0.2855( 5.6%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000804 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000804
 MPI-TASK:   1 -    315568041 BYTES IN      2 RECORDS TRANSFERRED IN    0.0004 SECONDS788920.1 Mbytes/s, TOTAL TIME=    0.0067( 6.0%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000804 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000804
 MPI-TASK:   1 -    324141674 BYTES IN      4 RECORDS TRANSFERRED IN    0.0007 SECONDS463059.5 Mbytes/s, TOTAL TIME=    0.0130( 5.4%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   804 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   804 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000804 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000804
 MPI-TASK:   1 -    324141665 BYTES IN      7 RECORDS TRANSFERRED IN    0.0012 SECONDS270118.1 Mbytes/s, TOTAL TIME=    0.0178( 6.7%)
 15:13:10 STEP  804 H= 201:00 +CPU= 60.560
ISTEP=     805 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     805 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   805 STEPO   0AAA00AAA
 15:14:01 STEP  805 H= 201:15 +CPU= 51.806
 NSTEP =   806 STEPO   0AAA00AAA
 15:14:54 STEP  806 H= 201:30 +CPU= 52.286
 NSTEP =   807 STEPO   0AAA00AAA
 15:15:48 STEP  807 H= 201:45 +CPU= 53.835
 NSTEP =   808 STEPO   0AAA00AAA
 15:16:42 STEP  808 H= 202:00 +CPU= 54.357
 NSTEP =   809 STEPO   0AAA00AAA
 15:17:37 STEP  809 H= 202:15 +CPU= 55.428
ISTEP=     810 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     810 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   810 STEPO   0AAA00AAA
 15:18:32 STEP  810 H= 202:30 +CPU= 54.559
 NSTEP =   811 STEPO   0AAA00AAA
 15:19:25 STEP  811 H= 202:45 +CPU= 53.167
 NSTEP =   812 STEPO   0AAA00AAA
 15:20:18 STEP  812 H= 203:00 +CPU= 52.455
 NSTEP =   813 STEPO   0AAA00AAA
 15:21:09 STEP  813 H= 203:15 +CPU= 51.723
 NSTEP =   814 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 15:15:48 0AAA00AAA STEPO      808    53.836   53.836   53.842    785:51    786:00 0.25329051571174E-04       2GB       0MB
 15:16:42 0AAA00AAA STEPO      809    54.357   54.357   54.366    786:45    786:54 0.25329051571174E-04       2GB       0MB
 15:17:37 0AAA00AAA STEPO      810    55.429   55.429   55.433    787:40    787:50 0.25329051571174E-04       2GB       0MB
 15:18:32 0AAA00AAA STEPO      811    54.559   54.559   54.569    788:35    788:44 0.25329051571174E-04       2GB       0MB
 15:19:25 0AAA00AAA STEPO      812    53.168   53.168   53.173    789:28    789:38 0.25329051571174E-04       2GB       0MB
 15:20:18 0AAA00AAA STEPO      813    52.455   52.455   52.465    790:21    790:30 0.25329051571174E-04       2GB       0MB
 15:21:09 0AAA00AAA STEPO      814    51.723   51.723   51.728    791:12    791:22 0.25329051571174E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     768",
 CTIME   = "00080000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         756, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/57/progress_file_12211215.xml

<running_values>
  <last_cpu_time>47500.360000</last_cpu_time>
  <upload_file_number>8</upload_file_number>
  <last_iter>814</last_iter>
  <last_upload>691200</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net