climateprediction.net home page
Task 22295392

Task 22295392

Name oifs_43r3_ps_0121_1992050100_123_961_12177765_1
Workunit 12177765
Created 3 Jan 2023, 22:43:42 UTC
Sent 21 Jan 2023, 10:17:41 UTC
Report deadline 22 Mar 2023, 10:17:41 UTC
Received 26 Jan 2023, 3:27:45 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1517434
Run time 1 days 0 hours 52 min 35 sec
CPU time 22 hours 22 min 55 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 4.91 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,392.21 MB
Peak swap size 4,978.07 MB
Peak disk usage 3,613.30 MB
Stderr
<core_client_version>7.9.3</core_client_version>
<![CDATA[
<stderr_txt>
TEP 2413 H=2413:00 +CPU= 26.521
  23:11:49 STEP 2414 H=2414:00 +CPU= 19.705
  23:12:17 STEP 2415 H=2415:00 +CPU= 28.096
  23:12:36 STEP 2416 H=2416:00 +CPU= 19.193
  23:12:54 STEP 2417 H=2417:00 +CPU= 17.758
  23:13:22 STEP 2418 H=2418:00 +CPU= 28.729
  23:13:41 STEP 2419 H=2419:00 +CPU= 18.108
  23:13:59 STEP 2420 H=2420:00 +CPU= 18.042
  23:14:26 STEP 2421 H=2421:00 +CPU= 27.828
  23:14:45 STEP 2422 H=2422:00 +CPU= 18.173
  23:15:03 STEP 2423 H=2423:00 +CPU= 18.507
  23:15:54 STEP 2424 H=2424:00 +CPU= 49.234
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_100.zip
Uploading the intermediate file: upload_file_100.zip
  23:16:21 STEP 2425 H=2425:00 +CPU= 27.372
  23:16:49 STEP 2426 H=2426:00 +CPU= 28.186
  23:17:27 STEP 2427 H=2427:00 +CPU= 37.929
  23:17:46 STEP 2428 H=2428:00 +CPU= 18.663
  23:18:04 STEP 2429 H=2429:00 +CPU= 17.963
  23:18:48 STEP 2430 H=2430:00 +CPU= 43.804
  23:19:16 STEP 2431 H=2431:00 +CPU= 27.790
  23:19:44 STEP 2432 H=2432:00 +CPU= 28.460
  23:20:26 STEP 2433 H=2433:00 +CPU= 42.029
  23:20:55 STEP 2434 H=2434:00 +CPU= 28.495
  23:21:22 STEP 2435 H=2435:00 +CPU= 27.776
  23:22:10 STEP 2436 H=2436:00 +CPU= 47.416
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002436
  23:22:33 STEP 2437 H=2437:00 +CPU= 23.060
  23:22:51 STEP 2438 H=2438:00 +CPU= 18.268
  23:23:26 STEP 2439 H=2439:00 +CPU= 34.624
  23:23:53 STEP 2440 H=2440:00 +CPU= 27.245
  23:24:20 STEP 2441 H=2441:00 +CPU= 26.923
  23:25:02 STEP 2442 H=2442:00 +CPU= 42.178
  23:25:29 STEP 2443 H=2443:00 +CPU= 26.555
  23:25:56 STEP 2444 H=2444:00 +CPU= 27.518
  23:26:24 STEP 2445 H=2445:00 +CPU= 28.242
  23:26:51 STEP 2446 H=2446:00 +CPU= 26.785
  23:27:19 STEP 2447 H=2447:00 +CPU= 28.218
  23:28:08 STEP 2448 H=2448:00 +CPU= 48.634
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_101.zip
Uploading the intermediate file: upload_file_101.zip
  23:28:35 STEP 2449 H=2449:00 +CPU= 26.325
  23:29:02 STEP 2450 H=2450:00 +CPU= 27.612
  23:29:44 STEP 2451 H=2451:00 +CPU= 40.878
  23:30:11 STEP 2452 H=2452:00 +CPU= 27.032
  23:30:38 STEP 2453 H=2453:00 +CPU= 27.544
  23:31:21 STEP 2454 H=2454:00 +CPU= 42.984
  23:31:49 STEP 2455 H=2455:00 +CPU= 27.494
  23:32:17 STEP 2456 H=2456:00 +CPU= 28.349
  23:32:58 STEP 2457 H=2457:00 +CPU= 41.138
  23:33:25 STEP 2458 H=2458:00 +CPU= 27.024
  23:33:51 STEP 2459 H=2459:00 +CPU= 26.172
  23:34:37 STEP 2460 H=2460:00 +CPU= 45.736
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002460
  23:35:03 STEP 2461 H=2461:00 +CPU= 26.031
  23:35:32 STEP 2462 H=2462:00 +CPU= 28.661
  23:36:14 STEP 2463 H=2463:00 +CPU= 42.061
  23:36:42 STEP 2464 H=2464:00 +CPU= 28.437
  23:37:10 STEP 2465 H=2465:00 +CPU= 27.441
  23:37:53 STEP 2466 H=2466:00 +CPU= 42.751
  23:38:20 STEP 2467 H=2467:00 +CPU= 27.084
  23:38:47 STEP 2468 H=2468:00 +CPU= 27.471
  23:39:29 STEP 2469 H=2469:00 +CPU= 41.369
  23:39:50 STEP 2470 H=2470:00 +CPU= 20.825
  23:40:08 STEP 2471 H=2471:00 +CPU= 18.867
  23:40:59 STEP 2472 H=2472:00 +CPU= 50.180
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_102.zip
Uploading the intermediate file: upload_file_102.zip
  23:41:26 STEP 2473 H=2473:00 +CPU= 27.025
  23:41:51 STEP 2474 H=2474:00 +CPU= 24.638
  23:42:18 STEP 2475 H=2475:00 +CPU= 27.755
  23:42:37 STEP 2476 H=2476:00 +CPU= 18.315
  23:42:54 STEP 2477 H=2477:00 +CPU= 17.167
  23:43:21 STEP 2478 H=2478:00 +CPU= 27.447
  23:43:39 STEP 2479 H=2479:00 +CPU= 17.372
  23:43:57 STEP 2480 H=2480:00 +CPU= 18.066
  23:44:28 STEP 2481 H=2481:00 +CPU= 31.327
  23:44:55 STEP 2482 H=2482:00 +CPU= 27.225
  23:45:13 STEP 2483 H=2483:00 +CPU= 17.704
  23:45:43 STEP 2484 H=2484:00 +CPU= 30.089
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002484
  23:46:00 STEP 2485 H=2485:00 +CPU= 17.341
  23:46:18 STEP 2486 H=2486:00 +CPU= 17.504
  23:46:46 STEP 2487 H=2487:00 +CPU= 27.572
  23:47:04 STEP 2488 H=2488:00 +CPU= 18.099
  23:47:29 STEP 2489 H=2489:00 +CPU= 25.285
  23:47:57 STEP 2490 H=2490:00 +CPU= 27.548
  23:48:14 STEP 2491 H=2491:00 +CPU= 17.826
  23:48:32 STEP 2492 H=2492:00 +CPU= 17.930
  23:49:00 STEP 2493 H=2493:00 +CPU= 27.284
  23:49:17 STEP 2494 H=2494:00 +CPU= 17.711
  23:49:45 STEP 2495 H=2495:00 +CPU= 27.827
  23:50:33 STEP 2496 H=2496:00 +CPU= 48.024
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_103.zip
Uploading the intermediate file: upload_file_103.zip
  23:51:01 STEP 2497 H=2497:00 +CPU= 27.255
  23:51:28 STEP 2498 H=2498:00 +CPU= 27.239
  23:52:10 STEP 2499 H=2499:00 +CPU= 41.805
  23:52:37 STEP 2500 H=2500:00 +CPU= 27.280
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1992050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0121
(argv4) batchid: 961
(argv5) wuid: 12177765
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8d1c17390c90e2e9d39ee85aa332604e to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0121_1992050100_123_961_12177765.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0121_1992050100_123_961_12177765.zip
ic_ancil_file: ic_ancil_12177765
ifsdata_file: ifsdata_12177765
climate_data_file: clim_data_12177765
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a9e0670ab83217debb5e37a2126dd437 to: /var/lib/boinc-client/slots/4/ic_ancil_12177765.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12177765.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3ba38a9a102552e6a5e4e08bc9b7d3a0 to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12177765.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12177765.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12177765.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12177765.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12177765.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12177765.xml
-- Model is restarting --
Adjusting last_iter, 2501, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12177765.xml
last_cpu_time: 67844
upload_file_number: 104
last_iter: 2495
last_upload: 8985600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465
The child process has been launched with process id: 25151
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ewen:1:1:25151:25151] [20230122:015852:1674381532:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  02:00:29 STEP 2496 H=2496:00 +CPU= 56.693
  02:01:07 STEP 2497 H=2497:00 +CPU= 37.369
  02:02:04 STEP 2498 H=2498:00 +CPU= 31.313
  02:02:49 STEP 2499 H=2499:00 +CPU= 41.808
  02:03:11 STEP 2500 H=2500:00 +CPU= 21.850
  02:03:32 STEP 2501 H=2501:00 +CPU= 21.139
  02:04:23 STEP 2502 H=2502:00 +CPU= 51.063
  02:04:57 STEP 2503 H=2503:00 +CPU= 33.183
  02:05:31 STEP 2504 H=2504:00 +CPU= 34.044
  02:06:21 STEP 2505 H=2505:00 +CPU= 50.103
  02:06:54 STEP 2506 H=2506:00 +CPU= 33.301
  02:07:19 STEP 2507 H=2507:00 +CPU= 24.968
  02:08:17 STEP 2508 H=2508:00 +CPU= 56.997
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002508
  02:08:51 STEP 2509 H=2509:00 +CPU= 33.685
  02:09:24 STEP 2510 H=2510:00 +CPU= 33.812
  02:10:15 STEP 2511 H=2511:00 +CPU= 50.124
  02:10:51 STEP 2512 H=2512:00 +CPU= 33.941
  02:11:13 STEP 2513 H=2513:00 +CPU= 21.515
  02:11:47 STEP 2514 H=2514:00 +CPU= 34.753
  02:12:14 STEP 2515 H=2515:00 +CPU= 26.597
  02:12:34 STEP 2516 H=2516:00 +CPU= 20.212
  02:13:13 STEP 2517 H=2517:00 +CPU= 38.505
  02:13:45 STEP 2518 H=2518:00 +CPU= 32.213
  02:14:19 STEP 2519 H=2519:00 +CPU= 33.821
  02:15:01 STEP 2520 H=2520:00 +CPU= 40.333
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_104.zip
Uploading the intermediate file: upload_file_104.zip
  02:15:33 STEP 2521 H=2521:00 +CPU= 30.657
  02:16:04 STEP 2522 H=2522:00 +CPU= 30.873
  02:16:54 STEP 2523 H=2523:00 +CPU= 50.250
  02:17:31 STEP 2524 H=2524:00 +CPU= 36.826
  02:18:00 STEP 2525 H=2525:00 +CPU= 29.366
  02:18:34 STEP 2526 H=2526:00 +CPU= 33.809
  02:18:56 STEP 2527 H=2527:00 +CPU= 22.228
  02:19:19 STEP 2528 H=2528:00 +CPU= 22.665
  02:19:51 STEP 2529 H=2529:00 +CPU= 32.502
  02:20:13 STEP 2530 H=2530:00 +CPU= 21.531
  02:20:35 STEP 2531 H=2531:00 +CPU= 22.173
  02:21:11 STEP 2532 H=2532:00 +CPU= 36.082
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002532
  02:21:33 STEP 2533 H=2533:00 +CPU= 21.745
  02:21:55 STEP 2534 H=2534:00 +CPU= 22.344
  02:22:39 STEP 2535 H=2535:00 +CPU= 43.664
  02:23:09 STEP 2536 H=2536:00 +CPU= 30.032
  02:23:43 STEP 2537 H=2537:00 +CPU= 33.474
  02:24:35 STEP 2538 H=2538:00 +CPU= 51.878
  02:25:03 STEP 2539 H=2539:00 +CPU= 28.268
  02:25:22 STEP 2540 H=2540:00 +CPU= 19.631
  02:25:53 STEP 2541 H=2541:00 +CPU= 30.699
  02:26:16 STEP 2542 H=2542:00 +CPU= 22.499
  02:26:37 STEP 2543 H=2543:00 +CPU= 21.426
  02:27:37 STEP 2544 H=2544:00 +CPU= 56.295
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_105.zip
Uploading the intermediate file: upload_file_105.zip
  02:27:58 STEP 2545 H=2545:00 +CPU= 21.311
  02:28:26 STEP 2546 H=2546:00 +CPU= 28.443
  02:28:57 STEP 2547 H=2547:00 +CPU= 30.618
  02:29:17 STEP 2548 H=2548:00 +CPU= 19.773
  02:29:35 STEP 2549 H=2549:00 +CPU= 18.685
  02:30:05 STEP 2550 H=2550:00 +CPU= 29.758
  02:30:24 STEP 2551 H=2551:00 +CPU= 18.992
  02:30:45 STEP 2552 H=2552:00 +CPU= 20.445
  02:31:14 STEP 2553 H=2553:00 +CPU= 29.870
  02:31:34 STEP 2554 H=2554:00 +CPU= 19.528
  02:31:53 STEP 2555 H=2555:00 +CPU= 19.142
  02:32:26 STEP 2556 H=2556:00 +CPU= 32.569
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002556
  02:32:45 STEP 2557 H=2557:00 +CPU= 19.118
  02:33:05 STEP 2558 H=2558:00 +CPU= 20.454
  02:33:38 STEP 2559 H=2559:00 +CPU= 32.908
  02:34:10 STEP 2560 H=2560:00 +CPU= 32.100
  02:34:41 STEP 2561 H=2561:00 +CPU= 30.625
  02:35:26 STEP 2562 H=2562:00 +CPU= 45.339
  02:35:55 STEP 2563 H=2563:00 +CPU= 29.019
  02:36:28 STEP 2564 H=2564:00 +CPU= 33.139
  02:37:04 STEP 2565 H=2565:00 +CPU= 35.672
  02:37:42 STEP 2566 H=2566:00 +CPU= 24.853
  02:38:06 STEP 2567 H=2567:00 +CPU= 22.657
  02:38:53 STEP 2568 H=2568:00 +CPU= 38.326
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_106.zip
Uploading the intermediate file: upload_file_106.zip
  02:39:10 STEP 2569 H=2569:00 +CPU= 17.326
  02:39:28 STEP 2570 H=2570:00 +CPU= 18.200
  02:39:57 STEP 2571 H=2571:00 +CPU= 28.260
  02:40:15 STEP 2572 H=2572:00 +CPU= 18.752
  02:40:33 STEP 2573 H=2573:00 +CPU= 18.157
  02:41:04 STEP 2574 H=2574:00 +CPU= 30.360
  02:41:30 STEP 2575 H=2575:00 +CPU= 25.787
  02:42:07 STEP 2576 H=2576:00 +CPU= 27.822
  02:42:48 STEP 2577 H=2577:00 +CPU= 39.956
  02:43:13 STEP 2578 H=2578:00 +CPU= 25.011
  02:43:31 STEP 2579 H=2579:00 +CPU= 18.082
  02:44:18 STEP 2580 H=2580:00 +CPU= 46.823
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002580
  02:44:41 STEP 2581 H=2581:00 +CPU= 22.582
  02:44:59 STEP 2582 H=2582:00 +CPU= 18.112
  02:45:31 STEP 2583 H=2583:00 +CPU= 32.811
  02:45:50 STEP 2584 H=2584:00 +CPU= 18.938
  02:46:08 STEP 2585 H=2585:00 +CPU= 17.958
  02:46:36 STEP 2586 H=2586:00 +CPU= 28.067
  02:46:55 STEP 2587 H=2587:00 +CPU= 18.173
  02:47:13 STEP 2588 H=2588:00 +CPU= 18.651
  02:47:42 STEP 2589 H=2589:00 +CPU= 29.134
  02:48:10 STEP 2590 H=2590:00 +CPU= 27.918
  02:48:38 STEP 2591 H=2591:00 +CPU= 28.043
  02:49:29 STEP 2592 H=2592:00 +CPU= 48.044
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_107.zip
Uploading the intermediate file: upload_file_107.zip
  02:49:58 STEP 2593 H=2593:00 +CPU= 28.517
  02:50:28 STEP 2594 H=2594:00 +CPU= 29.764
  02:50:57 STEP 2595 H=2595:00 +CPU= 29.508
  02:51:16 STEP 2596 H=2596:00 +CPU= 18.602
  02:51:34 STEP 2597 H=2597:00 +CPU= 18.596
  02:52:03 STEP 2598 H=2598:00 +CPU= 29.007
  02:52:21 STEP 2599 H=2599:00 +CPU= 17.347
  02:52:39 STEP 2600 H=2600:00 +CPU= 18.621
  02:53:11 STEP 2601 H=2601:00 +CPU= 32.025
  02:53:39 STEP 2602 H=2602:00 +CPU= 28.146
  02:54:07 STEP 2603 H=2603:00 +CPU= 27.959
  02:54:49 STEP 2604 H=2604:00 +CPU= 40.309
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002604
  02:55:17 STEP 2605 H=2605:00 +CPU= 28.325
  02:55:45 STEP 2606 H=2606:00 +CPU= 28.044
  02:56:21 STEP 2607 H=2607:00 +CPU= 36.385
  02:56:43 STEP 2608 H=2608:00 +CPU= 21.162
  02:57:02 STEP 2609 H=2609:00 +CPU= 19.314
  02:57:30 STEP 2610 H=2610:00 +CPU= 28.420
  02:57:48 STEP 2611 H=2611:00 +CPU= 18.130
  02:58:07 STEP 2612 H=2612:00 +CPU= 18.823
  02:58:35 STEP 2613 H=2613:00 +CPU= 28.094
  02:58:54 STEP 2614 H=2614:00 +CPU= 18.891
  02:59:13 STEP 2615 H=2615:00 +CPU= 18.197
  03:00:04 STEP 2616 H=2616:00 +CPU= 49.516
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_108.zip
Uploading the intermediate file: upload_file_108.zip
  03:00:32 STEP 2617 H=2617:00 +CPU= 27.861
  03:01:00 STEP 2618 H=2618:00 +CPU= 28.261
  03:01:34 STEP 2619 H=2619:00 +CPU= 33.804
  03:01:52 STEP 2620 H=2620:00 +CPU= 18.359
  03:03:07 STEP 2621 H=2621:00 +CPU= 22.901
  03:05:59 STEP 2622 H=2622:00 +CPU= 37.745
  03:06:24 STEP 2623 H=2623:00 +CPU= 20.169
  03:06:42 STEP 2624 H=2624:00 +CPU= 18.473
  03:07:12 STEP 2625 H=2625:00 +CPU= 27.679
  03:07:31 STEP 2626 H=2626:00 +CPU= 18.135
  03:07:48 STEP 2627 H=2627:00 +CPU= 17.305
  03:08:24 STEP 2628 H=2628:00 +CPU= 34.864
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002628
  03:08:41 STEP 2629 H=2629:00 +CPU= 17.314
  03:08:59 STEP 2630 H=2630:00 +CPU= 17.462
  03:09:26 STEP 2631 H=2631:00 +CPU= 26.830
  03:09:44 STEP 2632 H=2632:00 +CPU= 17.737
  03:10:01 STEP 2633 H=2633:00 +CPU= 17.823
  03:10:29 STEP 2634 H=2634:00 +CPU= 27.625
  03:10:46 STEP 2635 H=2635:00 +CPU= 16.955
  03:11:04 STEP 2636 H=2636:00 +CPU= 17.967
  03:11:31 STEP 2637 H=2637:00 +CPU= 27.106
  03:13:14 STEP 2638 H=2638:00 +CPU= 20.926
  03:13:37 STEP 2639 H=2639:00 +CPU= 20.871
  03:14:43 STEP 2640 H=2640:00 +CPU= 46.158
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_109.zip
Uploading the intermediate file: upload_file_109.zip
  03:16:02 STEP 2641 H=2641:00 +CPU= 25.072
  03:16:30 STEP 2642 H=2642:00 +CPU= 25.609
  03:17:05 STEP 2643 H=2643:00 +CPU= 27.670
  03:17:33 STEP 2644 H=2644:00 +CPU= 28.212
  03:18:00 STEP 2645 H=2645:00 +CPU= 26.644
  03:18:39 STEP 2646 H=2646:00 +CPU= 37.496
  03:19:00 STEP 2647 H=2647:00 +CPU= 21.131
  03:19:28 STEP 2648 H=2648:00 +CPU= 28.280
  03:20:01 STEP 2649 H=2649:00 +CPU= 32.066
  03:20:55 STEP 2650 H=2650:00 +CPU= 22.822
  03:21:18 STEP 2651 H=2651:00 +CPU= 21.396
  03:22:05 STEP 2652 H=2652:00 +CPU= 46.993
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002652
  03:22:57 STEP 2653 H=2653:00 +CPU= 25.730
  03:23:36 STEP 2654 H=2654:00 +CPU= 24.078
  03:24:09 STEP 2655 H=2655:00 +CPU= 32.843
  03:24:53 STEP 2656 H=2656:00 +CPU= 26.748
  03:25:17 STEP 2657 H=2657:00 +CPU= 22.941
  03:25:53 STEP 2658 H=2658:00 +CPU= 36.295
  03:26:22 STEP 2659 H=2659:00 +CPU= 28.239
  03:26:47 STEP 2660 H=2660:00 +CPU= 24.019
  03:27:20 STEP 2661 H=2661:00 +CPU= 32.972
  03:27:50 STEP 2662 H=2662:00 +CPU= 21.020
  03:28:13 STEP 2663 H=2663:00 +CPU= 22.796
  03:28:58 STEP 2664 H=2664:00 +CPU= 39.229
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_110.zip
Uploading the intermediate file: upload_file_110.zip
  03:29:20 STEP 2665 H=2665:00 +CPU= 21.351
  03:29:39 STEP 2666 H=2666:00 +CPU= 18.836
  03:30:30 STEP 2667 H=2667:00 +CPU= 32.350
  03:31:00 STEP 2668 H=2668:00 +CPU= 27.532
  03:31:21 STEP 2669 H=2669:00 +CPU= 20.522
  03:32:14 STEP 2670 H=2670:00 +CPU= 42.280
  03:32:43 STEP 2671 H=2671:00 +CPU= 27.946
  03:33:11 STEP 2672 H=2672:00 +CPU= 28.526
  03:33:48 STEP 2673 H=2673:00 +CPU= 36.022
  03:34:10 STEP 2674 H=2674:00 +CPU= 22.165
  03:34:53 STEP 2675 H=2675:00 +CPU= 21.908
  03:35:44 STEP 2676 H=2676:00 +CPU= 46.335
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002676
  03:36:12 STEP 2677 H=2677:00 +CPU= 27.961
  03:36:41 STEP 2678 H=2678:00 +CPU= 28.364
  03:37:09 STEP 2679 H=2679:00 +CPU= 28.020
  03:37:28 STEP 2680 H=2680:00 +CPU= 18.895
  03:37:52 STEP 2681 H=2681:00 +CPU= 24.289
  03:38:27 STEP 2682 H=2682:00 +CPU= 34.687
  03:38:55 STEP 2683 H=2683:00 +CPU= 27.919
  03:39:23 STEP 2684 H=2684:00 +CPU= 23.166
  03:40:01 STEP 2685 H=2685:00 +CPU= 36.661
  03:40:30 STEP 2686 H=2686:00 +CPU= 28.817
  03:41:15 STEP 2687 H=2687:00 +CPU= 23.815
  03:42:17 STEP 2688 H=2688:00 +CPU= 47.261
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_111.zip
Uploading the intermediate file: upload_file_111.zip
  03:42:39 STEP 2689 H=2689:00 +CPU= 21.339
  03:43:12 STEP 2690 H=2690:00 +CPU= 19.184
  03:43:56 STEP 2691 H=2691:00 +CPU= 42.332
  03:44:24 STEP 2692 H=2692:00 +CPU= 27.657
  03:44:43 STEP 2693 H=2693:00 +CPU= 18.823
  03:45:49 STEP 2694 H=2694:00 +CPU= 38.835
  03:46:18 STEP 2695 H=2695:00 +CPU= 28.299
  03:46:47 STEP 2696 H=2696:00 +CPU= 28.653
  03:47:30 STEP 2697 H=2697:00 +CPU= 43.159
  03:47:59 STEP 2698 H=2698:00 +CPU= 28.863
  03:48:31 STEP 2699 H=2699:00 +CPU= 21.924
  03:49:15 STEP 2700 H=2700:00 +CPU= 42.863
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002700
  03:49:40 STEP 2701 H=2701:00 +CPU= 23.784
  03:50:48 STEP 2702 H=2702:00 +CPU= 20.693
  03:51:28 STEP 2703 H=2703:00 +CPU= 35.712
  03:51:47 STEP 2704 H=2704:00 +CPU= 18.465
  03:52:04 STEP 2705 H=2705:00 +CPU= 17.401
  03:55:36 STEP 2706 H=2706:00 +CPU= 30.769
  03:56:05 STEP 2707 H=2707:00 +CPU= 28.549
  03:56:35 STEP 2708 H=2708:00 +CPU= 29.248
  03:57:14 STEP 2709 H=2709:00 +CPU= 38.113
  03:57:42 STEP 2710 H=2710:00 +CPU= 27.504
  03:58:09 STEP 2711 H=2711:00 +CPU= 23.507
  03:59:10 STEP 2712 H=2712:00 +CPU= 49.479
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_112.zip
Uploading the intermediate file: upload_file_112.zip
  03:59:37 STEP 2713 H=2713:00 +CPU= 26.799
  03:59:56 STEP 2714 H=2714:00 +CPU= 18.782
  04:01:27 STEP 2715 H=2715:00 +CPU= 40.077
  04:02:19 STEP 2716 H=2716:00 +CPU= 27.149
  04:02:46 STEP 2717 H=2717:00 +CPU= 25.132
  04:03:30 STEP 2718 H=2718:00 +CPU= 43.689
  04:03:58 STEP 2719 H=2719:00 +CPU= 27.968
  04:04:28 STEP 2720 H=2720:00 +CPU= 29.408
  04:05:07 STEP 2721 H=2721:00 +CPU= 39.304
  04:05:36 STEP 2722 H=2722:00 +CPU= 28.317
  04:06:03 STEP 2723 H=2723:00 +CPU= 27.387
  04:06:51 STEP 2724 H=2724:00 +CPU= 46.749
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002724
  04:07:16 STEP 2725 H=2725:00 +CPU= 25.040
  04:07:45 STEP 2726 H=2726:00 +CPU= 29.040
  04:08:28 STEP 2727 H=2727:00 +CPU= 42.684
  04:08:58 STEP 2728 H=2728:00 +CPU= 29.657
  04:09:26 STEP 2729 H=2729:00 +CPU= 27.943
  04:10:04 STEP 2730 H=2730:00 +CPU= 36.684
  04:10:23 STEP 2731 H=2731:00 +CPU= 19.193
  04:11:14 STEP 2732 H=2732:00 +CPU= 23.899
  04:11:58 STEP 2733 H=2733:00 +CPU= 43.829
  04:12:27 STEP 2734 H=2734:00 +CPU= 29.189
  04:12:58 STEP 2735 H=2735:00 +CPU= 29.920
  04:14:05 STEP 2736 H=2736:00 +CPU= 50.834
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_113.zip
Uploading the intermediate file: upload_file_113.zip
  04:14:34 STEP 2737 H=2737:00 +CPU= 29.011
Uploading trickle at timestep: 9849600
  04:15:04 STEP 2738 H=2738:00 +CPU= 30.316
  04:15:49 STEP 2739 H=2739:00 +CPU= 44.537
  04:16:19 STEP 2740 H=2740:00 +CPU= 30.105
  04:16:47 STEP 2741 H=2741:00 +CPU= 28.322
  04:17:25 STEP 2742 H=2742:00 +CPU= 37.040
  04:18:08 STEP 2743 H=2743:00 +CPU= 20.878
  04:18:31 STEP 2744 H=2744:00 +CPU= 20.871
  04:19:06 STEP 2745 H=2745:00 +CPU= 32.523
  04:19:32 STEP 2746 H=2746:00 +CPU= 26.026
  04:19:53 STEP 2747 H=2747:00 +CPU= 20.205
  04:20:41 STEP 2748 H=2748:00 +CPU= 47.873
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002748
  04:21:03 STEP 2749 H=2749:00 +CPU= 21.559
  04:21:31 STEP 2750 H=2750:00 +CPU= 28.312
  04:22:10 STEP 2751 H=2751:00 +CPU= 39.244
  04:22:36 STEP 2752 H=2752:00 +CPU= 25.939
  04:24:00 STEP 2753 H=2753:00 +CPU= 25.099
  04:24:42 STEP 2754 H=2754:00 +CPU= 39.092
  04:25:06 STEP 2755 H=2755:00 +CPU= 23.961
  04:25:29 STEP 2756 H=2756:00 +CPU= 22.321
  04:26:13 STEP 2757 H=2757:00 +CPU= 43.701
  04:26:43 STEP 2758 H=2758:00 +CPU= 30.333
  04:27:14 STEP 2759 H=2759:00 +CPU= 30.739
  04:28:25 STEP 2760 H=2760:00 +CPU= 43.579
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_114.zip
Uploading the intermediate file: upload_file_114.zip
  04:28:46 STEP 2761 H=2761:00 +CPU= 20.741
  04:29:15 STEP 2762 H=2762:00 +CPU= 28.316
  04:29:46 STEP 2763 H=2763:00 +CPU= 30.561
  04:30:13 STEP 2764 H=2764:00 +CPU= 26.828
  04:30:38 STEP 2765 H=2765:00 +CPU= 25.518
  04:31:08 STEP 2766 H=2766:00 +CPU= 29.569
  04:31:27 STEP 2767 H=2767:00 +CPU= 18.652
  04:31:56 STEP 2768 H=2768:00 +CPU= 29.073
  04:32:26 STEP 2769 H=2769:00 +CPU= 29.663
  04:32:55 STEP 2770 H=2770:00 +CPU= 28.827
  04:33:42 STEP 2771 H=2771:00 +CPU= 21.922
  04:34:19 STEP 2772 H=2772:00 +CPU= 34.849
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002772
  04:34:47 STEP 2773 H=2773:00 +CPU= 28.592
  04:35:16 STEP 2774 H=2774:00 +CPU= 28.862
  04:35:51 STEP 2775 H=2775:00 +CPU= 34.969
  04:36:20 STEP 2776 H=2776:00 +CPU= 28.279
  04:37:04 STEP 2777 H=2777:00 +CPU= 25.232
  04:38:08 STEP 2778 H=2778:00 +CPU= 41.040
  04:38:29 STEP 2779 H=2779:00 +CPU= 20.373
  04:38:57 STEP 2780 H=2780:00 +CPU= 28.315
  04:39:42 STEP 2781 H=2781:00 +CPU= 44.127
  04:40:10 STEP 2782 H=2782:00 +CPU= 27.842
  04:40:37 STEP 2783 H=2783:00 +CPU= 26.633
  04:41:32 STEP 2784 H=2784:00 +CPU= 50.251
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_115.zip
Uploading the intermediate file: upload_file_115.zip
  04:42:01 STEP 2785 H=2785:00 +CPU= 29.322
  04:42:31 STEP 2786 H=2786:00 +CPU= 30.129
  04:43:16 STEP 2787 H=2787:00 +CPU= 44.898
  04:43:37 STEP 2788 H=2788:00 +CPU= 19.706
  04:44:10 STEP 2789 H=2789:00 +CPU= 28.326
  04:44:46 STEP 2790 H=2790:00 +CPU= 36.585
  04:45:05 STEP 2791 H=2791:00 +CPU= 18.247
  04:45:24 STEP 2792 H=2792:00 +CPU= 19.370
  04:45:56 STEP 2793 H=2793:00 +CPU= 32.194
  04:46:16 STEP 2794 H=2794:00 +CPU= 19.252
  04:46:42 STEP 2795 H=2795:00 +CPU= 26.083
  04:47:15 STEP 2796 H=2796:00 +CPU= 31.580
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002796
  04:47:34 STEP 2797 H=2797:00 +CPU= 18.513
  04:48:03 STEP 2798 H=2798:00 +CPU= 29.055
  04:48:34 STEP 2799 H=2799:00 +CPU= 30.953
  04:49:02 STEP 2800 H=2800:00 +CPU= 27.497
  04:49:54 STEP 2801 H=2801:00 +CPU= 26.807
  04:50:36 STEP 2802 H=2802:00 +CPU= 40.172
  04:50:55 STEP 2803 H=2803:00 +CPU= 19.267
  04:51:18 STEP 2804 H=2804:00 +CPU= 22.756
  04:51:53 STEP 2805 H=2805:00 +CPU= 34.486
  04:52:12 STEP 2806 H=2806:00 +CPU= 19.670
  04:53:12 STEP 2807 H=2807:00 +CPU= 26.537
  04:54:10 STEP 2808 H=2808:00 +CPU= 42.169
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_116.zip
Uploading the intermediate file: upload_file_116.zip
  04:54:31 STEP 2809 H=2809:00 +CPU= 21.296
  04:54:53 STEP 2810 H=2810:00 +CPU= 21.285
  04:55:29 STEP 2811 H=2811:00 +CPU= 34.379
  04:55:50 STEP 2812 H=2812:00 +CPU= 21.221
  04:56:12 STEP 2813 H=2813:00 +CPU= 21.057
  04:56:46 STEP 2814 H=2814:00 +CPU= 34.017
  04:57:11 STEP 2815 H=2815:00 +CPU= 24.808
  04:58:08 STEP 2816 H=2816:00 +CPU= 21.995
  04:59:04 STEP 2817 H=2817:00 +CPU= 42.080
  04:59:28 STEP 2818 H=2818:00 +CPU= 24.174
  04:59:55 STEP 2819 H=2819:00 +CPU= 22.754
  05:00:50 STEP 2820 H=2820:00 +CPU= 49.444
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002820
  05:01:14 STEP 2821 H=2821:00 +CPU= 23.286
  05:01:34 STEP 2822 H=2822:00 +CPU= 19.263
  05:02:10 STEP 2823 H=2823:00 +CPU= 35.544
  05:02:38 STEP 2824 H=2824:00 +CPU= 28.009
  05:03:06 STEP 2825 H=2825:00 +CPU= 27.983
  05:03:50 STEP 2826 H=2826:00 +CPU= 43.870
  05:04:09 STEP 2827 H=2827:00 +CPU= 18.932
  05:04:28 STEP 2828 H=2828:00 +CPU= 18.861
  05:05:15 STEP 2829 H=2829:00 +CPU= 34.256
  05:06:08 STEP 2830 H=2830:00 +CPU= 27.913
  05:06:36 STEP 2831 H=2831:00 +CPU= 24.988
  05:07:48 STEP 2832 H=2832:00 +CPU= 46.842
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_117.zip
Uploading the intermediate file: upload_file_117.zip
  05:08:21 STEP 2833 H=2833:00 +CPU= 24.364
  05:08:48 STEP 2834 H=2834:00 +CPU= 26.445
  05:09:18 STEP 2835 H=2835:00 +CPU= 30.731
  05:09:38 STEP 2836 H=2836:00 +CPU= 19.588
  05:09:58 STEP 2837 H=2837:00 +CPU= 20.023
  05:10:34 STEP 2838 H=2838:00 +CPU= 32.264
  05:10:58 STEP 2839 H=2839:00 +CPU= 24.190
  05:11:16 STEP 2840 H=2840:00 +CPU= 18.249
  05:11:53 STEP 2841 H=2841:00 +CPU= 36.909
  05:12:32 STEP 2842 H=2842:00 +CPU= 19.815
  05:12:53 STEP 2843 H=2843:00 +CPU= 20.452
  05:13:37 STEP 2844 H=2844:00 +CPU= 43.650
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002844
  05:14:04 STEP 2845 H=2845:00 +CPU= 26.743
  05:14:32 STEP 2846 H=2846:00 +CPU= 28.438
  05:16:20 STEP 2847 H=2847:00 +CPU= 34.125
  05:16:39 STEP 2848 H=2848:00 +CPU= 17.840
  05:16:56 STEP 2849 H=2849:00 +CPU= 17.692
  05:17:52 STEP 2850 H=2850:00 +CPU= 33.986
  05:18:20 STEP 2851 H=2851:00 +CPU= 26.578
  05:18:45 STEP 2852 H=2852:00 +CPU= 25.207
  05:19:16 STEP 2853 H=2853:00 +CPU= 30.496
  05:19:35 STEP 2854 H=2854:00 +CPU= 18.879
  05:19:55 STEP 2855 H=2855:00 +CPU= 19.430
  05:20:49 STEP 2856 H=2856:00 +CPU= 45.165
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_118.zip
Uploading the intermediate file: upload_file_118.zip
  05:21:20 STEP 2857 H=2857:00 +CPU= 26.672
  05:22:03 STEP 2858 H=2858:00 +CPU= 27.378
  05:22:48 STEP 2859 H=2859:00 +CPU= 44.991
  05:23:18 STEP 2860 H=2860:00 +CPU= 30.195
  05:23:46 STEP 2861 H=2861:00 +CPU= 27.937
  05:24:29 STEP 2862 H=2862:00 +CPU= 41.200
  05:24:56 STEP 2863 H=2863:00 +CPU= 27.050
  05:25:24 STEP 2864 H=2864:00 +CPU= 27.562
  05:26:00 STEP 2865 H=2865:00 +CPU= 36.135
  05:26:21 STEP 2866 H=2866:00 +CPU= 20.784
  05:26:41 STEP 2867 H=2867:00 +CPU= 20.113
  05:28:10 STEP 2868 H=2868:00 +CPU= 37.449
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002868
  05:28:30 STEP 2869 H=2869:00 +CPU= 20.701
  05:28:55 STEP 2870 H=2870:00 +CPU= 24.798
  05:29:39 STEP 2871 H=2871:00 +CPU= 43.618
  05:30:04 STEP 2872 H=2872:00 +CPU= 24.206
  05:30:23 STEP 2873 H=2873:00 +CPU= 19.238
  05:30:59 STEP 2874 H=2874:00 +CPU= 35.762
  05:31:18 STEP 2875 H=2875:00 +CPU= 18.954
  05:31:46 STEP 2876 H=2876:00 +CPU= 28.346
  05:32:27 STEP 2877 H=2877:00 +CPU= 39.191
  05:32:57 STEP 2878 H=2878:00 +CPU= 30.090
  05:33:28 STEP 2879 H=2879:00 +CPU= 31.022
  05:34:20 STEP 2880 H=2880:00 +CPU= 46.509
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_119.zip
Uploading the intermediate file: upload_file_119.zip
  05:34:46 STEP 2881 H=2881:00 +CPU= 26.488
  05:35:15 STEP 2882 H=2882:00 +CPU= 28.278
  05:35:51 STEP 2883 H=2883:00 +CPU= 35.831
  05:36:09 STEP 2884 H=2884:00 +CPU= 17.988
  05:36:49 STEP 2885 H=2885:00 +CPU= 19.140
  05:37:30 STEP 2886 H=2886:00 +CPU= 40.476
  05:38:34 STEP 2887 H=2887:00 +CPU= 24.072
  05:39:01 STEP 2888 H=2888:00 +CPU= 18.420
  05:40:41 STEP 2889 H=2889:00 +CPU= 39.030
  05:41:05 STEP 2890 H=2890:00 +CPU= 21.508
  05:41:23 STEP 2891 H=2891:00 +CPU= 17.647
  05:42:12 STEP 2892 H=2892:00 +CPU= 44.592
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002892
  05:42:32 STEP 2893 H=2893:00 +CPU= 19.437
  05:43:02 STEP 2894 H=2894:00 +CPU= 30.146
  05:43:47 STEP 2895 H=2895:00 +CPU= 44.610
  05:44:09 STEP 2896 H=2896:00 +CPU= 22.059
  05:45:57 STEP 2897 H=2897:00 +CPU= 19.549
  05:47:22 STEP 2898 H=2898:00 +CPU= 26.105
  05:47:40 STEP 2899 H=2899:00 +CPU= 16.668
  05:50:11 STEP 2900 H=2900:00 +CPU= 16.405
  05:51:07 STEP 2901 H=2901:00 +CPU= 25.253
  05:54:05 STEP 2902 H=2902:00 +CPU= 17.006
  05:54:51 STEP 2903 H=2903:00 +CPU= 15.647
  05:56:34 STEP 2904 H=2904:00 +CPU= 28.981
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_120.zip
Uploading the intermediate file: upload_file_120.zip
  05:56:49 STEP 2905 H=2905:00 +CPU= 15.421
  05:57:06 STEP 2906 H=2906:00 +CPU= 15.830
  05:57:30 STEP 2907 H=2907:00 +CPU= 24.752
  05:57:48 STEP 2908 H=2908:00 +CPU= 17.500
  05:58:05 STEP 2909 H=2909:00 +CPU= 15.973
  05:58:31 STEP 2910 H=2910:00 +CPU= 26.006
  05:58:49 STEP 2911 H=2911:00 +CPU= 16.952
  05:59:06 STEP 2912 H=2912:00 +CPU= 17.355
  05:59:32 STEP 2913 H=2913:00 +CPU= 25.918
  05:59:50 STEP 2914 H=2914:00 +CPU= 17.310
  06:00:07 STEP 2915 H=2915:00 +CPU= 16.739
  06:00:36 STEP 2916 H=2916:00 +CPU= 28.834
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002916
  06:00:53 STEP 2917 H=2917:00 +CPU= 16.834
  06:01:10 STEP 2918 H=2918:00 +CPU= 16.957
  06:05:39 STEP 2919 H=2919:00 +CPU= 28.358
  06:05:59 STEP 2920 H=2920:00 +CPU= 17.557
  06:06:16 STEP 2921 H=2921:00 +CPU= 16.504
  06:06:47 STEP 2922 H=2922:00 +CPU= 26.519
  06:07:08 STEP 2923 H=2923:00 +CPU= 16.795
  06:07:40 STEP 2924 H=2924:00 +CPU= 16.318
  06:09:34 STEP 2925 H=2925:00 +CPU= 25.413
  06:09:53 STEP 2926 H=2926:00 +CPU= 15.817
  06:10:10 STEP 2927 H=2927:00 +CPU= 15.710
  06:11:06 STEP 2928 H=2928:00 +CPU= 27.174
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_121.zip
  06:11:22 STEP 2929 H=2929:00 +CPU= 15.021
Uploading the intermediate file: upload_file_121.zip
  06:11:38 STEP 2930 H=2930:00 +CPU= 15.778
  06:12:04 STEP 2931 H=2931:00 +CPU= 24.477
  06:12:20 STEP 2932 H=2932:00 +CPU= 15.770
  06:12:36 STEP 2933 H=2933:00 +CPU= 15.274
  06:13:02 STEP 2934 H=2934:00 +CPU= 24.723
  06:13:18 STEP 2935 H=2935:00 +CPU= 15.523
  06:13:35 STEP 2936 H=2936:00 +CPU= 16.205
  06:14:00 STEP 2937 H=2937:00 +CPU= 24.917
  06:14:17 STEP 2938 H=2938:00 +CPU= 16.014
  06:14:35 STEP 2939 H=2939:00 +CPU= 16.715
  06:15:26 STEP 2940 H=2940:00 +CPU= 30.758
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002940
  06:15:54 STEP 2941 H=2941:00 +CPU= 17.437
  06:16:12 STEP 2942 H=2942:00 +CPU= 16.738
  06:16:39 STEP 2943 H=2943:00 +CPU= 24.928
  06:16:55 STEP 2944 H=2944:00 +CPU= 15.622
  06:17:12 STEP 2945 H=2945:00 +CPU= 15.479
  06:17:40 STEP 2946 H=2946:00 +CPU= 25.381
  06:17:56 STEP 2947 H=2947:00 +CPU= 15.360
  06:18:12 STEP 2948 H=2948:00 +CPU= 16.324
  06:18:38 STEP 2949 H=2949:00 +CPU= 25.232
  06:18:58 STEP 2950 H=2950:00 +CPU= 17.365
  06:19:20 STEP 2951 H=2951:00 +CPU= 17.978
  06:20:05 STEP 2952 H=2952:00 +CPU= 28.263
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12177765/ICMSHhq0f+002928
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0121_1992050100_123_961_12177765_1_r1179725465_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
06:23:39 (25135): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:35:47 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 10,623,600 80,575 0.0076
09 May 2023 16:35:40 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 9,849,600 74,837 0.0076
09 May 2023 16:35:05 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 8,640,000 64,954 0.0075
09 May 2023 16:34:48 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 7,776,000 58,015 0.0075
09 May 2023 16:34:36 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 6,912,000 50,885 0.0074
09 May 2023 16:34:19 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 6,048,000 43,995 0.0073
09 May 2023 16:34:05 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 5,184,000 38,336 0.0074
09 May 2023 16:33:50 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 4,320,000 31,859 0.0074
09 May 2023 16:33:45 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 3,456,000 25,359 0.0073
09 May 2023 16:33:34 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 2,592,000 18,695 0.0072
09 May 2023 16:33:19 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 1,728,000 12,276 0.0071
09 May 2023 16:33:13 1517434 22295392 oifs_43r3_ps_0121_1992050100_123_961_12177765_1 864,000 6,338 0.0073


©2024 climateprediction.net