climateprediction.net home page
What happens near the end of a task?

What happens near the end of a task?

Message boards : Number crunching : What happens near the end of a task?
Message board moderation

To post messages, you must log in.

AuthorMessage
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 381
Credit: 3,690,501
RAC: 647
Message 61506 - Posted: 9 Nov 2019, 13:33:32 UTC

One of my work-units just uploaded its last .zip file, yet the boinc client says I have almost three hours to go. And the task is still running. What could it be doing now? After running all four trickles, is it not done?

Name hadam4h_a0pg_200811_4_842_011905372_0
Workunit 11905372

 Sat 09 Nov 2019 08:03:11 AM EST | climateprediction.net | Started upload of hadam4h_a0pg_200811_4_842_011905372_0_r1707431134_restart.zip
Sat 09 Nov 2019 08:03:13 AM EST | climateprediction.net | Finished upload of hadam4h_a0pg_200811_4_842_011905372_0_r1707431134_restart.zip
Sat 09 Nov 2019 08:03:24 AM EST | climateprediction.net | Sending scheduler request: To send trickle-up message.
Sat 09 Nov 2019 08:03:24 AM EST | climateprediction.net | Not requesting tasks: "no new tasks" requested via Manager
Sat 09 Nov 2019 08:03:25 AM EST | climateprediction.net | Scheduler request completed
Sat 09 Nov 2019 08:03:57 AM EST | climateprediction.net | Started upload of hadam4h_a0pg_200811_4_842_011905372_0_r1707431134_4.zip
Sat 09 Nov 2019 08:14:18 AM EST | climateprediction.net | Finished upload of hadam4h_a0pg_200811_4_842_011905372_0_r1707431134_4.zip

ID: 61506 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 2722
Credit: 3,313,664
RAC: 960
Message 61507 - Posted: 9 Nov 2019, 13:42:21 UTC - in response to Message 61506.  

There are a couple of other files that sometimes get created after the last zip. The restart.zip and stdaeout.zip (might not have the name of the last one exactly right. The first of the two is of the same order of magnitude as the monthly zips while the other is KB in size rather than MB. The restart files in some cases are used to seed yet more work.
ID: 61507 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 381
Credit: 3,690,501
RAC: 647
Message 61508 - Posted: 9 Nov 2019, 14:35:16 UTC - in response to Message 61507.  

There are a couple of other files that sometimes get created after the last zip. The restart.zip and stdaeout.zip (might not have the name of the last one exactly right. The first of the two is of the same order of magnitude as the monthly zips while the other is KB in size rather than MB.


There was one more file:

Sat 09 Nov 2019 09:18:28 AM EST | climateprediction.net | Computation for task hadam4h_a0pg_200811_4_842_011905372_0 finished
Sat 09 Nov 2019 09:18:28 AM EST | climateprediction.net | Resuming task hadam4h_a1i3_201211_4_842_011906403_0 using hadam4h version 852 in slot 1
Sat 09 Nov 2019 09:18:31 AM EST | climateprediction.net | Started upload of hadam4h_a0pg_200811_4_842_011905372_0_r1707431134_out.zip
Sat 09 Nov 2019 09:18:36 AM EST | climateprediction.net | Finished upload of hadam4h_a0pg_200811_4_842_011905372_0_r1707431134_out.zip
Sat 09 Nov 2019 09:18:38 AM EST | climateprediction.net | Sending scheduler request: To report completed tasks.


Notice how long the _4.zip file took to upload?
Sat 09 Nov 2019 08:03:57 AM EST | climateprediction.net | Started upload of hadam4h_a0pg_200811_4_842_011905372_0_r1707431134_4.zip
Sat 09 Nov 2019 08:14:18 AM EST | climateprediction.net | Finished upload of hadam4h_a0pg_200811_4_842_011905372_0_r1707431134_4.zip


A bit over 10 minutes. Since my Internet connection is about 75 Megabits/second, I infer that either the Internet backbone speeds are clogged or the CPDN server is none too speedy. I could go about 40x faster than that.

Date 	Download 	Upload 	IP Address 	Test Server 	Export All Results
	11/9/2019 9:32:03 AM	82.44 Mbps	36.14 Mbps	100.35.165.125	New York City, NY	 	 
	10/4/2019 7:45:44 AM	83.98 Mbps	87.33 Mbps	100.35.165.125	New York City, NY	 	 
	10/1/2019 5:17:42 PM	83.84 Mbps	66.32 Mbps	100.35.165.125	Seattle, WA

ID: 61508 · Report as offensive     Reply Quote
Profile geophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 1929
Credit: 41,067,747
RAC: 14,699
Message 61509 - Posted: 9 Nov 2019, 14:40:33 UTC
Last modified: 9 Nov 2019, 14:46:44 UTC

The other thing that might cause this is that while the model may upload the monthly file right after the last day of the month, the model continues on for another day before terminating itself. If you have about 3 hours between checkpoints, that would be about right.

You can check which model day the model run is on by opening a terminal window and going into the

"boincdir"/projects/climateprediction.net/hadam4hrestofmodelname

directory and typing tail -f stdout_mon.txt

It lists the model timesteps, the model date, model time, cumulative time, average sec/TS for the whole run so far, and individual sec/TS for each forecast timestep.
ID: 61509 · Report as offensive     Reply Quote

Message boards : Number crunching : What happens near the end of a task?

©2020 climateprediction.net