climateprediction.net home page
Which upload server?

Which upload server?

Message boards : Number crunching : Which upload server?
Message board moderation

To post messages, you must log in.

AuthorMessage
old_user8065

Send message
Joined: 1 Sep 04
Posts: 24
Credit: 10,865,773
RAC: 0
Message 35477 - Posted: 13 Nov 2008, 7:33:02 UTC

As we can see from project\'s status page there are approximately 5 upload servers. When a client receiiiives work unit (model) to run, upload server for every expected output file is defined. As I can see from client_state.xml, upload server for different output files of the same model can be different.

Due to some problems on my side it may happen that intermediate files get uploaded to wrong upload server (eg. to climateapps3.oucs.ox.ac.uk instead of uploader.oerc.ox.ac.uk or the other way around).

Does it create a problem to project?
Metod ...
ID: 35477 · Report as offensive     Reply Quote
old_user8065

Send message
Joined: 1 Sep 04
Posts: 24
Credit: 10,865,773
RAC: 0
Message 35558 - Posted: 20 Nov 2008, 16:51:10 UTC - in response to Message 35477.  
Last modified: 20 Nov 2008, 16:52:07 UTC

Does it create a problem to project?


I assume it doesn\'t since nobody responded in a week. Or at least it\'s not the biggest problem of the project.
Metod ...
ID: 35558 · Report as offensive     Reply Quote
Profile mo.v
Volunteer moderator
Avatar

Send message
Joined: 29 Sep 04
Posts: 2363
Credit: 14,611,758
RAC: 0
Message 35559 - Posted: 20 Nov 2008, 18:09:22 UTC

Hi Metod

For some reason I didn\'t notice your post, sorry. Perhaps someone else will correct me, but I\'m not sure how it\'s possible for zip files to go to the wrong upload server. Why do you think this happened?
Cpdn news
ID: 35559 · Report as offensive     Reply Quote
old_user8065

Send message
Joined: 1 Sep 04
Posts: 24
Credit: 10,865,773
RAC: 0
Message 35565 - Posted: 21 Nov 2008, 14:31:02 UTC - in response to Message 35559.  
Last modified: 21 Nov 2008, 14:37:47 UTC

Perhaps someone else will correct me, but I\'m not sure how it\'s possible for zip files to go to the wrong upload server. Why do you think this happened?


It\'s a continuation of troubles I was complaining about in this thread. I\'ve found a workaround which involves tunnels and port forwarding to specific upload server and then setting forwarding host as proxy in client_state.xml (I don\'t want to go into details, there might be some corporate IT guy lurking around ). What I overlooked was that at one moment there were a couple of intermediate files waiting to be uploaded and back-off timer of a wrong file expired while the workaround was in place.

As URLs of upload handlers are the same (except for the host names of course), the request went through just fine. AFAIK HTTP servers are allowed to ignore host name part of URL (especially if they are HTTP 1.0) unless they know how to deal with it - either when they act as any kind of proxy servers or when they run in several instances (virtual servers). Somehow I don\'t expect CPDN upload servers to do any of these.
Metod ...
ID: 35565 · Report as offensive     Reply Quote
Profile Milo Thurston
Volunteer moderator
Volunteer developer

Send message
Joined: 2 Mar 06
Posts: 253
Credit: 363,646
RAC: 0
Message 35568 - Posted: 21 Nov 2008, 22:29:30 UTC

It doesn\'t really matter to which upload server the files go. Each week the files on each upload server are indexed and the results entered into a central database specifically for the purpose of tracking files; this is separate from the BOINC database. With this database the physicists can find any of the files they might need and download them directly from the servers.
ID: 35568 · Report as offensive     Reply Quote
old_user8065

Send message
Joined: 1 Sep 04
Posts: 24
Credit: 10,865,773
RAC: 0
Message 35599 - Posted: 25 Nov 2008, 17:23:01 UTC - in response to Message 35568.  

Milo, thanks for the response. I\'ll stop to worry about past mishaps and will try to avoid them in future.
Metod ...
ID: 35599 · Report as offensive     Reply Quote
Profile Milo Thurston
Volunteer moderator
Volunteer developer

Send message
Joined: 2 Mar 06
Posts: 253
Credit: 363,646
RAC: 0
Message 35600 - Posted: 25 Nov 2008, 20:22:44 UTC - in response to Message 35599.  

Milo, thanks for the response. I\'ll stop to worry about past mishaps and will try to avoid them in future.


You\'re welcome, and don\'t worry about it.
Apologies for the delay but I usually don\'t get much chance to check the boards (the volunteer moderators do an excellent job anyway).
ID: 35600 · Report as offensive     Reply Quote

Message boards : Number crunching : Which upload server?

©2024 climateprediction.net