climateprediction.net home page
Posts by Mephist0

Posts by Mephist0

1) Message boards : Number crunching : Scheduler request too recent (Message 61944)
Posted 9 Jan 2020 by Mephist0
Post:
Might be a noob question, how do i change the resource between the projects? i want to have like 95% for CPDN and 5% for a backup project... i wish there would have been a setting to finish the backup project and then only do CPDN until all WU is processed and then request new work from the backup project.....
2) Message boards : Number crunching : New work Discussion (Message 61695)
Posted 17 Dec 2019 by Mephist0
Post:
This project does not seem to have a lack of CPU processing power though :) .. Thats a good thing for the project :) .. If you keep trying you will eventually get work. If no new WU is released you will probably get an old one wich has timed out :)
3) Message boards : Number crunching : Permanent HTTP error (Message 60817)
Posted 8 Aug 2019 by Mephist0
Post:
Thx :)
4) Message boards : Number crunching : Permanent HTTP error (Message 60814)
Posted 8 Aug 2019 by Mephist0
Post:
Since i have had so much problems with this project i had to give up on processing it on many of my servers.

I have a testserver running connected straight to internet to use it as reference to identify if problems is at my site or the project. Other projects works just fine on my other servers.

Now when i tried to download a WU i got this error:

2019-08-08 13:04:01 | climateprediction.net | Scheduler request completed: got 1 new tasks
2019-08-08 13:04:03 | climateprediction.net | Started download of wah2_cam25_a0d2_200405_18_823_011890237.zip
2019-08-08 13:04:03 | climateprediction.net | Started download of atmos_restart_batch_694_cam25_m01l_2004-05-01.gz
2019-08-08 13:04:03 | climateprediction.net | Started download of region_restart_batch_694_cam25_m01l_2004-05-01.gz
2019-08-08 13:04:03 | climateprediction.net | Started download of ic19610309_16_N96.gz
2019-08-08 13:04:03 | climateprediction.net | Started download of NATclim_ancil_134months_OSTIA_sst_MMM_1994-12-01_2006-01-30.gz
2019-08-08 13:04:03 | climateprediction.net | Started download of NATclim_ancil_14months_OSTIA_ice_0000P.gz
2019-08-08 13:04:03 | climateprediction.net | Started download of so2dms_prei_N96_1855_0000P.gz
2019-08-08 13:04:03 | climateprediction.net | Started download of oxi.addfa.gz
2019-08-08 13:04:03 | climateprediction.net | Started download of ozone_preind_N96_1879_0000Pv5.gz
2019-08-08 13:04:05 | climateprediction.net | Giving up on download of wah2_cam25_a0d2_200405_18_823_011890237.zip: permanent HTTP error
2019-08-08 13:04:05 | climateprediction.net | Giving up on download of atmos_restart_batch_694_cam25_m01l_2004-05-01.gz: permanent HTTP error
2019-08-08 13:04:05 | climateprediction.net | Giving up on download of region_restart_batch_694_cam25_m01l_2004-05-01.gz: permanent HTTP error
2019-08-08 13:04:05 | climateprediction.net | Finished download of ic19610309_16_N96.gz
2019-08-08 13:04:05 | climateprediction.net | Finished download of NATclim_ancil_14months_OSTIA_ice_0000P.gz
2019-08-08 13:04:05 | climateprediction.net | Finished download of ozone_preind_N96_1879_0000Pv5.gz
2019-08-08 13:04:06 | climateprediction.net | Finished download of so2dms_prei_N96_1855_0000P.gz
2019-08-08 13:04:14 | climateprediction.net | Finished download of NATclim_ancil_134months_OSTIA_sst_MMM_1994-12-01_2006-01-30.gz
2019-08-08 13:04:21 | climateprediction.net | Finished download of oxi.addfa.gz
5) Message boards : Number crunching : Upload failures (Message 60763)
Posted 31 Jul 2019 by Mephist0
Post:
Ha! You call that slow?
For years I was getting about 40 KB/s on a good day, and around 10-14 KB/s on a bad day.

Now that I've got super fast fiber, I get somewhere over a 100KB/s. (It's been so long since there was work that I've forgotten the exact number.)


Haha.. Yeah it was slow for me since i had to be in the datacenter until the uploads completed and it was hundreds of files to upload at once ;)

You say "super fast fiber" ... Now i guess that the bottleneck is not your fibre connection but the CPDN bandwith cap instead :)

I'm on fibre too. I can get 1 Gbit/s but i dont need that kind of speed. I'm paying 25$/month for 100 Mbit down and 10 Mbit up. I don't need faster than that.. An awesome plus for this connection is that my real speed is 135/17 Mbit so a lot more than i pay for :)
6) Message boards : Number crunching : Upload failures (Message 60762)
Posted 31 Jul 2019 by Mephist0
Post:
For everyones info..

sam50 units are limited to 1 MiB/s in upload speed
anz50 units are limitet to 100 KiB/s in upload speed..

Observed sitting waiting for 98 files to upload :D

To see if you can increase your upload speed you could try changing Max files per project to 1. It may not have any effect but it's worth a try
<max_file_xfers_per_project>1</max_file_xfers_per_project>


Yeah i had it at 1 when i was trying to fix my problems to not cause to many attempts on the climate servers. i got Max 1 MiB/s .. I had to increase the transfers slots to 10 since i was uploading both anz50 and sam50 units and it was taking forever. When i was transfering sam50 units only i got 10 MiB/s upload speed when i only were transfering anz50 i got a total of 1 MiB/s for all 10 transfers so there is defenatly a cap :) I normally have my transfers set to 2 (default)
7) Message boards : Number crunching : Upload failures (Message 60751)
Posted 30 Jul 2019 by Mephist0
Post:
For everyones info..

sam50 units are limited to 1 MiB/s in upload speed
anz50 units are limitet to 100 KiB/s in upload speed..

Observed sitting waiting for 98 files to upload :D


So for me, my bored band is the limiting factor.


So whats your connection speed?
A while ago i had only mobile dongle to upload from a server park.. 3 Mbit (aprox 300KB/s).. THAT took a LOOOOONG time.... :D
8) Message boards : Number crunching : Upload failures (Message 60749)
Posted 30 Jul 2019 by Mephist0
Post:
For everyones info..

sam50 units are limited to 1 MiB/s in upload speed
anz50 units are limitet to 100 KiB/s in upload speed..

Observed sitting waiting for 98 files to upload :D
9) Message boards : Number crunching : Upload failures (Message 60748)
Posted 30 Jul 2019 by Mephist0
Post:
Sorry for all the posts..

My issues is resolved now. Turns out the network i ran all my computers on have a proxy and that one interferes with the transfers in some way. When i moved the files to my laptop and connected direct to the internet instead of my work network problem was resolved!

My 98 files are on the way to upload now. Getting around 10 Mbit in speed on a 100 Mbit line :) Hope they will register since they will upload on another machine...
10) Message boards : Number crunching : Upload failures (Message 60747)
Posted 30 Jul 2019 by Mephist0
Post:
OK, update:

The issue is one of bandwidth onto the system. We are constantly running 146 upload processes at the moment and have been since I announced on the board that we were restarting jasmin. This is one of the issues around backing up when not running.
We are investigating how we can setup another upload server within the infrastructure that can be run as part of a load balancing pair behind the jasmin-upload name.


David


Out of curiosity.. How much bandwidth does this server / or connection have available?

Is there some load graph available to view? :) I wanted to know if there is any hour during the day where it might be easier to upload results (since im manually uploading).
11) Message boards : Number crunching : Upload failures (Message 60746)
Posted 30 Jul 2019 by Mephist0
Post:
Ok, so now on a new computer.. But still same issue.. Good now atleast i know its not the proxy server :)

So i guess the project still have issues with transient HTTP errors. :(
12) Message boards : Number crunching : Upload failures (Message 60744)
Posted 30 Jul 2019 by Mephist0
Post:
I will move my boinc installation to another computer that does not need a proxy server in between to see if the behaviour changes since i have had issues a long time now... :)
13) Message boards : Number crunching : Upload failures (Message 60743)
Posted 30 Jul 2019 by Mephist0
Post:
So the backlog of the sam50 (Jasmin) server is now resolved? .. I still get transient http error. So it seems Climateprediction.net cannot handle a proxy server anymore?

Or should i just keep trying?

Seems like one file succeded in uploading though..

2019-07-30 09:35:18 | climateprediction.net | Temporarily failed upload of wah2_sam50_n3ei_201412_24_808_011812012_0_r1312120817_restart.zip: transient HTTP error
2019-07-30 09:35:18 | climateprediction.net | Backing off 00:02:34 on upload of wah2_sam50_n3ei_201412_24_808_011812012_0_r1312120817_restart.zip
2019-07-30 09:35:18 | climateprediction.net | Started upload of wah2_sam50_n3ei_201412_24_808_011812012_0_r1312120817_24.zip
2019-07-30 09:35:19 | | Internet access OK - project servers may be temporarily down.
2019-07-30 09:35:21 | | Project communication failed: attempting access to reference site
2019-07-30 09:35:21 | climateprediction.net | Temporarily failed upload of wah2_sam50_n3ei_201412_24_808_011812012_0_r1312120817_24.zip: transient HTTP error
2019-07-30 09:35:21 | climateprediction.net | Backing off 00:02:57 on upload of wah2_sam50_n3ei_201412_24_808_011812012_0_r1312120817_24.zip
2019-07-30 09:35:21 | climateprediction.net | Started upload of wah2_sam50_n3ei_201412_24_808_011812012_0_r1312120817_out.zip
2019-07-30 09:35:22 | | Internet access OK - project servers may be temporarily down.
2019-07-30 09:35:23 | climateprediction.net | Finished upload of wah2_sam50_n3ei_201412_24_808_011812012_0_r1312120817_out.zip

2019-07-30 09:35:24 | climateprediction.net | Started upload of wah2_anz50_a0k6_201612_20_793_011761372_1_r261502111_2.zip
2019-07-30 09:35:28 | | Project communication failed: attempting access to reference site
2019-07-30 09:35:28 | climateprediction.net | Temporarily failed upload of wah2_anz50_a0k6_201612_20_793_011761372_1_r261502111_2.zip: transient HTTP error
14) Message boards : Number crunching : Upload failures (Message 60570)
Posted 3 Jul 2019 by Mephist0
Post:
Here is the log for this attempt with ANZ50 file..

Dont know if it is possible to see something there..

2019-07-03 12:05:11 | climateprediction.net | Started upload of wah2_anz50_a0k6_201612_20_793_011761372_1_r261502111_3.zip
2019-07-03 12:05:11 | climateprediction.net | [file_xfer] URL: http://upload4.cpdn.org/cpdn_cgi/file_upload_handler
2019-07-03 12:05:12 | | [http_xfer] [ID#0] HTTP: wrote 1210 bytes
2019-07-03 12:05:12 | | [http_xfer] [ID#0] HTTP: wrote 2819 bytes
2019-07-03 12:05:12 | | [http_xfer] [ID#0] HTTP: wrote 3523 bytes
2019-07-03 12:05:12 | | [http_xfer] [ID#0] HTTP: wrote 3596 bytes
2019-07-03 12:05:12 | | [http_xfer] [ID#0] HTTP: wrote 1734 bytes
2019-07-03 12:05:12 | | Internet access OK - project servers may be temporarily down.
2019-07-03 12:05:12 | | [http_xfer] [ID#232] HTTP: wrote 98 bytes
2019-07-03 12:05:13 | climateprediction.net | [file_xfer] http op done; retval 0 (Success)
2019-07-03 12:05:13 | climateprediction.net | [file_xfer] parsing upload response: <data_server_reply> <status>0</status> <file_size>262144</file_size></data_server_reply>
2019-07-03 12:05:13 | climateprediction.net | [file_xfer] parsing status: 0
2019-07-03 12:05:13 | climateprediction.net | [fxd] starting upload, upload_offset 262144
2019-07-03 12:05:15 | | Project communication failed: attempting access to reference site
2019-07-03 12:05:15 | climateprediction.net | [file_xfer] http op done; retval -184 (transient HTTP error)
2019-07-03 12:05:15 | climateprediction.net | [file_xfer] file transfer status -184 (transient HTTP error)
2019-07-03 12:05:15 | climateprediction.net | Temporarily failed upload of wah2_anz50_a0k6_201612_20_793_011761372_1_r261502111_3.zip: transient HTTP error
15) Message boards : Number crunching : Upload failures (Message 60569)
Posted 3 Jul 2019 by Mephist0
Post:
Damnit.. Seems i still have issues uploading ANZ50 files.. I dont get it.. I have tried 3 different proxy software now and i changed the project from HTTP to HTTPS deleting all my file transfers for that one.. But i still cannot upload it seems like...

2019-07-03 11:57:22 | climateprediction.net | Temporarily failed upload of wah2_anz50_a0k6_201612_20_793_011761372_1_r261502111_3.zip: transient HTTP error

The ANZ50 server should accept files without problems i guess? I dont get it why i have issues here...
16) Message boards : Number crunching : Upload failures (Message 60568)
Posted 3 Jul 2019 by Mephist0
Post:
Ok thanks, then i know. I have no rush. Its only that i had problems before the disk space problems started.. But issues seems to have been resolved now changing proxy software. I will try to send the work in 3 weeks when i get back from my vacation :)
17) Message boards : Number crunching : Upload failures (Message 60566)
Posted 3 Jul 2019 by Mephist0
Post:
ok thanks!
I have 75 files to upload (around 5GB of data). Good thing the timelimit is long then ;)

I had some problems with bad proxy software also that seemed to transfer the files even if the server would not accept it or something like that. I have changed proxy software now. It does not start to transfer anything and ends after a few minutes with "transient HTTP error" so it seems to be ok then :)

The old proxy software worked fine when transfers were working fine but seems now when project has issues the problem with the old proxy shows :)

Problem is that i cannot leave the proxy software running for too long since its not allowed on the network. But i will have to start it once in a while and see if it transfers or not then :)
18) Message boards : Number crunching : Upload failures (Message 60564)
Posted 3 Jul 2019 by Mephist0
Post:
Has anyone been able to upload anything to Jasmin recently?

I have some anz50 tasks that is not using Jasmin server i guess, any idea how i could get them to upload instead? It seems it tries the same files over and over again..
19) Message boards : Number crunching : Upload failures (Message 60544)
Posted 2 Jul 2019 by Mephist0
Post:
Ok. I will try tomorrow and the next day. Then im on vacation for 3 weeks then another try ;)
20) Message boards : Number crunching : Upload failures (Message 60542)
Posted 2 Jul 2019 by Mephist0
Post:
I have 71 files to upload. I will try to upload a few just to see if my tranfer is now working since i had issues with transient errors even before the space issue begun..

I have configured just to upload one file at a time and also max 1MiB/s in upload speed. Seems my upload was at 0 for 3,5 minutes then it started to upload.

1st upload: 74MB file, Transient HTTPS error :(
2nd upload: 43MB file (sam50) Transient HTTP error...
3rd upload: 43MB file (anz50) Transient HTTP error...

So.. Seems i will not be able to get Climate prediction to work with my Proxy setup. Other projects works fine.. So i dont get it why Climate Prediction is not working?

I also get the following error for some reason...
2019-07-02 09:23:39 | | Project communication failed: attempting access to reference site


Next 20

©2020 climateprediction.net