climateprediction.net home page
OpenIFS Discussion

OpenIFS Discussion

Message boards : Number crunching : OpenIFS Discussion
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 31 · Next

AuthorMessage
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1083
Credit: 16,671,999
RAC: 5,277
Message 66558 - Posted: 24 Nov 2022, 9:54:17 UTC

(If this is the wrong place for this question, where should I put it?)

Currently, I am using this for my app_config file for Climate Prediction models.
[/var/lib/boinc/projects/climateprediction.net]$ cat app_config.xml 
<app_config>
    <project_max_concurrent>5</project_max_concurrent>
</app_config>


If I wish to change this to two concurrent models to run at a time, it is obvious how to do it.
But how would I restrict my machine to do only two OpenIFS models, but still allow five traditional models to run if there are no OpenIFS models? Or to allow two OpenIFs models and three traditional models, or something like that. I have 65 GBytes RAM and a fast Internet connection.

Memory 	62.28 GB
Cache 	16896 KB
Total disk space 	488.04 GB
Free Disk Space 	482.25 GB
Measured floating point speed  6.13 billion ops/sec
Measured integer speed 	      26.09 billion ops/sec
Average upload rate 	  149.38 KB/sec
Average download rate 	11956.26 KB/sec

ID: 66558 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 947
Credit: 34,468,484
RAC: 4,162
Message 66559 - Posted: 24 Nov 2022, 10:12:19 UTC - in response to Message 66558.  

Refer to the user manual for project-level configuration, where the full list of options for app_config.xml are defined.

You would to add a separate <app>...</app> section for each IFS variant, once we know the exact application names in use. You could then use <max_concurrent> to limit each IFS type, but I don't see a way to limit the total IFS number of all types, once multiple versions are in play at the same time.
ID: 66559 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4365
Credit: 16,614,059
RAC: 1,518
Message 66560 - Posted: 24 Nov 2022, 10:31:47 UTC - in response to Message 66559.  

You would to add a separate <app>...</app> section for each IFS variant, once we know the exact application names in use. You could then use <max_concurrent> to limit each IFS type, but I don't see a way to limit the total IFS number of all types, once multiple versions are in play at the same time.
Depends how likely it is that batches of multiple variants will be out there at once I guess.
ID: 66560 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1083
Credit: 16,671,999
RAC: 5,277
Message 66562 - Posted: 24 Nov 2022, 13:50:14 UTC - in response to Message 66560.  

You would to add a separate <app>...</app> section for each IFS variant, once we know the exact application names in use. You could then use <max_concurrent> to limit each IFS type, but I don't see a way to limit the total IFS number of all types, once multiple versions are in play at the same time.

Depends how likely it is that batches of multiple variants will be out there at once I guess.


I have an upper limit of 12 (in the winter, and 8 in the summer) of total BOINC jobs. (No air conditioning.)
I am currently allowing an upper limit of 5 CPDN jobs. So now I have a prototype app_config.xml file that allows a max of 2 for each of the OpenIFS types and a max of 3 for each of the "traditional" ones. If there are only one variant of each distributed at a time, I should be OK. And the max limit of BOINC jobs will prevent disaster. I hope.

It looks, in part, like this:
$ cat app_config.xml 
<app_config>
<app>
    <name>OpenIFSname1</name>
    <max_concurrent>2</max_concurrent>
    </app>
<app>
    <name>OpenIFSname2</name>
    <max_concurrent>2</max_concurrent>
   </app>
<app>
    <name>hadam3_8.09</name>  <---<<<
    <max_concurrent>3</max_concurrent>
    </app>

I guessed at the names of the traditional tasks. One is marked with <---<<< . Is that the correct way the traditional ones are named?


This prototype is not in effect yet
ID: 66562 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4365
Credit: 16,614,059
RAC: 1,518
Message 66563 - Posted: 24 Nov 2022, 14:39:06 UTC

Last test batch were oifs_43r3_ps so perturbed surface ones.
ID: 66563 · Report as offensive     Reply Quote
Bryn Mawr

Send message
Joined: 28 Jul 19
Posts: 147
Credit: 12,827,924
RAC: 918
Message 66564 - Posted: 24 Nov 2022, 14:53:07 UTC - in response to Message 66562.  

You would to add a separate <app>...</app> section for each IFS variant, once we know the exact application names in use. You could then use <max_concurrent> to limit each IFS type, but I don't see a way to limit the total IFS number of all types, once multiple versions are in play at the same time.

Depends how likely it is that batches of multiple variants will be out there at once I guess.


I have an upper limit of 12 (in the winter, and 8 in the summer) of total BOINC jobs. (No air conditioning.)
I am currently allowing an upper limit of 5 CPDN jobs. So now I have a prototype app_config.xml file that allows a max of 2 for each of the OpenIFS types and a max of 3 for each of the "traditional" ones. If there are only one variant of each distributed at a time, I should be OK. And the max limit of BOINC jobs will prevent disaster. I hope.

It looks, in part, like this:
$ cat app_config.xml 
<app_config>
<app>
    <name>OpenIFSname1</name>
    <max_concurrent>2</max_concurrent>
    </app>
<app>
    <name>OpenIFSname2</name>
    <max_concurrent>2</max_concurrent>
   </app>
<app>
    <name>hadam3_8.09</name>  <---<<<
    <max_concurrent>3</max_concurrent>
    </app>

I guessed at the names of the traditional tasks. One is marked with <---<<< . Is that the correct way the traditional ones are named?


This prototype is not in effect yet


Add a project max concurrent to that and job's a good'un
ID: 66564 · Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 5 Aug 04
Posts: 1083
Credit: 16,671,999
RAC: 5,277
Message 66565 - Posted: 24 Nov 2022, 15:50:22 UTC - in response to Message 66564.  

Add a project max concurrent to that and job's a good'un


Do you mean like this?

$ cat app_config.xml 
<app_config>
    <project_max_concurrent>5</project_max_concurrent>
<app>
    <name>OpenIFSname1</name>
    <max_concurrent>2</max_concurrent>
    </app>
<app>
    <name>OpenIFSname2</name>
    <max_concurrent>2</max_concurrent>
   </app>
<app>
    <name>hadam3_8.09</name>
    <max_concurrent>3</max_concurrent>
    </app>
<app>
    <name>hadam3_8.52</name>
    <max_concurrent>3</max_concurrent>
    </app>
<app>
    <name>hadcm3s_8.36</name>
    <max_concurrent>3</max_concurrent>
    </app>
<app>
    <name>hadsm3_8.02</name>
    <max_concurrent>3</max_concurrent>
    </app>
</app_config>


If so, why have the the itemized list of the traditional work units in there at all?
ID: 66565 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 847
Credit: 13,960,536
RAC: 6,962
Message 66566 - Posted: 24 Nov 2022, 16:17:09 UTC

@Dave - could we move this discussion to a new thread 'OpenIFS Discussions' please? Would be more appropriate there, and keep this thread for FAQ only. Otherwise the FAQ will get lost in the long chain of messages.

I've already sent a list of OpenIFS app names to the forums. Please see this message: https://www.cpdn.org/forum_thread.php?id=9149&postid=66352

I will add these to the FAQ.
ID: 66566 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 947
Credit: 34,468,484
RAC: 4,162
Message 66568 - Posted: 24 Nov 2022, 17:01:22 UTC - in response to Message 66565.  

For the discussion thread:

For those adding app_config for the first time, and wanting to cover the 'traditional' app names, I have

hadam4
hadam4h
hadcm3s
hadsm4

In other words, the version numbers have not traditionally been included in the app_names.
ID: 66568 · Report as offensive     Reply Quote
Bryn Mawr

Send message
Joined: 28 Jul 19
Posts: 147
Credit: 12,827,924
RAC: 918
Message 66573 - Posted: 25 Nov 2022, 7:54:05 UTC - in response to Message 66565.  

Add a project max concurrent to that and job's a good'un


Do you mean like this?

$ cat app_config.xml 
<app_config>
    <project_max_concurrent>5</project_max_concurrent>
<app>
    <name>OpenIFSname1</name>
    <max_concurrent>2</max_concurrent>
    </app>
<app>
    <name>OpenIFSname2</name>
    <max_concurrent>2</max_concurrent>
   </app>
<app>
    <name>hadam3_8.09</name>
    <max_concurrent>3</max_concurrent>
    </app>
<app>
    <name>hadam3_8.52</name>
    <max_concurrent>3</max_concurrent>
    </app>
<app>
    <name>hadcm3s_8.36</name>
    <max_concurrent>3</max_concurrent>
    </app>
<app>
    <name>hadsm3_8.02</name>
    <max_concurrent>3</max_concurrent>
    </app>
</app_config>


If so, why have the the itemized list of the traditional work units in there at all?


Yes, the project max controls the overall count and the itemised list controls the individual apps and you have as much control as you want.
ID: 66573 · Report as offensive     Reply Quote
AndreyOR

Send message
Joined: 12 Apr 21
Posts: 247
Credit: 12,074,507
RAC: 2,078
Message 66574 - Posted: 25 Nov 2022, 10:48:10 UTC - in response to Message 66565.  

If so, why have the the itemized list of the traditional work units in there at all?

With that app_config you'll run at most 5 CPDN tasks total. They'll be on first come first serve basis. So it's possible, if work is available for multiple apps, that you won't get any work for some of them (may be able to download but not run) because the total tasks for CPDN will already be at 5. There's no way to control things conditionally (if...then) in app_config. You don't need to itemize, the only time it'll make a difference is if work is available for multiple apps at the same time.

Check Richard's post for correct Hadley model names.
ID: 66574 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4365
Credit: 16,614,059
RAC: 1,518
Message 66575 - Posted: 25 Nov 2022, 11:03:01 UTC

I expect the answer is negative but would the app_config file accept wildcards? If it did, that would be a way to limit the total number of OpenIFS tasks.
ID: 66575 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 947
Credit: 34,468,484
RAC: 4,162
Message 66577 - Posted: 25 Nov 2022, 11:17:12 UTC - in response to Message 66575.  
Last modified: 25 Nov 2022, 12:04:44 UTC

I expect the answer is negative but would the app_config file accept wildcards? If it did, that would be a way to limit the total number of OpenIFS tasks.
I think it's highly unlikely, but I can check the code. That's not the way David Anderson's mind works.

Edit - no, no sign of a wildcard handler in the code.
ID: 66577 · Report as offensive     Reply Quote
[AF] Kalianthys

Send message
Joined: 20 Dec 20
Posts: 11
Credit: 35,135,967
RAC: 56,721
Message 66590 - Posted: 26 Nov 2022, 19:28:53 UTC

Hello,

I have two task OpenIFS 43r3 Baroclinic Lifecycle v1.03 x86_64-pc-linux-gnu on my server : https://www.cpdn.org/results.php?hostid=1533394

The 2 tasks seem finished but they do not start on the server :

======== File transfers ========
1) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_0.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 18.170831
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
2) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_1.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 15.158657
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
3) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_2.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 6.125612
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
4) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_3.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 6.125960
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
5) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_4.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 3.013341
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
6) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_5.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.072802
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
7) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_6.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.069479
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
8) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_7.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.022983
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
9) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_8.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.023066
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
10) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_9.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.058683
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
11) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_10.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.058159
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
12) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_11.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.024752
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
13) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_12.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.024706
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
14) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_13.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.078303
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
15) -----------
   name: oifs_43r3_bl_k028_2016092300_15_943_12163082_0_r29951873_14.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 5.123589
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
16) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_0.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 10.116536
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
17) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_1.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 6.096991
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
18) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_2.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 6.122655
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
19) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_3.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 6.123920
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
20) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_4.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 37.340122
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
21) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_5.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 7.092978
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
22) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_6.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 8.045169
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
23) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_7.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 6.124228
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
24) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_8.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 6.086962
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
25) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_9.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.025336
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
26) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_10.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 6.041083
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
27) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_11.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.023235
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
28) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_12.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.058181
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
29) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_13.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.061807
   bytes_xferred: 113.000000
   xfer_speed: 0.000000
30) -----------
   name: oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_14.zip
   direction: upload
   sticky: no
   xfer active: no
   time_so_far: 4.024040
   bytes_xferred: 113.000000
   xfer_speed: 0.000000



I tried to send it back but without success.

Error logs :


817: 26-Nov-2022 13:16:15 (low) [climateprediction.net] Temporarily failed upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_13.zip: transient upload error
818: 26-Nov-2022 13:16:15 (low) [climateprediction.net] Backing off 00:04:34 on upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_13.zip
819: 26-Nov-2022 13:16:15 (low) [climateprediction.net] Temporarily failed upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_14.zip: transient upload error
820: 26-Nov-2022 13:16:15 (low) [climateprediction.net] Backing off 00:07:17 on upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_14.zip
821: 26-Nov-2022 13:16:16 (low) [climateprediction.net] Started upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_6.zip
822: 26-Nov-2022 13:16:18 (internal error) [climateprediction.net] [error] Error reported by file upload server: can't write to upload_dir
823: 26-Nov-2022 13:16:18 (low) [climateprediction.net] Temporarily failed upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_6.zip: transient upload error
824: 26-Nov-2022 13:16:18 (low) [climateprediction.net] Backing off 00:13:19 on upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_6.zip
825: 26-Nov-2022 13:16:20 (low) [climateprediction.net] Started upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_5.zip
826: 26-Nov-2022 13:16:22 (internal error) [climateprediction.net] [error] Error reported by file upload server: can't write to upload_dir
827: 26-Nov-2022 13:16:22 (low) [climateprediction.net] Temporarily failed upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_5.zip: transient upload error
828: 26-Nov-2022 13:16:22 (low) [climateprediction.net] Backing off 00:13:51 on upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_5.zip
829: 26-Nov-2022 13:16:26 (low) [climateprediction.net] Started upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_4.zip
830: 26-Nov-2022 13:16:28 (internal error) [climateprediction.net] [error] Error reported by file upload server: can't write to upload_dir
831: 26-Nov-2022 13:16:28 (low) [climateprediction.net] Temporarily failed upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_4.zip: transient upload error
832: 26-Nov-2022 13:16:28 (low) [climateprediction.net] Backing off 00:08:58 on upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_4.zip
833: 26-Nov-2022 13:16:30 (low) [climateprediction.net] Started upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_3.zip
834: 26-Nov-2022 13:16:32 (internal error) [climateprediction.net] [error] Error reported by file upload server: can't write to upload_dir
835: 26-Nov-2022 13:16:32 (low) [climateprediction.net] Temporarily failed upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_3.zip: transient upload error
836: 26-Nov-2022 13:16:32 (low) [climateprediction.net] Backing off 00:14:24 on upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_3.zip
837: 26-Nov-2022 13:16:35 (low) [climateprediction.net] Started upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_2.zip
838: 26-Nov-2022 13:16:37 (internal error) [climateprediction.net] [error] Error reported by file upload server: can't write to upload_dir
839: 26-Nov-2022 13:16:37 (low) [climateprediction.net] Temporarily failed upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_2.zip: transient upload error
840: 26-Nov-2022 13:16:37 (low) [climateprediction.net] Backing off 00:13:46 on upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_2.zip
841: 26-Nov-2022 13:16:40 (low) [climateprediction.net] Started upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_1.zip
842: 26-Nov-2022 13:16:42 (internal error) [climateprediction.net] [error] Error reported by file upload server: can't write to upload_dir
843: 26-Nov-2022 13:16:42 (low) [climateprediction.net] Temporarily failed upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_1.zip: transient upload error
844: 26-Nov-2022 13:16:42 (low) [climateprediction.net] Backing off 00:10:22 on upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_1.zip
845: 26-Nov-2022 13:16:44 (low) [climateprediction.net] Started upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_0.zip
846: 26-Nov-2022 13:16:46 (internal error) [climateprediction.net] [error] Error reported by file upload server: can't write to upload_dir
847: 26-Nov-2022 13:16:46 (low) [climateprediction.net] Temporarily failed upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_0.zip: transient upload error
848: 26-Nov-2022 13:16:46 (low) [climateprediction.net] Backing off 00:47:14 on upload of oifs_43r3_bl_k029_2016092300_15_943_12163083_0_r1290750425_0.zip


Kali.
ID: 66590 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 947
Credit: 34,468,484
RAC: 4,162
Message 66591 - Posted: 26 Nov 2022, 20:24:56 UTC

822: 26-Nov-2022 13:16:18 (internal error) [climateprediction.net] [error] Error reported by file upload server: can't write to upload_dir
That's critical, but should be easy to fix.
ID: 66591 · Report as offensive     Reply Quote
Profile Dave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4365
Credit: 16,614,059
RAC: 1,518
Message 66592 - Posted: 26 Nov 2022, 20:51:25 UTC - in response to Message 66577.  

I expect the answer is negative but would the app_config file accept wildcards? If it did, that would be a way to limit the total number of OpenIFS tasks.
I think it's highly unlikely, but I can check the code. That's not the way David Anderson's mind works.

Edit - no, no sign of a wildcard handler in the code.
Thanks for checking Richard. I thought it highly unlikely but also that it would be silly not to check.
ID: 66592 · Report as offensive     Reply Quote
[AF] Kalianthys

Send message
Joined: 20 Dec 20
Posts: 11
Credit: 35,135,967
RAC: 56,721
Message 66595 - Posted: 28 Nov 2022, 9:44:53 UTC - in response to Message 66591.  

822: 26-Nov-2022 13:16:18 (internal error) [climateprediction.net] [error] Error reported by file upload server: can't write to upload_dir
That's critical, but should be easy to fix.


Hi,

I hope this will be corrected before December 2 (deadline for sending).

Kali.
ID: 66595 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 847
Credit: 13,960,536
RAC: 6,962
Message 66597 - Posted: 28 Nov 2022, 12:18:59 UTC - in response to Message 66591.  

822: 26-Nov-2022 13:16:18 (internal error) [climateprediction.net] [error] Error reported by file upload server: can't write to upload_dir
That's critical, but should be easy to fix.
I've let Andy@CPDN know (he may know already). These are new apps, which is why they are being tested with small batches on the production site before the much bigger batches go out.

The setup on the 'dev-test' site has to be moved over to the main production site. There are small differences between the two so sometimes things can get missed, esp for new apps.
ID: 66597 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 1 Jan 07
Posts: 947
Credit: 34,468,484
RAC: 4,162
Message 66598 - Posted: 28 Nov 2022, 13:58:41 UTC - in response to Message 66597.  

Understood. Unfortunately the batches are so small, and are snapped up so quickly, that I haven't seen a single one yet, on either site. When I do, I plan to make a full copy of the task specifications (app_version, workunit, result), so I can inspect details like the upload urls and give further guidance if needed.

Mind you, even that wouldn't be enough to diagnose a permissions problem, which I suspect this may be.
ID: 66598 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 847
Credit: 13,960,536
RAC: 6,962
Message 66599 - Posted: 28 Nov 2022, 17:41:07 UTC

Why doesn't this app_config.xml work?

My WSL/Ubuntu downloaded 4 oifs_43r3_bl apps and despite the following app_config, decided to start them all at once - any ideas why? My reading of: https://boinc.berkeley.edu/wiki/Client_configuration says this is all I need. The app names match that in the client_state.xml. I don't need a project_max_concurrent as that's optional. So why didn't this work?

<app_config>
   <app>
      <name>hadsm4</name>
      <max_concurrent>2</max_concurrent>
   </app>
   <app>
      <name>oifs_43r3</name>
      <max_concurrent>1</max_concurrent>
   </app>
   <app>
      <name>oifs_43r3_ps</name>
      <max_concurrent>1</max_concurrent>
   </app>
   <app>
      <name>oifs_43r3_bl</name>
      <max_concurrent>1</max_concurrent>
   </app>
</app_config>
ID: 66599 · Report as offensive     Reply Quote
1 · 2 · 3 · 4 . . . 31 · Next

Message boards : Number crunching : OpenIFS Discussion

©2024 climateprediction.net