Changes between Version 11 and Version 12 of UM/Configurations/UKESM/RelNotes1.0


Ignore:
Timestamp:
22/02/19 11:04:54 (8 months ago)
Author:
jwalton
Comment:

—

Legend:

Unmodified
Added
Removed
Modified

  • UM/Configurations/UKESM/RelNotes1.0

    v11 v12  
    104104When running on Met Office machines (including Monsoon), the suite will, by default, archive ''a single copy'' of its data to MOOSE.  For critical model runs, this setting may be changed to archive two copies of the data (i.e. duplex) by switching `non_duplexed_set` in `postproc -> Post Processing-common settings -> Moose Archiving` to `false`.  Further guidance on when to choose this option is available at http://www-twiki/Main/MassNonDuplexPolicy (note that this link only works from within the Met Office). 
    105105 
    106 == Compute node usage 
     106== Compute resource usage 
    107107 
    108 The compute resources used by the suite can be set via controls on the `Machine Options` and `Domain Decomposition` control panels under `suite conf`.  The following discussion is specific to the Met Office HPC for the most part, but may still be helpful for users of other machines. 
     108The compute resources used by the suite can be set via parameters on the `Machine Options` and `Domain Decomposition` control panels under `suite conf`.  The following discussion is specific to the Met Office HPC for the most part, but may still be helpful for users of other machines. 
    109109 
    110 The type of compute node can be set via `XC40 core type`: a `Haswell` node has 32 cores, while `Broadwell` has 36.   
     110The type of compute node can be set via `suite conf -> Machine Options -> XC40 core type`: a `Haswell` node has 32 cores, while `Broadwell` has 36.   
    111111 
    112 On `Domain Decomposition -> Atmosphere`, the number of processes used by the UM can be set via `Atmosphere: Processes East-West` and `Atmosphere: Processes North-South`; additional processes for the IO Server may be requested using `IO Server Processes`.  Finally, `OpenMP threads for the atmosphere` sets the number of threads for each process; multiplying this by the number of processes gives the number of compute tasks.  Using the default values for these parameters, the number of tasks used by the UM is `(32 * 18 + 0) * 2 = 1152`. Dividing by the number of cores per node (in this case `36`) and rounding up (because different executables cannot run on the same node) gives `32` compute nodes used by the atmosphere. 
     112The suite is currently set up in `suite conf -> Domain Decomposition` to use 36 nodes (see [#Calculationofnodecount below] for more details on how this is calculated).  An alternative setup uses 19 nodes.  Parameter settings for both setups are: 
     113 
     114||=''Parameter''=||=''36 node suite''=||=''19 node suite''=|| 
     115||=Atmosphere: Processes East-West=||=32=||=32=|| 
     116||=Atmosphere: Processes North-South=||=18=||=18=|| 
     117||=IO Server Processes=||=0=||=0=|| 
     118||=OpenMP threads for the atmosphere=||=2=||=1=|| 
     119||=NEMO: Number of processes East-West=||=12=||=9=|| 
     120||=NEMO: Number of processes North-South=||=9=||=8=|| 
     121||=NEMO: Number of processes in XIOS server=||=6=||=6=|| 
     122||=OpenMP threads for the ocean=||=1=||=1=|| 
     123 
     124Note that the ocean must be rebuilt (by setting `suite conf -> Build and Run -> Build Ocean` to `true`) whenever the NEMO parameters in the table are changed during a run.  
     125 
     126Setting these parameters to other values may require load balancing to ensure that HPC resources are being used in the most efficient fashion.  
     127 
     128=== Calculation of node count 
     129 
     130On `Domain Decomposition -> Atmosphere`, the number of processes used by the UM can be set via `Atmosphere: Processes East-West` and `Atmosphere: Processes North-South`; additional processes for the IO Server may be requested using `IO Server Processes`.  Finally, `OpenMP threads for the atmosphere` sets the number of threads for each process; multiplying this by the number of processes gives the number of compute tasks.   
     131 
     132Using the parameter values for the ''36 node suite'', the number of tasks used by the UM is `(32 * 18 + 0) * 2 = 1152`. Dividing by the number of cores per node (in this case `36`) and rounding up (because different executables cannot run on the same node) gives `32` compute nodes used by the atmosphere. 
    113133 
    114134A similar calculation may be performed for the settings on `Domain Decomposition -> Ocean` using `NEMO: Number of processes East-West`, `NEMO: Number of processes North-South` and `OpenMP threads for the ocean` to give `12 * 9 * 1 = 108` tasks, or `3` compute nodes used by the ocean. 
    … …  
    117137 
    118138Thus, the total number of nodes used by the suite is `32 + 3 + 1 = 36`. 
    119  
    120 Changing the number of nodes used by the suite may require load balancing to ensure that HPC resources are being used in the most efficient fashion.  
    121139 
    122140== Tests in the suite