Setting up UM 8.4 to run on Polaris

1. Central Installation Directory Structure

The vn 8.4 installation is under /home/polaris_lds1/earhum/vn8.4.

2. Your Setup on Polaris

Copy the UM Setup section from /home/polaris_lds1/ldsgl/.bashrc to your .bashrc. In order to use vn 8.4 utilities make sure you source .umsetvars_8.4.

3. Model Build Information and UMUI Settings

The model build information is kept in configuration files which are read by the FCM build system. The appropriate compiler flags are used to build the model along with information about which libraries to link. We have used the following flags for the Polaris intel compiler for both model and communications builds

-i8 -r8 -fp-model precise -O1 -openmp

FCM Branches

Navigate to model selection → FCM configuration → FCM options for atmosphere and reconfiguration.

Until we introduce keywords, specify revision number 11979 for the UM code base and include the following branch


in the User Modifications table. The naming of this necessary branch may change in future - this branch contains a script change needed to ensure CRUNS behave correctly.

General UMUI Settings

Navigate to model selection → FCM configuration → FCM extract directories and output levels . Experience will help determine the most convenient places where the extracted model should reside. Explicitly set UM_ROUTDIR to be your user directory on /nobackup, however, note that files left in this directory may be marked for deletion in accordance with Polaris data policy.

Navigate to model selection → user information and submit method → general details Set User-id to be your Polaris id. The Tic Code is not relevant to Polaris. We have not tested end of run email notification.

Navigate to model selection → user information and target machine → job submission method Choose the option 'qsub' for SGE(SGI) for the submission method, enter for the Host name, select Change machine config file and enter sgi-intel-polaris.

Navigate to model selection → input/output control and resources → user hand edit files and include /home/grenville/umui_jobs/hand_edits/polaris_8.4.1 in the Hand edits list. This is currently necessary to mange the way Polaris handles the module command in the k shell and selects the mvapich2 module to ensure that IO servers function correctly. Future work should see the need for this to disappear.

4 Example UMUI Job

The following job is currently under user grenville in the UMUI

Job Id Model
xidemHadGEM3-GA4.0 Polaris

This job uses the UM I/O servers which require that the model run with at least two OpenMP threads. We have successfully run the model using mvapich2 as the mpi implementation which supports multi-threaded mpi. The model runs with single threading with both openmpi and intemlpi mpi implemantations - in these cases OpenMP code directives are obeyed but the model can not support I/O servers.

Last modified 6 years ago Last modified on 08/02/16 16:21:01