wiki:Projects/INSPECT/ShaCoNEMO-ORCA1-ARCHER

Version 5 (modified by annette, 3 years ago) (diff)

IPSL ORCA1 on ARCHER

Instructions for running the ShaCoNEMO ORCA1 configuration on ARCHER.

Access

You need to register with the NEMO website to extract the code: http://www.nemo-ocean.eu/user/register

The ORCA1 configuration is held in the ShaCoNEMO repository. You need to register for access, and this uses the same username and password as the main NEMO repository.

To create a branch to hold your code changes on the NEMO repository, you will need additional permissions.

References

The main NEMO website gives more general instructions on building & running NEMO with the standard confgurations included in the code: http://www.nemo-ocean.eu/Using-NEMO/User-Guides/Basics/NEMO-Quick-Start-Guide

The ORCA1_LIM3_PISCES configuration is documented on the ShaCoNEMO (Shared Configuration NEMO) wiki: https://forge.ipsl.jussieu.fr/shaconemo/wiki/README_ORCA1_LIM3_PISCES

  • LIM3 is the sea-ice
  • PISCES is the biogeochemical model

NEMO uses an external library to handle model output diagnostics called XIOS, which is documented here: http://forge.ipsl.jussieu.fr/ioserver/wiki

Code versions

Here we have used the following versions, based on the NEMO/ShaCoNEMO pages:

  • XIOS-1.0 r703
  • NEMO_3.6_stable r6969
  • ShaCoNEMO r60
  • ORCA1 input data set v9

Note: Although XIOS-2.0 is now recommended, revision 932 which is suggested for ORCA1 gave a link-time error which was not investigated.

Note: This is based on the latest version of ShaCoNEMO as of October 2016, and the recommended version of NEMO for this. Both are under active development, so you may wish to use newer versions as they become available.

Code extract

First you need to check out the code from the NEMO repository.

  1. On ARCHER, navigate to the directory that you wish to hold the code. If you are making changes to the standard configuration, you may wish to use the home directory as this is backed up.
  1. To checkout the code to a directory called NEMOGCM_3.6_r6969, run the following, replacing mylogin with your NEMO username (you will be prompted for your password):
    svn --username "mylogin" co -r6969 http://forge.ipsl.jussieu.fr/nemo/svn/branches/2015/nemo_v3_6_STABLE/NEMOGCM NEMOGCM_3.6_r6969
    

This creates a subversion "working copy". Running svn info lists the revision that was checked out, and svn diff lists the changes that have been made.

  1. To setup and extract the ORCA1 configuration:
    cd NEMOGCM_3.6_r6969/CONFIG
    

Then edit cfg.txt to add the line:

ORCA1_LIM3_PISCES OPA_SRC LIM_SRC_3 NST_SRC TOP_SRC
  1. Next checkout the ORCA1 configuration using the same username (mylogin) and password as for the NEMO checkout:
    svn --username "mylogin" co http://forge.ipsl.jussieu.fr/shaconemo/svn/trunk/ORCA1_LIM3_PISCES
    

This creates the directory ORCA1_LIM3_PISCES, which is another "working copy".

Code build

To build NEMO, you need to point to the location of XIOS, and the parallel-netCDF libraries. A version of XIOS-1 can be found here:

/work/n02/n02/annette/nemo/xios-1.0_r703

It is advisable to build NEMO with the same key modules as the library. The above version was built with:

  • cce/8.4.1
  • cray-netcdf-hdf5parallel/4.3.3.1
  • cray-mpich/7.2.6

(To build your own XIOS, see the notes below.)

To build the NEMO code:

  1. Navigate to the directory holding the build definition files:
    cd NEMOGCM_3.6_r6969/ARCH
    

Then create a new ARCHER configuration file in this directory arch-XC30_NCAS_ARCHER.fcm:

wget http://cms.ncas.ac.uk/attachment/wiki/CmsOnly/Annette/NEMO-howto/arch-XC30_NCAS_ARCHER.fcm

Note this uses the compile flags used for the Met Office GO6 runs.

  1. Go to the config directory:
    cd ../CONFIG
    
    And create a batch submission script for the compilation build_nemo.pbs:
    wget http://cms.ncas.ac.uk/attachment/wiki/CmsOnly/Annette/NEMO-howto/build_nemo.pbs
    
    Edit the PBS headers in the file to set your own project code:
    #PBS -A <xxx>
    
    This build script sets the ARCHER modules required to build with the XIOS library described above, plus it removes the pre-processor flag key_xios2 for building with XIOS-1, and adds key_nosignedzero.
  1. Then submit the build script:
    qsub build_nemo.pbs
    
    This creates a new directory under CONFIG/ called O1L3P. This differentiates between changes made to the original configuration. O1L3P will contain the executable plus namelists to be read in by the model.

Model run

Once the build has completed successfully, set up the inputs required for the model to run.

  1. Download the ORCA1 input data set, and untar. This needs to go on your work space on ARCHER (/work/n02/n02/<username>/):
    wget http://prodn.idris.fr/thredds/catalog/ipsl_public/romr006/SHACONEMO/eORCA1/catalog.html?dataset=DatasetScanipsl_public/romr006/SHACONEMO/eORCA1/INPUTS_ORCA1_LIM3_PISCES_V9.tar
    mkdir INPUTS_ORCA1_LIM3_PISCES_V9
    tar -xvf INPUTS_ORCA1_LIM3_PISCES_V9.tar -C INPUTS_ORCA1_LIM3_PISCES_V9
    
  1. Go to the run directory:
    cd NEMOGCM_3.6_r6969/CONFIG/O1L3P/EXP00
    
    NEMO expects the input data in its working directory, so create symlinks:
    ln -s <path-to-data>/INPUTS_ORCA1_LIM3_PISCES_V9/* . 
    
  1. In the same directory, copy in the XIOS-1 XML files (by default the configuration is set up for XIOS-2):
    mv ../../ORCA1_LIM3_PISCES/EXP00/ForXIOS1/* . 
    
  1. Make namelist changes:
  1. Copy over to work filesystem
  1. Submit run.

Working practices

Use of home Branches Making changes in ORCA1_LIM3 …

Building XIOS

http://forge.ipsl.jussieu.fr/ioserver/wiki/documentation

Edit the build file to point to your version. If using xios-2 … don't remove flag or copy in files.

Attachments (3)

Download all attachments as: .zip