Version 7 (modified by annette, 8 years ago) (diff)

NEMOAM test run

12 August 2013

We installed and ran the example ORCA2 NEMO-TAM experiment provided with the NEMO 3.4.1 source code on one of the local machines (um1, identical to um2). We followed the instructions listed on the main NEMO website:

Information and instructions for running locally are given below:

  • Input data
  • Timing runs
  • Instructions

Input data

The input files are located here: /net/jasmin/users/annette/nemo/ORCA2_LIM_nemo_v3.4

This is the ORCA2_LIM v3.4 data set mentioned here:

Timing runs

The times taken to run 75 timesteps (5 model days), based on the 'time' command around the model executable. This includes writing of restart files but not diagnostic history files.


1 proc  -> 3m17.43s
4 proc  -> 1m6.12s
8 proc  -> 0m55.86s
16 proc -> 1m29.75s


1 proc  -> 1m29.24s
4 proc  -> 0m34.25s
8 proc  -> ERROR
16 proc -> 0m43.27s

Instructions for running locally

Warning these instructions are under development

Step 1) Set up environment for um[12]

You should be on the local machine on which you wish to run the experiment.

a) Add the following lines to your .kshrc file:

if [ `uname -n` = 'um1' -o `uname -n` = 'um2' ]
  ulimit -c unlimited
  ulimit -s unlimited
  . /net/jasmin/users/simon/intel/composerxe\-2011.4.191/bin/ intel64
  export PATH=/home/jeff/linux_x86_64/mpich2-install/bin:$PATH
  export LD_LIBRARY_PATH=/panfs/jasmin/users/annette/netcdf/intel/lib:$LD_LIBRARY_PATH   
  export PATH=/net/jasmin/users/annette/fcm/bin:$PATH
  export PATH=/net/jasmin/users/annette/nemo/rebuild:$PATH

This ensures the supporting software is available.

Step 2) Extract code to um servers

You need to be on puma for this step.

a) Create a directory to extract NEMO code to eg:

mkdir ~/nemotam/v3.4.1_test
cd ~/nemotam/v3.4.1_test

b) Copy the following fcm configuration file:

cp ~annette/nemotam/v3.4.1_test/fcm-make.cfg . 

This contains all the information about where to get the source code from and how to build it. For more information see:

c) Edit the line in the file. This should use your own username and point to an experiment directory that can be seen on the local um machines. = annette@remote:/path/to/data/space/v3.4.1_test

d) Run extract (ignore warnings about SYMLINKS). This should be done in the directory where the fcm-make.cfg file sits.

fcm make 

This extracts the v3.4.1 branch and copies the source code over to your local data-space.

Step 3) Build ORCA2 NEMO and NEMOTAM on um servers

Log in to the um servers.

a) Go to the experiment directory created by the extract step:

cd /net/jasmin/users/annette/nemotam/v3.4.1_test

You should see another fcm-make.cfg file. This one just includes the build information, and you should not change anything in here.

b) Build the model executables:

fcm make -j 8 -v

This uses 8 parallel threads.

Step 4) Run direct NEMO model

Assuming the build completed successfully you should have two executables in the build/bin directory: nemo.exe and nemo_tam.exe.

a) Create an experiment directory to perform the run.

cd /path/to/data
mkdir v3.4.1_test_run
cd v3.4.1_test_run

In this example we assume this is at the same level as the v3.4.1_test directory created for the code, for example:

$ ls ../
OLD/  ORCA2_TAM_EXAMPLE/  v3.4.1_test/  v3.4.1_test_run/

b) Copy over example run scripts and control files:

cp /net/jasmin/users/annette/nemotam/ORCA2_TAM_EXAMPLE/* . 

c) Edit the run_opa.ksh} file to point to the executables you have just built, eg:


This sets the model to run for 75 timesteps (5 days) on 4 processors.

d) Run the direct NEMO model first:


It should take just over a minute to complete.

As well as the usual restart files *restart* and diagnostic output *grid_[TUVW]*, NEMO also produces (*tam_trajectory* files for use with TAM. Note that the diagnostic files will be empty as the output frequency is set to annual.

To concatenate the per processor output use the rebuild_nemo command. For example to generate a global restart file:

rebuild_nemo ORCA2_00000075_restart 4

Step 5) Run NEMO-TAM model

a) Go back to experiment directory EXP00

cd ../  

b) Copy over example run script:

cp /net/jasmin/users/annette/nemotam/run_tam.ksh . 

change CCFGDIR to pint to your MY_ORCA_TAM

c) Run TAM model:


Output in same place as main model: