wiki:Projects/INSPECT/NEMOTAM

Version 5 (modified by annette, 8 years ago) (diff)

NEMOAM test run

12 August 2013

We installed and ran the example ORCA2 NEMO-TAM experiment provided with the NEMO 3.4.1 source code on one of the local machines (um1, identical to um2). We followed the instructions listed on the main NEMO website: http://www.nemo-ocean.eu/Using-NEMO/User-Guides/Basics/Tangent-and-Adjoint-quick-start-guide

Information and instructions for running locally are given below:

  • Input data
  • Timing runs
  • Instructions

Input data

The input files are located here: /net/jasmin/users/annette/nemo/ORCA2_LIM_nemo_v3.4

This is the ORCA2_LIM v3.4 data set mentioned here:

http://www.nemo-ocean.eu/Using-NEMO/Configurations/ORCA2_LIM-and-AGRIF-demonstrator

Timing runs

The times taken to run 75 timesteps (5 model days), based on the 'time' command around the model executable. This includes writing of restart files but not diagnostic history files.

NEMO

1 proc  -> 3m17.43s
4 proc  -> 1m6.12s
8 proc  -> 0m55.86s
16 proc -> 1m29.75s

TAM

1 proc  -> 1m29.24s
4 proc  -> 0m34.25s
8 proc  -> ERROR
16 proc -> 0m43.27s

Instructions for running locally

Warning these instructions are under development

Step 1) Set up environment for um[12]

You should be on the local machine on which you wish to run the experiment.

a) Add the following lines to your .kshrc file:

if [ `uname -n` = 'um1' -o `uname -n` = 'um2' ]
then 
  ulimit -c unlimited
  ulimit -s unlimited
  . /net/jasmin/users/simon/intel/composerxe\-2011.4.191/bin/compilervars.sh intel64
  export PATH=/home/jeff/linux_x86_64/mpich2-install/bin:$PATH
  export LD_LIBRARY_PATH=/panfs/jasmin/users/annette/netcdf/intel/lib:$LD_LIBRARY_PATH   
  export PATH=/net/jasmin/users/annette/fcm/bin:$PATH
  export PATH=/net/jasmin/users/annette/nemo/rebuild:$PATH
fi

This ensures the supporting software is available to you.

Step 2) Extract code to um servers

You need to be on puma for this step.

a) Create a directory to extract NEMO code to eg:

mkdir ~/nemotam/v3.4.1_test
cd ~/nemotam/v3.4.1_test

b) Copy the following fcm configuration file:

cp ~annette/nemotam/v3.4.1_test/fcm-make.cfg . 

This contains all the information about where to get the source code from and how to build it. For more information see: http://cms.ncas.ac.uk/chrome/site/FCM/user_guide/make.html

c) Edit the mirror.target line in the file. This should use your own username and point to an experiment directory that can be seen on the local um machines.

mirror.target = annette@remote:/path/to/data/space/v3.4.1_test

d) Run extract (ignore warnings about SYMLINKS). This should be done in the directory where the fcm-make.cfg file sits.

fcm make 

This extracts the v3.4.1 branch and copies the source code over to your local data-space.

Step 3) Build ORCA2 NEMO and NEMOTAM on um servers

Log in to the um servers.

a) Go to the experiment directory created by the extract step:

cd /net/jasmin/users/annette/nemotam/v3.4.1_test

You should see another fcm-make.cfg file. This one just includes the build information, and you should not need to change anything in here.

b) Build the model executables:

fcm make -j 8 

This uses 8 parallel threads to do the make.

Step 4) Run direct NEMO model

Assuming the build completed successfully you should have two executables in the build/bin directory: nemo.exe and nemo_tam.exe.

a) Create an experiment directory to perform the run.

cd /path/to/data
mkdir v3.4.1_test_run
cd v3.4.1_test_run

In this example we assume this is at the same level as the v3.4.1_test directory created for the code, but it can be in any location. For example:

$ ls ../
OLD/  ORCA2_TAM_EXAMPLE/  v3.4.1_test/  v3.4.1_test_run/

b) Copy over example run scripts and control files:

cp /net/jasmin/users/annette/nemotam/ORCA2_TAM_EXAMPLE/* . 

change CCFGDIR to pint to your MY_ORCA_TAM

c) Run model:

./run_opa.ksh 

Output in:

./Run1/

Step 5) Run NEMO-TAM model

a) Go back to experiment directory EXP00

cd ../  

b) Copy over example run script:

cp /net/jasmin/users/annette/nemotam/run_tam.ksh . 

change CCFGDIR to pint to your MY_ORCA_TAM

c) Run TAM model:

./run_tam.ksh 

Output in same place as main model:

./Run1/