wiki:UmFamous

Version 16 (modified by simon, 3 years ago) (diff)

Standalone UM system and FAMOUS

This is a standalone version of FAMOUS based on version 4.5 of the Met Office's Unified Model (UM).

Two standard FAMOUS versions are supplied, one with the MOSESI land-surface scheme, and the other with MOESEII.2. The base runids for these configurations are xhmkq and xfhcr (under robin) respectively.

It has a complete UM system but without the need for the complicated installation process. It is linux only as it comes with pre-built small executables. It also comes with prebuilt model and reconfiguration executables. If no model code changes are needed, then the model executables can be used for model integration without the need for any compilation at all on the installation machine.

As the executables are statically linked, they will run on any modern linux machine.

The system has the flexibility for local compilation, if required. This does need extra work, however. The communications library, gcom will need installing, together with a MPI system, see below for full details.

All of the standard FAMOUS input ancillaries are supplied, together with initial dumps.

The UM and FAMOUS can be installed on any linux system, from laptops to clusters. Currently the only job control software supported is the at daemon. It is possible to run with other job control systems such as Sun Grid Engine and SLURM, and versions which support these will be released in the future.

umui configurations are available for both MOSESI and MOSESII. These have been simplified as much as possible with the mods and handedits concatenated and superfluous options removed.

The package tarball is located at: /net/jasmin/users/simon/um_famous.tgz

This document assumes the user is familiar with the UM and umui.

Prerequisites:

Running without compilation:

  • Linux system
  • at daemon
  • ksh

(atd and ksh may not be installed by default)

Running with compilation (above plus):

  • Intel fortran
  • A MPI implementation

Quick start guide:

  1. Copy /net/jasmin/users/simon/um_famous.tgz and unpack on the machine on which you want to run FAMOUS. The system will be unpacked into a directory called um_famous
  1. Edit example.profile to change the values of UMDIR and DATA_DIR, and append to your $HOME/.profile . (If $HOME/.profile doesn't exist, create it).
  • Ensure that there is no other UM setup commands in .profile
  • UMDIR should always have um_famous at the end of the path.
  1. Then
    . $HOME/.profile
    
  1. On puma, copy the job xmjne (MOSESI) or xmjnf (MOSESII) from user simon, save and process. The "Submit" button DOES NOT WORK. As prebuilt executables are provided, there is no need to build the model. For testing, these model configs require no changes.
  1. On the installation machine type:
    umsubmit_local -u pumauserid runid
    

You may be asked for your puma password.

  1. And the model should be running. Check by:
    cd $DATA_DIR/runid
    tail -f *pe0
    

FAMOUS configuration

MOSESI and MOSESII

Seperate MOSESI and MOSESII.2 configurations are available. The base umui jobs can be used to build and run each of these.

umui

FAMOUS is configured via the umui as usual. The standard FAMOUS jobs have be rationalised to make their configuration as simple as possible. Details on this are below. The standard jobs are xmjne (MOSESI) or xmjnf (MOSESII) under user simon. All standard umui options are available, practically it is no different from running FAMOUS using a standard UM installation.

The "Submit" button DOES NOT WORK.

umsubmit_local -u pumauserid runid

on the local machine must be used instead.

Simplistically there are three different modes of running, increasing in complexity.

Simplest
Run a copy of the base job without any alterations. This will work using the default start date and dumps and is ideal for testing the installation and timing studies.
Normal
The standard umui jobs can be altered as usual changing items such STASH, start dumps, output directory location…, infact anything which doesn't require model recompilation.
Full installation
If changes to the model or reconfiguration code are required, then a full installation is needed. This requires the Intel compiler, and a MPI installation. See below.

modsets

The difference between MOSESI and MOSESII modsets are considerable. In fact, MOSESII code does not exist as part of the standard UM4.5 release and can only be configured as mods. Therefore it was decided that there should be different standard mods for each configuration. All of the various model mods for the two FAMOUS configurations are now held in 5 mods, these are:

source_common.mod
FORTRAN mods which are common to both MOSESI and MOSESII
source_common_c.mod
C mods which are common to both MOSESI and MOSESII
source_MOSESI.mod
MOSESI exclusive mods
source_MOSESII.mod
MOSESII exclusive mods
port_end_f.mod
mod which allows IO on little endian machine, such as Linux. UM files are natively bigendian. It was decided to keep this separate to allow easy porting of a model configuration to a bigendian machine such as IBM Power.

Similarly there mods for the reconfiguration.

recon_common.mod
FORTRAN mods which are common to both MOSESI and MOSESII
recon_MOSESI.mod
MOSESI exclusive mods
recon_MOSESII.mod
MOSESII exclusive mods

These mods are concatenations of the standard mods for each model. The tool $UMDIR/tools/make_mod.sh was used for this.

handedits

The handedits where concatenated in a similar way to the modsets. There is a single handedit for each model configuration. They are ~simon/famous/handedit_MOSESI.sh and ~simon/famous/handedit_MOSESII.sh . The tool make_handedit.sh was used to create them. In those cases where the original handedit changed parameter values which could be set in the umui, these values where set directly in the umui and that part of the handedit discarded.

Directories

An number of directories can be set by the user via environment variables. Depending on the directory type, this needs to be done either via the umui or via the user's $HOME/.profile . Certain variables have to be set in the .profile as these are required by the UM system and need to be set before the umui job is copied from puma for submission.

$HOME/.profile environment variables

Required:

UMDIR
Directory of unpacked tarball. This should end with um_famous.
DATA_DIR
Base model output directory.

See example.profile in $UMDIR for example settings.

Opitional:

TMPDIR
Temporary files, default $DATA_DIR/tmp . Do not set this to a shared filespace such as /tmp as this could cause issues if more than one person is running the UM on the machine.
MY_OUTPUT
Output .leave files produced as the model runs. Default $DATA_DIR/umui_out
MPICH_DIR
Set only if using local MPI version. See below. Default $DATA_DIR/mpich3.2

umui environment variables

ARCHIVE_DIR
Directory where archived data and dumps are stored. Default $DATA_DIR/um_archive/$RUNID
FAMOUS_DATA
Directory of input data. Only needs to be changed to the input data directories are moved from under $UMDIR.

Archiving

The model configurations uses the standard umui archiving system. Output data files are translated into PP before archiving, while the dumps are just copied. It can be turned off via the umui, if required.

Intel FORTRAN, MPI and gcom

The model tarball comes complete with a pre-built MPICH3.2 system. The default installation is configured to use this. If the model needs recompilation, or a different MPI system than MPICH3.2 is required to run the model (for example on some clusters), then the model needs to be recompiled on the local system. Firstly Intel FORTRAN needs to be installed. In theory it is possible for this FAMOUS version to use other compilers, but currently only Intel FORTRAN is supported. MPI has to be installed (if it hasn't been already). FAMOUS has been tested with MPICH3.2 but should work with other implementations. For MPICH3.2 follow the installation instructions at https://www.mpich.org/ . It needs to be configured for Intel FORTRAN for both F77 and F90. Once it has been installed, add the line

export MPICH_DIR=/path/to/top/level/mpich

to your .profile, before the call to setvars, changing the path as required. If MPI is already installed, then add this line to point to the local installation.

Next gcom (the system the model uses for multi-processor communications) needs to be compiled using MPI. Type

cd $UMDIR/gcom3.8/gcom
make

The file $UMDIR/gcom3.8/gcom/lib/libgcom_buffered_mpi.a should now exist.

There is no need to recompile the reconfiguration unless there is a code change. The included executable independent of MPI.

The reconfiguration or model can now be compiled as usual. The standard compile system will use the MPI version defined in MPICH_PATH and the gcom version in $UMDIR/gcom3.8/gcom/lib.

IMPORTANT when compiling the model and reconfiguration, change the name of the output executable to $DATAW/$RUNID.exec of $DATAW/$RUNID.recon otherwise there is a danger that the standard executable under UMDIR will be overwritten.

Changes from standard UM/FAMOUS

  • Linux only.
  • UM comes as a single pre-configured tarball.
  • UM installation process no longer required.
  • UM tools included in tarball.
  • Scripts are hard coded, no script mods.
  • All small executables are prebuilt as static executables.
  • All mods grouped together into a small number of files.
  • Compile system changed so that pre-built object files no longer required.
  • All handedit file collated into a single file.
  • umui jobs cleaned up.
  • umui updated to set values directly which handedits overwrote. These handedits removed.
  • umui system code updated.
  • Dumps and ancils co-located.
  • MPICH built and included with system for multi-processor models.
  • Changed paths so that only minimal only two (UMDIR DATA_DIR) need changing.
  • Ensured that system is flexible enough to allow UM experts to configure as they would a standard UM system.
  • Archiving system fully integrated with model scripts.
  • Fixed a bug feature in recon where environment variables cannot be used in use ancils paths.