wiki:Docs/MirroringStdAncilFiles

Mirroring Standard Ancillary Files

The UM requires ancillary files to run. These include the land-sea mask, the orography, vegetation and ozone ancillary files, among others.

The Met Office produces and maintains a standard set of ancillary files on their supercomputers in a comprehensive collection of directories known as the ancillary tree. There are sets of ancillary files for global and limited area domains. Currently, the global domains are,

  • n2004
  • n216
  • n216e
  • n320
  • n48e
  • n512
  • n512e
  • n768e
  • n96
  • n96e

and the limited area domains are,

  • e4_11001000_euro
  • m4_288360_uk
  • my_600360_nae
  • ukv

CMS manually mirror these monthly onto ARCHER. This note describes the method.

On ARCHER, the standard ancillary tree is stored in $UMDIR/ancil, a mirror of the corresponding directories at the Met Office.

Currently (Dec 2015) the ancillary tree is 3.7 TB of data.

The Details

The mirroring script is launched on my Met Office PC. The required directories are not visible, but have to be "auto mounted":

cd /cray_hpc/projects/um1

The script is executed as follows,

ssh els049 $HOME/bin/mirror_anc_archer >> archer_ancmir_$(date +%F_%T).log 2>&1

i.e. it is run on the subsidiary machine els049 which ensures that the Met Office Reading/Exeter link is not overloaded. Thus the mirror is directly from Exeter to Edinburgh.

The script mirror_anc_archer:

SSH_ENV=$HOME/.ssh/environment.$(hostname)

. $SSH_ENV

UMDIR=${UMDIR:-/projects/um1}

# need this because the dir is automounted on local machines
cd /cray_hpc/projects/um1

echo "ancil"
echo "====="
cd  ancil
time rsync -az --stats --exclude-from=$HOME/bin/rsync_excludes.txt --partial \
               -e "ssh -qi $HOME/.ssh/id_rsa -o 'BatchMode yes'"  \
               data wmcginty@login.archer.ac.uk:/work/n02/n02/wmcginty/ancil

echo ""
echo "========="
echo""

echo "atmos"
echo "====="
cd atmos
anc_list="KGO e4_11001000_euro master n48e n96 n96e my_600360_nae \
                    m4_288360_uk ukv n320 n216 n216e n512 n512e n768e n2004"
for i in $anc_list
do
  echo $i:
  time rsync -az --stats --exclude-from=$HOME/bin/rsync_excludes.txt --partial \
              -e "ssh -qi $HOME/.ssh/id_rsa -o 'BatchMode yes'" \
              $i wmcginty@login.archer.ac.uk:/work/n02/n02/wmcginty/ancil/atmos
 echo "========"
 echo ""
done 

This sends the data to my ARCHER directory. The $UMDIR/ancil is a link to my ARCHER directory.

Last modified 15 months ago Last modified on 11/02/16 13:47:08