Changes between Version 2 and Version 3 of Docs/MirroringStdAncilFiles


Ignore:
Timestamp:
06/06/17 13:37:39 (2 years ago)
Author:
willie
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Docs/MirroringStdAncilFiles

    v2 v3  
    11== Mirroring Standard Ancillary Files == 
    22 
    3 The UM requires ancillary files to run.  These include the land-sea mask, the orography, vegetation and ozone ancillary files, among others. 
     3        The UM requires ancillary files to run. These include the land-sea mask, the orography, vegetation and ozone ancillary files, among others. 
     4         
     5        The Met Office produces and maintains a standard set of ancillary files on their supercomputers in a comprehensive collection of directories known as the ancillary tree. There are sets of ancillary files at various resolutions for global and limited area domains. The global domains include n512, n96 among others and the limited area domains include m4_288360_uk, ukv. 
     6         
     7        We automatically synchronise a subset of the ancillary tree at the Met Office with the shared directory /nerc/n02/shared/ancil on the ARCHER RDF.  This is done weekly on Mondays at 19:30 using a cron job on my Met Office PC.  This job launches a script, do_xfer, which, in turn, executes the mirroring script, mirror_anc_archer, on the computer els038 at the Met Office in Exeter.  This ensures that the Met Office !Reading/Exeter link is not overloaded. The mirror is directly from Exeter to ARCHER at Edinburgh. 
     8         
     9        A record of the transfer is kept on my Met Office PC. 
     10         
     11        On ARCHER in /home/y07/y07/umshared/scripts there is a PBS script, sync_ancil, which is run weekly on Thursday at 19:30 which mirrors again the RDF directory into the  shared $UMDIR/ancil directory on the Lustre file system. 
     12         
     13        The section of the ancillary tree on ARCHER comprises roughly 5.5 TB of data and is growing. 
    414 
    5 The Met Office produces and maintains a standard set of ancillary files on their supercomputers in a comprehensive collection of directories known as the ancillary tree. There are sets of ancillary files for global and limited area domains.  Currently, the global domains are, 
     15        The scripts involved are stored on PUMA in my scripts directory. 
    616 
    7  * n2004 
    8  * n216 
    9  * n216e 
    10  * n320 
    11  * n48e 
    12  * n512 
    13  * n512e 
    14  * n768e 
    15  * n96 
    16  * n96e 
    17  
    18 and the limited area domains are, 
    19  
    20  * e4_11001000_euro 
    21  * m4_288360_uk 
    22  * my_600360_nae 
    23  * ukv 
    24  
    25 CMS manually mirror these monthly onto ARCHER.  This note describes the method. 
    26  
    27 On ARCHER, the standard ancillary tree is stored in $UMDIR/ancil, a mirror of the corresponding directories at the Met Office. 
    28  
    29 Currently (Dec 2015) the ancillary tree is 3.7 TB of data. 
    30  
    31 == The Details == 
    32  
    33 The mirroring script is launched on my Met Office PC.  The required 
    34 directories are not visible, but have to be "auto mounted": 
    35  
    36 {{{ 
    37 cd /cray_hpc/projects/um1 
    38 }}} 
    39  
    40 The script is executed as follows, 
    41  
    42 {{{ 
    43 ssh els049 $HOME/bin/mirror_anc_archer >> archer_ancmir_$(date +%F_%T).log 2>&1 
    44 }}} 
    45 i.e. it is run on the subsidiary machine els049 which ensures that the 
    46 Met Office !Reading/Exeter link is not overloaded.  Thus the mirror is 
    47 directly from Exeter to Edinburgh. 
    48  
    49 The script mirror_anc_archer: 
    50 {{{ 
    51 SSH_ENV=$HOME/.ssh/environment.$(hostname) 
    52  
    53 . $SSH_ENV 
    54  
    55 UMDIR=${UMDIR:-/projects/um1} 
    56  
    57 # need this because the dir is automounted on local machines 
    58 cd /cray_hpc/projects/um1 
    59  
    60 echo "ancil" 
    61 echo "=====" 
    62 cd  ancil 
    63 time rsync -az --stats --exclude-from=$HOME/bin/rsync_excludes.txt --partial \ 
    64                -e "ssh -qi $HOME/.ssh/id_rsa -o 'BatchMode yes'"  \ 
    65                data wmcginty@login.archer.ac.uk:/work/n02/n02/wmcginty/ancil 
    66  
    67 echo "" 
    68 echo "=========" 
    69 echo"" 
    70  
    71 echo "atmos" 
    72 echo "=====" 
    73 cd atmos 
    74 anc_list="KGO e4_11001000_euro master n48e n96 n96e my_600360_nae \ 
    75                     m4_288360_uk ukv n320 n216 n216e n512 n512e n768e n2004" 
    76 for i in $anc_list 
    77 do 
    78   echo $i: 
    79   time rsync -az --stats --exclude-from=$HOME/bin/rsync_excludes.txt --partial \ 
    80               -e "ssh -qi $HOME/.ssh/id_rsa -o 'BatchMode yes'" \ 
    81               $i wmcginty@login.archer.ac.uk:/work/n02/n02/wmcginty/ancil/atmos 
    82  echo "========" 
    83  echo "" 
    84 done  
    85 }}} 
    86  
    87 This sends the data to my ARCHER directory.  The $UMDIR/ancil is a 
    88 link to my ARCHER directory. 
    89