Changes between Version 10 and Version 11 of Docs/MirroringStdAncilFiles


Ignore:
Timestamp:
15/07/19 14:59:57 (3 months ago)
Author:
willie
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Docs/MirroringStdAncilFiles

    v10 v11  
    33        The UM requires ancillary files to run. These include the land-sea mask, the orography, vegetation and ozone ancillary files, among others. 
    44         
    5         The Met Office produces and maintains a standard set of ancillary files on their supercomputers in a comprehensive collection of directories known as the ancillary tree. There are sets of ancillary files at various resolutions for global and limited area domains. The global domains include n512, n96 among others and the limited area domains include m4_288360_uk, ukv. 
     5        The Met Office produces and maintains a standard set of ancillary files on their supercomputers in a comprehensive collection of directories known as the ancillary tree ($UMDIR/ancil). There are sets of ancillary files at various resolutions for global and limited area domains. The global domains include n512, n96 among others and the limited area domains include m4_288360_uk, ukv. 
    66         
    7         We automatically synchronise a subset of the ancillary tree at the Met Office with the shared directory `/nerc/n02/shared/ancil` on the ARCHER RDF.  This is done weekly on Mondays at 06:00 using a `cron` job on a Met Office PC (account william.mcginty, Linux user name `frmy`).  This job launches a script, [attachment: do_xfer2], which, in turn, executes the mirroring script, [attachment: mirror_anc_archer], on the computer els003 at the Met Office in Exeter.  This ensures that the Met Office !Reading/Exeter link is not overloaded. The mirror is directly from Exeter to the ARCHER RDF shared account `/nerc/n02/shared/ancil` at Edinburgh. 
     7          Each week, on Mondays at 06:00, a `cron` job on a Met Office PC (account william.mcginty, Linux user name `frmy`) launches [attachment: do_xfer2], which, in turn, executes the mirroring script, [attachment: mirror_anc_archer], on the computer els003 at the Met Office in Exeter.  This ensures that the Met Office !Reading/Exeter link is not overloaded. The mirror is directly from Exeter to the ARCHER RDF shared account `/nerc/n02/shared/ancil` at Edinburgh. 
     8 
     9To ensure that this happens, login to els003 and add your keys 
     10 
     11{{{ 
     12ssh-add 
     13ssh-add ~/.ssh/<your-key> 
     14}}} 
     15 
     16The first is to support Rose/Cylc jobs and is not necessary here.  You will need to manually check the status of the transfers each week. A record of the transfers is kept on my Met Office PC account (user name `frmy`).  The keys will need to be added again whenever els003 is rebooted, or is taken out of commission.  It is not possible to send mail from els003 or the Met Office PC, so manual checking is the only option. 
    817         
    9         A record of the transfers is kept on my Met Office PC account (user name `frmy`). 
     18The second step of the process is run on ARCHER and takes the ancillary files from the shared account and synchronises them with those on the Lustre file system ($UMDIR/ancil). On ARCHER in `/home/y07/y07/umshared/scripts` there is a PBS script, sync_ancil, which is run weekly on Thursday at 19:30 which mirrors again the RDF directory into the  shared $UMDIR/ancil directory on the Lustre file system. 
    1019         
    11         On ARCHER in `/home/y07/y07/umshared/scripts` there is a PBS script, sync_ancil, which is run weekly on Thursday at 19:30 which mirrors again the RDF directory into the  shared $UMDIR/ancil directory on the Lustre file system. 
    12          
    13         The scripts involved are stored on PUMA (test) in `/home/willie/Documents/remote_scripts`. 
    14  
    1520 
    1621[https://people.ncas.ac.uk/people/view/109 W. McGinty]