wiki:Docs/MirroringStdAncilFiles

Mirroring Standard Ancillary Files

The UM requires ancillary files to run. These include the land-sea mask, the orography, vegetation and ozone ancillary files, among others.

The Met Office produces and maintains a standard set of ancillary files on their supercomputers in a comprehensive collection of directories known as the ancillary tree ($UMDIR/ancil). There are sets of ancillary files at various resolutions for global and limited area domains. The global domains include n512, n96 among others and the limited area domains include m4_288360_uk, ukv.

A subset of the ancillary tree, agreed with the Keir Bovis rsync_excludes.txt, is mirrored on ARCHER. The transfer method requires access to a Met Office PC.

Each week, on Mondays at 06:00, cron on the Met Office server els003 at Exeter launches do_xfer2, which, in turn, executes the mirroring script, mirror_anc_archer. This ensures that the Met Office Reading/Exeter link is not used. The mirror is directly from Exeter to the ARCHER RDF shared account /nerc/n02/shared/ancil at Edinburgh. The server shares the same directories as the Met Office PCs.

To ensure that this happens, login to els003 and add your keys,

ssh-add
ssh-add ~/.ssh/<your-key>

The first is to support Rose/Cylc jobs and is not necessary here. You will need to manually check the status of the transfers each week. A record of the transfers is kept on your Met Office PC account. The keys will need to be added again whenever els003 is rebooted, or is taken out of commission. It is not possible to send mail from els003 or the Met Office PC, so manual checking is the only option.

The second step of the process is run on ARCHER and takes the ancillary files from the shared account and synchronises them with those on the Lustre file system ($UMDIR/ancil). On ARCHER in /home/y07/y07/umshared/scripts there is a PBS script, sync_ancil, which is run weekly on Thursday at 19:30 which mirrors again the RDF directory into the shared $UMDIR/ancil directory on the Lustre file system.

The ancillary tree grew from 3.7 TB in December 2015 to 6.5 TB at July 2019.

W. McGinty

16, July 2019

Last modified 4 months ago Last modified on 16/07/19 08:46:39

Attachments (4)

Download all attachments as: .zip