wiki:Docs/MirroringStdAncilFiles

Mirroring Standard Ancillary Files

The UM requires ancillary files to run. These include the land-sea mask, the orography, vegetation and ozone ancillary files, among others.

The Met Office produces and maintains a standard set of ancillary files on their supercomputers in a comprehensive collection of directories known as the ancillary tree. There are sets of ancillary files at various resolutions for global and limited area domains. The global domains include n512, n96 among others and the limited area domains include m4_288360_uk, ukv.

We automatically synchronise a subset of the ancillary tree at the Met Office with the shared directory /nerc/n02/shared/ancil on the ARCHER RDF. This is done weekly on Mondays at 19:30 using a cron job on my Met Office PC. This job launches a script, do_xfer, which, in turn, executes the mirroring script, mirror_anc_archer, on the computer els038 at the Met Office in Exeter. This ensures that the Met Office Reading/Exeter link is not overloaded. The mirror is directly from Exeter to ARCHER at Edinburgh.

A record of the transfer is kept on my Met Office PC.

On ARCHER in /home/y07/y07/umshared/scripts there is a PBS script, sync_ancil, which is run weekly on Thursday at 19:30 which mirrors again the RDF directory into the shared $UMDIR/ancil directory on the Lustre file system.

The section of the ancillary tree on ARCHER comprises roughly 5.5 TB of data and is growing.

The scripts involved are stored on PUMA in my scripts directory.

Last modified 4 months ago Last modified on 06/06/17 13:37:39