wiki:UM/GettingInitialData

Version 2 (modified by willie, 4 years ago) (diff)

Getting Initial Data

Start dumps, LBC files and ensemble data can be obtained from the Met Office MOOSE database. This covers the latest five years.

Start dumps from earlier periods can be constructed as GRIB files from ECMWF data.

Getting data from the Met Office

The start dumps are now stored in MOOSE, which can be accessed via MONSooN.

To get data for 28/Sep/2011 we would type,

moo get moose:/opfc/atm/global/rerun/201109.file/20110928_qwqg00.T+0 .

(Note the dot). This takes about 15 to 20 minutes.

To find out whether the data exists in moose:

moo ls moose:/opfc/atm/global/rerun/201109.file

Generally we prefer to get the output from the global runs - these have "qwqg" in the title; otherwise we get the global update - these have "qwqu" in the file name.

Older files are stored in a packed archive format - these have the extension ".pax". These usually contain several dumps and have names like,

coprr.udQU06.20060721.pax

where the "ud" denotes "unified model dump" and QU06 denotes global update output at 06Z. These have to be unpacked and the files relabelled. There is a script to do this in

 
#!/bin/ksh
# i will be like coprr.udQU06.20060725.pax
for i in *.pax
do
print $i
dat=${i%.*}                   # take .pax off the end
datestring=${dat##*.}        # remove bit to left of the rightmost dot
pax -r -f $i
cd opdaily/datawgl
for j in *
do
mv $j $TMPDIR/${datestring}_$j
done
cd -
done

This assumes a list of pax files and unpacks, relabels and stores the results in $TMPDIR.

UKV start dumps

The UKV start dumps are in MASS with pathnames of the type:

moose:/opfc/atm/ukv/rerun/201108.file/20110810_qwqv03.T+1

moose:/opfc/atm/ukv/rerun/201108.file/20110810_qwqv06.T+1 etc…

(you obviously need to replace year, month, day, time) and are of size approx 4G.

I gather we can get the boundary condition files by rerunning the NAE but - it might be easier to copy them if they're available as we're only considering a small number of cases here. I don't know the filenames though.

UKV forecasts and analyses

Can we have the analyses and 36 hr forecasts from the same times as the start dump times above plus

09Z 20110112 15Z 20110827 03Z 20111101 09Z 20111101

The UKV forecasts and analyses are in the MASS-R with pathnames of the type

moose:/opfc/atm/ukv/prodm/YYYY.pp/prodm_op_ukv_YYYYMMDD_AA_NNN.pp for the multilevel fields and

moose:/opfc/atm/ukv/prods/YYYY.pp/prods_op_ukv_YYYYMMDD_AA_NNN.pp

where YYYY = year e.g. 2011, MM = month e.g. 05, DD = day e.g. 21 AA = analysis time = 03, 09, 15 or 21 NNN = the part of the forecast - goes up in two's from 000 to 036. e.g. 000, 002, 004 …. 036

NAE start dumps and LBCs

MOGREPS-R files

Analysis files and perturbations files at 18Z 20110111

Ensemble fields are in:

moose:/opfc/atm/mogreps-g/rerun/YYYYMM.file moose:/opfc/atm/mogreps-r/rerun/YYYYMM.file

Where YYYY = year e.g. 2011, MM = month e.g. 07

At minimum to re-run MOGREPS-R You want the analysis files in the mogreps-r path: E.g. 20110722_qwqy06.T+0.18km 20110722_qwqy18.T+0.18km (All files containing 'T+0' in the filename

And all the perturbations files E.g. 20110722_perts.qwey06.oper??.pp1 where ?? Goes from 00 to 23 members (All files containing 'perts' in the filename)

Also in the mogreps-g path you need the boundary files: E.g. 20110722_qweg00.oper??.FRAMES.EY.tar.gz (at the corresponding times to above)

Getting data from ECMWF

W. McGinty, 15 Oct 2015