Opened 8 years ago
Closed 7 years ago
#1120 closed help (completed)
Extracting a subdomain from pp files using convsh/xconv
Reported by: | cbirch | Owned by: | um_support |
---|---|---|---|
Component: | UM Tools | Keywords: | |
Cc: | Platform: | Other | |
UM Version: | <select version> |
Description
Hi,
I have just uploaded pp files from three year-long, 12km-resolution global simulations to Jasmin from the Met Office. The files are massive and I would like to extract a subdomain of certain variables, save them as netcdf files and then scp to Leeds. There are multiple files so I can't use the xconv interface.
I have attached the (convsh) script which extracts selected variables at selected levels and converts them to netcdf. I got this from Nick Dixon, who I think got it from Grenville originally. Extracting a variable at a given level works fine on Jasmin but I can't get it to extract the subdomain properly.
Lines 72-76 in the file appear to do the sub-domain extraction. The numbers I have set in setgrid 1 and 2 are correct and the subdomain is saved out successfully. However there is something wrong with the values it extracts. Rather than simply extracting the grid boxes I want, it is doing some kind of interpolation (line 76). If I comment out this line the subdomain is not extracted. The comment on line 73 suggests that someone at some point has been playing with this (I didn't add that comment). I was wondering if you knew how to get this working?
Thanks,
Cathryn
Attachments (1)
Change History (7)
Changed 8 years ago by cbirch
comment:1 Changed 8 years ago by jeff
comment:2 Changed 8 years ago by cbirch
Hi Jeff,
Thanks for the info. The interpolation/roudning error is a bit of a risk. I'm looking into doing it with Python instead. This will take me longer so I just wanted to check there wasn't an easy way of doing it with convsh.
Thanks,
Cathryn
comment:3 Changed 8 years ago by grenville
Cathryn
You could convert to netcdf, then use the NCO operators - I don't know how much data you'd need to convert, so this may be impractical?
Grenville
comment:4 Changed 8 years ago by cbirch
Thanks for the suggestion. I hadn't thought of that. As you say the issue may be that I don't have the space on Jasmin to convert enough files to netcdf. Python would be better because I could read directly from the pp files (I think). I'll take a look at anyway NCO though.
Cathryn
comment:5 Changed 8 years ago by grenville
Cathryn
David Hassell has written some very clever data handling tools (in python) - they take the pain out of reading huge files and lots of files (they can read pp, netcdf..) . I'm not sure if they can slice data exactly the way you want, but they are will wort a look. David is on vacation currently.
Grenville
comment:6 Changed 7 years ago by annette
- Resolution set to completed
- Status changed from new to closed
Hi Cathryn
xconv can only subset over the level and time dimension. This script tries to use bilinear interpolation to do the same thing over the lat/lon dimensions. This might work if you choose the output grid to have exactly the same lat/lon values as contained in the input grid. Then the bilinear interpolation routine will choose a weight of 1 for the point you want and a weight of 0 for the other interpolation points. Of course rounding errors may make this not work as you might expect. Maybe you should use some other tool to do this.
Jeff.