Opened 6 years ago

Closed 6 years ago

#1120 closed help (completed)

Extracting a subdomain from pp files using convsh/xconv

Reported by: cbirch Owned by: um_support
Component: UM Tools Keywords:
Cc: Platform: Other
UM Version: <select version>



I have just uploaded pp files from three year-long, 12km-resolution global simulations to Jasmin from the Met Office. The files are massive and I would like to extract a subdomain of certain variables, save them as netcdf files and then scp to Leeds. There are multiple files so I can't use the xconv interface.

I have attached the (convsh) script which extracts selected variables at selected levels and converts them to netcdf. I got this from Nick Dixon, who I think got it from Grenville originally. Extracting a variable at a given level works fine on Jasmin but I can't get it to extract the subdomain properly.

Lines 72-76 in the file appear to do the sub-domain extraction. The numbers I have set in setgrid 1 and 2 are correct and the subdomain is saved out successfully. However there is something wrong with the values it extracts. Rather than simply extracting the grid boxes I want, it is doing some kind of interpolation (line 76). If I comment out this line the subdomain is not extracted. The comment on line 73 suggests that someone at some point has been playing with this (I didn't add that comment). I was wondering if you knew how to get this working?


Attachments (1)

extractField0Level0Aust (2.0 KB) - added by cbirch 6 years ago.

Download all attachments as: .zip

Change History (7)

Changed 6 years ago by cbirch

comment:1 Changed 6 years ago by jeff

Hi Cathryn

xconv can only subset over the level and time dimension. This script tries to use bilinear interpolation to do the same thing over the lat/lon dimensions. This might work if you choose the output grid to have exactly the same lat/lon values as contained in the input grid. Then the bilinear interpolation routine will choose a weight of 1 for the point you want and a weight of 0 for the other interpolation points. Of course rounding errors may make this not work as you might expect. Maybe you should use some other tool to do this.


comment:2 Changed 6 years ago by cbirch

Hi Jeff,

Thanks for the info. The interpolation/roudning error is a bit of a risk. I'm looking into doing it with Python instead. This will take me longer so I just wanted to check there wasn't an easy way of doing it with convsh.


comment:3 Changed 6 years ago by grenville


You could convert to netcdf, then use the NCO operators - I don't know how much data you'd need to convert, so this may be impractical?


comment:4 Changed 6 years ago by cbirch

Thanks for the suggestion. I hadn't thought of that. As you say the issue may be that I don't have the space on Jasmin to convert enough files to netcdf. Python would be better because I could read directly from the pp files (I think). I'll take a look at anyway NCO though.


comment:5 Changed 6 years ago by grenville


David Hassell has written some very clever data handling tools (in python) - they take the pain out of reading huge files and lots of files (they can read pp, netcdf..) . I'm not sure if they can slice data exactly the way you want, but they are will wort a look. David is on vacation currently.


comment:6 Changed 6 years ago by annette

  • Resolution set to completed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.