Opened 5 weeks ago

Closed 3 weeks ago

#2609 closed help (fixed)

cdo error

Reported by: charlie Owned by: um_support
Priority: normal Component: UM Tools
Keywords: Cc:
Platform: Other UM Version:

Description

Hi,

Sorry to bother you, but I'm having problems getting cdo to work - I'm trying to re-grid a curvilinear NEMO file to a standard N96 grid. I have done this several times in very recent past and it has worked, but this time I get the following error:

[cwilliams2011@jasmin-sci2 cell_area.d]$ cdo -remapnn,global_lonlat_n96 nemo_as392o_1y_18501201-18511201_grid-T_new.nc new_file.nc

cdo remapnn (Abort): Unsupported generic coordinates (Variable: area)!
[cwilliams2011@jasmin-sci2 cell_area.d]$ 

Please would you advise what this means, and what I can do about it in order to deal with this file? The file in question, as well as the latlon file, is at /group_workspaces/jasmin2/ncas_climate/users/cwilliams2011/data1_for_use.d/pacmedy.d/gc31n96orca1_mh.d/test_equilib.d/atmos.d/cell_area.d

Thanks,

Charlie

Change History (11)

comment:1 Changed 4 weeks ago by andy

Hi Charlie,

Can you allow access to nemo_as392o_1y_18501201-18511201_grid-T_new.nc - at the moment it is 600 in terms of permissions. I will then have a look and see if I can replicate your issue.

There's nothing obvious on a quick Google search using the error that you get. I've asked around and we don't have any in-house knowledge of cdo. You could try the CEDA helpdesk or better still the authors of cdo to see if they know what's going on.

Cheers
Andy

comment:2 Changed 4 weeks ago by charlie

Done, permissions opened. Many thanks.

comment:3 Changed 4 weeks ago by andy

Hi Charlie,

I get the same error as you on jasmin-sci3.

ncdump -h nemo_as392o_1y_18501201-18511201_grid-T_new.nc
netcdf nemo_as392o_1y_18501201-18511201_grid-T_new {
dimensions:

y = 332 ;
x = 362 ;

variables:

float area(y, x) ;

area:standard_name = "cell_area" ;
area:units = "m2" ;

How does this compare with previous fields that worked with regridding in cdo?

comment:4 Changed 4 weeks ago by charlie

Hi Andy,

See below for the ncdump from a file which cdo worked with (containing 5 fields) and, after that, the ncdump of the resulting regridded file.

Charlie

netcdf nemo_as392o_1y_18501201-18511201_grid-T_new
dimensions:
	y = 332 ;
	x = 362 ;
	nvertex = 4 ;
	deptht = 75 ;
	axis_nbounds = 2 ;
	time_counter = UNLIMITED ; // (1 currently)
variables:
	float bounds_lat(y, x, nvertex) ;
	float bounds_lon(y, x, nvertex) ;
	float deptht(deptht) ;
		deptht:standard_name = "depth" ;
		deptht:long_name = "Vertical T levels" ;
		deptht:units = "m" ;
		deptht:axis = "Z" ;
		deptht:positive = "down" ;
		deptht:bounds = "deptht_bounds" ;
	float deptht_bounds(deptht, axis_nbounds) ;
	float nav_lat(y, x) ;
		nav_lat:standard_name = "latitude" ;
		nav_lat:long_name = "Latitude" ;
		nav_lat:units = "degrees_north" ;
		nav_lat:nav_model = "grid_T" ;
		nav_lat:bounds = "bounds_lat" ;
	float nav_lon(y, x) ;
		nav_lon:standard_name = "longitude" ;
		nav_lon:long_name = "Longitude" ;
		nav_lon:units = "degrees_east" ;
		nav_lon:nav_model = "grid_T" ;
		nav_lon:bounds = "bounds_lon" ;
	float so(time_counter, deptht, y, x) ;
		so:standard_name = "sea_water_salinity" ;
		so:long_name = "Sea Water Salinity" ;
		so:units = "0.001" ;
		so:online_operation = "average" ;
		so:interval_operation = "1 month" ;
		so:interval_write = "1 month" ;
		so:cell_measures = "area: area" ;
		so:_FillValue = 1.e+20f ;
		so:missing_value = 1.e+20f ;
		so:coordinates = "time_centered deptht nav_lat nav_lon" ;
		so:cell_methods = "time: mean (thickness weighted)" ;
	float sos(time_counter, y, x) ;
		sos:standard_name = "sea_surface_salinity" ;
		sos:long_name = "Sea Surface Salinity" ;
		sos:units = "0.001" ;
		sos:online_operation = "average" ;
		sos:interval_operation = "2700 s" ;
		sos:interval_write = "1 month" ;
		sos:cell_methods = "time: mean (interval: 2700 s)" ;
		sos:cell_measures = "area: area" ;
		sos:_FillValue = 1.e+20f ;
		sos:missing_value = 1.e+20f ;
		sos:coordinates = "time_centered nav_lat nav_lon" ;
	float thetao(time_counter, deptht, y, x) ;
		thetao:standard_name = "sea_water_potential_temperature" ;
		thetao:long_name = "Sea Water Potential Temperature" ;
		thetao:units = "degree_C" ;
		thetao:online_operation = "average" ;
		thetao:interval_operation = "1 month" ;
		thetao:interval_write = "1 month" ;
		thetao:cell_measures = "area: area" ;
		thetao:_FillValue = 1.e+20f ;
		thetao:missing_value = 1.e+20f ;
		thetao:coordinates = "time_centered deptht nav_lat nav_lon" ;
		thetao:cell_methods = "time: mean (thickness weighted)" ;
	float thkcello(time_counter, deptht, y, x) ;
		thkcello:standard_name = "cell_thickness" ;
		thkcello:long_name = "Ocean Model cell Thickness" ;
		thkcello:units = "m" ;
		thkcello:online_operation = "average" ;
		thkcello:interval_operation = "2700 s" ;
		thkcello:interval_write = "1 month" ;
		thkcello:cell_methods = "time: mean (interval: 2700 s)" ;
		thkcello:cell_measures = "area: area" ;
		thkcello:_FillValue = 1.e+20f ;
		thkcello:missing_value = 1.e+20f ;
		thkcello:coordinates = "time_centered deptht nav_lat nav_lon" ;
	double time_centered(time_counter) ;
		time_centered:standard_name = "time" ;
		time_centered:long_name = "Time axis" ;
		time_centered:calendar = "360_day" ;
		time_centered:units = "seconds since 1900-01-01 00:00:00" ;
		time_centered:time_origin = "1900-01-01 00:00:00" ;
		time_centered:bounds = "time_centered_bounds" ;
	double time_centered_bounds(time_counter, axis_nbounds) ;
	double time_counter(time_counter) ;
		time_counter:axis = "T" ;
	float tos(time_counter, y, x) ;
		tos:standard_name = "sea_surface_temperature" ;
		tos:long_name = "Sea Surface Temperature" ;
		tos:units = "degree_C" ;
		tos:online_operation = "average" ;
		tos:interval_operation = "2700 s" ;
		tos:interval_write = "1 month" ;
		tos:cell_methods = "time: mean (interval: 2700 s)" ;
		tos:cell_measures = "area: area" ;
		tos:_FillValue = 1.e+20f ;
		tos:missing_value = 1.e+20f ;
		tos:coordinates = "time_centered nav_lat nav_lon" ;

// global attributes:
		:name = "as392o_1m_18500701_18501230" ;
		:description = "ocean T grid variables" ;
		:title = "ocean T grid variables" ;
		:Conventions = "CF-1.5" ;
		:production = "An IPSL model" ;
		:timeStamp = "2018-Jan-05 01:07:56 UTC" ;
		:TimeStamp = "05/01/2018 06:48:31 +0000" ;
		:file_name = "nemo_as392o_1y_18501201-18511201_grid-T.nc" ;
		:history = "Wed Sep 19 15:20:51 2018: ncks -v sos,tos,so,thetao,thkcello nemo_as392o_1y_18501201-18511201_grid-T.nc -O nemo_as392o_1y_18501201-18511201_grid-T_new.nc" ;
		:NCO = "20180919" ;

and

netcdf new_nemo_as392o_1y_18501201-18511201_grid-T_new {
dimensions:
	time_counter = UNLIMITED ; // (1 currently)
	lon = 360 ;
	lat = 180 ;
	deptht = 75 ;
	bnds = 2 ;
variables:
	double time_counter(time_counter) ;
		time_counter:standard_name = "time" ;
		time_counter:units = "day as %Y%m%d.%f" ;
		time_counter:calendar = "proleptic_gregorian" ;
		time_counter:axis = "T" ;
	double lon(lon) ;
		lon:standard_name = "longitude" ;
		lon:long_name = "longitude" ;
		lon:units = "degrees_east" ;
		lon:axis = "X" ;
	double lat(lat) ;
		lat:standard_name = "latitude" ;
		lat:long_name = "latitude" ;
		lat:units = "degrees_north" ;
		lat:axis = "Y" ;
	float deptht(deptht) ;
		deptht:standard_name = "depth" ;
		deptht:long_name = "Vertical T levels" ;
		deptht:units = "m" ;
		deptht:positive = "down" ;
		deptht:axis = "Z" ;
		deptht:bounds = "deptht_bnds" ;
	float deptht_bnds(deptht, bnds) ;
	float so(time_counter, deptht, lat, lon) ;
		so:standard_name = "sea_water_salinity" ;
		so:long_name = "Sea Water Salinity" ;
		so:units = "0.001" ;
		so:_FillValue = 1.e+20f ;
		so:missing_value = 1.e+20f ;
		so:online_operation = "average" ;
		so:interval_operation = "1 month" ;
		so:interval_write = "1 month" ;
		so:cell_measures = "area: area" ;
		so:cell_methods = "time: mean (thickness weighted)" ;
	float sos(time_counter, lat, lon) ;
		sos:standard_name = "sea_surface_salinity" ;
		sos:long_name = "Sea Surface Salinity" ;
		sos:units = "0.001" ;
		sos:_FillValue = 1.e+20f ;
		sos:missing_value = 1.e+20f ;
		sos:online_operation = "average" ;
		sos:interval_operation = "2700 s" ;
		sos:interval_write = "1 month" ;
		sos:cell_methods = "time: mean (interval: 2700 s)" ;
		sos:cell_measures = "area: area" ;
	float thetao(time_counter, deptht, lat, lon) ;
		thetao:standard_name = "sea_water_potential_temperature" ;
		thetao:long_name = "Sea Water Potential Temperature" ;
		thetao:units = "degree_C" ;
		thetao:_FillValue = 1.e+20f ;
		thetao:missing_value = 1.e+20f ;
		thetao:online_operation = "average" ;
		thetao:interval_operation = "1 month" ;
		thetao:interval_write = "1 month" ;
		thetao:cell_measures = "area: area" ;
		thetao:cell_methods = "time: mean (thickness weighted)" ;
	float thkcello(time_counter, deptht, lat, lon) ;
		thkcello:standard_name = "cell_thickness" ;
		thkcello:long_name = "Ocean Model cell Thickness" ;
		thkcello:units = "m" ;
		thkcello:_FillValue = 1.e+20f ;
		thkcello:missing_value = 1.e+20f ;
		thkcello:online_operation = "average" ;
		thkcello:interval_operation = "2700 s" ;
		thkcello:interval_write = "1 month" ;
		thkcello:cell_methods = "time: mean (interval: 2700 s)" ;
		thkcello:cell_measures = "area: area" ;
	float tos(time_counter, lat, lon) ;
		tos:standard_name = "sea_surface_temperature" ;
		tos:long_name = "Sea Surface Temperature" ;
		tos:units = "degree_C" ;
		tos:_FillValue = 1.e+20f ;
		tos:missing_value = 1.e+20f ;
		tos:online_operation = "average" ;
		tos:interval_operation = "2700 s" ;
		tos:interval_write = "1 month" ;
		tos:cell_methods = "time: mean (interval: 2700 s)" ;
		tos:cell_measures = "area: area" ;

// global attributes:
		:CDI = "Climate Data Interface version 1.9.4 (http://mpimet.mpg.de/cdi)" ;
		:history = "Tue Sep 04 14:24:57 2018: cdo -L -remapnn,global_lonlat_1deg /group_workspaces/jasmin2/ncas_climate/users/cwilliams2011/data1_for_use.d/pacmedy.d/gc31n96orca1_mh.d/test_equilib.d/ocean.d/nemo_as392o_1y_18501201-18511201_grid-T_new.nc /group_workspaces/jasmin2/ncas_climate/users/cwilliams2011/data1_for_use.d/pacmedy.d/gc31n96orca1_mh.d/test_equilib.d/ocean.d/new_nemo_as392o_1y_18501201-18511201_grid-T_new.nc\n",
			"Tue Sep  4 11:34:36 2018: ncks -v sos,tos,so,thetao,thkcello nemo_as392o_1y_18501201-18511201_grid-T.nc -O nemo_as392o_1y_18501201-18511201_grid-T_new.nc" ;
		:Conventions = "CF-1.5" ;
		:name = "as392o_1m_18500701_18501230" ;
		:description = "ocean T grid variables" ;
		:title = "ocean T grid variables" ;
		:production = "An IPSL model" ;
		:timeStamp = "2018-Jan-05 01:07:56 UTC" ;
		:TimeStamp = "05/01/2018 06:48:31 +0000" ;
		:file_name = "nemo_as392o_1y_18501201-18511201_grid-T.nc" ;
		:NCO = "20180904" ;
		:CDO = "Climate Data Operators version 1.9.4 (http://mpimet.mpg.de/cdo)" ;

comment:5 Changed 4 weeks ago by andy

Hi Charlie,

It looks like you have no coordinates in your date file so understandably cdo doesn't know what to do and aborts.

In your data that works:

float so(time_counter, deptht, y, x) ;

so:standard_name = "sea_water_salinity" ;
so:long_name = "Sea Water Salinity" ;
so:units = "0.001" ;
so:online_operation = "average" ;
so:interval_operation = "1 month" ;
so:interval_write = "1 month" ;
so:cell_measures = "area: area" ;
so:_FillValue = 1.e+20f ;
so:missing_value = 1.e+20f ;
so:coordinates = "time_centered deptht nav_lat nav_lon" ;
so:cell_methods = "time: mean (thickness weighted)" ;

The coordinates are related to nav_lon and nav_lat which are also defined.

In your nemo_as392o_1y_18501201-18511201_grid-T_new.nc data you have no nav_lon or nav_lat or coordinates attributes.

Cheers
Andy

comment:6 Changed 4 weeks ago by charlie

Hi Andy,

Okay, so is there a way of putting the co-ordinates in? I did another google search last night, and came across the following link:

https://code.mpimet.mpg.de/boards/1/topics/55

It says to use ncatted to resolve this problem, but I'm not entirely sure I understand the instructions. Are you able to explain?

Thanks,

Charlie

comment:7 Changed 4 weeks ago by charlie

Okay, possibly making progress. I have now managed to add in the co-ordinate attribute following the website: ncatted -a coordinates,area,c,c,"nav_lon nav_lat" nemo_as392o_1y_18501201-18511201_grid-T_new_cell.nc

If I know run cdo, I get the following error:

[cwilliams2011@jasmin-sci2 ocean.d]$ cdo -remapnn,global_lonlat_1deg nemo_as392o_1y_18501201-18511201_grid-T_new_cell.nc zz
Warning (cdf_scan_var_attr): NetCDF: Variable not found - >nav_lon<
Warning (cdf_scan_var_attr): NetCDF: Variable not found - >nav_lat<
cdo remapnn (Abort): Unsupported generic coordinates (Variable: area)!

This makes sense, because the variables above indeed don't exist. So is there an easy way of adding in these variables, either using ncatted or something else?

Charlie

PS. I'm now doing all of this in my home directory, /home/users/cwilliams2011/analysis.d/projects.d/pacmedy.d/gc3.1_runs.d/test_equilib.d/ocean.d, to avoid the space issue (I also tried it on the previous file which worked, and again it worked fine)

comment:8 Changed 4 weeks ago by andy

Hi Charlie,

I've never added a variable into a netCDF file using ncatted. Have a look on the web as I'm sure there's some examples out there. You'll also need to make sure that the grid that your data is on is the same as the grid defined by nav_lon and nav_lat. For this you'll need to go back to see how the data was generated.

Cheers
Andy

comment:9 Changed 4 weeks ago by charlie

I've just checked ncatted, and you can't add variables using this. It's only for attributes, nothing else. Is there another way, perhaps using cf-python, of adding in variables?

comment:10 Changed 4 weeks ago by andy

Hi Charlie,

You could try combining the file that doesn't have the nav_lon and nav_lat in with one that does.

ncks -A file1.nc file2.nc

Cheers
Andy

comment:11 Changed 3 weeks ago by charlie

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.