Opened 7 weeks ago

Last modified 7 weeks ago

#2847 accepted help

Plotting the observations for latent heat at fluxnet site US_Ha1

Reported by: NoelClancy Owned by: pmcguire
Component: JULES Keywords:
Cc: Platform: JASMIN
UM Version:

Description

I can plot observations for GPP, Reco, NEE and SH but not LE.
I have attached my python file that can plot all variables but the LE.

Attachments (1)

US_Ha1_run_ba137b_v_run_ba137f_ALL_gb.py (15.8 KB) - added by NoelClancy 7 weeks ago.

Download all attachments as: .zip

Change History (5)

Changed 7 weeks ago by NoelClancy

comment:1 Changed 7 weeks ago by pmcguire

Hi Noel

I looked at your Python code (that uses iris), and looked for the LE variables and compared them to the SH variables.

Maybe after line 354 you're missing a
LE_obs_cube_mean = obs_cube_mean
Can you try adding that line?
Does that work?
Patrick

comment:2 Changed 7 weeks ago by NoelClancy

TypeError? Traceback (most recent call last)
/home/users/nmc/FLUXNET2015/JULES_12313_30_sites/US_Ha1/scr/US_Ha1_run_ba137b_v_run_ba137f_ALL_gb.py in <module>()

352 obs_cube = iris.util.squeeze(obs_cube)
353 #obs_cube.data = np.ma.maximum(obs_cube.data, 0.0)

—> 354 obs_cube_mean = obs_cube.aggregated_by(["year", "month"], iris.analysis.MEAN, mdtol=0.5)

355 LE_obs_cube_mean = obs_cube_mean
356 print('LE_obs_cube_mean.data = ', obs_cube_mean.data)

/usr/lib/python2.7/site-packages/iris/cube.pyc in aggregated_by(self, coords, aggregator, kwargs)

3412 result = aggregator.aggregate(groupby_sub_cube.data,
3413 axis=dimension_to_groupby,

→ 3414 kwargs)

3415
3416 # Determine aggregation result data type for the aggregate-by cube

/usr/lib/python2.7/site-packages/iris/analysis/init.pyc in aggregate(self, data, axis, kwargs)

508 mdtol = kwargs.pop('mdtol', None)
509

—> 510 result = self.call_func(data, axis=axis, kwargs)

511 if (mdtol is not None and ma.isMaskedArray(data)):
512 fraction_not_missing = data.count(axis=axis) / data.shape[axis]

/usr/lib/python2.7/site-packages/numpy/ma/extras.pyc in average(a, axis, weights, returned)

574
575 if weights is None:

—> 576 avg = a.mean(axis)

577 scl = avg.dtype.type(a.count(axis))
578 else:

/usr/lib/python2.7/site-packages/numpy/ma/core.pyc in mean(self, axis, dtype, out, keepdims)

5155 dtype=dtype, kwargs)[()]
5156 else:

→ 5157 dsum = self.sum(axis=axis, dtype=dtype, kwargs)

5158 cnt = self.count(axis=axis, kwargs)
5159 if cnt.shape == () and (cnt == 0):

/usr/lib/python2.7/site-packages/numpy/ma/core.pyc in sum(self, axis, dtype, out, keepdims)

4995 # No explicit output
4996 if out is None:

→ 4997 result = self.filled(0).sum(axis, dtype=dtype, kwargs)

4998 rndim = getattr(result, 'ndim', 0)
4999 if rndim:

/usr/lib/python2.7/site-packages/numpy/core/_methods.pyc in _sum(a, axis, dtype, out, keepdims)

30
31 def _sum(a, axis=None, dtype=None, out=None, keepdims=False):

—→ 32 return umr_sum(a, axis, dtype, out, keepdims)

33
34 def _prod(a, axis=None, dtype=None, out=None, keepdims=False):

TypeError?: cannot perform reduce with flexible type

comment:3 Changed 7 weeks ago by pmcguire

  • Cc mcguirepatr@… removed
  • Owner changed from um_support to pmcguire
  • Status changed from new to accepted

I removed my email address from this ticket, since the JULES or Land Surface Processes tickets should go to me automatically anyways.

comment:4 Changed 7 weeks ago by pmcguire

Hi Noel
This is likely a known problem/bug with iris's aggregated_by() function and its handling partial days of data.
See the discussion in ticket #2637 and in ​https://github.com/SciTools/iris/issues/3190#issuecomment-427386106

In my testing, Python didn't have this problem for all the FLUXNET sites for GPP , only a couple of them.

Karina Williams implemented a partial solution (in her u-al752 suite's Python code) of 'skipping the two FLUXNET sites that iris's aggregated_by() can't handle with partial days of data'.

We have asked the iris team to fix this problem, but we don't know how long this will take.

Another solution would be to set up your Python on JASMIN with miniconda or Py Pip to use different versions of the numpy and iris packages, as suggested in the github analysis that I posted to the iris developers.

Patrick

Note: See TracTickets for help on using tickets.