Opened 5 months ago

Closed 5 months ago

#2735 closed help (fixed)

Vegetation fraction ancillary file problem, yet again

Reported by: charlie Owned by: um_support
Component: UM Model Keywords:
Cc: Platform: NEXCS
UM Version: 10.7

Description

Hi,

Sorry about this, but I'm having new problems with my vegetation fraction ancillary file. I suspect this closely relates to ticket #2558 (especially comments 8-14) which Simon solved, but I have gone through this and checked everything, so perhaps it's a different error.

The error I'm getting is:

[0] ???!!!???!!!???!!!???!!!???!!!       ERROR        ???!!!???!!!???!!!???!!!???!!!
[0] ?  Error code: 3
[0] ?  Error from routine: GLUE_CONV_6A
[0] ?  Error message: Mid conv went to the top of the model at point            3 in seg on call  1
[0] ?  Error from processor: 568
[0] ?  Error number: 12
[0] ????????????????????????????????????????????????????????????????????????????????

I have checked all the things suggested by ticket #2558 and these should be fine - the various tiles do sum to 1, there is no missing data, and the latitudes go from South to North as they should. I have checked the output coming out of the reconfiguration, and it looks fine and does it should i.e. my data on the correct mask. You will notice that there is banding in the data, and it is filled, but that's how it should be.

The only thing I wonder about is whether, as you can see, the data is upside down relative to the latitudes. I'm wondering if this is again due to using the room version in xancil - I created these with version 10.7 but maybe it should be earlier?

My files are at:

/home/d05/cwilliams/gc31/new_ancils/vegetation/fractions_igbp/qrparm.veg.frac and /home/d05/cwilliams/gc31/new_ancils/vegetation/func_type_modis/qrparm.veg.func

Please would you help?

Charlie

Change History (28)

comment:1 Changed 5 months ago by simon

Have you tried swapping the data in the ancil and rerunning?

comment:2 Changed 5 months ago by charlie

By swapping do you mean flipping the data, but not the latitudes?

comment:3 Changed 5 months ago by simon

Yes,

For ancillary files, the latitude (and longitude for that matter) values are ignored, it's the orientation of the stored data (N-S vs S-N) with matters.

comment:4 Changed 5 months ago by charlie

Hi Simon,

Okay, tried that (flipping the data but not the latitudes), and get exactly the same error as above.

Charlie

comment:5 Changed 5 months ago by charlie

Hi again,

Just a quick update: I had an idea, which was to reimpose the land-sea mask back onto the veg fraction and function fields. This is because, when modifying another ancillary (soil parameters), I was getting a very similar error and it turned out to be because the field was filled (i.e. no mask). With that ancillary, when I added the mask back in, it worked. No-one I spoke to could understand why it worked - it was almost as if the model was expecting a mask and was falling over when there wasn't one - but it did work. So I wondered if that might be the problem here. So I reimposed the mask, and I also know that the metadata is correct as well this time, because my modified data was simply inserted into a copy of the original i.e. it should be identical to the original (e.g. it's attributes), other than the actual values. However, exactly the same error as above, at exactly the same point.

Please can you help?!

Charlie

comment:6 Changed 5 months ago by simon

Hi,

What's the name of the new ancil?

Simon.

comment:7 Changed 5 months ago by charlie

The fractions are at /home/d05/cwilliams/gc31/new_ancils/vegetation/fractions_igbp where qrparm.veg.frac is my modified version and qrparm.veg.frac_orig is the original.

Likewise, the functions are at /home/d05/cwilliams/gc31/new_ancils/vegetation/func_type_modis where qrparm.veg.func is my modified version and qrparm.veg.func_orig is the original.

By eye in xconv at least, they are identical apart from the actual data. What am I missing?

comment:8 Changed 5 months ago by simon

Erm, isn't the data obviously inverted in the new ancils? Looking at the unmasked eocene2 .nc files, the data over the land masses looks OK, and the unmasked "bad" data is over the oceans. In your new ancils, the data is inverted (this is obvious when looking at Africa), so some of the unmasked "bad" data appears on the land.

comment:9 Changed 5 months ago by charlie

I see what you mean. So shall I flip the data in my masked version?

comment:10 Changed 5 months ago by charlie

Sorry for the delay. I have now done this, flipping both the fractions and the functions (data, not latitudes) and keeping the mask. The new versions are now qrparm.veg.frac and qrparm.veg.func which should now be the same way up as the filled versions (eocene3*)

However, after resubmitting, I get exactly the same error message at exactly the same point.

Please help!

comment:11 Changed 5 months ago by charlie

Sorry, typo: eocene2*

comment:12 Changed 5 months ago by simon

Your "missing data" in the final ancil isn't missing data, it's Nans. These could be fed into the model when it runs, causing the crash. These Nans need to be replaced by -32768*32768 which is the missing data value used in the header. Also ensure that you are using the same mask as is used in the model. Any mismatch between the land sea mask used in the model and the land sea mask used to generate the ancillary is likely to cause issues.

comment:13 Changed 5 months ago by charlie

Okay, so I need to replace my NaNs? with -32768*32768? I have already checked that the masks match, they do.

comment:14 Changed 5 months ago by charlie

Nope, tried that, same error :-(

comment:15 Changed 5 months ago by charlie

Further to this, do you think it's worth me resubmitting when just one of my new fraction or function files, and keeping the other to the original (which works)? Just because at the moment we don't know which one, or possibly both, is causing the blowup.

comment:16 Changed 5 months ago by simon

Sounds like a reasonable thing to do.

comment:17 Changed 5 months ago by charlie

Okay, as suggested I have now run it with the original vegetation fraction and the new function, and vice versa.

The good news is that it would appear the new function is fine - it runs perfectly well if I use the original fraction and the new function.

The bad news is, therefore, that the error must lie with the fraction, because when I run it with the new fraction and the original function, I get the same error as above.

So something must still be wrong with the fraction. It can't be a problem with the mask matching the mask used by the model, because they were both created at the same time (as was the function, and all the other ancillaries which contain masks e.g. soil parameters). All of them were created at the same time by a Rose ancillary suite, using the same input mask. I have checked my qrparm.mask (also created at the same time), and it is identical to the mask in the fraction.

comment:18 Changed 5 months ago by charlie

Hi again,

Just to say that, because I was doubting myself, I have now checked the masks in detail, comparing the original version of the fraction with my new version, as well as the file coming out of the model (at /home/d05/cwilliams/cylc-run/u-be195/share/data/be195.astart - here, I have checked both the land mask and the fraction field). All 3 are identical - they all have exactly the same number of ocean points, in exactly the same place.

The only difference between them is the ocean points in my new fraction (the netcdf version) are all -32768*32768 (or rather -1.0737e+09), which is what you said to do, whereas in the original and in the restart dump (again converted to netcdf) they are 2.0000e+20. In the original fraction (the UM version), they are again -1.0737e+09. I'm assuming this is just the way xconv/xancil interprets missing values definitely, and isn't causing the problem?

Charlie

comment:19 Changed 5 months ago by charlie

Hi again,

Further to this, I have been browsing the various other tickets given my error, and came across #2079. This person also not the same error message when trying to implement a new vegetation fraction file, and it was resolved by the following:

This has been fixed: the new vegetation ancillary was too different to the old one and caused too big a jump between the start dump and the vegetation field I wanted to force the model with. I created a new transitional vegetation fraction ancillary that shifts from the present-day to the 1905 vegetation fractions over a period of 90 days, updating daily. I've only tested it for the first 3 days but checking the .out file suggests that the model is picking it up and using it and the job now runs without crashing.

Might this be the reason here, simply because I am modifying the original data a fair bit, such that there is significant banding in certain places? If this is the reason, could you advise how I can create a transitional field, as described above?

Thanks,

Charlie

comment:20 Changed 5 months ago by simon

What's the suite ID of your run? It'll be useful to look at your start dump.

comment:21 Changed 5 months ago by charlie

It's u-be195. I have just resubmitted it, using the new fraction, so expecting it to fail any minute. The start dump should be available within the next 10 minutes.

comment:22 Changed 5 months ago by simon

One comment so far, your new ancil has no land ice, while the old one has Antarctica covered in land ice, and there is also some in the northern hemisphere. If look at all fields in your soil param file, qrparm.soil, all the values are constant, apart from the land ice points found in the old veg fraction file. As you now have no land ice, perhaps all land values in the qrparm.soil should be set to the constant value globally? I'm not an expert on this, however.

comment:23 Changed 5 months ago by charlie

Hi Simon,

The land ice in the fraction is correct, there should be no land ice anywhere. So I purposely set these to 0. In the soil parameters file, I have replaced everywhere with the area weighted global mean, except those areas that contain land ice which have also been zeroed. This is how it should be to reflect the time period I'm modelling, the early Eocene, when there was no land ice anywhere and therefore whatever soil parameters (which we have to estimate/idealise) should also be zero over those land ice regions.

Charlie

comment:24 Changed 5 months ago by simon

So even if there's no land ice, there are still regions where the soil parameters are all set to zero? In this case, is the model expected to run with all soil parameters set to zero?

comment:25 Changed 5 months ago by charlie

Errr… Yes, I suppose, is that a problem? In my original versions (where the fraction file has land ice over Antarctica/Greenland?), the soil parameters were zero over these same regions. So when I filled these parameters for the rest of the world with the global mean, I thought I should keep the zeros as zeros. And just remove the land ice from the fraction. Is this likely to be a problem?

I'm going to be in the department this morning, so are you around for a quick chat? If so, where is your office these days?

comment:26 Changed 5 months ago by simon

Sometime 1100-1200 is best. I'm in HP113 in the Harry Pitt, right at the other end of the building.

comment:27 Changed 5 months ago by charlie

Hi Simon,

Just to say that yes, you were right, that was the problem. I have now changed my soil parameters and dust such that they now contain the constant value everywhere, including Antarctica/Greenland?, and it works.

Many thanks,

Charlie

comment:28 Changed 5 months ago by simon

  • Resolution set to fixed
  • Status changed from new to closed

Excellent, I'll close the ticket.

Simon.

Note: See TracTickets for help on using tickets.