Opened 6 months ago

Last modified 5 months ago

#3511 new help

Odd land results- possibly to do with ancilleries

Reported by: ha392 Owned by: um_support
Component: UM Model Keywords:
Cc: Platform: ARCHER2
UM Version: 11.4

Description

Hi,
I am currently having some troubles with an atmospheric model, where I am getting odd results over land. I am using a copy of suite u-ca369, running as u-cb624 (control) and u-cc554 (experiment). I have created ancilleries from sea ice and SST data using xancil. The model uses these ancilleries to make a small change, that would not logically cause a large global difference over land. I am not changing anything else in the models.

I originally wondered if it was to do with the extrapolating over land for the SSTs, but again, this should not make a difference, as the control and experiment are created in the same way, with only a very small difference (same values set to NaN etc), and after checking the ancilleries against each other, the only difference in the SIC and SST model inputs are the expected small differences in the region expected. I have tried a couple of experiments to test some theories around this, but I am still getting the odd land result.

I still think it must be something to do with how I am making the ancilleries, as the control and experiment suites are exact copies of each other, apart from these ancilleries. Note- the most recent runs were not successful for the experiment versions of the model, but worked fine for the control- these were attempts using the land mask option in xancil, so the fact that this option worked for the control and not the experiment tells me something, I am just not sure what. Any advice is welcome after a few weeks of getting nowhere.

Thank you,
Holly

Attachments (1)

Day1.jpg (88.9 KB) - added by ha392 6 months ago.
Temperature at surface at the first day

Download all attachments as: .zip

Change History (19)

comment:1 Changed 6 months ago by jeff

Hi Holly

At the moment I can't see any of your archer2 files, can you run these commands on archer2 so I read your files

chmod -R g+rX /home/n02/n02/ha392
chmod -R g+rX /work/n02/n02/ha392

Thanks.

Jeff.

comment:2 Changed 6 months ago by ha392

Hi Jeff,

Ah my apologies, I had not done this yet for Archer2. You should now be able to see them.

Brief update since yesterday, I have fixed the latter issue with the ancillary files, so now the control and experiment files are exactly the same- and the experiment file works with land mask applied. There should now be absolutely no difference between the two. However, after running the simulations and checking the control against the experiment this morning, I am still getting the issue with the model output difference, where the sea ice and SST output is as expected (the same as the input), but not surface temperature and other variables (strange signal over land in surface temperature, related odd signals in MSLP/ precip etc - all expected to be minimal).

Thank you,
Holly

comment:3 Changed 6 months ago by jeff

Hi Holly

It looks like the problem is being caused by a mismatch between the land/sea mask your original data is on and the the mask used by the UM. When you created the ancil files you choose the option to use the missing data to calculate the mask, but this mask is not the same as the UM one.

You will need the land fraction ancil file (/work/y07/shared/umshared/ancil/atmos/n96e/orca1/land_sea_mask/etop01/v2/qrparm.landfrac), use xconv to turn this into a netCDF file. Then in xancil under the Land fraction section set "Use Land Fraction NetCDF variable" to yes and add the land fraction nc file. Next for SST and Sea Ice select "Use land fraction NetCDF file to calculate land mask". This should now create the ancil files on the correct mask.

You also need to make sure your input data doesn't having missing data where the UM has sea points. you can use xconv to do this by using the extrapolation option on the input nc files.

Jeff.

comment:4 Changed 6 months ago by ha392

Hi Jeff,

Ah, that makes sense. When I have made ancilleries before I had used UM data to do so, therefore, I had not encountered this problem before. Yes, I am aware of the extrapolating, the most recent attempts were just trials without, I will make sure to do this with the new files. Hopefully this will fix the issue.

Thank you,
Holly

comment:5 Changed 6 months ago by ha392

Hi Jeff,

This unfortunately did not seem to make any difference to the output, and I am still getting an odd signal over land.

Thank you,
Holly

comment:6 Changed 6 months ago by jeff

Hi Holly

Where are you seeing this? I looked at the surface temperature fields in the cc554a.pa files in directory /work/n02/n02/ha392/cylc-run/u-cc554/share/data/History_Data and they look ok to me, or at least not like before.

Jeff.

comment:7 Changed 6 months ago by ha392

Hi Jeff,

In the comparrison between the cc55a.pm and cb624a.pm values, such as surface temperature, pressure etc. The differences are coming out illogical for the change in SST and SIC, when nothing else is changed in the model. for example, there is an odd difference signal in temperature over land, that cannot be explained by the small SST and SIC perturbation. I have checked over my analyis scripts in case it was some kind of time lag, but they seem fine, and I see it when looking at the two raw data files. This occurs from the first month of output, where it makes no logical sense for them to have a large difference over land points and regions that are spatially far away from the perturbation. The SST and SIC outputs are coming out of the model as expected.

Thank you,
Holly

comment:8 Changed 6 months ago by jeff

Hi Holly

The fields I looked at before you updated the ancil fields had obviously bad data points around the coast and this has now gone so the ancil file fix has at least worked for that.

Looking at the difference in the surface temperature for the first daily mean shows only a small difference if you think this isn't correct then maybe try outputting surface temperature for every timestep for the first day and compare those and see if the time evolution of the fields make sense. You could also check the time evolution of the differences in the daily mean surface temperature.

Jeff.

Changed 6 months ago by ha392

Temperature at surface at the first day

comment:9 Changed 6 months ago by ha392

Hi Jeff,

I have had a quick check through, and I am finding that it still doesn't make sense. The only difference between the two models should be the SST and SIC ancilleries, and these come out as expected, with a small perturbation in the Weddell Sea. I have attached the difference of the daily mean surface temperature for the first day, which evolves from there and the difference continues to diverge in the time series. As you can see, this difference is unexpected, as it is global in reach and quite random. This is a very odd and illogical response from such a small localised perturbation. Something is definitely going wrong, still possibly with my ancilleries, I just cannot work out what.

Thank you,
Holly

comment:10 Changed 6 months ago by jeff

Hi Holly

I tried to have another look at your files but they seem to have been deleted now. I couldn't see anything wrong with your jobs and I think the output could be correct, if not what you expect. I am by no means an expert in this though.

Jeff.

comment:11 Changed 6 months ago by ha392

Hi Jeff,

Thank you for looking into it. I am still not entirely convinced, so I am running a few more tests. The temperature difference plot at day 1 between the two models attached to this ticket still does not quite seem right for this type of perturbation. With the perturbating being a small difference in the Weddell Sea, it does not make sense (even with internal variaibltiy) to get such a large difference over Siberia.

Thank you,

Holly

comment:12 Changed 6 months ago by ha392

Hi again Jeff,

I have just run these models again for one month, this time using the SIC and SST outputs from previous versions, in case it was something to do with the original files somehow not agreeing with the model (my logic is that the model output would be in the correct format at the very least). Unfortunately, I am still getting the same result. Perhaps it is some kind of particularly large internal variaibltiy, but if you are willing to have another quick look over, then the recent runs are now there. Thank you for your help so far, perhaps I am chasing dead ends after all!

Thank you,
Holly

comment:13 Changed 6 months ago by jeff

Hi Holly

I ran cb624 and cc554 for 1 day outputting surface temperature every timestep. The difference between the 2 jobs for that field can be found here /work/n02/n02/jwc/helpdesk/3511/safe/diff_temp_ts.nc

Looking at the output after 1 timestep (20 minutes) there appear to be many small differences appearing everywhere. These small differences of course start to grow and explain why you get unexpected results. Whether this is the way the UM is meant work, I'm not really sure. It would probably need a UM expert at the Met Office to answer this.

Jeff.

comment:14 Changed 5 months ago by ha392

Hi Jeff,

Interesting. Thank you for testing this yourself, at least I now know that it isn't that I am doing something obviously wrong with the ancillaries. I plan to run a test using daily ancilleries next (once I can work out how to get that working), then I will see if I get the same issues.

Thank you,
Holly

comment:15 Changed 5 months ago by ha392

Hi again Jeff,

Seeing as it does seem to be as issue with the UM, who could I contact to discuss this with? You mention contacting a possible expert at the Met Office- to which I would have no idea where to start.

I have run similar experiments in the past, where I do not get these issues, using version 10.7 (e.g u-bp258 based on u-av321). I know that there was talk of not bringing these versions over to ARCHER2, but I noticed, for example, that u-as037 has recently been converted for ARCHER2. Is there a change in this decision?

I am unsure where to go from here, as we have deliverables coming up, and if it is seemingly not a mistake in my ancilleries or a silly mistake, I do not know what more I can do to get the models to work correctly.

Thank you,
Holly

comment:16 Changed 5 months ago by grenville

Hi Holly

We'd planned to not install UM 10.X versions, but did install 10.7 since it is associated with CMIP6 runs. Other version 10 installations are incomplete or for a specific CMS purpose. We plan to only move in step with new UM releases.

My feeling is that the solution to the SST perturbation problem will not be a quick fix. Porting your 10.7 suite to ARCHER2 sounds like it might be the fastest way to proceed. Having a job that doesn't give strange results will potentially be a help in determining the cause of the observed behaviour.

Porting a 10.7 suite should be straightforward.

Grenville

comment:17 Changed 5 months ago by ha392

Hi Grenville,

Thanks for your reply. I spent yesterday trying to track down my results from the 10.7 suite, as I had a thought that I had only looked at the ensemble mean, and not the single members for daily results. I seem to actually get the same issue (that is smoothed out in the ensemble mean- i.e internal variablity), so I do not think this would make a difference to try with 10.7.

My guess now is that it must be some kind of very large internal variablity. I guess the question is whether the model is supposed to be like this (as this seems like a fairly simple check for the experts?), or I have been doing something very wrong- which is a possibility, although I can't think of what that could be, as my changes to suites have been very minimal, with mostly just ancillary changes in SST and SIC (which have now been checked). If this is the case and the models are supposed to have these strange results/ very large, random internal variablity, then I apologies for wasting any time.

Thank you,
Holly

comment:18 Changed 5 months ago by grenville

Hi Holly

I talked to a modeller with vast amounts of UM experience - he was not overly surprised that changing ice fractions and temperatures would have a potentially large impact on model behaviour, so I think your case is not an anomaly. Andrew Orr (BAS) might have some more insight.

Grenville

Note: See TracTickets for help on using tickets.