Opened 3 months ago

Closed 8 days ago

#2929 closed help (fixed)

Error with l_trap_uv, and EG_BICGSTAB

Reported by: MJohnston1 Owned by: um_support
Component: UM Model Keywords:
Cc: Platform: ARCHER
UM Version: 10.9

Description

Hi Helpdesk,

I am trying to run an idealised simulation at coarser horizontal resolution. To do this I have copied a working simulation at 100 m horizontal resolution (u-bg387) and changed the resolution to 800 m (u-bi963).

I have successfully done this in the past (e.g. u-bh553) by changing the following

UM> namelist> UM Science Settings> Idealised> Initialisation> Horizontal Grid> delta_xi1 and delta_xi2

I have also changed the number of grid points in the domain (so that the total domain size is the same as the high resolution case), the number of processors, and the required ancillaries for land-sea mask and fraction of vegetation types (I assume that these are done correctly as they look like coarse versions of the working ancillaries in xconv).

The model successfully reconfigures, but crashes after the first time step. I was getting the error that my 'halos were too small for advection' which wasn't a problem when I ran the model without any ancillaries on a smaller test domain at this resolution.

So I reduced the time step from 24 seconds to 3 seconds (the time step I used for 100 m). I continued to get this error, so I turned on l_trap_uv as reccommended. Now I'm getting an 'EG_BICSTAB' error that NaNs? or inftys have been introduced.

So I've turned on diagnostic level output and print max winds but its not obvious to me where NaNs? would be coming from.

Doing diff between the …/app/um/rose-app.conf files for the working high resolution suite (u-bg387) and the not working coarse resolution suite (u-bi963), I can only spot the differences in grid, ancillaries, initial conditions, and STASH which I expect.

Maybe I'm missing something obvious here? Could there be something unrelated that is triggering the original halos/advection/l_trap_uv error?

Best Regards,
Michael

Change History (14)

comment:1 Changed 3 months ago by grenville

Michael

u-bh553 does not have any land, u-bi963 does have land but not all the required fields are in the start file. Why not simply reconfigure the start file from u-bg387 - that way you won't need any ancillary files; all the fields will be re-gridded to 800m resolution.

Grenville

comment:2 Changed 3 months ago by MJohnston1

Grenville,

Thank you for your response. I'm unfamiliar with changing the start file, but I have just given this a try. I am using /work/n02/n02/xb899100/cylc-run/u-bg387/share/data/atmos.astart as the start file for u-bi963.

At the reconfiguration step I get the following error:

????????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!!       ERROR        ???!!!???!!!???!!!???!!!???!!!
?  Error code: 4
?  Error from routine: LAM_INCLUSION
?  Error message: Southern boundary at   -5.6033430 not within source LAM with
50.0000000
?  Error from processor: 20
?  Error number: 2
????????????????????????????????????????????????????????????????????????????????

Seems like the reconfiguration is trying to use lat/lon, but the start dump I supplied uses a cartesian grid. I end up with the u-bi963 start dump with a cartesian grid when I was using the ancil files (which is what I expect).

Cheers,
Michael

comment:3 Changed 2 months ago by grenville

Michael

I'm a bit confused - both u-bg387 and u-bi963 are configured with the same domain?

Grenville

comment:4 Changed 2 months ago by MJohnston1

Grenville,

Double checking - the domains aren't exactly the same: the 100 m case is 1160 x 319 grid points (116.0 x 31.9 km) which doesn't map directly onto an 800 m grid. That plus the requirement to have even number of grid points in the east-west direction means that I've used a grid that contains 146 x 40 grid points (116.8 x 32.0 km) for the 800 m case.

I suspect that the reconfiguration expects a start file on a lat-lon grid, but then the idealised reconfiguration converts it to a cartesian grid.

Michael

comment:5 Changed 2 months ago by grenville

Michael

This crmstyle app is new to me - I see how the domains differ, but I can't even see how the resolution is set.

Grenville

comment:6 Changed 2 months ago by MJohnston1

Hi Grenville,

I have been setting the resolution using the following within the gui:

UM> namelist> UM Science Settings> Idealised> Initialisation> Horizontal Grid> delta_xi1 and delta_xi2

where delta_xi1 is the dx resolution, and delta_xi2 is the dy resolution. There are other places in the gui which sound like they set the resolution but they are either overwritten or ignored as they don't seem to do anything.

Hope that helps,
Michael

comment:7 Changed 2 months ago by grenville

Michael

It appears that the reconfiguration won't do the cartesian-cartesian case (the Rose metadata is unhelpful on the topic).

How did you create the landsea mask for the 800m model? - at this stage (short of changing back to the usual UM coordinates), I can only suggest that you repeat the procedure you used for the landsea mask for all the other land fields (check the 100m model for those required).

Grenville

comment:8 Changed 2 months ago by grenville

Michael - pl ease see below for advice between Rachel Stratton — can you upgrade your job to 11.3?

Grenville,

Until about UM11.0 cartesian output dumps from the UM were not correctly labelled as Cartesian. Once this had been done I started to add code to allow the reconfiguration of correctly labelled dumps.

UM11.1 - it is possible to reconfigure a Cartesian dump at one resolution to another resolution using bilinear interpolation. The new grid cannot cover a bigger domain than the original grid. A smaller grid new grid always starts from 0.0, 0.0 x and y, so you cannot chose which part of the grid you get.
Um11.3 - it is possible to reconfigure a Cartesian dump from one resolution to another using area averaging.

Please note that pre UM11.0 cartesian dumps cannot be correctly reconfigured as they don't have the correct information in their headers.
In the same way as for lat-lon dumps the fields are either taken from the source dump or ancillary files. Unfortunately I am not sure that Cartesian ancillary files are correctly supported. For some recent Idealised island tests I did I could create an ancillary land sea mask at 250m resolution but had problems with 1000m resolution. I think this may be because this resolution exceeds "360 degrees" resolution as far as some part of the ancillary file creation was concerned.

So the answer to your question is, if the 100m dump was created at UM11.0 or after it can be reconfigured to 800m. Before that and the user would need to try to modify code in some way to make the reconfiguration recognise it as a Cartesian dump.

Best wishes

Rachel

Hi Rachel

We have a user who has run an idealised model at 100m and would like to run the same thing at 800m. The 100m domain has a blob of land in it, but the 800m start file was missing lots of associated fields (I'm not sure how it was created). I suggested simply reconfiguring the 100m start file on to 800m - without understanding cartesian coordinates & reconfiguration. Can you advise how to reconfigure the file to 800m — the UM help is a bit ambiguous about cartesian-cartesian reconfiguration.

Thanks

Best wishes

Grenville

comment:9 Changed 2 months ago by MJohnston1

Hi Grenville,

I will give this a try this afternoon.

Thanks,
Michael

comment:10 Changed 2 months ago by MJohnston1

I've first copied the suite to u-bj874, then I've done the following:

cd /home/MJohnston1/roses/u-bj874/app/fcm_make/
rose app-upgrade -a vn11.3
cd ../um
rose app-upgrade -a vn11.3

Which produced the following error:

Traceback (most recent call last):
  File "/usr/local/python/lib/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/python/lib/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/home/fcm/rose-2016.11.1/lib/python/rose/upgrade.py", line 692, in <module>
    main()
  File "/home/fcm/rose-2016.11.1/lib/python/rose/upgrade.py", line 658, in main
    combined_config_map, meta_config, macro_function, macro_name=macro_id)
  File "/home/fcm/rose-2016.11.1/lib/python/rose/macro.py", line 1173, in apply_macro_to_config_map
    return_value = macro_function(macro_config, meta_config, conf_key)
  File "/home/fcm/rose-2016.11.1/lib/python/rose/upgrade.py", line 650, in <lambda>
    conf, meta, opts.non_interactive)
  File "/home/fcm/rose-2016.11.1/lib/python/rose/upgrade.py", line 469, in transform
    upgrade_macro_result = func(config, meta_config, **res)
  File "/home/fcm/rose-meta/um-atmos/version112_113.py", line 662, in upgrade
    step_secs = int(self.get_setting_value(config, ["namelist:nlstcgen", "steps_per_periodim"]))
ValueError: invalid literal for int() with base 10: '$NTSTEPS'

I'm not sure what to make of this.

Michael

comment:11 Changed 2 months ago by grenville

Michael

I upgraded your apps successfully - I went to 11.2 then to 11.3 (I was just testing) - try jumping directly to 11.3:

rose app-upgrade vn11.3

You will need to fix up branches/dev/michaeljohnston/vn10.9_idlsurffluxlandoption - for 11.3 (possibly)

Grenville

comment:12 Changed 8 weeks ago by MJohnston1

Grenville,

Is this in u-bj874? It is showing vn11.3 in ../app/fcm_make and vn11.2 in ../app/um.

When I use rose app-upgrade vn11.3 in ../app/um on this suite, it gives the same error as above.

Michael

comment:13 Changed 8 days ago by MJohnston1

To update on this situation:

I have found a satisfactory solution. I was unable to successfully upgrade my suite to vn11.2 or 11.3, however I redesigned my branch at vn10.9 so that ancil files are no longer required for my purposes, and the I get the expected results.

For grid spacing greater than 100 m, I now use branches/dev/michaeljohnston/vn10.9_idealised_island@73565

Michael

comment:14 Changed 8 days ago by willie

  • Resolution set to fixed
  • Status changed from new to closed

Hi Michael,
Thanks for letting us know
Willie

Note: See TracTickets for help on using tickets.