Opened 3 months ago

Closed 7 days ago

#2971 closed help (fixed)

AINITIAL issue from UKESM coupled to atmos-only

Reported by: ChrisWells Owned by: um_support
Component: UM Model Keywords: UKESM, AMIP
Cc: Platform: Monsoon2
UM Version: 11.1

Description

Hi,

I tried to run the AMIP UKESM suite with an AINITIAL file of my own, output from a UKESM run. I turned off dynamic vegetation in my coupled run, and that seems to cause problems for the atmos-only model. when I compare my AINITIAL file (/home/d00/chwel/expts/cmip6/ensembles/bh765a.da21500101_00) with the defaul AMIP one (/projects/ukesm/jwalton/startdumps/u-az513/az513a.da19790101_00) in xconv I can see the default one has ~200 more variables in, seemingly all related to the vegetation scheme in some way.

Is there some way for me to get the suite to accept my AINITIAL file? The veg scheme is already off in the AMIP suite.

Cheers,
Chris

Attachments (1)

xancil.png (63.6 KB) - added by ChrisWells 2 months ago.

Download all attachments as: .zip

Change History (39)

comment:1 Changed 3 months ago by grenville

Chris

What is the suite id of the failing jib?

Grenville

comment:2 Changed 3 months ago by ChrisWells

Hi Grenville,

It's suite u-bl172 - the only changes from u-bj557, an AMIP copy which I've ran ok, were the ainitial file and the start date.

Cheers,
Chris

comment:3 Changed 3 months ago by grenville

Chris

Please don't move any files used in u-bl172, while I try to run it.

Grenville

comment:4 Changed 3 months ago by grenville

Chris

The error is

????????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!! ERROR ???!!!???!!!???!!!???!!!???!!!
? Error code: 800
? Error from routine: RCF_ANCIL_ATMOS
? Error message: replanca_rcf_replanca:ERR:LAND FRAC & MASK ARE INCONSISTENT
? Error from processor: 0
? Error number: 10
????????????????????????????????????????????????????????????????????????????????

The reconfiguration appears to find missing value indicators for the land fraction at valid land points.

Could you try running a reconfiguration of the start file in the u-bh765 suite (ie in the suite that generated the start file.)

Grenville

comment:5 Changed 3 months ago by ChrisWells

Hi Grenville,

Thanks for looking into this. I'm unsure what you mean by running a reconfiguration, sorry - do you mean to run the bh765 suite without the model on?

Cheers,
Chris

comment:6 Changed 3 months ago by grenville

Chris

I ran /home/d00/chwel/expts/cmip6/ensembles/bh765a.da21500101_00 through a reconfiguration in u-bh765 successfully, so the problem appears to be UM version related. We are still looking, but upgrading the AMIP job to 11.2 may be advisable.

Grenville

comment:7 Changed 3 months ago by ChrisWells

Hi Grenville,

Thanks for that - I'm unsure how to upgrade the AMIP job to vn11.2, apologies; happy to have a go if there are instructions somewhere. I can see the metadata in Um says 11.1 in the atmos-only, but 11.2 in coupled - that could be a problem if I can't change it, since I would really like the atmos-only version to have identical science to the bh765 job in the atmosphere; depends how much difference there is.

One extra thing; I turned off the ch4 link between ukca and radaer in bh765; I was planning on making this change to the atmos-only version too once I had it running, but I don't know if this will cause another problem when trying to initialise that current atmos-only run with l_ukca_radch4 on, from a dump from bh765 with it off.

Cheers,
Chris

comment:8 Changed 3 months ago by grenville

Chris

The upgrade worked OK (for the reconfiguration - I didn't run the model). Do this (in your suite):

~/roses/u-bl172> cd app
~/roses/u-bl172/app> cd um
~/roses/u-bl172/app/um> rose app-upgrade vn11.2

… accept all the changes

then do the same thing in ~/roses/u-bl172/app/fcm_make_um

Then remove the branches from the build (they are not needed at 11.2) - then rebuild.

Grenville

comment:9 Changed 3 months ago by ChrisWells

Hi Grenville,

Thanks for sorting that! I'm afraid I'm not sure what you mean by removing the branches - is this in the GUI? and which branches?

Cheers,
Chris

comment:10 Changed 3 months ago by grenville

Chris

go to fcm_make_um→env→sources and remove all the branches listed under um_sources.

Grenville

comment:11 Changed 3 months ago by ChrisWells

Hi Grenville,

I think I did those steps right - I ran that command line and removed those branches, but I get this error on Recon:

????????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!!       ERROR        ???!!!???!!!???!!!???!!!???!!!
?  Error code: 4324
?  Error from routine: CHECK_IOSTAT
?  Error message:
?        Error reading NAMELIST io_control
?        IoMsg: The variable name 'PRINT_MEMORY_INFO' is unrecognized in namelist input.
?        Please check input list against code.
?  Error from processor: 0
?  Error number: 0
????????????????????????????????????????????????????????????????????????????????

I've looked at tracs with errors in the same routine, but can't see anything obvious I should try. I noticed that in my job.out file I have these lines:

[WARN] UM version (VN=x.y) defined in the environment.
[INFO] Overriding $VN to 11.1
[WARN] Using default STASHmaster as none provided "/projects/um1/vn11.1/ctldata/STASHmaster".
[WARN] Using default STASH2CF as none provided "/projects/um1/vn11.1/ctldata/STASH2CF/STASH_to_CF.txt".

Which might mean the version hasn't fully changed to 11.2? Or that I need to point the suite to some other files?

Cheers,
Chris

comment:12 Changed 3 months ago by ros

Chris,

You need to upgrade both the um and fcm_make_um apps. You have not done the fcm_make_um one as it is still at vn11.1

Cheers,
Ros.

comment:13 Changed 3 months ago by ChrisWells

Hi Ros,

My bad - thanks for pointing that out. I ran it with that change, also having to change i_override_date_time, and recon worked but I got this error in atmos_main:

????????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!!       ERROR        ???!!!???!!!???!!!???!!!???!!!
?  Error code: 201
?  Error from routine: UP_ANCIL
?  Error message: REPLANCA: PP HEADERS ON ANCILLARY FILE DO NOT MATCH
?  Error from processor: 376
?  Error number: 59
????????????????????????????????????????????????????????????????????????????????

From #807 I guess this is a problem with field 201-200=1, which I should find in the ANCILmaster file. I don't know what ancil reference file my suite is using; there isn't an ANCILmaster folder under /projects/um1/vn11.2/ctldata/, as there are for 7.3, 8.2, and 8.4. In all of those ancil fields files, the stash 1 is land-sea mask.

Do you know what might be going wrong here?

Cheers,
Chris

comment:14 Changed 2 months ago by ChrisWells

Hi,

Just wondering if there had been time to take a look at this yet?

Cheers,
Chris

comment:15 Changed 2 months ago by willie

Hi Chris,

I have just caught up with this ticket. The pe_output has the following,

REPLANCA: UPDATE REQUIRED FOR FIELD 1
REPLANCA - time interpolation for ancillary field, stashcode  31
targ_time,time1,time2  2418852.,  2418480.,  2419200.
hours,int,period  2418852,  720,  -1
Information used in checking ancillary data set: position of lookup table in dataset: 3360
Position of first lookup table referring to data type  1
Interval between lookup tables referring to data type  1  Number of steps 3359
STASH code in dataset  24   STASH code requested  31
'start' position of lookup tables for dataset  1 in overall lookup array  1
24,  31,  1
UP_ANCIL : Error in REPLANCA.

If you look at STASH item 24 in the Configure ancils and initialise dump fields table, it points to the ancillary $CMIP6_ANCILS/n96e/timeseries_1870-2016/SstSeaIce/sst_amip_n96e.anc. But your dump is for the year 2150. So you don't have ancillaries for this time.

STASH items 96, 132 and others will also suffer this problem.

Willie

comment:16 Changed 2 months ago by willie

  • Keywords UKESM, AMIP added
  • Platform set to Monsoon2
  • UM Version set to 11.1

comment:17 Changed 2 months ago by ChrisWells

Hi Willie,

Many thanks for that. I just changed the start year to 2000 and it worked, but I need to be able to use my own ancil files; I replaced the sst ancil with my own, and got this error on atmos-main:

????????????????????????????????????????????????????????????????????????????????
??????????????????????????????      WARNING       ??????????????????????????????
?  Warning code: -10
?  Warning from routine: ANCIL_CHECK_GRID_STAGGER
?  Warning message: Ancil file mismatch in fixed header(9) grid stagger value
?          Model grid stagger = 6
?          Ancil file grid stagger = 2
?          Ancil file path = /home/d00/chwel/expts/cmip6/tropics_perturbations/atmos_only/u-bh765m_1year_sst.anc
?          PLEASE READ - this warning will be converted to an error
?          in future. Please update ancil file to specify the correct
?          grid stagger value.
?  Warning from processor: 0
?  Warning number: 59
????????????????????????????????????????????????????????????????????????????????

Do you know what might be causing this?

Cheers,
Chris

comment:18 Changed 2 months ago by willie

Hi Chris,

This is just a warning - you can ignore it. Your start dump and SST file are both for the year 2150, but some of the other ancillary files don't go that far. Your atmosphere models is crashing with a segmentation fault, probably as a result of this.

Willie

comment:19 Changed 2 months ago by ChrisWells

Hi Willie,

But I'm trying to run it at 2000 now, with it set to overwrite the ancil values (I turned on i_override_date_time and set it to year 2000). Should I not expect this to sort out the date issues? And if not, should I recreate the sst ancil and give it a date within the others? I want to run the model for a few decades with repeating sst and sea ice from 1 year - can I get the model to expect that?

Cheers,
Chris

comment:20 Changed 2 months ago by willie

Hi Chris,

Overriding the date in um → Reconfiguration and Ancillary Control → Output dump fixed header override only changes the date in the reconfigured start dump. This is in share/data/bl172a.astart. This is then passed on to the atmosphere model. So it is incompatible with your SST ancillary year 2150.

I think you need to extend the ancillary files up to 2150 plus several decades. Avoid overriding the start date in the dump.

Willie

comment:21 Changed 2 months ago by ChrisWells

Hi Willie,

Many thanks for that info and advice - I'll give that a go.

Cheers,
Chris

comment:22 Changed 2 months ago by ChrisWells

Hi Willie,

I remade my ancil file to be in the year 2000 (/home/d00/chwel/expts/cmip6/tropics_perturbations/atmos_only/bh765_2000_sst.anc), to see if that would make it run, but I get a similar error to before:

????????????????????????????????????????????????????????????????????????????????
??????????????????????????????      WARNING       ??????????????????????????????
?  Warning code: -10
?  Warning from routine: ANCIL_CHECK_GRID_STAGGER
?  Warning message: Ancil file mismatch in fixed header(9) grid stagger value
?          Model grid stagger = 6
?          Ancil file grid stagger = 2
[384] exceptions: An exception was raised:11 (Segmentation fault)
[388] exceptions: An exception was raised:11 (Segmentation fault)
[386] exceptions: An exception was raised:11 (Segmentation fault)
[384] exceptions: the exception reports the extra information: Address not mapped to object.

I'm also not sure I understand the ancillary file issues - for example the atmos only run uses the ancil

$CMIP6_ANCILS/model_derived/ukesm1.0_historical_r5i1p1f3_u-az513/n96e/clim_1979-2014/ocean_bgc/v1/CHL_clim_az513_1979_2014_n96.anc

which is a 1-year-long file with time set to 1979; but it works in the model at 2000 when I use the default sst one - is there some attribute which lets it repeat? And I think my error must be a different one since it wont work with my new 2000-based sst ancil?

Many thanks,
Chris

comment:23 Changed 2 months ago by willie

Hi Chris,

The atmosphere model is failing with a segmentation fault. I think this is because your SST ancillary

/home/d00/chwel/expts/cmip6/tropics_perturbations/atmos_only/bh765_2000_sst.anc

has the wrong data and format. This is for UM4.5 and the data appears to be REF SEA SURF TEMPERATURE in deg C, stash 24. These range in value -40 to +40. I think you need vn 11.2 SURFACE TEMPERATURE AFTER TIMESTEP.

Willie

Changed 2 months ago by ChrisWells

comment:24 Changed 2 months ago by ChrisWells

Hi Willie,

I must be using the wrong method for generating the ancil file. I'm using xancil with file /home/d00/chwel/expts/cmip6/tropics_perturbations/atmos_only/u-bh765m_1year.nc, which is from monthly Stash 24 data downloaded from MASS. The xancil option I use I've attached. Where should I get the correct data to generate the file with, and do my xancil options seem right?

Cheers,
Chris

comment:25 Changed 2 months ago by willie

Hi Chris,

From memory, in Xancil on the general page, you need to set the UM version.

Willie

comment:26 Changed 2 months ago by willie

PS The Xancil user guide is here http://cms.ncas.ac.uk/documents/xancil/

comment:27 Changed 8 weeks ago by ChrisWells

Hi Willie,

I'm afraid I still haven't got this working. I played around with the xancil settings, following the documentation, but before I was choosing "Reference SST, SSS, Air-Temp and Ice Depth" as the field to be created, and now I think I should be making just SST and Sea ice files, separately. But when I try and create the SST file, using an nc file of Surface Temperature output from the coupled model, I get

 Writing Sea Surface Temperature ancillary file
 /home/d00/chwel/expts/cmip6/tropics_perturbations/atmos_only/sst
 ERROR: in procedure get_ncdiminfo 1 : NetCDF error number          -49 :
 NetCDF: Variable not found :

The same issue occurs with sea ice, with the same error number

I think I might be using the wrong input data? The documentation doesn't mention what should be input - do you know what I should be using? I have just been using Stash 24 for temperature, and 31 for sea ice.

I've selected the variables ("temp" and "iceconc") in the xancil gui, so I don't know how it doesn't find the variables.

Cheers,
Chris

comment:28 Changed 8 weeks ago by willie

Hi Chris,

Stash 24 and 31 seem right. What is your input data to xancil?

Willie

comment:29 Changed 8 weeks ago by ChrisWells

Hi Willie,

In the directory /home/d00/chwel/expts/cmip6/tropics_perturbations/atmos_only , sst.nc and ice.nc are the nc files, which are those variables taken from u-bh765m_1year.nc, which is from data downloaded from MASS using the script download_template_multi_yonly_mine.sh in that directory.

sst.nc and ice.nc are both 1 year, 12 monthly timestep files with only the temp and iceconc variables in, from stashes 24 and 31.

Cheers,
Chris

comment:30 Changed 8 weeks ago by jeff

Hi Chris

Don't use "Reference SST, SSS, Air-Temp and Ice Depth" that is for the old UM ocean model in versions vn6.6 and earlier.

To see exactly how you have set up xancil can you create a job file, this allows you to save the exact state of xancil and reload it at a later date. Also looking at the xancil.namelist file it only has information from creating the ice ancil, you should create both sst and ice ancil files at the same time as they can interact with each other.

Jeff.

comment:31 Changed 8 weeks ago by ChrisWells

Hi Jeff,

Thanks for the info - that error in my comment 27 (1st one today) was from using the single sst and sea ice file creation options, not the Reference one, but I was using them one at a time - I didn't appreciate that they interact.

I just tried making them together, which I wasn't doing before, and can see that it has created 2 ancil files; I wasn't sure what options to pick in the individual variable panels, as I can't see any info on them in the documentation, so I used the default settings; I'll see if they work in my simulation.

Cheers,
Chris

comment:32 Changed 8 weeks ago by jeff

Hi Chris

Have you looked at the ancil files you created? They don't look correct to me, the ice fraction field is all zero and the sst looks to be frozen well into lower latitudes.

Looking at the job file in xancil you have set all the ice fields to be calculated, i.e. not using the input ice field at all. You probably want to set "Calculate Sea-Ice fraction values from SST value?" to no, this will then read it from the input file. You can then either calculate the ice depth and ice edge values from the ice fraction or set "Use AMIPII …" to yes and this will produce a file with only an ice fraction field.

If using AMIPII type ice ancil you will need to set l_amipii_ice_processing to TRUE in your UM job.

Jeff.

comment:33 Changed 7 weeks ago by ChrisWells

Hi Jeff,

Thanks - I followed those steps and it now produces ice and sst files which seem right, and work in the model.

However, the atmos-only suite has ancil files for DMS and CHL (stashes 132 and 96), and also 3 vegetation ones (stashes 216, 217, 218). I can download data for DMS and CHL from my coupled run, but in xancil I can't see the option to create those ancils.

And the veg stashes aren't output from my coupled runs (e.g. u-bh765), or from other ukesm historical suites, but they should be constant values anyway for my suites as I turned the dynamic veg scheme off, so could I output them from a 1-year coupled run? Another issue is that xconv gives me an error when I try and convert the veg ancils to nc to have a look at them.

Do you have any advice on these?

Many thanks,
Chris

comment:34 Changed 7 weeks ago by jeff

Hi Chris

If there isn't a panel to create an ancillary file you can use the "Generalised Ancillary Files" option, this can be used for creating any ancillary file.

I'm not sure about the veg stashes maybe someone else can help.

I would need to know which veg files have a problem with xconv to see what is wrong, but I'm away until Sept. 18th so won't be able to look until then.

Jeff.

comment:35 Changed 6 weeks ago by ChrisWells

Hi Jeff,

The veg files in the default AMIP UKESM suite (u-be303, my u-bl924) are:

/projects/ancils/cmip6/ancils/model_derived/ukesm1.0_historical_r5i1p1f3_u-az513/n96e/timeseries_1979-2014/vegetation/v1/u-az513_m01s00i216_1979-2014_annual_timeseries_land_cover_frac.anc

/projects/ancils/cmip6/ancils/model_derived/ukesm1.0_historical_r5i1p1f3_u-az513/n96e/clim_1979-2014/vegetation/v1/u-az513_m01s00i217_1979-2014_monthly_climatology_LAI.anc

/projects/ancils/cmip6/ancils/model_derived/ukesm1.0_historical_r5i1p1f3_u-az513/n96e/clim_1979-2014/vegetation/v1/u-az513_m01s00i218_1979-2014_time_mean_canopy_height.anc

on monsoon, for stashes 216, 217, and 218. Using Xconv, the 1st file doesn't open, the 2nd one has 3 years of monthly data, and the 3rd seems to be a single timeslice. The 2 which open are on 5 levels (Plant Functional Types, I think).

Cheers,
Chris

comment:36 Changed 2 weeks ago by ChrisWells

Hi,

I've got veg ancils sorted now, with help from Alistair Sellar, but I am still having issues with the DMS and CHL ancils.

I made them in /home/d00/chwel/expts/cmip6/tropics_perturbations/atmos_only/ancils/veg_ocean/CHL_DMS - the xancil jobs are in there, and the ancils (DMS_2000.anc and CHL_2000.anc) look similar to the defaults (CHL_clim_az513_1979_2014_n96.anc, DMS_clim_az513_1979_2014_n96.anc) although the DMS variable has a different name.

I ran them in u-bn509, which failed on recon with what I think is a segmentation error? But I can't seem to get much info from the output. I ran the same setup with the default DMS and CHL files in u-bn514, which worked.

Do you know how I should fix this?

Cheers,
Chris

comment:37 Changed 12 days ago by jeff

Hi Chris

You created these ancils for UM version 4.5, try changing that to 11.1.

Jeff.

comment:38 Changed 7 days ago by ChrisWells

  • Resolution set to fixed
  • Status changed from new to closed

Hi Jeff,

Thanks for spotting that - I thought I'd changed that but must've not. I've made new CHL and DMS ancils and the model seems to run fine with them.

Thanks Jeff, Grenville, Willie, and Ros, and Alistair via email, for helping me get this atmos-only suite running - I'll close this ticket now.

Cheers,
Chris

Note: See TracTickets for help on using tickets.