Opened 5 months ago

Closed 5 months ago

#2816 closed help (fixed)

UM looking for non-existing days and ancillary problem

Reported by: cbellisario Owned by: willie
Component: UM Model Keywords: nudging
Cc: Platform: NEXCS
UM Version: 11.0

Description (last modified by willie)

Dear helpdesk team,

I am currently trying to run a whole year with the UM (suites bd467, be515, bg907 and bg908) however they all crash in February 1989 with the error

???????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!!       ERROR        ???!!!???!!!???!!!???!!!???!!!
?  Error code: 1
?  Error from routine: EM_FOPEN
?  Error message: NetCDF error
?  Error from processor: 0
?  Error number: 81
????????????????????????????????????????????????????????????????????????????????

[0] exceptions: An non-exception application exit occured.
[0] exceptions: whilst in a serial region
[0] exceptions: Task had pid=70942 on host nid04392
[0] exceptions: Program is "/home/d04/chrbe/cylc-run/u-bd467/share/fcm_make_um/build-atmos/bin/um-atmos.exe"
Warning in umPrintMgr: umPrintExceptionHandler : Handler Invoked
[NID 04392] 2019-03-14 20:39:05 Apid 59087547: initiated application termination
[FAIL] um-atmos # return-code=137
2019-03-14T20:39:12Z CRITICAL - failed/EXIT

Following http://cms.ncas.ac.uk/ticket/2365, I tried to turn it off and on again without any success, and following http://cms.ncas.ac.uk/ticket/2763, I looked at the pe_output to see that the UM was looking for non existing file:

NUDGING_NETCDF_LOADER: filename 1: /projects/ukca-admin/analyses/era-in//ecm-e40_1deg-model-levs_1989022900_all.nc
NUDGING_NETCDF_LOADER: filename 2: /projects/ukca-admin/analyses/era-in//ecm-e40_1deg-model-levs_1989022906_all.nc
Error in EM_FOPEN: No such file or directory - /projects/ukca-admin/analyses/era-in//ecm-e40_1deg-model-levs_1989022900_all.nc

And there was no February the 29th in 1989.
My idea is that had to switch to "Gregorian" in the /suite conf/Model Configuration/Calendar? instead of 360 days.

However, I am facing now another problem with sea ice ancillaries:

????????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!!       ERROR        ???!!!???!!!???!!!???!!!???!!!
?  Error code: 1
?  Error from routine: io:file_open
?  Error message: Failed to open file /projects/um1/ancil/atmos/n96e/orca1/seaice/reynolds/1981_2009_greg/v1/qrclim.seaice
?  Error from processor: 0
?  Error number: 67
????????????????????????????????????????????????????????????????????????????????

[0] exceptions: An non-exception application exit occured.
[0] exceptions: whilst in a serial region
[0] exceptions: Task had pid=35010 on host nid05383
[0] exceptions: Program is "/home/d04/chrbe/cylc-run/u-be515/share/fcm_make_um/build-atmos/bin/um-atmos.exe"
[0] exceptions: calling registered handler @ 0x00417e80
Warning in umPrintMgr: umPrintExceptionHandler : Handler Invoked
[0] exceptions: Done callbacks
[NID 05383] 2019-03-15 10:48:08 Apid 59193084: initiated application termination
[FAIL] um-atmos # return-code=137
2019-03-15T10:48:12Z CRITICAL - failed/EXIT

The only gregorian seaice ancillary is in
/projects/um1/ancil/atmos/n96e/orca025/seaice/reynolds/1981_2009_greg/v1

So before going into further ducktape fixing, how can I make the model run for a whole year?

Thank you in advance,

Best regards,

Christophe

Change History (6)

comment:1 Changed 5 months ago by willie

  • Keywords nudging added; UM, EM_OPEN removed
  • Owner changed from um_support to willie
  • Status changed from new to accepted
  • UM Version set to 11.0

Hi Christophe,

The UMDP 083 on nudging the UM is worth a read https://code.metoffice.gov.uk/doc/um/latest/papers/umdp_083.pdf. In order to nudge the UM you must have a Gregorian calendar.

As for the sea ice ancillary, you may be able to get away with using,

 /projects/um1/ancil/atmos/n96e/orca025/seaice/reynolds/1981_2009_greg/v1

or maybe even regridding it using xancil, if not.

Willie

comment:2 Changed 5 months ago by cbellisario

Dear Willie,

Thank you for your help,

I have set
[namelist:items(4a4f86c3)]
ancilfilename='/projects/um1/ancil/atmos/n96e/orca025/seaice/reynolds/1981_2009_greg/v1/$UM_ANCIL_SEAICE_FILE'

[namelist:items(dbe5f9b1)]
ancilfilename='/projects/um1/ancil/atmos/n96e/orca025/sst/reynolds/1981_2009_greg/v1/$UM_ANCIL_SST_FILE'

However, I now get a new error:
????????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!! ERROR ???!!!???!!!???!!!???!!!???!!!
? Error code: 65536
? Error from routine: EM_GET_TIME_INFO
? Error message: NetCDF calendar 360_day does not match model calendar, and attribute calendar_flexible /= 1 or not set
? Error from processor: 0
? Error number: 80
????????????????????????????????????????????????????????????????????????????????

Any idea of where I have to remove this 360_day reading?

Thank you

comment:3 Changed 5 months ago by cbellisario

Addition:
Not sure if that helps but the first timestep of the UM is working. It crashes at timestep 1 and this is what appears just before the crash in job.out.

Maximum vertical velocity at timestep 1 Max w this run
w_max level proc position run w_max level timestep
0.755E+00 39 293 127.5deg E 31.9deg N 0.853E+00 79 0
n_band: 6
UKCA AGE-OF-AIR: Reset method= 1. Tracer will be reset upto level 10
5 files found in offline namelist

Still looking for fresh ideas….
????????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!! ERROR ???!!!???!!!???!!!???!!!???!!!
? Error code: 65536
? Error from routine: EM_GET_TIME_INFO
? Error message: NetCDF calendar 360_day does not match model calendar, and attribute calendar_flexible /= 1 or not set
? Error from processor: 0
? Error number: 81
????????????????????????????????????????????????????????????????????????????????

I tried to switch off few things without any success, I tried to turn .false. of
um/namelist/top level model control/run control and time settings/ lcal360
but also
um/namelist/UM science settings/short term logicals/l_rm_hardwire_gas360
still the same error. I have no idea to which file it is related.

comment:4 Changed 5 months ago by willie

Hi Christophe,

I'm not sure what suite you're modifying. If you want to change the ancillary file, then you should do this in the GUI Reconfig and Ancillary control -> Ancil options -> configure ancils and initialise dump fields. Do not edit the rose-app.conf directly. Once you have made your change you must then run Metadata -> um -> um/stash_indices.TidyStashTransform.transform macro. This will re-index the ancillary table and allow the model to recognise your changes.

Also, make sure that you have sent the calendar to Gregorian in suite conf -> Model Configuration.

Willie

comment:5 Changed 5 months ago by cbellisario

Dear Willie,

Thank you for your help,

I was working on suites bd467, be515, bg907 and bg908, trying most of the things on the first one.

I ask the creator the suite (Paul Earnshaw) about the problem and I was able to help me. Following his advice, I could make the code run:
What you need to do is set the suite to run with aerosol climatologies only. I think there are calendar dependent input files that have only been set up for the 360day calendar if you run with the GLOMAP MODE aerosol scheme. In your suite this can be done by adding "aeroclim" to the UM_OPT_KEYS variable in the rose-suite.conf file.

Best regards,

Christophe

comment:6 Changed 5 months ago by willie

  • Description modified (diff)
  • Resolution set to fixed
  • Status changed from accepted to closed

Hi Christophe,

Thanks for letting us know. I'll close this ticket.

Willie

Note: See TracTickets for help on using tickets.