Opened 3 years ago

Closed 3 years ago

Last modified 2 years ago

#1951 closed help (completed)

Unknown time profile error

Reported by: mattjbr123 Owned by: um_support
Component: UM Model Keywords:
Cc: Platform: MONSooN
UM Version: 10.3

Description

Hi,

Having some trouble with STASH in Rose.
I have some variables in the suite I copied assigned to a time profile TALLTS (all timesteps) which are also assigned to usage profiles that send the output to a 'secondary store with user supplied TAG', which in my case is 98. I have no idea where this goes, so I cloned some of these variables/STASH requests and changed the usage profile to one that outputs to a fieldsfile. I have run all the STASH Macros that transform, validate and remove duplicates. Now whenever I try to run the model it throws up the following error:

????????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!!       ERROR        ???!!!???!!!???!!!???!!!???!!!
?  Error code: 16
?  Error from routine: rdbasis
?  Error message: UNRECOGNISED time profile request TALLTS
?  Error from processor: 0
?  Error number: 5
????????????????????????????????????????????????????????????????????????????????

which disappears when I disable the variables going to a fields file with TALLTS as the time profile.

As far as I can tell it is failing because 'rdbasis' cannot find TALLTS as a time profile in the STASHC file, which lists the STASH requests along with the domain, time and usage profiles required by these. For some reason the new STASH requests are getting written to this file, but the fact that they require TALLTS as their time profile isn't, and so TALLTS as a required time profile isn't written to the STASHC file.

Yet when the variables are just going to this 'secondary store' there is no problem. I have checked the STASHC file for this case, and TALLTS is still not in it as a time profile, but here it doesn't matter because there are no STASH requests requiring TALLTS in the STASHC file. So presumably having it output to a secondary store means it does not require the requests to be written to the STASHC file, and maybe some conflict is then causing it not to write TALLTS to the STASHC file when I request the same variables (cloned) outputted to a fieldsfile (with a different usage profile). I have tried just changing the usage profile of the variables with TALLTS, rather than cloning them, and this just brings up the same problem…

I've had a chat with William Ingram who has tried the same stuff as me, to no avail.

Any ideas?
I'm running suite u-af404 with Rose on MONSooN (username mabro, project solar)

Change History (19)

comment:1 Changed 3 years ago by willie

  • Platform set to MONSooN

comment:2 Changed 3 years ago by mattjbr123

Update: After deleting the TALLTS time profile and creating one identical to it with a different name and assigning all the variables to that instead I got it to appear in the STASHC file, but now it's complaining about an unknown usage profile, with the exactly the same type of error as above.

When I did the same with the usage profile however, it didn't fix the problem.

comment:3 Changed 3 years ago by willie

Hi Matt,

This is proving difficult. I am looking at the possibility that it is nothing to do with STASH, but an earlier error. I did the same as you, deleting TALLTS and then recreating it, but this did not solve the problem as it did for you: I still get error 16. TBC…

Willie

comment:4 Changed 3 years ago by willie

  • UM Version changed from <select version> to 10.3

comment:5 Changed 3 years ago by willie

Hi Matt,

I think I've got it working now. I switched the debugger on and ran it for only five minutes but it got to the first time step. To do this go to fcm_make_um > env > Advanced compilation and put a

-g

in the fcflags_overrides and the ldflags_overrides_prefix boxes and then run you job again.

So no other changes, just recompile with the debugger on.

Let me know how it goes.

Willie

comment:6 Changed 3 years ago by mattjbr123

Hi Willie,

Thanks for looking in to this - unfortunately I'm still getting the usage profile error with the debugger on. When I changed the usage profile I changed UPUKCA to UPUKCA2, assigned the variables that were on UPUKCA to UPUKCA2 and reran all the STASH macros. I figured if I changed the name like I did for the time profile that would fix it, but no luck :/

It looks to be the same problem, as the UPUKCA2 usage profile doesn't appear in the STASHC file (though ALLTS, what I renamed TALLTS to, now does…)

????????????????????????????????????????????????????????????????????????????????
???!!!???!!!???!!!???!!!???!!!       ERROR        ???!!!???!!!???!!!???!!!???!!!
?  Error code: 18
?  Error from routine: rdbasis
?  Error message: UNRECOGNISED usage profile request UPUKCA2
?  Error from processor: 0
?  Error number: 5
????????????????????????????????????????????????????????????????????????????????

This warning was also present in stderr, not sure if it's relevant, I just hadn't seen it/noticed it before.

????????????????????????????????????????????????????????????????????????????????
??????????????????????????????      WARNING       ??????????????????????????????
?  Warning code: -10
?  Warning from routine: check_configid
?  Warning message:
?          itab is set to missing data, pp_head(LBEXP) will be set to zero
?  Warning from processor: 0
?  Warning number: 0
????????????????????????????????????????????????????????????????????????????????

comment:7 Changed 3 years ago by mattjbr123

Hi Willie,

Any luck with this? Can't say I've had any my end…

Matt

comment:8 Changed 3 years ago by grenville

Matt

We are still looking at this — it's proving to be quite tricky

Grenville

comment:9 Changed 3 years ago by grenville

Matt

Where did this suite come from? Did it ever run?

Grenville

comment:10 Changed 3 years ago by mattjbr123

Hi Grenville,

Yes, the suite ran fine previous to me changing the time profiles. It came from suite u-ad273@trunk/11216 (martinandrews) which itself came from u-ad131/trunk@10853 (jeremywalton).

Many thanks!

Matt

comment:11 Changed 3 years ago by grenville

MAtt

You say "so I cloned some of these variables/STASH requests and changed the usage profile to one that outputs to a fieldsfile."

what exactly have you added to the stash? What are you trying to do?

Grenville

comment:12 Changed 3 years ago by mattjbr123

There were some diagnostics that were being output to 'secondary store with user supplied tag' (98), which I wanted to actually view. I couldn't work out where this option was sending the diagnostics so I:

1) Changed the usage profile associated with these diagnostics to pp4 (I think), so that they would be output to a pp file rather than seemingly disappearing into the unknown. At least some of these diagnostics had a time profile of TALLTS.
2) This brought up the error, so I backtracked, reinstated the original usage profile for these diagnostics, and instead cloned them and changed the usage profile of the cloned variables. At this point there are 2 copies of each variable in STASH, one going to 'secondary store', the other to pp4 (or whatever pp file it was).
3) The error still came up and at this point I had a good look in the documentation to try and fathom what STASH was doing and I realised that TALLTS, the time profile associated with the diagnostics, was not being written to the STASHC file which is where rdbasis was reading from, but the diagnostics requesting TALLTS as their time profile, were, causing it to fail as it could not find TALLTS. I then removed the diagnostics that were outputting to a fieldsfile so that we were back to one copy of each diagnostic going to the 'secondary store'.
4) Next, I removed TALLTS completely, created a new one with name ALLTS that was identical to it and assigned all the diagnostics that were on TALLTS to ALLTS. This fixed the unrecognised time profile error but brought up the same error with the usage profile associated with most of these diagnostics - UPUKCA.
5) I repeated step 4) for the usage profile, creating one identical to UPUKCA called UPUKCA2, but this time it did not fix the problem.

So in summary, I have not added any new diagnostics to STASH, I have just changed the usage profile of some of the diagnostics, which causes an upset because the time profile associated with these is not written to the STASHC file when it presumably should be.

The diagnostics in question, the ones I was originally after, were diagnostics particular to the nudging part of the UM - U COMPNT OF WIND AFTER NUDGING (and V COMPNT OF WINF AFTER NUDGING), section 39 items 7 and 12 respectively. Perhaps this has something to do with the problem? Maybe these variables are not designed to be output to fieldsfiles or something…

Martin Andrews at the Met Office said the following about the suite which may be useful:
The original job was a GA7.0 N216 UM10.3 AMIP Gregorian Climate Assessment (1981 – 2008). I made following changes:

  • Added nudging
  • Corrected stash specification and added output nudging diagnostics to apa.pp
  • Included n216 settings
  • Transferred to MONSooN

The particular nudging diagnostics I wanted were not being output to apa.pp, I was after ones that were output every timestep and the ones in apa.pp were not (and were less useful anyway).

Hope some of that helps/is useful.

Matt

Last edited 3 years ago by mattjbr123 (previous) (diff)

comment:13 Changed 3 years ago by luke

Hi Matt,

I can help with what Tag98 is for - this is the way UKCA copies diagnostics into the model. Rather than passing them through a subroutine call or via a module, instead they are written to a Tagged STASH stream, which UKCA then reads from every timestep. This is why they would be TALLTS, as they are needed every timestep when UKCA is called. If you change any of these the model will crash, most likely in a UKCA routine, which you have already encountered.

As for why the job was failing otherwise, have you validated your STASH changes after you made them? In the UMUI you could just add entries, but in Rose you need to run the TidyStashTransform macro, otherwise you will probably have issues.

Before you do this, I would suggest that you save all changes in Rose and also run a fcm commit in the suite directory, so that you can backtrack if needed. Indeed, this is good advice when making changes to suites generally.

I hope this helps,
Luke

comment:14 Changed 3 years ago by mattjbr123

Great, thanks for the info Luke. Presumably changing the name of the usage profile from 'UPUKCA' to 'UPUKCA2' shouldn't upset it (provided I run the STASH macros, which I have)?

Some other things I have learnt:

  • Any diagnostics with a usage profile outputting to a secondary store do not need the time profiles associated with them to be written in the STASHC namelist
  • When you clone a diagnostic with the above usage profile and change it to a pp file usage profile, if its required time profile is not used with any other diagnostics that have pp file usage profiles, then the time profile will not be output to STASHC file and the model will fail
  • A way to get it to write the required time profile to the STASHC file in such a situation is to, seemingly, create a new time profile that is identical, save for the name, and assign the diagnostics to this one instead, deleting the old time profile. This seems to work regardless of whether the diagnostics with this time profile are going to a pp file or secondary store.

Things that I'll try next:

  • Now try and run with a diagnostic with time profile ALLTS and usage profile outputting to a pp file (at the moment they are all going to the secondary store)
  • See if I can get it to write the usage profile to the STASH file, as that's where it's failing at the moment…
  • Firstly by changing the usage profile name in the same way as the time profile

Before I do this I will commit the current version of the suite so that you can maybe download it and reproduce the errors/mess around with it yourself.

comment:15 Changed 3 years ago by mattjbr123

Also here's a link to the notes I made when I first had the problem. Maybe there's something useful in there… https://www.dropbox.com/s/iwgq33kg578szel/crashnotes.docx?dl=0

comment:16 follow-up: Changed 3 years ago by mattjbr123

One other, hopefully simple, question whilst I remember:

Is there any easy way to change the usage/time/domain profiles of multiple diagnostics at once on the STASH Requests page in Rose? I can't seem to find a way…

Thanks once again!
Matt

comment:17 in reply to: ↑ 16 Changed 3 years ago by ros

See also #1971

Replying to mattjbr123:

One other, hopefully simple, question whilst I remember:

Is there any easy way to change the usage/time/domain profiles of multiple diagnostics at once on the STASH Requests page in Rose? I can't seem to find a way…

Thanks once again!
Matt

comment:18 Changed 3 years ago by ros

  • Resolution set to completed
  • Status changed from new to closed

comment:19 Changed 2 years ago by mattjbr123

I have worked out what caused this.

If you create a copy of the UPUKCA usage profile and assign some/all STASH requests to it, but DON'T run the rose transform macro it doesn't have an issue, but when you do run the macro, then it complains about the 'unknown usage profile'. The same seems to apply for the time/domain profiles.
I'm not sure whether this bug(?) is specific to my suite or more general - but I don't think it's the correct behaviour. It can be reproduced (hopefully) by checking out suite u-af404 at revision 28090 and following the above steps.

Note: See TracTickets for help on using tickets.