Opened 8 weeks ago

Closed 5 weeks ago

#3477 closed help (fixed)

meta data for JULES

Reported by: katieblackford Owned by: jules_support
Component: JULES Keywords: metadata, JULES, JASMIN
Cc: Platform: JASMIN
UM Version:

Description

Hi Patrick and Carolina,

Thank you for offering to help me with this.

At the moment I am working off a copy of the u-bx723 suite (u-ca315), which runs successfully, but I am trying to upgrade this suite to version 5.4 of JULES. However, when I try to upgrade the suite using rose app-upgrade I get an error saying, "could not find meta flag". The meta is referred to in the app/jules/rose-app.conf file, and points to the meta data on the group workspace, so I am unsure as to why this isn't working. Also, when I open the suite with rose edit, I get the pop up saying that it can't find the metadata. I have tried following the guidance in step 2 here ​https://jules.jchmr.org/content/modify-suite, but I cannot find the appropriate place to add in a meta entry in the app/fcm_make.

I have also been trying to run a copy of u-bv321. I was having problems with an old copy of this suite (u-cb141) where I get an error relating to the namelist files during the fcm-make stage. I have attached the job.err file I get.
The original suite has been updated since, and I am currently seeing if I can get this updated version to run (u-cc141). And I will let you know if I need any help with this.

Thanks again for your help.

Best,
Katie

Change History (25)

comment:1 Changed 8 weeks ago by pmcguire

Hi Katie
What is the path in your suite to the meta data?
The meta data path was wrong in the u-bx723 suite until a couple of weeks ago, when I fixed it.

This is the path in my copy of that suite:
grep -r meta ~pmcguire/roses/u-bx723/

/home/users/pmcguire/roses/u-bx723/app/jules/rose-app.conf:meta=/gws/nopw/j04/jules/admin/JULES/vn5.3/trunk/rose-meta/jules-standalone/vn5.3

Until a couple of weeks ago, this was pointing to an old non-existent path with the prefix: /group_workspaces/jasmin4/jules/

Patrick

comment:2 Changed 8 weeks ago by pmcguire

Hi Patrick,

The meta path is similar to the old path:

meta=/group_workspaces/jasmin2/jules/admin/JULES/vn5.3/trunk/rose-meta/jules-standalone/vn5.3

I shall try changing it to match what you are using.

Thanks,
Katie

comment:3 Changed 8 weeks ago by pmcguire

Hi Katie:
Hopefully that helps!
Let me know.
Patrick

comment:4 Changed 8 weeks ago by pmcguire

Hi Katie
Did it work after you made that change?
Patrick

comment:5 Changed 8 weeks ago by pmcguire

Hi Patrick,

I have made that change, and I no longer get the error pop up when I use rose edit. And I can now see what upgrades I can do to the suite using:
rose app-upgrade -M $JULES_ROOT/rose-meta -C ~/roses/u-ca315/app/jules --all-versions
However, when I do this for /app/fcm_make I still get the error saying, "could not find meta flag". Also, I am wanting to upgrade to version 5.4, and this isn't listed as an option with the above call. Is it ok to just try and upgrade to this version anyway? Using rose app-upgrade -M $JULES_ROOT/rose-meta -C /path/to/rose/suite/app/jules <version> && rose macro --fix -C app/jules. And is it ok to just run this for app/jules? Or is there a way to solves the issue with not being able to find the meta in app/fcm_make to be able to run both the commands suggested here ​http://jules-lsm.github.io/latest/building-and-running/rose.html#upgrading-an-existing-jules-rose-suite

Thanks,
Katie

comment:6 Changed 8 weeks ago by pmcguire

Hi Katie:
Are you doing this on the cylc1.jasmin computer or on a sci*.jasmin machine?
The former is needed, I think.
Patrick

comment:7 Changed 8 weeks ago by pmcguire

Hi Patrick,

I am running it on cylc1.jasmin

Katie

Last edited 8 weeks ago by pmcguire (previous) (diff)

comment:8 Changed 8 weeks ago by pmcguire

Hi Katie
OK. Thanks. I will try to do that soon.
Do you have $JULES_ROOT defined?
What is its value?
you can see this by using the command:
echo $JULES_ROOT
Patrick

comment:9 Changed 8 weeks ago by pmcguire

Hi Patrick,

Thank you.
When I run echo $JULES_ROOT, I get no output. I'm assuming this means it's not defined, does this need to be the case? I have been running this suite recently with no problems.

Thanks,
Katie

comment:10 Changed 8 weeks ago by pmcguire

Hi Katie:
I figured out that I need to change the u-bx723/app/jules/rose-app.conf file so that it has meta=jules-standalone/vn5.3 instead of meta=/gws/nopw/j04/jules/admin/JULES/vn5.3/trunk/rose-meta/jules-standalone/vn5.3.

And since neither of us have $JULES_ROOT defined, you can check for the versions for the upgrade with this command:

rose app-upgrade -M /gws/nopw/j04/jules/admin/JULES/vn5.4/trunk/rose-meta -C ~/roses/u-ca315/app/jules —all-versions

Does this help?
Patrick

comment:11 Changed 8 weeks ago by katieblackford

Thanks Patrick,

I have made this change and the command now works and I can view the versions I can upgrade to. How do I go about correctly upgrading this suite to vn5.4 now?

Thanks,
Katie

comment:12 Changed 8 weeks ago by pmcguire

Hi Katie:
You could follow further along in that tutorial. The next step is something like:

rose app-upgrade -M /gws/nopw/j04/jules/admin/JULES/vn5.4/trunk/rose-meta -C ~/roses/u-ca315/app/jules vn5.4

How does that work?
Patrick

comment:13 Changed 8 weeks ago by katieblackford

Hi Patrick,

I ran the upgrade using:

rose app-upgrade -M /gws/nopw/j04/jules/admin/JULES/vn5.4/trunk/rose-meta -C ~/roses/u-ca315/app/jules vn5.4

And this appears to have worked, is there a way to check this and is this all I need to have done to upgrade the suite? Or do I also need to run this but for app/fmc_make as some of the tutorials suggest?
I also haven't run:

rose macro --fix -C $RSUITE/app/jules

Do I need to do this as well?

comment:14 Changed 8 weeks ago by pmcguire

Hi Katie:
I am glad the app-upgrade worked.
You should probably follow the rest of the tutorial, including doing the macro --fix.

After doing the upgrade, you might need to change the relative path for the meta data in your u-bx723/app/jules/rose-app.conf file to the absolute path:
meta=jules-standalone/vn5.4
to:
meta=/gws/nopw/j04/jules/admin/JULES/vn5.4/trunk/rose-meta/jules-standalone/vn5.4
Otherwise, you might not be able to properly open up the modified suite with rose edit.

This is what I see when I compare my old u-bx723new1 suite which was 5.3 to my new u-bx723 suite which is 5.4:

diff ~pmcguire/roses/u-bx723new1/app/jules/rose-app.conf ~pmcguire/roses/u-bx723/rose-app.conf

1c1,5

< meta=/gws/nopw/j04/jules/admin/JULES/vn5.3/trunk/rose-meta/jules-standalone/vn5.3

—-

#meta=/gws/nopw/j04/jules/admin/JULES/vn5.3/trunk/rose-meta/jules-standalone/vn5.3

#when doing "rose app-upgrade -M /gws/nopw/j04/jules/admin/JULES/vn5.4/trunk/rose-meta -C ~/roses/u-ca315/app/jules β€”all-versions",

#then switch the metadata to:

meta=jules-standalone/vn5.4

74a79,81

[file:science_fixes.nml]

source=namelist:jules_temp_fixes

369a377

!!l_riv_overbank=.false.

438a447

!!fire_mort_io=5*1.0

442a452

!!g1_stomata_io=5*2.0

660c670,671

< !!l_riv_overbank=.false.

—-

!!i_river_vn=2

!!l_inland=.false.

661a673

!!nstep_rivers=-32768

666,667d677

< !!rivers_timestep=-32768

< !!rivers_type='rfm'

668a679

!!slfac=0.0

859a871,880

[namelist:jules_temp_fixes]

l_dtcanfix=.true.

l_fix_alb_ice_thick=.true.

l_fix_albsnow_ts=.true.

l_fix_ctile_orog=.true.

l_fix_moruses_roof_rad_coupling=.false.

l_fix_osa_chloro=.false.

l_fix_ustar_dust=.true.

l_fix_wind_snow=.true.

869c890

< const_val=1.0,10.0,1.0,

—-

const_val=1.0,10.0,1.0

943a965

stomata_model=1

You might want to do something similar with your changed suite to see if the app-upgrade made the same changes that it did for mine.

You can also make sure that when you open it in rose edit that no errors are reported.
Some of the changed flags during the upgrade need to be inspected, to make sure that they are what you want.

Upgrading a suite is sometimes not easy. And it might be best to run the 5.3 version of the suite and the 5.4 version of the suite under otherwise identical setups, to see if the changes made in the upgrade produce reasonable results.
Patrick

comment:15 Changed 8 weeks ago by katieblackford

Hi Patrick,

At present my suite is showing the same changes as yours. I shall continue with the remaining steps of the tutorial and see if this works. I'll let you know if I have any problems with this.

Thanks,
Katie

comment:16 Changed 7 weeks ago by katieblackford

Hi Patrick,

I have tried continuing updating the suite, and the macro —fix works successfully. However, when I try to do the app upgrade for fcm_make (rose app-upgrade -M /gws/nopw/j04/jules/admin/JULES/vn5.4/trunk/rose-meta -C ~/roses/u-ca315/app/fcm_make vn5.4) I get an error saying "cannot find meta flag". Any ideas on how to fix this?
The update applied to app/jules appears to have worked successfully, so I am unsure why this isn't now working for fcm_make.

Thanks,
Katie

comment:17 Changed 7 weeks ago by pmcguire

Hi Katie:
I don't think the fcm_make app needs to be updated for this suite from 5.3 to 5.4.
But I will check on that.
Patrick

comment:18 Changed 7 weeks ago by pmcguire

Hi Katie:
These are the meta data changes from 5.3 to 5.4 for the fcm_make app, as found in the file:
/gws/nopw/j04/jules/admin/JULES/vn5.4/trunk/rose-meta/jules-fcm-make/versions.py

I don't see anything immediately that needs to be changed for the fcm_make app, especially for JASMIN, to go from JULES 5.3 to JULES 5.4. And even so, I am not completely sure that the rose app-upgrade was used to for the fcm_make app to get it up to date previously to JULES 5.3.

So, I wouldn't worry right now about updating the fcm_make app from 5.3 to 5.4. Does everything work otherwise?
Patrick

class vn53_t854(rose.upgrade.MacroUpgrade):
    """Upgrade macro for #854 by Paul Cresswell"""

    BEFORE_TAG = "vn5.3"
    AFTER_TAG = "vn5.3_t854"

    def upgrade(self, config, meta_config=None):
        # Change the default meto-xc40 build to a remote extract
        # (i.e. 'local' to that platform)
        platform = self.get_setting_value(config, ["env", "JULES_PLATFORM"])
        if platform is not None:
            if platform == 'meto-xc40-cce':
                self.change_setting_value(config, ["env", "JULES_REMOTE"], "local")

        return config, self.reports

class vn53_t855(rose.upgrade.MacroUpgrade):
    """Upgrade macro for #855 by Paul Cresswell"""

    BEFORE_TAG = "vn5.3_t854"
    AFTER_TAG = "vn5.3_t855"

    def upgrade(self, config, meta_config=None):
        # (Re-)Add compulsory=true variables to apps that don't use them
        self.add_setting(config, ['env', 'JULES_BUILD'], '$JULES_BUILD')
        self.add_setting(config, ['env', 'JULES_OMP'], '$JULES_OMP')
        self.add_setting(config, ['env', 'JULES_PLATFORM'], '$JULES_PLATFORM')
        return config, self.reports

class vn53_vn54(rose.upgrade.MacroUpgrade):
    """Version bump macro"""

    BEFORE_TAG = "vn5.3_t855"
    AFTER_TAG = "vn5.4"

    def upgrade(self, config, meta_config=None):
        # Nothing to do
        return config, self.reports

comment:19 Changed 7 weeks ago by katieblackford

Hi Patrick.

Thanks for this. Everything else has appeared to have worked successfully. Thanks for your help getting the suite updated.

Katie

comment:20 Changed 7 weeks ago by katieblackford

Hi Patrick,

I have tried running the updated suite but I have been getting the following error:
"{MPI Task 1} [FATAL ERROR] init_vegetation: Error reading namelist JULES_VEGETATION (IOSTAT=19 IOMSG=invalid reference to variable in NAMELIST input, unit 1, file /work/scratch-pw/katie_b1/cylc-run/u-ca315/work/18600101T0000Z/RECON/./jules_vegetation.nml, line 27, position 14)"

From looking at the file referenced above, I think this may be due to the stomata_model switch in the jules_vegetation namelist. I have tried looking at the app/jules/rose-app.conf file, but nothing in there appears to be wrong with how this switch is set up. Also nothing in the documentation suggests to me that anything else needs to be turned on to get this to work.
The stomata_model was added in when I upgraded from vn5.3 to vn5.4, so perhaps something went wrong or was missed out when trying to do this upgrade?

Thanks again,
Katie

Last edited 7 weeks ago by katieblackford (previous) (diff)

comment:21 Changed 7 weeks ago by pmcguire

Hi Katie:
Maybe your JULES source code version is not right?
What does it say in your ~/roses/rose-suite.conf file?

Does it say this?:

JULES_FCM='fcm:jules.x_tr'
JULES_REVISION='13249'

If it says that, then I think that is JULES 5.3.

If you want to update it to JULES 5.4, you might try:

JULES_FCM='fcm:jules.x_tr'
JULES_REVISION='vn5.4'

I think that might work.

If that doesn't work then you can find the all-numeric MOSRS JULES_REVISION number (i.e., for vn5.3, it was 13249), and use that number instead of JULES_REVISION='vn5.4'.
Patrick

comment:22 Changed 6 weeks ago by pmcguire

Hi Katie:
Is it working better now?
Patrick

comment:23 Changed 5 weeks ago by katieblackford

Hi Patrick,
Yes, it is all working now. Thanks for your help.
Katie

comment:24 Changed 5 weeks ago by pmcguire

Excellent! That's great to hear, Katie!
I will close the ticket now.
If you need more help in the future, just create a new ticket!
Patrick

comment:25 Changed 5 weeks ago by pmcguire

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.