Opened 7 months ago

Closed 7 months ago

#2690 closed help (fixed)

XCS-C Jasmin Transfer

Reported by: m.couldrey Owned by: ros
Component: Other Keywords: JASMIN transfer, ssh keys
Cc: Platform: NEXCS
UM Version: 10.7

Description

Hi CMS

I’m setting up a suite (u-az004) to transfer output to Jasmin after running. I was following these helpful instructions:
http://cms.ncas.ac.uk/wiki/Docs/PostProcessingAppNexcsSetup
and saw at the bottom that there are additional steps to configure non-interactive login between NEXCS and Jasmin. What would those steps be?

As an aside, I had a problem with setting up the cylc graph when I followed those instructions. When I make the suggested edits to the cylc graph section of the suite.rc and run the suite in simulation mode, it fails at the following error.

[FAIL] cylc validate -v -o /var/tmp/tmpRPt2U4 —strict u-az004 # return-code=1, stderr=
[FAIL] 2018-12-05T10:02:39Z ERROR - POSTPROC:succeed-all ⇒ pptransfer
[FAIL] 'ERROR, self-edge detected: pptransfer:succeed ⇒ pptransfer'

To work around it for now, I’ve ignored the edits to the cylc graph, and that seems to allow the suite to run successfully in simulation mode. How can I fix this so the graph includes the transfer process?

Many thanks!
Matt

Change History (6)

comment:1 Changed 7 months ago by ros

  • Component changed from UM Model to Other
  • Owner changed from um_support to ros
  • Status changed from new to accepted

Hi Matt,

I'll send you the steps to sort configure the non-interactive login by email in a mo.

It's difficult to say what the problem with the pptransfer in the graph was without being able to see what you'd done. Please try adding the lines to the graph again and let me know when you've done and if you still get the above error message. One thing to note, make sure there is no extra whitespace at the end of the lines you add as this can cause cylc errors with the graph.

Cheers,
Ros.

comment:2 Changed 7 months ago by m.couldrey

Hi Ros

Thanks for the help! I've had a look for whitespace but still get the same problem. I've added back in the lines as suggested in Section 3 of http://cms.ncas.ac.uk/wiki/Docs/PostProcessingAppNexcsSetup

I also wasn't sure exactly where to add in the final task definition lines as per Section 4. For the moment I've put them right at the very end in the {% if POSTPROC %} par. But I was wondering if the lines should instead go just above line 211 {# * TASKS IN THE ORDER THEY ARE RUN ….

Cheers!
Matt

comment:3 Changed 7 months ago by ros

Hi Matt,

That was a bit of an obscure one. I've taken a copy of your suite.rc file and fixed it up for you. Please copy ~rhatcher/roses/u-az004/suite.rc and give it another go.

The problem is due to the way the POSTPROC tasks have been grouped. pptransfer inherits [[POSTPROC]] settings and thus was binned as a POSTPROC task. The graph was then effectively saying "Run all POSTPROC tasks including pptransfer and then run pptransfer" which is impossible.

I'll update the instructions.

Cheers,
Ros.

comment:4 Changed 7 months ago by ros

Note for CMS:

The fix for this is to create a new family to group only the postproc_ tasks together.

  • create new family:

[[POSTPROC_GROUP]]

  • Add POSTPROC_GROUP to the inherit line for all the postproc_ tasks.

E.g.

    [[postproc_atmos]]
        inherit = None, POSTPROC_GROUP, POSTPROC
  • Edit the graph to change the POSTPROC task to be POSTPROC_GROUP.
    Caveat this DOES NOT mean replace all occurences of POSTPROC with POSTPROC_GROUP. Leave "if POSTPROC" occurences as is - this is the variable POSTPROC not a task.

E.g.

coupled[-{{FMT}}] => coupled {{ '=> \\' if POSTPROC or HOUSEKEEP else '' }}
  {% if POSTPROC %}
POSTPROC{{ ':succeed-all => \\' if PPTRANSFER or HOUSEKEEP else '' }}
  {% endif %}

would become:

coupled[-{{FMT}}] => coupled {{ '=> \\' if POSTPROC or HOUSEKEEP else '' }}
  {% if POSTPROC %}
POSTPROC_GROUP{{ ':succeed-all => \\' if PPTRANSFER or HOUSEKEEP else '' }}
  {% endif %}

comment:5 Changed 7 months ago by m.couldrey

Ah, I see! Thanks very much for figuring that one out, I wouldn't have spotted that! The suite now runs in simulation mode for me, so this seems like the right fix. I'm currently waiting for it to run for real, so hopefully that'll go through too.
Cheers!
Matt

comment:6 Changed 7 months ago by ros

  • Resolution set to fixed
  • Status changed from accepted to closed

I'm assuming this is all working now. You can reopen the ticket if required.

Regards,
Ros.

Note: See TracTickets for help on using tickets.