wiki:Docs/PostProcessingAppNexcsSetup

Version 5 (modified by ros, 7 months ago) (diff)

NEXCS specific setup instructions for data transfer to JASMIN

Suite modifications

  1. Edit rose-suite-conf:

Add PPTRANSFER=true

  1. The PPTRANSFER variable will, by default, appear under "suite conf → jinja2". To tell Rose to place it with all the other suite control switches edit meta/rose-meta.conf to add in the metadata for the PPTRANSFER variable. (This step is optional.)
[jinja2:suite.rc=PPTRANSFER]
compulsory=true
description=Transfer files archived with PostProc to a remote machine
help=
ns=<namespace>
sort-key=runPostproc1
title=PP Transfer
type=boolean

Where <namespace> is the name of the panel in which you want the variable listed. For example if want it listed under panel "suite conf → Build and Run" replace <namespace> with Build and Run.

  1. Add the pptransfer task into the cylc graph. Edit suite.rc file:

[To Do: Add in/change to cater for suites with the "SITE" setup]

Add the pptransfer task into the graph for all subsequent cycles such that it runs after the postproc task and also waits for the previous pptransfer task to complete. As an example for a coupled suite (All added lines indicated with "⇐"):

        [[[ {{FMT}} ]]]
            graph = """
{% if RUN %}
coupled[-{{FMT}}] => coupled {{ '=> \\' if POSTPROC or HOUSEKEEP else '' }}
  {% if POSTPROC %}
postproc {{ '=> \\' if PPTRANSFER or HOUSEKEEP else '' }}     <= "PPTRANSFER or" added here
  {% endif %}
  {% if PPTRANSFER %}                                         <=
pptransfer {{ '=> \\' if HOUSEKEEP else '' }}                 <=
  {% endif %}                                                 <=
  {% if HOUSEKEEP %}
housekeeping
  {% endif %}
  {% if POSTPROC %}
postproc[-{{FMT}}] => postproc
  {% endif %}
  {% if PPTRANSFER %}                                         <=
pptransfer[-{{FMT}}] => pptransfer                            <=
  {% endif %}                                                 <=
{% endif %}
"""
  1. Finally at the end of the suite.rc file add the pptransfer task definitions:
  • Rename the existing [[postproc]] section [[POSTPROC]]
  • Add:
        [[postproc]]
            inherit = None, POSTPROC
    
        [[pptransfer]]
            inherit = None, POSTPROC
            [[[environment]]]
    	    ROSE_TASK_APP = postproc
    

Set up ssh-key to connect to JASMIN

Some setup is required to enable non-interactive authentication from NEXCS to JASMIN. Please contain cms_support@… for details.