wiki:Archer/Transition2020/PPTransfer

Version 19 (modified by ros, 4 months ago) (diff)

Configuring pptransfer app and UM suite to pull data on /work from JASMIN

The pptransfer task used to run on the ARCHER Data Transfer Node (dtn02) which we no longer have access to from February 19th. The instructions below modify a suite to run the pptransfer task on the JASMIN transfer nodes and pull the data across from ARCHER /work disk.

Note: Instructions cannot cover all possible suite setup combinations so you may need to adjust them accordingly. For example tasks may be named slightly differently or inherit differently.

Suite Changes

In the rose suite editor go to "postproc → Post processing - common settings":

  • In panel "Archer Archiving" change archive_root_path to be a directory in your $DATADIR on /work disk (e.g. /work/n02/n02/<username>/archive where <username> is your ARCHER username). This will be a temporary area to stage your data before transfer to JASMIN.
  • In panel "JASMIN Transfer" change:
    • transfer_type to Pull
    • remote_host to login.archer.ac.uk
    • Set gridftp to false

In the ~/roses/<SUITEID>/site/archer.rc file edit the section [[PPTRANSFER_RESOURCE]]:

  • Remove the line pre-script = "module load anaconda". (Note this may appear in other pptransfer related sections)
  • Replace the line host = dtn02.rdf.ac.uk with host = jasmin-xfer2.ceda.ac.uk

Setup required on PUMA and JASMIN

You now need to setup ssh-agent to be able to login to jasmin-xfer2.ceda.ac.uk and from there to login.archer.ac.uk.

PUMA:

These instructions assume you already have an ssh-agent running on PUMA which you would already have needed in order to run a UM suite on ARCHER.

  • Log on to PUMA.
  • Make sure you have a copy of your JASMIN ssh-key in your ~/.ssh directory. This is the key you generated when you applied for access to JASMIN. The remainder of the instructions assume it is called id_dsa_jasmin please replace this with the name of your key as appropriate.
  • Setup ~/.ssh/config file to login to the jasmin-xfer node. For example:
    Host jasmin-xfer2.ceda.ac.uk
    Hostname jasmin-xfer2.ceda.ac.uk
    User <jasmin-username>
    IdentityFile ~/.ssh/id_dsa_jasmin
    ForwardAgent no
    

Replace <jasmin-username> with your JASMIN username.

  • Run ssh-add ~/.ssh/id_dsa_jasmin to attach the key to your existing ssh-agent.

Check that you can now ssh to jasmin-xfer2.ceda.ac.uk without prompt for password/passphrase by running:

ssh jasmin-xfer2.ceda.ac.uk

JASMIN:

  1. Log on to JASMIN: ssh jasmin-xfer2.ceda.ac.uk
  1. Setup PATH to Rose, Cylc and FCM:
  • In your ~/.bash_profile add the following:
    if [[ `hostname` = "jasmin-cylc.ceda.ac.uk" ]]; then
      # Rose/cylc on jasmin-cylc
      export PATH=/apps/contrib/metomi/bin:$PATH
    else
      # Rose/cylc on jasmin-xfer
      export PATH=/home/users/rshatcher/software/bin:$PATH 
    fi
    
  1. Setup ssh-agent to access ARCHER Login Nodes:
  • Copy the private key you use on PUMA to access ARCHER over to JASMIN. This will be something like ~/.ssh/id_dsa.
    scp <puma-username>@puma.nerc.ac.uk:~/.ssh/id_dsa ~/.ssh
    
  • Copy the ssh-agent start up script:
    cp ~rshatcher/setup ~/.ssh
    
  • In your ~/.bashrc add the following to start up ssh-agent:
    if [[ `hostname` = "jasmin-xfer2.ceda.ac.uk" ]] ; then
      # ssh-agent setup for transfer app
      . ~/.ssh/setup
    fi
    
  • In your ~/.ssh/config add the following lines:
    Host login.archer.ac.uk
      Hostname login.archer.ac.uk
      User <archer-username>
    

Replacing <archer-username> with your ARCHER username.

  • Log out of jasmin-xfer2.ceda.ac.uk and back in again.
  • Run ssh-add. You will be prompted for your passphrase.

Check that you can now ssh to ARCHER without prompt for password/passphrase by running:

ssh login.archer.ac.uk

Updating a Running Suite

  1. Reload the suite: rose suite-run --reload
  2. Hold the whole suite, or just the next pptransfer task
  3. In the Cylc GUI: Control —> Insert Task(s)…
  4. Set TASK-NAME.CYCLE-POINT=fcm_make_pptransfer.<YYYYMMDDT0000Z>, where <YYYYMMDDT0000Z> is an active cycle point
  5. Leave stop-point=POINT blank
  6. Check the "Do not check if a cycle point is valid or not" box
  7. Insert, and wait for the task to complete.
  8. If nothing happens: You probably typed something incorrectly! Try again.
  9. In the Cylc GUI: Control —> Insert Task(s)…
  10. Set TASK-NAME.CYCLE-POINT=fcm_make2_pptransfer<YYYYMMDDT0000Z>, where <YYYYMMDDT0000Z> is an active cycle point
  11. Leave stop-point=POINT blank
  12. Check the "Do not check if a cycle point is valid or not" box
  13. Insert, and wait for the task to complete.
  14. Release the held suite/pptransfer task