Opened 4 months ago

Closed 3 months ago

#3417 closed help (answered)

Cylc Remote Access

Reported by: luciana Owned by: ros
Component: Rose/Cylc Keywords: Cylc Remote Access
Cc: Platform: JASMIN
UM Version:

Description

Hello.

I went a step back and tried to run the example for remote access in tut/oneoff/remote.

—-
host = server1.niwa.co.nz ⇒ It doesn't work for me.
—-

Which remote server I have access to test it? I've also tried puma, without success.

Kind regards.

Luciana.

Change History (15)

comment:1 Changed 4 months ago by ros

Hi Luciana,

server1.niwa.co.nz is just there as an example for you change as required.

If you set up your ssh-agent so you can login to JASMIN from PUMA without prompt for passphrase you can run the example on PUMA submitting to JASMIN.

Regards,
Ros.

comment:2 Changed 4 months ago by luciana

Hi Rosalyn.

I just had this idea; it didn't work as well. I tried with login1.jasmin.ac.uk and login.jasmin.ac.uk. And I do have access from Puma to Jasmin without a password. However, even if that had worked, it wouldn't solve my original problem because I need to run from Jasmin (Puma doesn't have Cylc broadcast -F FILE).

COMMAND FAILED (255): ssh -oBatchMode=yes -oConnectTimeout=10 -n login1.jasmin.ac.uk mkdir -p '$HOME/cylc-run/test-remote-jasmin' '$HOME/cylc-run/test-remote-jasmin/log/job'

Which server am I allowed to access from Jasmin? I'm just looking something for simple tests, not a big task to run elsewhere.

Kind regards.

Luciana.

PS. When I have multiple questions, do you prefer that I combine them in just one thread or open one ticket per topic?

comment:3 Changed 4 months ago by luciana

Hi Rosalyn.

Still about this problem. I have access to Archer from my computer, but I cannot access it from Puma or Jasmin. Archer is the expected candidate to execute the work after I'm familiar with the systems. Maybe we can make this big step now, but I have no idea of how or where to start. Can you point me in the right direction?

Kind regards.

Luciana.

comment:4 Changed 4 months ago by ros

  • Owner changed from um_support to ros
  • Status changed from new to accepted

Hi Luciana,

Submission of cylc suites to ARCHER needs to be done from "PUMA", this is the platform all our users use to do this. We have the later cylc-7.8.x available on our pumatest server. Pumatest suffered a disk issue yesterday and we are currently waiting confirmation that this is resolved. Once pumatest is back we will give you an account on there.

There is, however, also a slight issue with cylc suite submission to ARCHER which I'm hoping ARCHER will fix soon. You will need to wait until that is fixed in order to proceed further.

Regards,
Ros.

CMS Note: See also #3416

comment:5 Changed 4 months ago by luciana

Hi Ros.

Thank you for your contact. I'll wait for access to pumatest.

Can you tell me why the test from puma to jasmin failed? My login is different in puma (luciana) and jasmin (lucy). Would that be a problem?

Kind regards.

Luciana.

comment:6 Changed 4 months ago by ros

Hi Luciana,

If you are talking about suites test-remote-jasmin and test-remote-jasmin-2, they both failed to ssh to jasmin and archer respectively. You can see the error messages in the log/suite/err file for both suites.

In answer to your question about different usernames on the 2 machines, that is potentially a problem yes. You either need to specify your username for each remote host in the suite.rc (owner = <username> from memory), or specify it in your ~/.ssh/config file

Regards,
Ros.

comment:7 Changed 4 months ago by luciana

Hi Ros.

Those are the tests I've tried, several times, in fact. They all give me the same error message that I copied here before: unauthorised access.

I've tried a different suite now, from cylc-examples/remote/minimal:

—-
[ scheduling ]

[ [ dependencies ] ]

graph = "foo ⇒ bar"

[ runtime ]

[ [ root ] ]

script = """

echo HELLO from $CYLC_TASK_NAME on $(hostname)
sleep 10
echo GOODBYE from $CYLC_TASK_NAME on $(hostname)"""

[ [ [ remote ] ] ]

# the remote host name:
host = login1.jasmin.ac.uk
owner = lucy

—-

Again, I do have access from puma to jasmin without a passphrase, but the suite doesn't work and I don't know what else to do to make it work. :/

I also run again:

—-
eval $(ssh-agent -s)
ssh-add ~/.ssh/id_rsa_jasmin
—-

but it didn't make any difference. Still unauthorised access.

Kind regards.

Luciana.

comment:8 Changed 4 months ago by ros

Hi Luciana,

You need to get your ssh-agent working properly first.

It looks like you already have the correct setup in your ~/.profile.

Please remove the file ~/.ssh/environment.puma. And then log out and back into PUMA. You should see the message: "Initialising new SSH agent…"

[If you don't get that message use ps -flu luciana| grep ssh-agent to list any already running ssh-agent processes and use kill <PID> to kill them. Then try logging out of PUMA and back in again.]

Edit your ~/.ssh/config file to update the jasmin login node settings following the name change recently:

Host login1.jasmin.ac.uk
User lucy
IdentityFile ~/.ssh/id_rsa_jasmin
ForwardAgent yes
ForwardX11 yes

Then run ssh-add to add your JASMIN key to your ssh-agent. You should then be able to ssh to the JASMIN login node without being prompted for a passphrase.

Once you have that you can then try running the suite again.

Regards,
Ros.

comment:9 Changed 4 months ago by luciana

Dear Ros.

I did everything you asked me and still no luck. :/ I'll play with other tests now. Thanks!

Kind regards.

Luciana.

comment:10 Changed 4 months ago by ros

Hi Luciana,

When you decide to try this again please run ssh login1.jasmin.ac.uk on the command line. You're getting a "Host Key Verification error" when you run the suite so I suspect when you run on the command line you will get the same message and need to answer yes or delete an entry from your known_hosts file before you will be able to submit a suite. You must be able to ssh from PUMA to JASMIN without prompt for any information, passphrase, etc.

Regards,
Ros.

comment:11 Changed 4 months ago by luciana

Hi Ros.

I'm getting the following message:

—-
luciana@puma:/home/luciana> ssh login1.jasmin.ac.uk
key_load_public: invalid format
—-

but even with this message I still can log in directly into Jasmin, without any extra information. I then removed the known_hosts file and tried again.

—-
luciana@puma:/home/luciana> ssh -A lucy@…
key_load_public: invalid format
The authenticity of host 'login1.jasmin.ac.uk (130.246.130.28)' can't be established.
ECDSA key fingerprint is SHA256:Cv5IBRfVB0nOPulh3zdtncSX1XPh3X/IfKOhozBuXrE.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'login1.jasmin.ac.uk,130.246.130.28' (ECDSA) to the list of known hosts.
—-

After doing that, I'm still getting the message: "key_load_public: invalid format".

—-
luciana@puma:/home/luciana> ssh -A lucy@…
key_load_public: invalid format
—-

and the message "COMMAND STDERR: Host key verification failed." on log/suite/err.

I searched on Google for "key_load_public: invalid format" and it looks like my keys are not configured properly. I've changed some other things, but I still have the same problems. I've created a folder "trash" in my main directory and I've moved some files from .ssh to it, to clean .ssh a bit. Still, no solution.

On Jasmin, now I can see some of the suites I ran on Puma, but their directory contains a file "cylc-suite-env" and an empty "log/job" folder.

Let's hope you can help me to make it work! :)

Kind regards.

Luciana.

comment:12 Changed 4 months ago by ros

Hi Luciana,

I don't know what's going on with the key_load issue but if you can login ok it probably will be fine so long as ssh is not returning an error code. Do you get the same message when you log into JASMIN from other machines? The issue with your cylc suite is that you have that set to go to xfer1.jasmin.ac.uk. So you need to check you can login into that server ok. I suspect you've probably not added the equivalent section to your ~/.ssh/config so ssh knows what key and user name to use.

Regards,
Ros.


comment:13 Changed 4 months ago by luciana

Dear Ros.

I've tried access Jasmin from Archer. Sometimes it works, sometimes I get this message in the terminal, seconds after logging in.

—-
[lucy@cylc1 ~]$ Killed
—-

The other error message, key_load_public: invalid format, doesn't appear when I log from Archer to Jasmin. I used to be able to log in into Jasmin directly from my computer using Pulse Secure. However, this is not working anymore. :/ As long as I have one way to access Jasmin, it's fine by me. :)

I've tried to access xfer1.jasmin.ac.uk from puma and it works smoothly, without any extra configurations or passphrases. So the suite should be working too. :/

Andy has just given me access to pumatest. I'll play with it a little bit and see if I can proceed with my tests from there.

Thank you for your support.

Kind regards.

Luciana.

comment:14 Changed 4 months ago by ros

Closing ticket - superceded by #3426

comment:15 Changed 3 months ago by ros

  • Resolution set to answered
  • Status changed from accepted to closed
Note: See TracTickets for help on using tickets.