Opened 4 months ago

Closed 3 months ago

#2342 closed help (fixed)

PUMA SVN repository on JASMIN?

Reported by: pmcguire Owned by: um_support
Priority: normal Component: PUMA
Keywords: PUMA, SVN, FCM, JASMIN, MONSooN Cc:
Platform: Other UM Version: <select version>

Description

Is the PUMA JULES SVN/FCM repository available on JASMIN?
I can access it from PUMA and from MONSooN, but not JASMIN.
For example, this doesn't work on JASMIN, but it works on PUMA and on MONSooN:

jasmin-sci1> svn list svn://puma.nerc.ac.uk/jules.xm_svn/main/trunk

Change History (10)

comment:1 Changed 4 months ago by ros

Hi Patrick,

The Mirror repositories on PUMA are only available from PUMA or Monsoon. JASMIN does not have access to a mirror so you will need to checkout directly from the MOSRS Jules repository.

If you are just trying to checkout a branch/trunk then you will simply need to supply your MOSRS username/password.

If you are extracting code from within a Rose suite you will need to set up gpg-agent caching. JASMIN instructions for this are available here - I assume they are still up-to-date - https://code.metoffice.gov.uk/trac/home/wiki/AuthenticationCaching#JASMIN

Cheers,
Ros.

comment:2 follow-up: Changed 4 months ago by pmcguire

Dear Ros
OK. Thanks for the advice.

I decided that my best mode of getting things work was to switch to the MOSRS JULES repository.
I also followed the instructions for gpg-agent caching on JASMIN.

This seems to work if I checkout interactively. But it doesn't work in batch non-interactive mode.
I tried both jasmin-sci1 and jasmin-cylc.

For example, when I do this on Jasmin-sci1:

fcm make -f fcm:jules.x_br/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353

it works.

But when I do this on jasmin-sci1, it doesn't work

bsub -oo tmp.out -eo tmp.err fcm make -f fcm:jules.x_br/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353

The resulting tmp.err file looks like:
[FAIL] config-file=https://code.metoffice.gov.uk/svn/jules/main/branches/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353
[FAIL] https://code.metoffice.gov.uk/svn/jules/main/branches/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353: cannot load config file
[FAIL] https://code.metoffice.gov.uk/svn/jules/main/branches/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353: cannot be read
[FAIL] svn: E215004: Authentication failed and interactive prompting is disabled; see the —force-interactive option
[FAIL] svn: E215004: Unable to connect to a repository at URL 'https://code.metoffice.gov.uk/svn/jules/main/branches/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg'
[FAIL] svn: E215004: No more credentials or we tried too many times.
[FAIL] Authentication failed

Any suggestions?
Patrick McGuire?

comment:3 in reply to: ↑ 2 Changed 4 months ago by pmcguire

Hi
In this twiki, they say: "All suites run on Monsoon extract the source code (UM, NEMO, SOCRATES, etc) from read-only mirror repositories located on PUMA. This is primarily because most suites run the extract and build on the shared nodes which can only access the PUMA repository due to the authentication mechanisms."
http://collab.metoffice.gov.uk/twiki/bin/view/Support/MONSooNRose#Rose%20on%20the%20Met%20Office%20Science%20R

My suite u-at414 runs fine on Monsoon because it was designed to take advantage of the PUMA read-only source code repository, which doesn't require authentication, because as they say above, it is being done from the shared nodes, where authentication is difficult.

The trouble is, I am supposed to get this suite working on JASMIN, which also has shared nodes, so I guess it is working the same way as on MONSooN. But you say that JASMIN doesn't have access to the PUMA mirror repository. I can't get the JASMIN version of the suite (u-at416) working with the original MOSRS repository instead of the PUMA mirror repository. I have authentication problems, as noted in my last post. I tried debugging this authentication problem, but I haven't had success, unfortunately. I tried following the instructions in your original response.

This must be a common issue with other suites that build on shared nodes from source code from the MOSRS repository. Or is this building on shared nodes after extracting from the original MOSRS repository just not done?

Maybe I am confused a bit….

Patrick

Replying to pmcguire:

Dear Ros
OK. Thanks for the advice.

I decided that my best mode of getting things work was to switch to the MOSRS JULES repository.
I also followed the instructions for gpg-agent caching on JASMIN.

This seems to work if I checkout interactively. But it doesn't work in batch non-interactive mode.
I tried both jasmin-sci1 and jasmin-cylc.

For example, when I do this on Jasmin-sci1:

fcm make -f fcm:jules.x_br/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353

it works.

But when I do this on jasmin-sci1, it doesn't work

bsub -oo tmp.out -eo tmp.err fcm make -f fcm:jules.x_br/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353

The resulting tmp.err file looks like:
[FAIL] config-file=https://code.metoffice.gov.uk/svn/jules/main/branches/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353
[FAIL] https://code.metoffice.gov.uk/svn/jules/main/branches/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353: cannot load config file
[FAIL] https://code.metoffice.gov.uk/svn/jules/main/branches/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353: cannot be read
[FAIL] svn: E215004: Authentication failed and interactive prompting is disabled; see the —force-interactive option
[FAIL] svn: E215004: Unable to connect to a repository at URL 'https://code.metoffice.gov.uk/svn/jules/main/branches/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg'
[FAIL] svn: E215004: No more credentials or we tried too many times.
[FAIL] Authentication failed

Any suggestions?
Patrick McGuire?

comment:4 Changed 4 months ago by grenville

Patrick

I can't build interactively (or in batch) - have you seen this:

[FAIL] make preprocess # 0.2s
[FAIL] make # 4.0s
[FAIL] preprocess.prop{fpp.defs}[] = SCMA BL_DIAG_HACK COMPILER_GFORTRAN NCDF_DUMMY MPI_DUMMY: bad name-space
[FAIL] preprocess.prop{file-ext.h}[] = .h .inc: bad name-space

Grenville

comment:5 Changed 4 months ago by pmcguire

No I haven't seen that. Are you running u-at416 on CEDA JASMIN?
I never got to the build phase on JASMIN. JUST to the extract phase.
Patrick

comment:6 Changed 4 months ago by grenville

I was looking at the authentication problem - I don't seem to have that problem; does it persist for you?

comment:7 Changed 4 months ago by grenville

Some oddness in the config file - it seems to be struggling with incremental builds.

This works OK for me

bsub -oo tmp.out -eo tmp.err fcm make —new -f fcm:jules.x_br/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353

Grenville

comment:8 Changed 4 months ago by pmcguire

That command with the
bsub -oo tmp.out -eo tmp.err fcm make —new -f fcm:jules.x_br/dev/karinawilliams/r9227_add_gpp_unstressed_diagnostic/etc/fcm-make/make.cfg@9353
does NOT work for me, unfortunately.
Patrick

comment:9 Changed 3 months ago by pmcguire

Just to close this ticket out, I will report on some offline communication here.
Grenville informed me that, in reality, he was having the same problem with authentication from the JASMIN LOTUS shared nodes as I was having.
He suggested that I run the authentication as a background process in ROSE/CYLC instead of from the LOTUS shared nodes. This seems to work!
Patrick

comment:10 Changed 3 months ago by willie

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.