Opened 3 months ago

Closed 3 months ago

#2963 closed help (fixed)

Failure of pptransfer when pushing data from Nexcs to Jasmin

Reported by: charlie Owned by: ros
Component: UM Model Keywords:
Cc: Platform: NEXCS
UM Version: 10.7

Description

Hi,

Sorry to run you, but I am having a failure with my pptransfer app. The error is below. I *thought* I had upgraded the postprocessing and installed pptransfer correctly, following your usual instructions, and I have have compared it to one of my other suites which works, and believe they are the same, but clearly something has gone wrong. Have I done something silly?

My suite is u-bk944.

At least the coupled cycle ran successfully, so hopefully this will be an easy problem to resolve. I have checked both directories (i.e. on JASMIN at NEXCS) and they appear to be okay e.g. on JASMIN:

[cwilliams2011@jasmin-sci2 gc31]$ pwd
/group_workspaces/jasmin2/nexcs/cwilliams2011/sweet/gc31

And NEXCS:

cwilliams@xcslc1:/projects/nexcs-n02/cwilliams/sweet/gc31> pwd
/projects/nexcs-n02/cwilliams/sweet/gc31
cwilliams@xcslc1:/projects/nexcs-n02/cwilliams/sweet/gc31> ls
u-be195 u-bg273 u-bh604 u-bi281 u-bi623 u-bk944
u-bg272 u-bh301 u-bh960 u-bi324 u-bk568

So it is clearly creating the correct suite ID's directory on NEXCS, just failing when it comes to pushing it to JASMIN.

Very many thanks

Charlie

[WARN] file:atmospp.nl: skip missing optional source: namelist:moose_arch
[WARN] file:nemocicepp.nl: skip missing optional source: namelist:moose_arch
[WARN]  [SUBPROCESS]: Command: rsync -av --stats --rsync-path=mkdir -p /group_workspaces/jasmin2/nexcs/cwilliams2011/sweet/gc31/u-bk944/18500101T0000Z && rsync /projects/nexcs-n02/cwilliams/sweet/gc31/u-bk944/18500101T0000Z/ jasmin-xfer2.ceda.ac.uk:/group_workspaces/jasmin2/nexcs/cwilliams2011/sweet/gc31/u-bk944/18500101T0000Z
[SUBPROCESS]: Error = 255:
	ssh_exchange_identification: read: Connection reset by peer
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(641) [sender=3.0.4]

[WARN]  Transfer command failed: rsync -av --stats --rsync-path="mkdir -p /group_workspaces/jasmin2/nexcs/cwilliams2011/sweet/gc31/u-bk944/18500101T0000Z && rsync" /projects/nexcs-n02/cwilliams/sweet/gc31/u-bk944/18500101T0000Z/ jasmin-xfer2.ceda.ac.uk:/group_workspaces/jasmin2/nexcs/cwilliams2011/sweet/gc31/u-bk944/18500101T0000Z
[ERROR]  transfer.py: Unknown Error - Return Code=255
[FAIL]  Command Terminated
[FAIL] Terminating PostProc...
[FAIL] transfer.py # return-code=1
2019-07-22T00:09:03Z CRITICAL - failed/EXIT

Change History (5)

comment:1 Changed 3 months ago by ros

Hi Charlie,

Really sorry for the delay. I swear I responded to this. :-( There was a problem with the reverse DNS lookup for Monsoon at the Met Office end which meant that jasmin-xfer2 was rejecting the connections. This has now been resolved.

Regards,
Ros.

comment:2 Changed 3 months ago by charlie

Thanks so much Ros, I was about to come and badger you in person about this, this morning!

So am I right in thinking that I don't need to do anything, other than shutting down the suite, then restarting (rose suite-run —restart), then triggering the failed pptransfer task? Will all my other tasks i.e. in the next cycles, which are currently "waiting", just pick up where they left off once the failed pptransfer task has succeeded?

Charlie

comment:3 Changed 3 months ago by ros

  • Owner changed from um_support to ros
  • Status changed from new to accepted

Hi Charlie,

You should just be able to trigger the failed pptransfer task without any stopping and restarting. You've not made any changes to the suite so no need to restart it.

Regards,
Ros.

comment:4 Changed 3 months ago by charlie

Okay, done, it is now running. I'll let you know if it fails again, otherwise I will close this ticket. Very many thanks.

Charlie

comment:5 Changed 3 months ago by charlie

  • Resolution set to fixed
  • Status changed from accepted to closed
Note: See TracTickets for help on using tickets.