Opened 13 days ago

Last modified 10 days ago

#2396 new error

Job failing at archive stage

Reported by: a.elvidge Owned by: um_support
Priority: normal Component: UM Model
Keywords: Cc:
Platform: UM Version: 10.6

Description

Hi,

I am just getting started using Monsoon again.
My test job u-au901 is getting all the way to the archiving stage, then failing. It seems to me that no data is being produced locally (despite setting up STASH to do so), hence the failure when attempting to send to MASS. Here is the error message:

[FAIL] moo put -F -c umpp /home/d00/aelvidge/cylc-run/u-au901/work/20151125T0000Z/IGP_4p0_GA7_archive/tmpHc7QZ7/20151125T0000Z_IGP_4p0_GA7_pa000.pp /home/d00/aelvidge/cylc-run/u-au901/work/20151125T0000Z/IGP_4p0_GA7_archive/tmpHc7QZ7/20151125T0000Z_IGP_4p0_GA7_pb000.pp /home/d00/aelvidge/cylc-run/u-au901/work/20151125T0000Z/IGP_4p0_GA7_archive/tmpHc7QZ7/20151125T0000Z_IGP_4p0_GA7_pc000.pp moose:/devfc/u-au901/field.pp/ # return-code=2, stderr=
[FAIL] put command-id=497626574 failed: (SSC_TASK_REJECTION) one or more tasks are rejected.
[FAIL]   /home/d00/aelvidge/cylc-run/u-au901/work/20151125T0000Z/IGP_4p0_GA7_archive/tmpHc7QZ7/20151125T0000Z_IGP_4p0_GA7_pa000.pp -> moose:/devfc/u-au901/field.pp/20151125T0000Z_IGP_4p0_GA7_pa000.pp: (TSSC_SET_DOES_NOT_EXIST) no such data set.
[FAIL] put: failed (2)
[FAIL] ! moose:/devfc/u-au901/field.pp/ [compress=None, t(init)=2018-02-12T15:19:43Z, dt(tran)=0s, dt(arch)=2s, ret-code=2]
[FAIL] !	20151125T0000Z_IGP_4p0_GA7_pa000.pp (umnsaa_pa000)
[FAIL] !	20151125T0000Z_IGP_4p0_GA7_pb000.pp (umnsaa_pb000)
[FAIL] !	20151125T0000Z_IGP_4p0_GA7_pc000.pp (umnsaa_pc000)

Any help much appreciated.

Cheers, Andy

Change History (3)

comment:1 Changed 12 days ago by willie

Hi Andy,

Do you have permission to write to this:

frmy@xcs-c$ moo ls  moose:/devfc/u-au901
ls command-id=498175736 failed: (SSC_TASK_REJECTION) one or more tasks are rejected.
  moose:/devfc/u-au901: (TSSC_SET_DOES_NOT_EXIST) no such data set.

Did the install_cold app fail? It should've created this.

Regards
Willie

comment:2 Changed 10 days ago by a.elvidge

Hi Willie,

Ah yes, I hadn't noticed that, but when I change my arch location to an accessible location, I still get an error:

[FAIL] moo put -F -c umpp /home/d00/aelvidge/cylc-run/u-au901/work/20151125T0000Z/IGP_4p0_GA7_archive/tmpZo88dR/20151125T0000Z_IGP_4p0_GA7_pb000.pp /home/d00/aelvidge/cylc-run/u-au901/work/20151125T0000Z/IGP_4p0_GA7_archive/tmpZo88dR/20151125T0000Z_IGP_4p0_GA7_pa000.pp /home/d00/aelvidge/cylc-run/u-au901/work/20151125T0000Z/IGP_4p0_GA7_archive/tmpZo88dR/20151125T0000Z_IGP_4p0_GA7_pc000.pp moose:/adhoc/projects/accacia/aelvidge/u-au901/field.pp/ # return-code=2, stderr=
[FAIL] put command-id=499302094 failed: (SSC_TASK_REJECTION) one or more tasks are rejected.
[FAIL]   /home/d00/aelvidge/cylc-run/u-au901/work/20151125T0000Z/IGP_4p0_GA7_archive/tmpZo88dR/20151125T0000Z_IGP_4p0_GA7_pb000.pp -> moose:/adhoc/projects/accacia/aelvidge/u-au901/field.pp: (TSSC_IS_NOT_DIRECTORY) target does not resolve to a directory.
[FAIL] put: failed (2)
[FAIL] ! moose:/adhoc/projects/accacia/aelvidge/u-au901/field.pp/ [compress=None, t(init)=2018-02-15T10:47:06Z, dt(tran)=0s, dt(arch)=1s, ret-code=2]
[FAIL] !	20151125T0000Z_IGP_4p0_GA7_pa000.pp (umnsaa_pa000)
[FAIL] !	20151125T0000Z_IGP_4p0_GA7_pb000.pp (umnsaa_pb000)
[FAIL] !	20151125T0000Z_IGP_4p0_GA7_pc000.pp (umnsaa_pc000)
2018-02-15T10:47:08Z CRITICAL - Task job script received signal EXIT

It is true that moose:/adhoc/projects/accacia/aelvidge/u-au901/field.pp is not yet a directory - but I'd have thought this directory should have been created? Note that moose:/adhoc/projects/accacia/aelvidge/ does exist, and I have permission to write here.

Any help much appreciated.

Cheers, Andy

comment:3 Changed 10 days ago by willie

Hi Andy,

Monsoon is having issues with MASS at the moment - see the Monsoon collaboration channel on Yammer.

Willie

Note: See TracTickets for help on using tickets.