#2927 closed help (fixed)

MASS @ JASMIN - Problem using SELECT and wildcards

Reported by: dan Owned by: um_support
Component: Other Keywords: MASS MOO JASMIN
Cc: Platform: JASMIN
UM Version:

Description

Hi All,

I'm attempting to extract some PP data from MASS on JASMIN using the SELECT option.
Previously this has worked fine. But I have now encountered an error:

[dlrhodso@mass-cli1 data]$ moo select N216_query_mslp moose:/crum/u-aq281/apm.pp N216_ATM/mslp/u-aq281/
select command-id=731076270 failed: (SSC_TASK_REJECTION) one or more tasks are rejected.

moose:/crum/u-aq281/apm.pp → N216_ATM/mslp/u-aq281/: (TSSC_SPANS_TOO_MANY_RESOURCES) retrieval request exceeds the maximum number of tape resources.

Tape count: 63, limit: 50.

select: failed (2)

In an attempt to limit the requests and avoid this error, I have been trying to use wildcards, as documented in the wiki (

http://gws-access.ceda.ac.uk/public/mohc_shared/moose-user-doc/external_user_guide.html)

moo select N216_query_mslp moose:/crum/u-aq281/apm.pp/aq281a.pm1968* N216_ATM/mslp/u-aq281/

But this gives me:

select command-id=731077227 failed: (SSC_TASK_REJECTION) one or more tasks are rejected.

moose:/crum/u-aq281/apm.pp/aq281a.pm1968* → N216_ATM/mslp/u-aq281/: (TSSC_INVALID_SELECTION_TARGET) metadata querying commands must specify appropriate files, collections or parents thereof, and must specify a suitable file type if it cannot be inferred.

select: failed (2)
[
if i specify PP using:

moo select -T=pp N216_query_mslp moose:/crum/u-aq281/apm.pp/aq281a.pm1968* N216_ATM/mslp/u-aq281/

I get the same error.
N216_query_mslp is simply:

begin

lbproc=128
stash=16222

end

Have you any idea what I might be doing wrong?

It seems that the wildcards are not working as expected. I thought that this might be because the wildcards are somehow expanded before they are passed to moo in the wrapper script, but I'm not sure that's the case now.

I can go and ask JASMIN, but I thought you might be better acquainted with MOO!

Cheers,

Dan

Change History (5)

comment:1 follow-up: Changed 13 months ago by grenville

Dan

Are you sure there is a file with aq281a.pm1968 in its name? Does aq281a.pm* fail too?

(not that I'm MOO acquainted)

Grenville

comment:2 follow-up: Changed 13 months ago by luke

I've only ever used moo select on directories within MASS, not separate files.

This webpage might be informative as it seems to give quite detailed information: https://collab.metoffice.gov.uk/twiki/bin/viewfile/Static/MASS/monsoon_user_guide.html#select-performs-filtered-retrieval-from-a-data-set

I would suggest having a select file (e.g. foo.stash) that looks like:

begin
    lbproc=128
    stash=16222
    year=1968
end

and then using the following command to just extract the 1968 values:

moo select -v foo.stash moose:/crum/u-aq281/apm.pp ./N216_ATM/mslp/u-aq281/apm.pp

as you also need to extract to a directory.

You can also specify a range of years like so:

    year=[1968..1970]

etc.

Thanks,
Luke

comment:3 in reply to: ↑ 1 Changed 13 months ago by dan

Replying to grenville:

Dan

Are you sure there is a file with aq281a.pm1968 in its name? Does aq281a.pm* fail too?

(not that I'm MOO acquainted)

Grenville

Yes - these files exist as:

moo ls moose:/crum/u-aq281/apm.pp/aq281a.pm1968*
moose:/crum/u-aq281/apm.pp/aq281a.pm1968apr.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968aug.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968dec.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968feb.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968jan.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968jul.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968jun.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968mar.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968may.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968nov.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968oct.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968sep.pp

but
moo select N216_query_mslp moose:/crum/u-aq281/apm.pp/aq281a.pm1968* N216_ATM/mslp/u-aq281/

fails!
I must be doing something else wrong!

comment:4 in reply to: ↑ 2 Changed 13 months ago by dan

Replying to luke:

I would suggest having a select file (e.g. foo.stash) that looks like:

begin
    lbproc=128
    stash=16222
    year=1968
end

and then using the following command to just extract the 1968 values:

moo select -v foo.stash moose:/crum/u-aq281/apm.pp ./N216_ATM/mslp/u-aq281/apm.pp

as you also need to extract to a directory.

You can also specify a range of years like so:

    year=[1968..1970]

etc.

Excellent - I hadn't thought of subsetting the files by dates via query file - this works!!

Thanks, Luke - saved me a frustrating morning!

Cheers,

Dan

comment:5 Changed 13 months ago by luke

  • Resolution set to fixed
  • Status changed from new to closed

Great! I'm glad that works.

Thanks,
Luke

Note: See TracTickets for help on using tickets.