Opened 20 months ago
Closed 20 months ago
#2927 closed help (fixed)
MASS @ JASMIN - Problem using SELECT and wildcards
Reported by: | dan | Owned by: | um_support |
---|---|---|---|
Component: | Other | Keywords: | MASS MOO JASMIN |
Cc: | Platform: | JASMIN | |
UM Version: |
Description
Hi All,
I'm attempting to extract some PP data from MASS on JASMIN using the SELECT option.
Previously this has worked fine. But I have now encountered an error:
[dlrhodso@mass-cli1 data]$ moo select N216_query_mslp moose:/crum/u-aq281/apm.pp N216_ATM/mslp/u-aq281/
select command-id=731076270 failed: (SSC_TASK_REJECTION) one or more tasks are rejected.
moose:/crum/u-aq281/apm.pp → N216_ATM/mslp/u-aq281/: (TSSC_SPANS_TOO_MANY_RESOURCES) retrieval request exceeds the maximum number of tape resources.
Tape count: 63, limit: 50.
select: failed (2)
In an attempt to limit the requests and avoid this error, I have been trying to use wildcards, as documented in the wiki (
http://gws-access.ceda.ac.uk/public/mohc_shared/moose-user-doc/external_user_guide.html)
moo select N216_query_mslp moose:/crum/u-aq281/apm.pp/aq281a.pm1968* N216_ATM/mslp/u-aq281/
But this gives me:
select command-id=731077227 failed: (SSC_TASK_REJECTION) one or more tasks are rejected.
moose:/crum/u-aq281/apm.pp/aq281a.pm1968* → N216_ATM/mslp/u-aq281/: (TSSC_INVALID_SELECTION_TARGET) metadata querying commands must specify appropriate files, collections or parents thereof, and must specify a suitable file type if it cannot be inferred.
select: failed (2)
[
if i specify PP using:
moo select -T=pp N216_query_mslp moose:/crum/u-aq281/apm.pp/aq281a.pm1968* N216_ATM/mslp/u-aq281/
I get the same error.
N216_query_mslp is simply:
begin
lbproc=128
stash=16222
end
Have you any idea what I might be doing wrong?
It seems that the wildcards are not working as expected. I thought that this might be because the wildcards are somehow expanded before they are passed to moo in the wrapper script, but I'm not sure that's the case now.
I can go and ask JASMIN, but I thought you might be better acquainted with MOO!
Cheers,
Dan
Change History (5)
comment:1 follow-up: ↓ 3 Changed 20 months ago by grenville
comment:2 follow-up: ↓ 4 Changed 20 months ago by luke
I've only ever used moo select on directories within MASS, not separate files.
This webpage might be informative as it seems to give quite detailed information: https://collab.metoffice.gov.uk/twiki/bin/viewfile/Static/MASS/monsoon_user_guide.html#select-performs-filtered-retrieval-from-a-data-set
I would suggest having a select file (e.g. foo.stash) that looks like:
begin lbproc=128 stash=16222 year=1968 end
and then using the following command to just extract the 1968 values:
moo select -v foo.stash moose:/crum/u-aq281/apm.pp ./N216_ATM/mslp/u-aq281/apm.pp
as you also need to extract to a directory.
You can also specify a range of years like so:
year=[1968..1970]
etc.
Thanks,
Luke
comment:3 in reply to: ↑ 1 Changed 20 months ago by dan
Replying to grenville:
Dan
Are you sure there is a file with aq281a.pm1968 in its name? Does aq281a.pm* fail too?
(not that I'm MOO acquainted)
Grenville
Yes - these files exist as:
moo ls moose:/crum/u-aq281/apm.pp/aq281a.pm1968*
moose:/crum/u-aq281/apm.pp/aq281a.pm1968apr.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968aug.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968dec.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968feb.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968jan.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968jul.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968jun.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968mar.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968may.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968nov.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968oct.pp
moose:/crum/u-aq281/apm.pp/aq281a.pm1968sep.pp
but
moo select N216_query_mslp moose:/crum/u-aq281/apm.pp/aq281a.pm1968* N216_ATM/mslp/u-aq281/
fails!
I must be doing something else wrong!
comment:4 in reply to: ↑ 2 Changed 20 months ago by dan
Replying to luke:
I would suggest having a select file (e.g. foo.stash) that looks like:
begin lbproc=128 stash=16222 year=1968 endand then using the following command to just extract the 1968 values:
moo select -v foo.stash moose:/crum/u-aq281/apm.pp ./N216_ATM/mslp/u-aq281/apm.ppas you also need to extract to a directory.
You can also specify a range of years like so:
year=[1968..1970]etc.
Excellent - I hadn't thought of subsetting the files by dates via query file - this works!!
Thanks, Luke - saved me a frustrating morning!
Cheers,
Dan
comment:5 Changed 20 months ago by luke
- Resolution set to fixed
- Status changed from new to closed
Great! I'm glad that works.
Thanks,
Luke
Dan
Are you sure there is a file with aq281a.pm1968 in its name? Does aq281a.pm* fail too?
(not that I'm MOO acquainted)
Grenville