Opened 3 years ago
Closed 3 years ago
#2001 closed help (fixed)
Moving files exceeds walltime limit
Reported by: | Leighton_Regayre | Owned by: | um_support |
---|---|---|---|
Component: | Other | Keywords: | |
Cc: | Platform: | ||
UM Version: |
Description
Hello,
I'm attempting to move a large number of files from /work to /nerc.
I move the files in batches using a script using the queue. It has worked perfectly in the past. Currently when attempting to move just 2 .pp files I get an error message:
walltime 27 exceeded limit 24
despite having entered a 24 hour limit. Could this be a /work space allocation issue?
The output file is here:
/work/n02/n02/lre/Dumps/mv2n_xmtna_sub_.e3994322
The /nerc space I'm attempting to move files to is here:
/nerc/n02/n02-CCPPE/lre
Change History (10)
comment:1 Changed 3 years ago by grenville
comment:2 Changed 3 years ago by Leighton_Regayre
Grenville,
I've changed permissions.
In case it's the cause of the problem the script I'm using to move the files is here:
/home/n02/n02/masara/bin/move2nerc.sh
Thanks,
Leighton.
comment:3 Changed 3 years ago by grenville
Leighton
Where is the output from move2nerc.sh, where is $subscript?
Grenville
comment:4 Changed 3 years ago by Leighton_Regayre
Grenville,
I use the default filepath:
/work/n02/n02/lre/Dumps
to move files to the directory:
/nerc/n02/n02-CCPPE/lre
The files I'm attempting to move at the moment have the root:
xmtn?
and contain:
*.pm*
There are only 2 of them with this root.
Leighton.
comment:5 Changed 3 years ago by grenville
Leighton
I'm not following this — I need to see the pbs script which was submitted to the machine. I'm not clear on why there is no /work/n02/n02/lre/Dumps/mv2n_xmtna_sub_.o3994322 file
I can't see which files you refer to here
The files I'm attempting to move at the moment have the root:
xmtn?
and contain:
*.pm*
Grenville
comment:6 Changed 3 years ago by Leighton_Regayre
Grenville,
Sorry for not explaining more fully.
The script requires me to enter the source and destination directories, followed by the folders (in my case xmtn*, meaning all directories starting with xmtn), then the type of file (*.pp).
The raw script is here:
home/n02/n02/masara/bin/move2nerc.sh
I'm not sure there's any /work/n02/n02/lre/Dumps/mv2n_xmtna_sub_.o3994322 file. That confuses me too.
Leighton.
comment:7 Changed 3 years ago by grenville
Leighton
Please run home/n02/n02/masara/bin/move2nerc.sh as you did previously and keep the file
subscript=$dirm2n/mv2n_${dirs[0]}_sub_$datim
That's the one I (think) I need to see.
Grenville
comment:8 Changed 3 years ago by grenville
Leighton
I see that some of your jobs have a wallclock time of 24 seconds —- could that be the problem?
Grenville
comment:9 Changed 3 years ago by Leighton_Regayre
Grenville,
Example output files are here:
/home/n02/n02/lre/umui_runs/scripts/move2nerc/mv2n_xmtoa_sub_20161019-083828
There was one submission where I used an alternative to the default wallclock because I was confused by an error message referring to a required 27 hours. This wasn't the cause of the transfer failing.
Today the jobs are copying fine using exactly the same submission process.
I'll let you know if the problem resurfaces.
Thanks for your help,
Leighton.
comment:10 Changed 3 years ago by ros
- Component changed from UM Model to Other
- Resolution set to fixed
- Status changed from new to closed
- UM Version <select version> deleted
No further comments received so closing this ticket.
Leighton
Please change permissions on /work/n02/n02/lre/Dumps/mv2n_xmtna_sub_.e3994322 so I can read it.
Grenville