Opened 3 years ago
Closed 3 years ago
#2371 closed help (fixed)
Start dump request
Reported by: | a.elvidge | Owned by: | um_support |
---|---|---|---|
Component: | Data | Keywords: | |
Cc: | Platform: | Monsoon2 | |
UM Version: |
Description
Hi,
I would like to request some (a lot of) UM start dumps, please:
Every 24 hours at 00 UTC between and inclusive of 20151125T0000Z and 20160531T0000Z.
Would this be possible?
Thanks, Andy
Change History (12)
comment:1 Changed 3 years ago by willie
comment:2 Changed 3 years ago by a.elvidge
Hi Willie,
Do you know how I go about finding out how much space is available on my Monsoon account / project space?
Cheers, Andy
comment:3 Changed 3 years ago by willie
Hi Andy,
There is useful information at http://collab.metoffice.gov.uk/twiki/bin/view/Support/FileSystems.
Regards
Willie
comment:4 Changed 3 years ago by ros
Just to add, if I'm reading the information I have correctly, both polar and accacia currently have a 6Tb group quota. polar has used 24% of its quota and accacia 7%.
comment:5 Changed 3 years ago by willie
- UM Version <select version> deleted
Hi Andy,
MASS is now back up and I'm starting to get the data. I miscalculated: it is actually 187 dumps and they are 22.84GB each, so 4.17TB in all. It takes 6.47 mins to get one dump, so 20.16 hours solid 'moo get' for them all. I am trying to get one months worth in one get to see if it's any faster.
Regards
Willie
comment:6 Changed 3 years ago by a.elvidge
Hi Willie,
That's quite a lot of data. Perhaps hold off for a bit. I'll have a think about alternatives, or whether to go ahead.
Cheers, Andy
comment:7 Changed 3 years ago by a.elvidge
Hi Willie, Ros,
Is my understanding correct that the start dumps are stored on MASS, and so are not part of the 6 gb allocated local disk space? In which case, please go ahead with the data transfer.
Thanks, Andy
comment:8 Changed 3 years ago by willie
Hi Andy,
Although they are stored in MASS they need to be extracted to be used. They will end up on, say, /projects/accacia/aelvidge, where they will use up quota.
For reference, extracting a months worth in one moo get is about four times faster than getting the files individually.
Let me know how you want to proceed.
Regards
Willie
comment:9 Changed 3 years ago by a.elvidge
Hi Willie,
Ok… the start dumps will only be there temporarily whilst I run the job. So I think it's ok to proceed, please.
Thanks, Andy
comment:10 Changed 3 years ago by willie
Hi Andy,
Your data is now ready on Monsoon in /projects/nexcs-n02/frmy/data_xfer. It is 189 dumps and 4.3TB.
Please copy it to your own work space as soon as possible and let me know when you've done that so I can free up my work space.
Regards
Willie
comment:11 Changed 3 years ago by a.elvidge
Hi Willie,
I have now copied all those start dumps across, thanks.
Cheers, Andy
comment:12 Changed 3 years ago by willie
- Resolution set to fixed
- Status changed from new to closed
Hi Andy,
Yes it certainly possible. This is about 156 dumps and I think each are 24GB, so that is 3,744GB or about 3.7 TB, so you'll need to make sure you have space to store this.
I'll get started on this once I can access MASS - its down at the moment, due to recent problems affecting Monsoon and its systems.
Regards,
Willie