Start dump request

[a.elvidge]

Hi,

I would like to request some (a lot of) UM start dumps, please:
Every 24 hours at 00 UTC between and inclusive of 20151125T0000Z and 20160531T0000Z.

Would this be possible?

Thanks, Andy

[willie]

Hi Andy,
Yes it certainly possible. This is about 156 dumps and I think each are 24GB, so that is 3,744GB or about 3.7 TB, so you'll need to make sure you have space to store this.

I'll get started on this once I can access MASS - its down at the moment, due to recent problems affecting Monsoon and its systems.

Regards,
Willie

[a.elvidge]

Hi Willie,
Do you know how I go about finding out how much space is available on my Monsoon account / project space?
Cheers, Andy

[willie]

Hi Andy,

There is useful information at ​http://collab.metoffice.gov.uk/twiki/bin/view/Support/FileSystems.

Regards
Willie

[ros]

Just to add, if I'm reading the information I have correctly, both polar and accacia currently have a 6Tb group quota. polar has used 24% of its quota and accacia 7%.

[willie]

Hi Andy,

MASS is now back up and I'm starting to get the data. I miscalculated: it is actually 187 dumps and they are 22.84GB each, so 4.17TB in all. It takes 6.47 mins to get one dump, so 20.16 hours solid 'moo get' for them all. I am trying to get one months worth in one get to see if it's any faster.

Regards
Willie

[a.elvidge]

Hi Willie,
That's quite a lot of data. Perhaps hold off for a bit. I'll have a think about alternatives, or whether to go ahead.
Cheers, Andy

[a.elvidge]

Hi Willie, Ros,

Is my understanding correct that the start dumps are stored on MASS, and so are not part of the 6 gb allocated local disk space? In which case, please go ahead with the data transfer.

Thanks, Andy

[willie]

Hi Andy,

Although they are stored in MASS they need to be extracted to be used. They will end up on, say, /projects/accacia/aelvidge, where they will use up quota.

For reference, extracting a months worth in one moo get is about four times faster than getting the files individually.

Let me know how you want to proceed.

Regards
Willie

[a.elvidge]

Hi Willie,

Ok… the start dumps will only be there temporarily whilst I run the job. So I think it's ok to proceed, please.

Thanks, Andy

[willie]

Hi Andy,

Your data is now ready on Monsoon in /projects/nexcs-n02/frmy/data_xfer. It is 189 dumps and 4.3TB.

Please copy it to your own work space as soon as possible and let me know when you've done that so I can free up my work space.

Regards
Willie

[a.elvidge]

Hi Willie,

I have now copied all those start dumps across, thanks.

Cheers, Andy