Opened 3 years ago

Closed 3 years ago

#2007 closed help (completed)

more disk space

Reported by: lboljka Owned by: grenville
Component: Disk Space Keywords:
Cc: Platform: ARCHER
UM Version: 8.6

Description (last modified by grenville)

Hi

I am trying to run a 30-year long run on Archer, but am afraid that it might crash due to disk space some time along the way. Thus I would probably need more space on Archer home and work space (I will start moving things to RDF when it becomes available, but I am not sure when this will happen). Note that I have cleared archer home and work as well as puma's space as much as I could before running the job.

My username is lboljka, job that I am running is xmxic.

Thank you!
Best wishes
Lina

Change History (7)

comment:1 Changed 3 years ago by grenville

  • Description modified (diff)

Lina

Can you be more specific - how much space will you need? You should be able to estimate the amount from the number of fields written and their output frequency.

Grenville

comment:2 Changed 3 years ago by lboljka

Hi Grenville

I am not sure how to estimate it correctly but this is what I need:

  • p, u, v , rho, w, theta (6 fields) ⇒ all at original grids of N96L63 (for all approx. 192 x 144 points for horizontal resolution, 63 levels and 10800 time points - one output per day). So 6 x 192 x 144 x 63 x 10800 points for these 6 fields.
  • and two surface fields (T,p) ⇒ 2x 192 x 144 x 1 x 10800 points for these 2 fields.

Plus I guess some space for dumps (every 360 days - when the run reruns).

Currently (for whatever reason) I have at least 55% full space and I doubt it can hold all of this data - but if it is enough than no worries.

Thank you!
Best wishes
Lina

comment:3 Changed 3 years ago by grenville

Lina

Each data point needs 8 bytes, so (please check) I think this adds up to 90GB? Plus some for dumps (you have 1000GB currently) so maybe you're OK?

Grenville

comment:4 Changed 3 years ago by lboljka

Hi Grenville

I calculated (144x192x63x10800x6 + 144x192x1x10800x2 ) x 8bytes = 9.1 x 1e11 bytes, which makes about 900GB (not 90GB) - due to staggered grid it is hard to make exact estimate. Then if we add the dumps we are basically at a limit where it might work or might not so maybe extra few hundred GB would probably help the job not crashing right at the end (I am not able to see at the moment how much space I still have or to verify the size of files as both RDF and Archer are down). Sorry about this.

Thanks!
Best wishes
Lina

comment:5 Changed 3 years ago by grenville

Ah - OK I increased your space to 2TB - let's keep an eye on usage.

Grenville

comment:6 Changed 3 years ago by lboljka

Great! Thanks!
Lina

comment:7 Changed 3 years ago by ros

  • Resolution set to completed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.