Opened 13 days ago

Closed 12 days ago

#3096 closed help (completed)

Disk I/O error - work quota

Reported by: Leighton_Regayre Owned by: ros
Component: Disk Space Keywords: allocation quota
Cc: Platform: ARCHER
UM Version: 11.1

Description

Hello,

I'm attempting to make a large ensemble of UKESM1 simulations. My suite crashed over the weekend because output from the jobs exceeded my /work quota. I'm running the jobs in batches and moving all data off /work manually because the postproc task isn't working properly in my Gregorian calendar simulations (e.g. #3075).

Could I please have an increase to my allocated /work quota? It would make it much easier to create my ensemble efficiently. The alternatives are to run in much smaller batches, or risk wasting resource by rerunning some batches (today's experience).

Thanks,

Leighton

Change History (5)

comment:1 Changed 13 days ago by ros

  • Owner changed from um_support to ros
  • Status changed from new to accepted

Hi Leighton,

You already have a large /work quota and are currently only using 16Tb of it. If this is not enough space for you to manage with manually moving your data you need to sort out postproc to do it automatically.

Regards,
Ros.

comment:2 Changed 13 days ago by Leighton_Regayre

Hi Ros,

Thanks for the reply. There's only 16Tb used at the moment because since making this ticket, I deleted a lot of the data created by my ensemble of failed jobs. I pointed out in ticket #3075 that I'm having trouble with the postproc task. Grenville said on comment 17 that the postproc guru is out, but I have no idea what that means. Some files are being created and moved by the postproc task, but not all. If it doesn't behave reliably then I need to transfer files manually, but still need to make a lot of data each day in order to create the ensemble before ARCHER goes down in Feb.

Since the postproc guru is down, and doesn't seem to be working properly for Gregorian calendar output from the UKESM (set up according to Mohit's adapted release version), I can't see any alternative than an increase in my /work allocation.

Could you please reconsider?

Thanks,

Leighton

comment:3 Changed 13 days ago by ros

Hi Leighton,

I was out on holiday the last couple of weeks, but will take a look at postproc now - there is no reason I know of why it shouldn't work with gregorian calendar. Managing that much data manually is going to be difficult even with an increase in quota. We have a finite amount of space on /work, but I have given you some extra to hopefully tide over until we get postproc working properly.

Regards,
Ros.

comment:4 Changed 13 days ago by Leighton_Regayre

Hi Ros,

That's fantastic news. Thanks!

I agree, I'd far prefer to be using the postproc for most efficient production of the ensemble, so that's welcome news as well.

All the best,

Leighton

comment:5 Changed 12 days ago by ros

  • Component changed from UM Model to Disk Space
  • Resolution set to completed
  • Status changed from accepted to closed

Discussion of postproc continued in original ticket #3075

Note: See TracTickets for help on using tickets.