Opened 10 years ago

Closed 10 years ago

#582 closed help (fixed)

slow listing of /work on HECToR

Reported by: jonathan Owned by: lois
Component: UM Model Keywords:
Cc: Platform:
UM Version: <select version>

Description

Dear Lois

The problem with slow listing of /work directories has not been alleviated, I don't think. For instance, it just took 4 min 9 sec to ls -l /work/n02/n02/gregoryj/um_archive/xdnyp. This is quite a large directory, but not huge (3000 files, all created since 4 Feb). With such terribly slow speeds, it is extremely difficult to manage the data. ls without -l is quite fast, but to check and decide what to do with the files I need to see their dates. It's hardly a usable filesystem for interactive purposes as things are. Can HECToR improve this, do you think?

Thanks

Jonathan

Change History (2)

comment:1 Changed 10 years ago by lois

  • Owner changed from um_support to lois
  • Status changed from new to assigned

I will pass on your complaint to HECToR Jonathan. HECToR are very aware of the limitations of the Lustre file system so I am trying to push then to monitor these issues more closely and that they propose potential solutions which might be possible to resolve with phase 3 funding. I am also raising this issue with NERC so that NERC can add weight to the need to resolve Lustre problems in order for them to get value for their science on HECToR. So please keep complaining and I will continue to put the pressure on HECToR, Cray and NERC to come up with solutions.

I am sorry that there is no instant answer to your problems.

Lois

comment:2 Changed 10 years ago by lois

  • Resolution set to fixed
  • Status changed from assigned to closed

HECToR and Cray have put in detailed monitoring systems and they will share the analysis of their results with us. I will prompt them in a couple of weeks, when hopefully they have enough data to come to some conclusion. This does not mean there will be a solution but hopefully it will be a step forward.

Lois

Note: See TracTickets for help on using tickets.