Opened 5 months ago

Last modified 3 weeks ago

#3210 accepted help

1.2T input directory for Global JULES on JASMIN

Reported by: NoelClancy Owned by: pmcguire
Component: JULES Keywords:
Cc: gerd.folberth@… Platform:
UM Version:

Description (last modified by pmcguire)

Hi Patrick,

Do you know of a suitable location on the JASMIN group workspaces for a large (1.2T) amount of input data?

I want to run a global ozone damage suite on JASMIN but I need to save a 1.2T directory of necessary input data files on JASMIN (OzDamDriverData). This input data will also be available of others who wish to run global JULES suites on JASMIN in the future.

[nclancy@eld504:/scratch/nclancy]$ du -sh OzDamDriverData
1.2T OzDamDriverData

scp: /group_workspaces/jasmin2/jules/nmc_data_transfer/OzDamDriverData/CRU-NCEP-v7/CRU-NCEP-v7-n96e/cruncepv7_uwind_1979_n96e.nc: Disk quota exceeded

Change History (10)

comment:1 Changed 5 months ago by pmcguire

Hi Noel
Ideally, we would be able to have a big enough jules group workspace on JASMIN to serve as a place to store driving data like this. But alas, for a long time, the jules group workspace has been rather full.

I sent an email to the University of Reading's met-land email list on January 7th asking opinions or desires etc. for setting up a land modelling group workspace on JASMIN, which could address this issue, as well as other data storage issues. Can you respond to that email, please? It could give more support to this initiative.

In the meantime, do you and Gerd both belong to any other group workspaces on JASMIN? If you let me know which ones, I can help you decide which of those GWS's or which of other GWS's might be the current best option.
Patrick McGuire

comment:2 Changed 5 months ago by NoelClancy

Hi Patrick,

As far as I know Gerd works mainly on the Met Office HPL and MONSOON, so I don't think he has access to any of the group workspaces on JASMIN.

In my case, obviously, I have access to the group_workspaces folder and I am currently storing relatively small amounts of data there.

However, I'm not completely sure what other spaces on JASMIN I have access to. I applied for access to some spaces at the start of my PhD but I cannot remember which ones. Do you know how I can find out off hand?

In the meantime, I will investigate this myself and get back to you shortly.

Thanks,

Noel

comment:3 Changed 5 months ago by pmcguire

Hi Noel
You can find out which group workspaces you currently belong to at the JASMIN portal.
Patrick

comment:4 Changed 5 months ago by NoelClancy

Patrick,

I just logged onto the JASMIN portal and I have access to the following:

Login Services: jasmin-login
Group Workspaces: jules
Group workspaces: ncas_generic

Noel

comment:5 Changed 5 months ago by pmcguire

Hi Noel:
There is currently 19TB of space on ncas_generic, which you have access to. You could put your driving data there, at least temporarily.

/gws/nopw/j04/ncas_generic

Remember it's a no-parallel-write hard drive.

This GWS is mainly for NCAS people, but I have been using it for storing WFDEI driving data for JULES, for use by people that use the GL6R JULES global-gridded suites. I believe it is ok for you to store the Ozone-Damage JULES driving data there, as well.
Patrick

comment:6 Changed 4 weeks ago by pmcguire

Hi Noel:
Do you have a group workspace (GWS) privilege for the GWS named landsurf_rdg? You could move your ancillary files and other files there, if you want? (By moving, I suggest copying, and then checking your copy, and then deleting the original).
This landsurf_rdg GWS just came online recently. You can apply for access if you don't have it at the JASMIN accounts portal.
Patrick

comment:7 Changed 4 weeks ago by pmcguire

  • Status changed from new to accepted

comment:8 Changed 4 weeks ago by pmcguire

  • Description modified (diff)

comment:9 Changed 4 weeks ago by NoelClancy

Yes, I have access to landsurf_rdg, I applied for it already on the JASMIN portal.
I have ozone damage data files on ncas_generic.

I can create a directory with my username nmc and then copy all my files to

/gws/nopw/j04/landsurf_rdg/nmc

comment:10 Changed 3 weeks ago by pmcguire

Hi Noel:
I am glad the landsurf_rdg GWS seems to work.
Do you need further help with this? Is everything working?
Patrick

Note: See TracTickets for help on using tickets.