Opened 12 years ago

Closed 12 years ago

#127 closed help (fixed)

Request for start dumps

Reported by: soutten Owned by: willie
Component: Data Keywords:
Cc: Platform:
UM Version:

Description

I'd like to request some start dumps for the following dates.

11 January 2008, 00Z
13 January 2008, 00Z
10 February 2007, 00Z
09 December 2007, 00Z

I am part of the GFDex project and the start dumps I currently use are
stored at

/hpcx/work/n02/n02-ncas/umx/dumps/weather/global

If these dumps could also be stored there, that would be excellent.

Thanks

Stephen Outten

Change History (18)

comment:1 Changed 12 years ago by Stephen Outten

  • id set to 127

This message has 0 attachment(s)

comment:2 follow-up: Changed 12 years ago by ros

  • Component changed from UM Model to Data
  • Owner changed from um_support to willie

comment:3 in reply to: ↑ 2 Changed 12 years ago by umdoc

Replying to ros:

Could you please let me know what the status of this request is.
I've been checking the folder, but I haven't seen any new start dumps.

Thank you.

Stephen

comment:4 Changed 12 years ago by willie

Stephen, I have started getting this data. I have extracted the data from MASS and unpacked it and I am now in the process of transferring it to HPCx. I estimate the transfer will take about 75.6 hours ( it started 13:42 Tuesday 22nd April), so it should complete on Friday at about 6pm, barring problems (and the transfer process is fragile). I will then do some checking so I expect middle of next week. Sorry it is taking so long.

I am storing the data I extract under Met_O_MASS in dumps/weather.

Regards,
Willie

comment:5 follow-up: Changed 12 years ago by willie

Stephen,
Your start dumps are now available in /home/hpcx/n02/n02/umx/dumps/weather/Met_O_MASS/global. I have checked that the files transferred correctly and looked at a couple with xconv. All seems OK.

Let me know how you get on with using them (successes as well as failures!).

Regards,
Willie

comment:6 Changed 12 years ago by umdoc

Willie,

One of the dates is wrong.
I asked for 10 February 2007, and the files in the folder are for 10 February 2008.
Could you produce the ones for 2007 please?

I've been building new ancillary files. I'll test the start dumps over the next couple of days.

Thanks

Stephen

comment:7 in reply to: ↑ 5 Changed 12 years ago by umdoc

I don't have permissions to access the start dumps.
Only the user (umx) has any permissions on them.

Stephen

Replying to willie:

Stephen,
Your start dumps are now available in /home/hpcx/n02/n02/umx/dumps/weather/Met_O_MASS/global. I have checked that the files transferred correctly and looked at a couple with xconv. All seems OK.

Let me know how you get on with using them (successes as well as failures!).

Regards,
Willie

comment:8 Changed 12 years ago by umdoc

Willie,

I've tried using the start dumps from where they are but I keep having a strange problem.
The file is called 20071209_qwqg00.T+0. I put this into the umui as the name of the start dump.
The run then crashes. The leave file says it can't open a file called 20071209_qwqg00
It seems to be ignoring the '.T+0' part of the name.
Is there a limit to the number of charcters that can be used in the start dump names?

I have read access on this one file. So I've copied it to my work space and renamed it 20071209.T+0
I changed the name in the umui and it appears to have successfully opened the file.
However, it still crashed, this time stating 'wrong no of atmos prognostic fields'.
I've compared the start dump to one that I know works, and the only difference appears to be the final field.
In the 20071209.T+0 start dump, there are 75 fields and the last one is 'Stash code = 321'
In the start dump that I know works, there are the same 75 fields apart from the last one which is 'NET DN SW RAD FLUX:OPEN SEA:SEA MEAN'.
I'm not sure if this is important.

I'll keep trying to get this to work, but I thought I'd better tell you what I am trying.

thanks

Stephen

comment:9 Changed 12 years ago by umdoc

Willie,

I've had a talk with Nina, and she has already spoken with you about the difference in size of the start dumps.
When my run fails, it says

' No of prog fields in Atmos dump 1042

No of prog fields expected 992 '

This is a difference of 50. I would have expected this to be a difference of 49, but I suspect that the presence of 'Stash code = 321' instead of the 'NET DN SW RAD FLUX:OPEN SEA:SEA MEAN' is to blame.
I think this difference is what is causing the run to crash.
I'm assuming her start dumps will have the same problem.

Is it possible to get the start dumps re-created with 'NET DN SW RAD FLUX:OPEN SEA:SEA MEAN' instead of 'stash code = 321'?
Perhaps re-create one of them and I will test it?

If I can help or if you need more information, please let me know.

Stephen

comment:10 Changed 12 years ago by willie

Stephen,

Apologies for these problems. In my rush to go on holiday, I forgot the 20070210 data - this is now in progress and I expect it to be ready Friday night. I have now changed the permissions on the start dumps so you should have access to them - mea culpa. Regarding the truncation of filenames, this can be avoided by using the environment variable section in the UMUI. It can be found in Sub Model Independent > File & Directory naming. Introduce a variable called START_DUMPS and point it at the directory. Then in the start dumps page, simply put $START_DUMPS in the directory slot and 20070210_qwqg00.T+0, for example, in the filename slot.

The start dumps are peculiar to the UM model that was extant at the time the forecast was made. The UM is ever evolving. This accounts for differences in the start dumps. I don't think re-creating them is possible, but I might be wrong. I am looking at the wrong number of prognostics variables issue …

Regards
Willie

comment:11 Changed 12 years ago by willie

Stephen, when I have had "wrong number of atmospheric prognostics" in the past, I have switched on reconfiguration to get rid of it. Go to Atmosphere > Ancil and Input Files > Start dump and highlight the "using the reconfiguration" radio button. Hope that helps.
Regards
Willie

comment:12 Changed 12 years ago by umdoc

Using a varibale has fixed the naming issue.
I've tried turning on the reconfiguration, but I get an error message that says it can't open the astart file.
I believe I know why.
This run is a global run designed to create a boundary file for our limited area.
We do not do reconfiguration for these global run.
We use an executable from Changa at /hpcx/devt/n02/n02-ncas/cwang/UM_EXEC/reconf.exe
We had some problems early on with global runs creating boundary conditions.
If I remember correctly, the reconfiguration required a modset that the Met. office had, but that we did not have access to, so Chang created the executable and told us to use that.
Since we've never needed to change the start dumps, we've never needed to do a reconfiguration on the global run.

The job is on PUMA, my username is soutten, and it is job xclvd.
Could you have a look at it and tell me what changes need to be made and specifically if I need any mod sets for the reconfiguration that I don't have.

Hope this makes sense.

Stephen

p.s. Thanks for all your help so far.

comment:13 Changed 12 years ago by willie

Stephen,
I am investigating this further. STASH code 321, it turns out, is a user stash code and should not have appeared in a Met Office dump. We are investigating. In the mean time I am trying to do a reconfiguration which will result in a new start dump which you can use. More later.
Regards
Willie

comment:14 Changed 12 years ago by willie

Stephen,

I now have a solution. Basically, you need to do a reconfiguration step and you will require some user STASH masters, which I have provided (see below). This problem occurs because the Met Office UM is a higher version than the currently released PUM 6.1. We expect this problem to disappear when PUM 6.6 is released in a few months time. I have tried the solution on two different start files and it works. However, please let me know if it fails for any of the others.

Modifications required

Difference in window subindep_Compile

→ Model Selection

→ Sub-Model Independent

→ Compilation and Modifications

→ Compile options for the model

Radio button: Choose option

Entry is set to 'Compile and build the executable named below,

Entry box: Specify the directory for the executable

Entry is set to '$DEVTDIR'

Difference in window subindep_CompRecon

→ Model Selection

→ Sub-Model Independent

→ Compilation and Modifications

→ Modifications for the reconfiguration

Entry box: Specify the directory for the executable

Entry is set to '$DEVTDIR'

Radio button: Choose option

Entry is set to 'Compile and build the executable named below'

Differences in Table mods

add the following
$PUM_MODS61/recon_fix_6.1.mf90 Y

Difference in window subindep_Compile_Mods

→ Model Selection

→ Sub-Model Independent

→ Compilation and Modifications

→ Modifications for the model

Differences in Table Mods

switch off the following
$PS11_MODS/atd6f602.mf77 N
$PS11_MODS/atd6h602.mh N
$PS11_MODS/oper_6.1_HiRes.mh N
$PS11_MODS/um601_qwaterload.mf77 N
$PUM_MODS61/ars9f601.mf77 N

Difference in window atmos_STASH_UserDiags

→ Model Selection

→ Atmosphere

→ STASH

→ User-STASHmaster files. Diags, Progs & Ancills.

Check box: Using user STASHmaster files for the Atmosphere.

Differences in Table Specify the STASHmaster files

add the following lines

/home/umui/interim_6.1_6.6_USTASH/GL_USTASH6.1.1_biogenic_ignore
/home/umui/interim_6.1_6.6_USTASH/USTASH_5_187_from6.6
/home/umui/interim_6.1_6.6_USTASH/USTASH_5_188_from6.6
/home/umui/interim_6.1_6.6_USTASH/USTASH_5_246_from6.6

End of required modifications

comment:15 Changed 12 years ago by umdoc

Willie,

I've used the changes you suggested and, with some adjustments, I got the global job to run successfully.
I'm now having a problem with the limited area run. It says 'Cant find STASH item 321 section 0 model 1 in STASHmaster'.
I'm trying some adjustments to the job to get it to work. Any suggestions would be appreciated. The job is xclve.

Could you please confirm that it is NOT possible the recreate the start dumps with the correct fields?
If it can be done, then could you please recreate the start dumps as soon as possible.
Searching for individual fixes for the jobs is time consuming, especially when this one change should solve all of my problems.
I'm sure other people will also have problems if their start dumps contain a different set of fields.

Obviously if it cannot be done, then I'll need to find ways to get the jobs to work with these new dumps.
I'll wait to hear from you.

Stephen

comment:16 follow-up: Changed 12 years ago by willie

Stephen,

I have now reconfigured all the start dumps to remove problematic fields. I replaced all the files in the dumps/weather/Met_O_MASS/global directory with new files with the same name on 28th May at noon. I hope this does not cause too much confusion.

Regards
Willie

comment:17 in reply to: ↑ 16 Changed 12 years ago by umdoc

Thanks Willie, all these start dumps are now working perfectly with my
old code.
They're much easier to use.

Take care and thanks again

Stephen

Replying to willie:

Stephen,

I have now reconfigured all the start dumps to remove problematic fields. I replaced all the files in the dumps/weather/Met_O_MASS/global directory with new files with the same name on 28th May at noon. I hope this does not cause too much confusion.

Regards
Willie

comment:18 Changed 12 years ago by willie

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.