Opened 7 years ago

Closed 7 years ago

#1112 closed help (fixed)

Running UKCA v7.3 (job xieph) outputting new diagnostics

Reported by: fcentoni Owned by: luke
Component: UM Model Keywords:
Cc: Platform: HECToR
UM Version: 7.3

Description

Dear Luke (cc Ros),

I made the code changes you suggested in order to output dry deposition resistance terms from the subroutine UKCA_surfddr.F90.

I first called the subroutine asad_output_3D_field within routine mentioned above for both the array r_stom and r_cut_o3.
Then I added the following blocks into the routine asad_flux_dat.F90:

TYPE(asad_flux_defn), DIMENSION(2), PARAMETER :: &

asad_extra_drydep_diags=(/ &

! R stomatal O3

asad_flux_defn('FLD',34364,'X',.FALSE.,0,1,&
(/'r_stom_o3',' '/),&
(/' ',' ',' ',' '/)),&

! R non-stomatal O3

asad_flux_defn('FLD',34365,'X',.FALSE.,0,1,&
(/'r_nonstom_o3',' '/),&
(/' ',' ',' ',' '/)),&
/)

and added 2 diagnostics to the total n_chemical_fluxes at the bottom.
Furthermore I created a new stashmaster file (s34_CheM_STASH_fluxes3D_v7.3_newoutputs) and selected in stash as well as choosing to run with my working copy in FCM options for Atmosphere and Reconfiguration.
Apparently the two new diagnostics turned up in STASH. I then ran the model and it turned out the job was properly compiled by HECTOR.
However the run ended up failing. The error is:

UM ERROR (Model aborting) :

Routine generating error: ASAD_CHEMICAL_DIAGNOSTICS
Error code: 34364
Error message:

STASH CODE NOT AVAILABLE

Have I picked up the wrong STASH CODE?. Browsing the 34 section in STASH, I thought they were available though.

Many thanks.
Federico

Change History (27)

comment:1 Changed 7 years ago by luke

  • Owner changed from um_support to luke
  • Status changed from new to accepted

comment:2 Changed 7 years ago by luke

Hi Federico,

There are a few issues with this job which need to be cleared up, and then hopefully you should be able to run.

1) The first problem is a mistake I made - there are a few errors in the asad_chem_flux_diags.F90 routine I gave you - you should take a new copy from

/home/luke/temp/fcentoni/asad_chem_flux_diags.F90

on PUMA. You can just copy this into your branch's working copy.

2) The second problem is a minor one to do with how the 3D fields are specified. The string which tells the diagnostic code which field is being uploaded needs to be exactly 10 characters in length (including spaces, which should be used to pad out at the end), so your 'r_stom_o3' is 1 character too short, and your 'r_nonstom_o3' is two characters too long. You will need to make both of these (and any future ones) 10 characters in length. The easiest way to check is to see if the ' match up with the line below.

3) You have included the diagnostics in your user-STASHmaster file, but you haven't initialised them (e.g. to zero - option 3) in the 'Initialisation of user prognostics' panel

4) You are currently running from the branch repository, rather than from a working copy. This means that the changes you have implemented in the code have not been picked up by the job. This means that the code knows nothing about the two STASH dignostics you have defined, hence the error. In the 'FCM Options for Atmosphere and Reconfiguration' you will need to select the 'Include modifications from user working copy' radio button and then set the path as

/home/fcentoni/vn7.3_DryDep_Wesely/src

You should copy accross the updated version of asad_chem_flux_diags.F90 first though. This *should* compile, but if you have any problems with the rouine and are not sure how to correct them, please let me know.

Hopefully this is all you should need to do.

Thanks,
Luke

comment:3 Changed 7 years ago by luke

  • Platform changed from <select platform> to HECToR

comment:4 Changed 7 years ago by fcentoni

Hi Luke,

I think I have fixed all the things you mentioned above. Thanks for that.

However I get this compilation error:

MODULE asad_flux_dat


ftn-855 crayftn: ERROR ASAD_FLUX_DAT, File = /home/n02/n02/fcentoni/um/xieph/ummodel/ppsrc/UM/atmosphere/UKCA/asad_flux_dat.f90, Line = 61, Column = 8

The compiler has detected errors in module "ASAD_FLUX_DAT". No module information file will be created for this module.

/)

ftn-1725 crayftn: ERROR ASAD_FLUX_DAT, File = /home/n02/n02/fcentoni/um/xieph/ummodel/ppsrc/UM/atmosphere/UKCA/asad_flux_dat.f90, Line = 447, Column = 8

Unexpected syntax while parsing the type-declaration statement : "operand" was expected but found "/".

I checkout if had missed some '/' or parenthesis within asad_flux_dat.F90 but it looks everything fine to me. I don't know why Hector is not happy with that.

Many thanks.
Federico

comment:5 Changed 7 years ago by luke

Hi Federico,

The problem here is the block

    TYPE(asad_flux_defn), DIMENSION(2), PARAMETER :: &
         asad_extra_drydep_diags=(/ &
  ! R stomatal O3
         asad_flux_defn('FLD',34364,'X',.FALSE.,0,1,&
         (/'r_stoma_o3','          '/),&
         (/'          ','          ','          ','          '/)),& 
  ! R non-stomatal O3
         asad_flux_defn('FLD',34365,'X',.FALSE.,0,1,&
         (/'r_cutic_o3','          '/),&
         (/'          ','          ','          ','          '/)),&          
         /) 

if you look at the last-but-one line, you see it ends '/)),& - this should be '/)) &, i.e. the comma needs to be removed. If you compare this to other blocks you will see they have this. The reason for need for the comma to be removed is that it tells the compiler to expect another array element (i.e., another asad_flux_defn statement), but as this is the last one in the block the comma needs to be removed.

Thanks,
Luke

comment:6 Changed 7 years ago by fcentoni

Hi Luke,

it is still not compiling. Looking at the file xieph000.xieph.d13219.t154008.comp.leave, it seems there is something wrong with the model_levels and ierr variables within the routine ukca_surfddr.F90.

Maybe I have to declare these variables at the top of the subroutine and then I have to read in model_levels along with row_length and rows through:

INTEGER, INTENT(IN) :: row_length,rows,model_levels

and declaring the counter ierr as integer.

Ignore the error r_cut_o3(i,n)(1:row_length,1:rows,1:model_levels), it was my mistake.

Is it correct?

Many thanks.
Federico.

comment:7 Changed 7 years ago by fcentoni

Hi Luke,

I think there may be some variable dimensionality problem.
In the surfddr.F90 subroutine you see:

REAL, DIMENSION(row_length,npft) :: r_cut_o3 ! Cuticular Resistance for O3
REAL, DIMENSION(row_length,rows,npft,5) :: r_stom ! Stomatal resistance

But when I call asad_output_3D_field:

r_stom(1:row_length,1:rows,1:model_levels), &

'r_stoma_o3',ierr)

The same is for r_cut_o3.

Should I have to redeclare r_stom and r_cut_o3 with a different dimensionality?

This is an error which I found out with the last attempt to run the model yesterday night.

Many thanks,
Federico.

comment:8 Changed 7 years ago by luke

Hi Federico,

That's correct. There are two ways you could do this:

1) You could have each PFT be a separate field (i.e. surface fields) for r_cut_o3, and have npft*5 fields for the r_stom array, i.e. for each of the NO2/O3/PAN/SO2/NH3 values. You should be able to call asad_output_3D_field for a 2D field so long as where model_levels is passed in, that is set to 1, and the array is defined as (1:row_length,1:rows,1:1). However, you would also need to ensure that you used 2D user-STASHmasterfiles for these fields in the UMUI/STASH, otherwise there could be problems.

2) The other option is to create a new temporary array of size (1:row_length,1:rows,1:model_levels) and then do something like

  temp_array(:,:,:) = 0.0
  temp_array(1:row_length,1:rows,1:npft) = r_cut_o3

and

  temp_array(:,:,:) = 0.0
  temp_array(1:row_length,1:rows,1:(npft*5)) = RESHAPE(r_stom,(/row_length,rows,npft*5/))

and then pass the temp_array out through the asad_output_3D_field routine. This has the advantage that all your current UMUI/STASH settings are correct. However, you will then need to unpick exactly what the array means at the end, as now each level corresponds to a different PFT or PFT/species combination.

In terms of ease of understanding, option 1 is clearer, but option 2 is the easiest to do. You could also do a third option:

3) Do a half-way house for r_stom, and output the values for each species separately, so do

  temp_array(:,:,:) = 0.0
  temp_array(1:row_length,1:rows,1:npft) = r_stom(:,:,:,1)

etc. over each of the 5 species. This would need some minor changes, but would still be clear.

In STASH as well, you only need to output the number of levels corresponding to the number of PFTs. I wouldn't use DPFTS though, as the STASHmaster file is not set up for this, you should use a variation of DALLTH, but where 'Range ending at' is set to the number of PFTs (5 I think), or 5 times this number for r_stom if you use option 2.

I hope this helps.

Thanks,
Luke

comment:9 Changed 7 years ago by fcentoni

Hi Luke,

thank you for your detailed reply. I am currently queuing to continue to run the model.
I adopted the third option (just for the array r_stom) creating a new domain profile from DALLTH setting the 'Ranging ending at' to 5. In this way, the job was properly compiled.

However, for the array r_cut_o3 it turned out to cause an error.

The r_cut_o3 is bidimensional :

REAL, DIMENSION(row_length,npft) :: r_cut_o3 ! Cuticular Resistance for O3

So is it correct if I do:

temp_array1(1:row_length,1:1,1:npft) = r_cut_o3

and then I use a 2D stashmaster file for the field r_cut_o3?
I suppose if I make a 2D stashmaster file with this field into it, I have to remove it from the 3D stashmaster field.

I will eventually make this changes for O3_cut_O3 with the next run.
Thanks.
Federico

comment:10 Changed 7 years ago by luke

Hi Federico,

You still need to define you array from 1:row_length,1:rows, as this defines the surface (x,y) dimensions. The freedom you have is over the 3rd ("height") dimension. You must pass in an array of size model_levels to the diagnostic routine, but it can be mostly zeros with only the first/lowest npft levels filled.

Thanks,
Luke

comment:11 Changed 7 years ago by fcentoni

Hi Luke,

Hector finally ran the job last night after a long queue but the run failed.
I think the compilation was OK.

The last .leave file is:

xieph000.xieph.d13220.t133312.comp.leave

What was the run failure caused by?

Many thanks.
Federico.

comment:12 Changed 7 years ago by luke

Hi Federico,

It has compiled OK, but it seems to die due to a segmentation fault on timestep 2.

You may be able to use totalview to see where the code died. There is a core file existing in your

  /work/n02/n02/fcentoni/um/xieph

directory. However, it is not from the last time you ran, but from the time before. However, you haven't recompiled in that time, so you may still be able to get useful information.

I can't read the core file (it can only be read by you, currently), so you will have to read it yourself. You should use my tv script to do this, like so

  cd /work/n02/n02/fcentoni/um/xieph
  /home/n02/n02/luke/bin/tv ./bin/x*.exe ./core

This will load Totalview, which should give you some information about the routine where the code died.

However, if this doesn't give any useful information you should try recompiling using the debug settings. To do this you need to

1) Delete the core file in the /work/n02/n02/fcentoni/um/xieph as a new one won't be created if an old one exists

2) In the UMUI, change the compile settings to debug in

  Model Selection
    -> Compilation and Modifications
      -> Compile options for the UM Model

and then set it to compile and sun, and change the level of optimisation to debug

3) You should either delete the

  /home/n02/n02/fcentoni/um/xieph

directory (note, this is on HOME, not WORK) as this contains the object files which need to be rebuilt, or you can go to

  Model Selection
    -> FCM Configuration
      -> FCM Extract and Build Directories and Output levels

and use the radio buttons to select 'Force FULL extract' and 'Force FULL build'. However, if you do this second option you will need to remember to turn it off again, otherwise it will do this everytime you compile and this will take more time.

4) You should also increase the level of output in the .leave/fort6 files, as this may also be helpful. Go to

  Model Selection
    -> Input/Output Control and Resources
      -> Output Choices

and select 'Extra diagnostic messages. You should probably also turn this back to 'Normal informative messages and warnings' once you have debugged your model, as this will make the output in the fort6/.leave files smaller again.

Once you have done this, save, process and submit, and then when a core file is created, view it in Totalview again. If you are still having trouble, you can change the permissions on the file by

  chmod a+r core

which should mean that I can then view it in Totalview.

If you make some code changes and recompile, remember to set the FCM options back (if you have done this from step 3) and delete the core file before sending the job off again.

Also, if you have any problems running the tv script from my bin please let me know.

Thanks,
Luke

comment:13 Changed 7 years ago by fcentoni

Hi Luke,

I followed you instructions but it is still not running.
Apparently, there is something wrong with the logical setting for surface types within the subroutine ukca_surfddr.

It is definitely quite dodgy why the code changes I made end up affecting the model like this.

Many thanks.
Federico.

comment:14 Changed 7 years ago by luke

Hi Federico,

I'm not quite sure what you mean about the logical setting for the surface types. Is this output from Totalview?

Could you make your core file readable so I can take a look?

Also, I'm going on leave until Tuesday next week, so I will probably not be able to reply until then I'm afraid.

Thanks,
Luke

comment:15 Changed 7 years ago by fcentoni

Hi Luke,

yes it is the output from totalview.

Within the this part of code :

! Set logical for surface types
{{{DO n = 1, ntype

DO k = 1, rows

DO i = 1, row_length

todo(i,k,n) = (gsf(i,k,n) > 0.0)

END DO

END DO

END DO}}}

the line todo(i,k,n) = (gsf(i,k,n) > 0.0) turns out to be highlighted in yellow.

I ran the commmand chmod a+r more so you should be able to have a look at 'core' file.

Many thanks again.
F.

comment:16 Changed 7 years ago by fcentoni

Hi,

I am trying to recompile the same job having made some code changes on it.
But the compilation ended up failing twice with the same compilation error (the last .comp.leave file is xieph000.xieph.d13224.t132232.comp.leave)

The error message is:

Build command finished on Mon Aug 12 12:24:24 2013.
Base build: OK
ERROR: /home/n02/n02/fcentoni/um/xieph/ummodel/fcm.bld.lock: lock file exists,

/home/n02/n02/fcentoni/um/xieph/ummodel: destination is busy.

Build failed on Mon Aug 12 12:24:27 2013.
Build command started on Mon Aug 12 12:24:24 2013.

May you help me out with that?

Many thanks,
Federico.

comment:17 Changed 7 years ago by luke

Hi Federico,

This means that, at some point, compilation was finished abruptly, either by a qdel command or because it ran out of time. To be able to compile again you will need to delete the lock file. If this file exists fcm will not run the compilation step. So you should

  rm /home/n02/n02/fcentoni/um/xieph/ummodel/fcm.bld.lock

Also, remember to delete the core file in your /work/n02/n02/fcentoni/um/xieph directory, otherwise a new one will not be created.

Thanks,
Luke

comment:18 Changed 7 years ago by fcentoni

Hi Luke,

I managed to compile the job now but the 'PE RANK 60 exit signal Segmentation fault' error occurred again…

What may it be the cause of this trouble?

I made the last code changes on the ukca_aerod.F90 in order to output Ra and Rb and on ukca_ddcalc.F90 to output Vd. I did not included the code changes I had previously made for r_stom and r_cut_o3 in ukca_surfddr.F90. This in order to find out whether the trouble causing the 'segmentation fault' was triggered by these first code changes or not.

In addition, how do I manage to detect which subroutines the arrays associated to the following diagnostics are outputted from?:

'Aerodynamic resistance' STASHCODE = 03054
'Resist_B' STASHCODE=03063
'Canopy conductance' STASHCODE=03259
'Stomatal conductance on PFTS' STASHCODE=03462

Many thanks.
Federico.


comment:19 Changed 7 years ago by luke

Hi Federico,

I can see from your branch that you are now passing in model_levels into

src/atmosphere/UKCA/ukca_ddcalc.F90
src/atmosphere/UKCA/ukca_surfddr.F90
src/atmosphere/UKCA/ukca_aerod.F90

as well as some other changes. However, you have not made changes to the routine that call these, namely src/atmosphere/UKCA/ukca_ddepctl.F90. You will need to pass in model_levels into these routines from ukca_ddeptctl. I suspect that this is the reason for this crash. It may also have been causing your other problems.

Thanks,
Luke

comment:20 Changed 7 years ago by fcentoni

Hi Luke (cc Ros),

I managed to fix up all the code errors, letting the model outputting the information I want to look into.
However, I still have a doubt about the concentration values which are actually used by the model, in order to calculate O3 dry deposition fluxes, which I want the model to output as well.

Is it the subroutine asad_cdrive.f90 the one I have to consider?
For the same purpose, considering the diagnostics currently available on STASH, I was wondering weather the concentration values (stash codes 34001 or 34353) correspond to the array I am interested in or not.

Finally, I tried to test the model with a short 2-days run activating the interactive vegetation distribution. It turned that the model ran but it did not produce any output file anymore. So I was wondering, is the coupling with land surface model working in the UKCA version I am currently running with?
In case it worked, is the LAI index supposed to vary?

I do not know if I had missed setting up something.

Many thanks.
Federico

comment:21 Changed 7 years ago by luke

Hi Federico,

I'm glad that you have managed to get the model to compile and run!

Both 34001 and 34353 should be the same (although there may be slight differences due to the time when the field is output), however, 34353 is a tropospheric diagnostic (i.e. the O3 concentration in the troposphere calculated every timestep) whereas 34001 is the prognostic UKCA O3 tracer field. For your calculations I would use 34001. 34353 is there if you want to calculate the tropospheric burden easily (in fact I should probably have written this so it outputted the burden and not the concentration). It should be noted that 34001 is held as mass-mixing ratio in the UM and so is output as this, but UKCA converts this to volume-mixing ratio for the solver. This is done in UKCA_CHEMISTRY_CTL when ZFTR is calculated from the ALL_TRACERS array.

ASAD_CDRIVE is the main ASAD routine in that it calls all the other ASAD routines. This is passed the ZDRYRT2 array (a reshape of ZDRYRT) from UKCA_CHEMISTRY_CTL, where ZDRYRT is calculated in either UKCA_DDEPCTL or UKCA_DDEPRT (depending on if you are using the 2D or interactive scheme). This is held as DRYRT in ASAD_CDRIVE which is then passed to UKCA_DRYDEP which just copies this into the DPD array, which is then used by the solver as loss.

Which output file do you mean? You xieph job is set to do 10-day dumps (the standard setting for climate jobs) and so after 2 days would not have created any of the xiepha.da* files. You should have created some xiepha.pb* files though.

I don't know enough about the land-surface scheme to know what would be changing or not on these timescales, or how easy it would be to see this.

Thanks,
Luke

comment:22 Changed 7 years ago by fcentoni

Hi Luke,

thank you very much for your always professional and effective help!

In terms of output files, I meant xieph.g* and xieph.h* files.

In a previous run, I had set up the dump file time frequency as 12 hours and I tried to run the model for 2-days switching on the 'interactive vegetation distribution' within the vegetation section. This in order to check out it may affect the O3 cuticular resistance or O3 stomatal resistance (which depends on LAI) instead of using values taken by an ancillary file (as for 'fixed vegetation distribution').

The model turned to properly compile and run but instead of UPG and UPH output files which I had previously got with another 2-days test run, the model only came out with xieph.fort6.pe* files and that was it.
The compilation file relative to that run should be xieph000.xieph.d13238.t165750.comp.leave. Unfortunately, I deleted the folder containing the files mentioned above.

Do you know who I may speak to in order to understand how to properly let model working coupled with the land-surface scheme on (MOSES II)?
Maybe Garry Hayman?

Many thanks.
Federico.

comment:23 Changed 7 years ago by luke

Hi Federico,

Are you meaning when running the xieph job? In your

/work/n02/n02/fcentoni/um/xieph/archive

directory you have 719 pg and ph files, with one of each in the directory above. This would correspond to 30 days of run though, rather than 2. All of these were made yesterday.

I'm not sure who would be best. Garry may be able to point you to the right person (if it isn't him).

Thanks,
Luke

comment:24 Changed 7 years ago by luke

Hi Federico,

Is this working for you now? If so, I will close this ticket.

Thanks,
Luke

comment:25 Changed 7 years ago by fcentoni

Hi Luke,

yes it is. I ran the model with a hourly temporal resolution (for a whole year) including all the changes I made on the code.

Many thanks for your help.
Federico.

comment:26 Changed 7 years ago by luke

Hi Federico,

That's great. I'll close this ticket now, and if you have any more problems, please open a new ticket.

Thanks,
Luke

comment:27 Changed 7 years ago by luke

  • Resolution set to fixed
  • Status changed from accepted to closed
Note: See TracTickets for help on using tickets.