Changes between Version 11 and Version 12 of Projects/OpenIFS-IO


Ignore:
Timestamp:
27/09/13 15:02:11 (7 years ago)
Author:
grenville
Comment:

—

Legend:

Unmodified
Added
Removed
Modified

  • Projects/OpenIFS-IO

    v11 v12  
    4343=== Setting the HECToR environment === 
    4444 
    45 You will need to set up the correct environment to build and run OIFS jobs. Since HECToR has different hardware on its compute and service nodes, there is an extra consideration when using some OIFS-related utilities on the service nodes (more later). Setting the environment simple involves loading the appropriate environment module; at the HECToR command line, type 
     45You will need to set up the correct environment to build and run OIFS jobs. Since HECToR has different hardware on its compute and service nodes, there is an extra consideration when using some OIFS-related utilities on the service nodes (more later). Setting the environment simply involves loading the appropriate environment module; at the HECToR command line, type 
    4646 
    4747{{{ 
    … …  
    7272}}} 
    7373 
    74 If your normal module set up is with the gnu or pgi programming environments, then an attempt to load openifs_cce/0.0.2 will fail, and it will also fail unless you have the Cray cce1/8.1.8 compiler loaded. Use '''module swap''' to ensure an appropriate starting module configuration. 
     74If your normal environment is not set up with the cray programming environment, then an attempt to load openifs_cce/0.0.2 will fail, and it will also fail unless you have the Cray cce1/8.1.8 compiler loaded. Use '''module swap''' to ensure an appropriate starting module configuration. 
     75 
     76We have created two other modules required for OpenIFS, shown below: 
     77 
     78{{{ 
     79module show grib_api_cce/0.0.2 
     80------------------------------------------------------------------- 
     81/usr/local/packages/oifs/modules/grib_api_cce/0.0.2: 
     82 
     83conflict         PrgEnv-gnu  
     84conflict         PrgEnv-pgi  
     85conflict         PrgEnv-pathscale  
     86prereq   cce/8.1.8  
     87 
     88        ECMWF library GRIB_API on HECToR Phase 3 
     89 Open IFS support libraries installed at /work/y07/y07/oifs/install 
     90 Platform is cce_8.1.8-il 
     91 Version of library GRIB_API is 1.9.18 
     92setenv           GRIB_API_PATH /work/y07/y07/oifs/install/grib_api/1.9.18/cce_8.1.8-il  
     93prepend-path     PATH /work/y07/y07/oifs/install/grib_api/1.9.18/cce_8.1.8-il/bin  
     94prepend-path     LD_LIBRARY_PATH /work/y07/y07/oifs/install/grib_api/1.9.18/cce_8.1.8-il/lib  
     95setenv           GRIB_DEFINITION_PATH /work/y07/y07/oifs/install/grib_api/1.9.18/cce_8.1.8-il/share/grib_api/definitions  
     96setenv           GRIB_SAMPLES_PATH /work/y07/y07/oifs/install/grib_api/1.9.18/cce_8.1.8-il/share/grib_api/ifs_samples/grib1_mlgrib2  
     97module-whatis    Support library for ECMWF software : GRIB_API on HECToR Phase 3  
     98------------------------------------------------------------------- 
     99}}} 
     100 
     101and 
     102 
     103{{{ 
     104module show fdb_cce/0.0.2 
     105------------------------------------------------------------------- 
     106/usr/local/packages/oifs/modules/fdb_cce/0.0.2: 
     107 
     108conflict         PrgEnv-gnu  
     109conflict         PrgEnv-pgi  
     110conflict         PrgEnv-pathscale  
     111prereq   cce/8.1.8  
     112prereq   grib_api_cce/0.0.2  
     113 
     114        ECMWF library FDB on HECToR Phase 3 
     115 Open IFS support libraries installed at /work/y07/y07/oifs/install 
     116 Platform is cce_8.1.8-il 
     117 Version of library FDB is 5.0.0 
     118 Version of library ECKIT is 0.3.0 
     119setenv           ECKIT_PATH /work/y07/y07/oifs/install/eckit/0.3.0/cce_8.1.8-il  
     120prepend-path     LD_LIBRARY_PATH /work/y07/y07/oifs/install/eckit/0.3.0/cce_8.1.8-il/lib  
     121setenv           FDB_PATH /work/y07/y07/oifs/install/fdb/5.0.0/cce_8.1.8-il  
     122prepend-path     PATH /work/y07/y07/oifs/install/fdb/5.0.0/cce_8.1.8-il/bin  
     123prepend-path     LD_LIBRARY_PATH /work/y07/y07/oifs/install/fdb/5.0.0/cce_8.1.8-il/lib  
     124module-whatis    ECMWF library FDB on HECToR Phase 3  
     125------------------------------------------------------------------ 
     126}}} 
    75127 
    76128=== Check out the OIFS code === 
    … …  
    88140=== Build the OIFS executable === 
    89141 
    90 After executing this command on HECToR, you will have a complete copy of the code in your /home space. A set of configuration files is provided in the oifs/make sub-directory for several possible compilers and options. When you loaded the openifs_cce/0.0.2 module, you set several environment variables which determine the particular configuration files to choose. In the make directory type 
     142After executing this command on HECToR, you will have a complete copy of the code in your /home space. A set of configuration files is provided in the oifs/make sub-directory for several possible compilers and options. When you loaded the openifs_cce/0.0.2 module, you set several environment variables (namely, OIFS_ARCH, OIFS_COMP, and OIFS_BUILD) which determine the particular configuration files to choose. In the make directory type 
    91143 
    92144{{{ 
    … …  
    100152=== Getting data to run the model === 
    101153 
    102 Now you have an executable, you will need some data. The source for data is Glenn Carver at ECMWF; Glenn can provide start data for configurations at several resolutions and for various times. We have several jobs available for model testing at t159, t511 and t1279 resolution. Each job comes with data required to start the model, a namelist defining a set of model parameters, a PBS script (called oifs_job) set up for a "standard" run and an example trace output file. We will give a description of the important parts of the PBS script used to submit jobs to the compute nodes. 
    103  
    104 The file has the usual PBS directives for reserving HPC resource, in this case we have requested 1024 processors with fully populated nodes, for a time limit of 25 minutes. You should edit this to specify your account and email details and change the HPC resource request as appropriate. 
     154Now you have an executable, you will need some data. The source for data is Glenn Carver at ECMWF; Glenn can provide start data for configurations at several resolutions and for various times. We have several jobs available for model testing at t159, t511 and t1279 resolution. Each job comes with data required to start the model, a namelist defining a set of model parameters, a PBS script (called oifs.job) set up for a "standard" run and an example trace output file. We will give a description of the important parts of the PBS script used to submit jobs to the compute nodes. 
     155 
     156=== Running the model === 
     157 
     158The file has the usual PBS directives for reserving HPC resource, in this case we have requested 1024 processors with fully populated nodes, for a time limit of 25 minutes. You should edit this to specify your HECToR account and email details and change the HPC resource request as appropriate. 
     159 
    105160{{{ 
    106161#!/bin/bash --login 
    … …  
    138193 
    139194Currently, you will need to set OIFS_HOME to your oifs code tree 
     195 
    140196{{{ 
    141197 
    … …  
    164220NAMELIST=ecmwf/$EXPID.namelistfc 
    165221 
    166 /work/n02/n02/emgdc/oifs/ifs_run.sh -m $MASTER -r 511 -e $EXPID -s $TSTEP -n $NPROC \ 
     222$OIFS_HOME/make/cfg/ifs_run.sh -m $MASTER -r 511 -e $EXPID -s $TSTEP -n $NPROC \ 
    167223                                    -t $NTHREADS -f d1 -x $RUN -l $NAMELIST 
    168224