wiki:UM/SingleColumnModel/ARCHERSCMTechnical

Version 3 (modified by willie, 7 months ago) (diff)

Technical Note on the ARCHER SCM

Rose suite changes

The Rose suite modifications are in u-bf976. The FCM make was modified to include the UM branch and remove FORTRAN flag overrides that were not relevant. The mirror mechanism was added to transfer the code to ARCHER and the platform configuration was set to that specified in the PLATFORM environment variable and the compute host was changed from self to ARCHER. The task graph in the suite configuration file was modified to introduce the "compile on ARCHER" step (second phase of FCM). The following ARCHER modules were set up in the initial script,

  • PrgEnv-intel/17.0.0.098,
  • cray-mpich/7.5.5,
  • cray-netcdf/4.3.2.

The PLATFORM environment variable was set to ncas-x86-ifort (This is not a good choice of name, the original intention was to emulate the Met Office PC setup which has the x86 architecture. Subsequently I switched to the compute nodes which have the Cray XC30 architecture). The environment for building on ARCHER was added.

GCOM

On ARCHER GCOM 6.6, built with the Intel compiler via ftn for the xc30 compute nodes, was used for the NCAS site. The parallel version mpp was used, thus ensuring that the SCM operates in parallel mode. Later versions of GCOM can be used.

The current build of GCOM is incorrectly labelled on ARCHER - the ifort15 version is actually built with the Intel version 17 compiler for the compute nodes.

UM branch

The SCM is part of the UM code and the build configuration is part of the code too.

Some modifications of the UM 11.1 code are required. See the development branch dev/williammcginty/vn11.1_NCAS_SCM_ARCHER. A new set of make configurations for the ARCHER post processor was created in ncas-x86-ifort, based on the Met Office version meto-x86-ifort.

The following FCM make configuration files were modified: under the inc directory, the external paths, libraries, parallel and serial configurations. These are shared with the UM. The remaining modifications were restricted to the SCM configuration files. There is one for each level of build: debug, high, rigorous or safe.

The main change in the libraries, parallel and serial configuration files was to change the Fortran compiler from mpif90 used by the Met Office to the compiler ftn for the compute nodes.

For the SCM configuration the change was to split the extract step into a mirror and extract.

The external paths file was modified as follows,

  • set the library root directory for ARCHER,
  • point to the GCOM parallel build,
  • the GRIB API path is not required and so everything after the equals sign is deleted,
  • the Shared UM library path is defined,
  • The GNU C library (glibc) path to libm is deleted.

The libraries configuration was modified to delete the path referencing the GNU C Library (glibc) libm: the ftn compiler takes care of this if -lm is included in the loader flags. In the serial configuration file, the reference to the GRIB API was removed. Bizarrely, this file includes the parallel configuration file. In the parallel configuration file, the reference to the GRIB API was removed. The MPI C and FORTRAN compilers were replaced by cpp and ftn. The -Werror flag for the C compiler was deleted. The remaining changes were in the location of the following libraries,

  • MPL library was added,
  • the GRIB Library was removed,
  • the GNU C Library version of libm was updated.

New Timings

The resulting suite u-bf976 has been run successfully on ARCHER. It is compiled for debug. The PUMA phase of the FCM build takes about one minute, the ARCHER phase about five minutes. The SCM model itself runs in about two minutes.