Posts in category news

HPC News (ARCHER, ARCHER2, and NEXCS)

  • ARCHER will end on Jan 27th - please note, this means no access whatsoever subsequently
  • NEXCS will now continue to late June - this represents a five month extension of the service in light of delays with ARCHER2

CMS Support over the Christmas Period

In line with university Christmas closure periods, the CMS team will be unavailable from 19th December 2020 until 4th January 2021.

We wish you all a Merry Christmas and Happy New Year.

Latest CF data tool releases with a new paper for cfdm

A set of new releases is now available of four of our CF data tool libraries. These versions are inter-compatible and we encourage users to upgrade to these latest and greatest libraries for new features and bug fixes:

Improvements made relative to previous versions can be viewed in the change logs, as linked above. Any further information and guidance is contained elsewhere in the respective documentation, including details on how to install the libraries:

Additionally, we are pleased to announce that a paper focusing on cfdm as a software library has just been published in the Journal of Open Source Software. The paper is open access and available to view here.

UM Version 11.7 released

The latest version of the UM, 11.7, is available on ARCHER. Those of you who are spending your last kAUs on this machine may wish to check it out and incorporate it into your suites.

Update: ARCHER Transition - Modifying PPTransfer

ARCHER has now been configured to allow outgoing connections from the post-processing nodes. This means that it is possible again to transfer data to JASMIN by pushing from ARCHER in the same way that we did from the RDF.

UM suites need to be modified to stage data for archiving on the /work disk and configured to run the pptransfer task in the ARCHER serial queue. You will also need to setup ssh-agent on both the post-processing nodes espp1 & espp2 to connect to JASMIN.

Full instructions on how to modify a UM suite can be found on our website

Update: ARCHER return to service - UM work flow

We continue to work with ARCHER to implement a robust solution to handle the 2FA access. Until we have that solution, this is a short-term alternative. In your PUMA or pumatest .ssh/config file (create one if you don't already have one), delete references to login.archer.ac.uk and then add:

Host login.archer.ac.uk
User <your ARCHER username>
IdentityFile ~/.ssh/<your private key to ARCHER>
ControlMaster auto
ControlPath /tmp/ssh-socket-%r@%h-%p
ControlPersist yes

Login to ARCHER (with passphrase and password).

Rose/Cylc suites should not use rose host-select - it will not work; suites must specify login.archer.ac.uk in the appropriate .rc file (suite.rc or archer.rc), for example, host = login.archer.ac.uk

Submit UMUI and Rose/Cylc jobs as usual.

The connection to ARCHER will persist after logging out of PUMA/pumatest and will enable the Cylc engine to manage your suite. However, the connection will be terminated (each day at ~5AM in our experience) - logging in to ARCHER will re-establish the connection and Rose/Cylc suites will pick up automatically. If a suite times out (over a weekend possibly), a normal rose suite-restart should get it going again.

ARCHER return to service - UM work flow

When ARCHER returns to operation on May 21st all users will be required to use two credentials to access the service: an SSH key with a passphrase and their ARCHER password.

Rose/Cylc suites and UMUI jobs will not run under this HPC access model. We are working closely with ARCHER to develop and implement a solution that will enable UM work flows to function with the new security scheme.

ARCHER Transition - Modifying PPTransfer

From February 19th the ARCHER RDF will become read-only. This means that to continue archiving and transferring data to JASMIN UM suites will need to be modified to stage data for archiving on the /work disk as well as modifying the pptransfer app to pull the data across from JASMIN rather than using the ARCHER data transfer nodes.

Full instructions on how to modify a UM suite can be found on our website

Single Column Model available in UM VM

The SCM can be run inside the Met Office's UM Virtual Machine. See SCMVM for instructions.

New cf-python release (3.1.0)

We have recently released a new minor version of cf-python, our earth science data analysis library built on a complete implementation of the CF data model.

Improvements made for this latest version, 3.1.0, can be viewed from the official change log (https://github.com/NCAS-CMS/cf-python/blob/master/Changelog.rst#version-310). Note cf-python v.3 is built with, & run using, Python 3.

cf-python 3.1.0 is available to download & view on PyPI & GitHub; instructions for installing it can be found on a dedicated installation page in the official documentation:

CMS Support over Christmas Period

In line with university Christmas closure periods, the CMS team will be unavailable from 21st December 2019 until 2nd January 2020. However, there will also be minimal support on Friday 20th December and after this period until Monday 6th January due to limited availability of staff.

We wish you all a Merry Christmas and Happy New Year.

New versions of cf-python and cf-plot for Python 3

As you probably know, Python 2 is being frozen at the end of this year - so Python 3 is the future!

We are happy to announce that the Python 3 versions of cf-python (v3.0.1) and cf-plot (v3.0.3) are now available for Linux and Mac.

The new cf-plot documentation is in its usual place (https://ajheaps.github.io/cf-plot), but the new cf-python documentation has moved to a new home: https://ncas-cms.github.io/cf-python

These new versions are only Python 3-only, and we would encourage new users to start with Python 3. All future training courses will be run using Python 3.

The Python 2 code is still available, of course, and will be fully supported for a limited period - but all new features from now on will only go into the Python 3 libraries.

Both the Python 2 and Python 3 versions of cf-python and cf-plot are already installed on

  • Jasmin@CEDA,
  • Archer
  • Archer Jasmin RDF
  • Reading University RACC cluster

See http://ajheaps.github.io/cf-plot/download.html for access instructions, as well as instructions for personal installations.

We hope that you enjoy using the new versions, and welcome feedback of any nature.

UM Introduction Course (6-8 November 2019)

NCAS-CMS will be delivering a 3-day course introducing new users to the Met Office Unified Model with the Rose/cylc system. The course is a mix of practical exercises and informal lectures covering a range of topics including running suites, code management, file formats, and troubleshooting common problems.

The course will be held at the University of Reading from 6-8 November 2019. There is no fee for attending the course, and lunch will be provided. However, attendees are responsible for their own travel, accommodation, and other subsistence costs.

Material from previous course can be found here:

http://cms.ncas.ac.uk/wiki/UmTraining

For further details, including how to apply, see the NCAS training page:

https://www.ncas.ac.uk/en/introduction-to-unified-model-course

UM 11.3

The Met Office have released version 11.3 of the UM. It's installed on ARCHER and MONSooN, so feel free to upgrade any suites as may be relevant. The list of CMS supported versions is here.

UKESM1 Released

UKESM1 is the first release of the UK Earth System Model. Built as a joint venture by the Met Office Hadley Centre and the Natural Environment Research Council (NERC), UKESM1 consists of the HadGEM3 coupled physical climate model plus additional components that model key biogeochemical, chemistry, aerosol and vegetation processes.

Two model configurations are now available for use following the CMIP6 pre-industrial and historical (1850-2014) experiment protocols. Instructions are also provided on how to run the model following either the CMIP6 1% transient CO2 or abrupt 4xCO2 protocols. An atmosphere-land only (AMIP) configuration of UKESM1 will also be released shortly.

Researchers wishing to use UKESM1 should read the instructions provided on how to access and configure the model to run on either NERC or Met Office computer systems.

For more information on the UKESM project please go to: https://ukesm.ac.uk/

UM Introduction Course (3-5 April 2019)

NCAS-CMS will be delivering a 3-day course introducing new users to the Met Office Unified Model with the Rose/cylc system. The course is a mix of practical exercises and informal lectures covering a range of topics including running suites, code management, file formats, and troubleshooting common problems.

The course will be held at the University of Reading from 3-5 April 2019. There is no fee for attending the course, and lunch will be provided. However, attendees are responsible for their own travel, accommodation, and other subsistence costs.

Material from previous course can be found here:

http://cms.ncas.ac.uk/wiki/UmTraining

For further details, including how to apply, see the NCAS training page:

https://www.ncas.ac.uk/en/introduction-to-unified-model-course

CMS Christmas Closure dates 2018

The CMS team will be unavailable from 22 December 2018 until 3 January 2019, due to University closure over the holiday period. We will deal with any helpdesk queries or other issues on our return. We wish you all an enjoyable holiday!

UM Introduction Course (7-9 November 2018)

NCAS-CMS will be delivering a 3-day course introducing new users to the Met Office Unified Model with the Rose/cylc system. The course is a mix of practical exercises and informal lectures covering a range of topics including running suites, code management, file formats, and troubleshooting common problems.

The course will be held at the University of Reading from 7-9 November 2018. There is no fee for attending the course, and lunch will be provided. However, attendees are responsible for their own travel, accommodation, and other subsistence costs.

Material from previous course can be found here:

http://cms.ncas.ac.uk/wiki/UmTraining

For further details, including how to apply, see the NCAS training page:

https://www.ncas.ac.uk/en/introduction-to-unified-model-course

Post-processing for ARCHER/NEXCS plus transfer to JASMIN released

Modifications to the Met Office post-processing app to enable archiving on ARCHER/NEXCS and automatic transfer of data from the RDF/NEXCS to JASMIN have now been incorporated into the Met Office trunk and released as part of postproc version postproc_2.2.

Instructions on how to upgrade the postprocessing in your Rose suites is available on the CMS website: Post-Processing App

CMS closure dates

The CMS team will be unavailable from 22 December 2017 until 2 January 2018, due to University closure over the holiday period. We will deal with any helpdesk queries or other issues on our return. We wish you all an enjoyable holiday!

NCAS Data Tools first training course on 21st September 2017

The first training course for 20 people on the NCAS data tools cf-python and cf-plot was held at Reading University on the 21st September 2017. Another training course to be held on 2nd October is already full. If you are interested in attending a one day course please register your interest by emailing Andy Heaps andy.heaps at ncas.ac.uk

The cf-python and cf-plot Python packages build on the CF metadata conventions to provide powerful extraction, regridding and visualization capabilities of netCDF, Met Office PP and fields files. Operations such as reading, grid transformation and plotting of contours and vectors can be performed in a single line making for quick analysis and plotting of data.

Further information on the tools and capabilities can be found at: https://cfpython.bitbucket.io http://ajheaps.github.io/cf-plot http://ajheaps.github.io/cf-view

training class at Reaing University

Rose/Cylc UM conversion course (5th May 2017)

NCAS-CMS will be running a 1-day Rose/Cylc UM conversion course on Friday 5th May at Reading University. Rose & Cylc are the replacement user interface and job submission/control system for the UMUI for UM version 10.x. The course is aimed at those already familiar with running the UM under the UMUI and will not cover how the UM itself works. (See our UM Training course page for details of other courses we run: http://cms.ncas.ac.uk/wiki/UmTraining). The majority of the course will be hands-on exercises supplemented with lectures.

Course contents:

  • Overview of Rose/Cylc
  • Creating & managing suites
  • How to configure, build and run a UM application
  • Examining Rose suite output
  • Common pitfalls and troubleshooting

Please only attend this course if you are planning on using Rose within the next few months. We will be running this course again later in the year, subject to demand.

We still have a few spaces left. To register for this course please email: cms-support@… including a brief summary of what model configurations/versions you intend to be working with.

Course attendees are responsible for paying their own travel costs.

CF Checker 3.0.0 Released

A new version of the CF Checker has been released and is now available for download from Github (https://github.com/cedadev/cf-checker/releases). A distribution will also be available on PyPI soon.

This is a major update to remove the dependency on CDAT; moving to use netcdf4-python instead.

UM Introduction Course (29-31 March 2017)

NCAS-CMS will be delivering a 3-day course introducing new users to the Met Office Unified Model with the new Rose/cylc system. The course is a mix of practical exercises and informal lectures covering a range of topics including running suites, code management, file formats and troubleshooting common problems.

The course will be held at the University of Reading from 29-31 March 2017. There is no fee for attending the course, and lunch will be provided. However attendees are responsible for their own travel, accommodation and other subsistence costs.

For further details and a link to the registration form see:

https://www.ncas.ac.uk/index.php/en/introduction-to-unified-model-course

Please note, this course is intended for new UM users only. CMS will also be running a 1 day Rose/cylc conversion course for users familiar with running the UM through the UMUI. Details of this will follow soon.

UM Training November 2016

We recently ran the first, this academic year, of our popular 3 day UM Introduction courses. The course was attended by researchers from around the UK, as well as 3 participants from the US.

For the first time the course focused entirely on running the UM through the Rose/cylc interface. Attendees worked through a series of exercises designed to help users become familiar with the UM software, including how to diagnose common problems. We also had 2 very interesting science talks from Carol Halliwell and Sylvia Bohnenstengel of the Met Office.

The course was very well received with everyone who responded to our request for feedback (11 out of 19 participants) rating the lectures, practicals and facilities good or excellent.

The next course will run 29-31 March 2017, with registration opening in January 2017. For more details see: http://cms.ncas.ac.uk/wiki/UmTraining

Mule utilities available on ARCHER

The Met Office recently released version 1.3 of Mule. Mule is a Python API for reading & writing UM format files, and a set of utilities to replace the UM executables such as cumf and pumf.

For more information on Mule, see the user guide: https://code.metoffice.gov.uk/doc/um/index.html. (Note you will need a MOSRS account to view this page.)

Mule is available on ARCHER for the serial and login nodes. To use, a one-off setup step is required to add the Mule library to your Python path. For instructions see: wiki:ToolsAndUtilities/mule

GA7 Optimizations

Work undertaken to optimise Pier Luigi Vidale's N512 GA7 runs by NCAS-CMS has resulted in a 15-20% speed up of the model with no loss of bit-comparison. The savings in ARCHER resource resulting from this effort are very significant - at ARCHER partner rate this is ~£60k, and twice that at the non-partner rate. Another way of looking at this is that we save ~100M AU (for these experiments) which enables a raft of ARCHER projects which may struggle otherwise.

Details follow:

In the standard GA7, stochastic physics is turned on. This holds a field in spectral space distributed over the PEs. Every timestep this field is gathered by PE0, converted to grid space and then distributed over the domain. This requires a gather and many scatters at everytime timestep. A month long N512 high resolution job was examined with drhoook and it was noted that FOR_PATTERN, the routine which does the gather/scatter and spectral→grid space transformation takes a significant proportion of the run time, 513s for a 3000s run.

FOR_PATTERN has been rewritten to remove all gathers and scatters of the spectral field. Instead of having the spectral rows distributed over all PEs, each PE holds the spectral rows equivalent to its own rows in grid space. It then does a local fourier transform to get back to grid space, and then extracts the its own longitude domain from the resultant field.

This requires extra compute as every PE over a latitude band does the same spectral→grid space transformation, but the savings communication times can far outweigh this. Gather/scatters when running on a large number (1000s) of PEs are best avoided as these can take many milliseconds to perform.

With it new code FOR_PATTERN takes 23s, compared with 513s.

Two test jobs have been run. A high resolution N512 and an AMIP N96 configuration. For both there is full bit compariability with the previous version.

The N512 GA7 is now being run on Archer with this branch. The speed increase, from Pier Luigi Vidale:

So, for two domain decompositions and for a 2-month dump at N512: 48x48: ~7 hours, but a few times as short as 6hrs40mins (down from ~8 hours) 48x72: 4hrs50m to 5hrs02m (down from ~6hours)

So a ~15%-20% speed up

For the AMIP GA7 runs, there was no discernable speed change, but this isn't surprising as the gather/scatter would be much faster whilst running on a low number of PEs

UM Training April 2016

We recently delivered the second of our bi-annual 3-day UM Training courses. This time we hosted 16 attendees from a wide range institutions from across the UK. Three days of hard work running and breaking UM jobs on ARCHER with specialist presentations from MO, and NCAS experts. This was our final UMUI-centric training course, next time Rose/Cylc will be the main focus. Many thanks to those attending and presenting.

Course details and presentations are available here.

The course photo sees us hard at work:

Training Course: An Introduction to the Unified Model

Dates: 13-15th April 2016

Location: University of Reading

Applications for this course are now open, please apply for places through the application form

This course, run by NCAS-CMS, introduces new users to the Met Office Unified Model systems and provides practical experience of setting up and running experiments. It comprises a mix of lecture-style delivery covering a range of topics, including UM overview, Running the UM, UM file formats, post processing tools, etc and a significant practical component with exercises run on ARCHER to introduce new UM users to HPC and to help users past the initial steep learning usually involved with UM modelling.

Closing date for applications: 5th April

Accommodation, subsistence costs and reasonable travel within the UK will be funded for eligible students and scientists.

MONC supersedes LEM

The Met Office/NERC Cloud Model (MONC) is a large eddy simulation. It's predecessor was the Met Office Large Eddy Model (LEM). MONC includes all the LEM science, but is written in modern Fortran, using modern programming techniques. It is much faster than the LEM, much more scalable and can deal with larger domains.

MONC can be run on a variety of computers, for example Linux boxes, MONSooN and ARCHER.

MONC has a BSD license and can be accessed at the Met Office Science Repository Service.

UM Training December 2015

December saw the successful delivery of one of our bi-annual 3-day UM Training courses. This time we hosted 25 attendees from a wide range institutions from across the UK. Three days of hard work running and breaking UM jobs on ARCHER with specialist presentations from EPCC, MO, and NCAS experts left us all ready for the Christmas break. Many thanks to those attending and presenting.

Course details and presentations are available here.

Those of us feeling photogenic appear in the course photo:

UM Training attendees

Large Eddy Model

The Met Office Large Eddy Model code is now available to NERC users with a PUMA account.

Two versions are available. One runs on single processor machines, the other on ARCHER. For details see The Met Office Large Eddy Model.

Performance analysis and Optimisation of the Met Unified Model on a Cray XC30

Here we present results from the optimisation work carried out by the UK National Centre for Atmospheric Science (NCAS) for a high resolution configuration (N512) on the UK ARCHER supercomputer, a Cray XC-30. On ARCHER, we use Cray Performance Analysis Tools (CrayPAT) to analyse the performance of UM and then Cray Reveal to identify and parallelise serial loops using OpenMP directives. We compare performance of the optimised version at a range of scales, and with a range of optimisations, including altered MPI rank placement, and addition of OpenMP directives.

Article available at

http://arxiv.org/abs/1511.03885

CAP9.0 Installed on ARCHER

The Central Ancillary Program (CAP) version ANCIL9.0 has now been installed on ARCHER. You can find the executables and scripts at $UMDIR/CAP9.0/build/bin. The scientific and technical guides can be found on the Met Office Science Repository.

As always, report any issues with this software to the CMS help desk.

New Xconv version (1.93) released

A new version of xconv (1.93) has been released and the on-line documentation has been updated. Xconv documentation can be found here

http://cms.ncas.ac.uk/documents/xconv

Xconv 1.93 executables are available for the following platforms

  • Linux x86 64 bit
  • OS-X x86 32 bit
  • OS-X x86 64 bit

and can be downloaded from here

http://cms.ncas.ac.uk/documents/xconv/download.html

[resolved] Network problems between Reading and ARCHER

The JANET engineers have identified an network issue and put a fix in place. Please let us know if you see any further network stalling between puma and Archer.

[updated] Network problems between Reading and ARCHER

JANET engineers are now investigating the cause of the slow connection between Reading and Archer.

(Read more)

Training Course: Introduction to Unified Model

Dates: December 16-18 2015

Location: University of Reading

Applications for this course are now open, please apply for places through the application form

This course, run by NCAS-CMS, introduces new users to the Met Office Unified Model systems and provides practical experience of setting up and running experiments. UM software management system, file formats, utilities, and configurations are discussed. Users will be led through a series of exercises designed to encourage best practice in scientific exploitation and HPC resource management of the UM.

The Met Office Unified Model is the model used for weather and climate prediction at the Met Office and also at other forecasting centres worldwide.

Accommodation, subsistence costs and reasonable travel within the UK will be funded for eligible students and scientists.

Network problems between Reading and ARCHER

We are currently experiencing a very slow network between Reading and ARCHER. This is affecting connections between PUMA and ARCHER, including submissions of UM jobs which may be very slow or fail.

ARCHER and Reading IT are investigating this issue.

It might be that this problem is a more general one between us in the south and ARCHER in the north as a similar intermittent slow network connection to ARCHER has been reported from Exeter, Oxford and London. It’s a bit early to say if this is the case and more information and tests need to be done.

Updates will be posted here and to the PUMA mailing list when we have further news.