Changes between Version 5 and Version 6 of Projects/OpenIFS-IO

Timestamp:

23/09/13 15:19:16 (7 years ago)

Author:

grenville

Comment:

—

Projects/OpenIFS-IO

@@ -18 +18 @@
 Considerable progress has been made with this project. The OpenIFS repository has been created on PUMA and the OpenIFS code installed. Mark has installed the necessary grib libraries on HECToR and the FDB libraries under a package account for maintainability and has set up environment modules to help manage the software installation. The FDB calls have been activated in the OpenIFS code and we have successfully extracted and built the model on HECToR. Glenn has made available several models at differing resolutions for testing. We have concentrated on the t1279 model (about 25km global resolution). Several scenarios have been investigated - the results are summarized in the figure below. A baseline performance with minimal IO is established for 1 and 8 OMP threads the model scales well, especially when run with 8 OMP threads (see figure below right). OpenIFS running with is 'single-writer' output scheme shows the characteristic performance slow down as a single processor throttles the rest in order to gather and output data. Our test generates hourly output for a total of 58GB/model day; time spent performing output is a very significant portion of the total (~25% of wallclock for 1024 processors, rising to ~40% wallclock for 4096 processors). Running with the FDB scheme shows an impressive performance improvement, whereby the time taken for asynchronous multiprocessor output is virtually hidden and amounts to not more than 5% of the wallclock time for the run.
 
-[[Image(oifs-t1279.png,left)]]
+[[Image(oifs-t1279.png)]]
 
 [[Image(oifs-t1279-scaling.png,right)]]
+
+
+
+
+
+
+
+
+=== How to get on IOFS job running on HECToR ===
+
+There are a couple of preliminary administrative tasks which need to be taken care of first. 
+
+* Your institution will need an OpenIFS license - check with Glenn Carver (ECMWF)
+* You will need an account on HECToR
+* You will need to be a member of the oifs package account on HECToR. Contact CMS to arrange to be added to this group. 
+* You will need to have access to the OpenIFS code repository on PUMA - contact CMS
+
+It will be helpful if you have some knowledge of the Flexible Configuration Management (FCM) system. We use FCM to manage the OpenIFS code and in addition to manage the OpenIFS build. Much of the work has been done which will enable you to get started on HECToR, but it will benefit you and make your work more efficient if you become familiar with some simple FCM operations, such as commit, check-out, add, revert, branch-create. FCM is described in depth in complete online documentation at http://metomi.github.io/fcm/doc/ and an FCM tutorial is available at http://cms.ncas.ac.uk/wiki/Fcm. 
+
+
+=== Check out the OIFS code ===
+
+You will need to set up the correct environment to build and run OIFS jobs. Since HECToR has different hardware on its compute and service nodes, there is an extra consideration when using some OIFS-related utilities on the service nodes (more later). Let's assume that you are interested in building and running the model only for now. At the HECToR command line, type
+
+{{{
+module use ~oifs/modules
+module load openifs_cce/0.0.2
+}}}
+
+Loading the openifs module sets some paths required for the build and run and has some checking to ensure that there is no conflict with currently loaded modules. Typing
+
+{{{
+module show openifs_cce/0.0.2
+}}}
+
+gives details of what the module does, namely
+
+{{{
+module-whatis    CCE versions of support libraires to build OpenIFS 
+conflict         PrgEnv-gnu 
+conflict         PrgEnv-pgi 
+prereq   cce/8.1.8 
+module           load grib_api_cce/0.0.2 
+module           load fdb_cce/0.0.2 
+prepend-path     PATH /work/n02/n02/hum/fcm/bin 
+setenv           OIFS_IFSDATA /work/y07/y07/oifs/data/ecmwf/ifsdata 
+setenv           OIFS_ARCH x86_64 
+setenv           OIFS_COMP cce_fdb 
+setenv           OIFS_BUILD opt 
+}}}
+
+If your normal module set up is with the gnu or pgi programming environments, then an attempt to load openifs_cce/0.0.2 will fail, and it will also fail unless you have the Cray cce1/8.1.8 compiler loaded. Use '''module swap''' to ensure an appropriate starting module configuration.
+
+
+On HECToR create a suitably named directory in your /home space. We recommend building in /home which handles small files more efficiently than does /work for a quicker build. Now check out the OIFS code tree with '''fcm co'''. Here is an example
+
+{{{
+fcm co fcm:oifs/branches/dev/grenville/38r1v02_fdb_updates@16
+}}}
+
+This cumbersome command can be made simpler by use of FCM keywords and of course you can use aliasing to reduce the amount of typing required. However, it is instructive to see the command in its entirety (actually we have already defined the keyword oifs to save some typing). The text after fcm co is the URL for the code. In this case the code is in a '''branch''' owned by '''dev'''eloper '''grenville'''; the breanch is called '''38r1v02_fdb_updates''' and the code is at ('''@''') revision number '''16'''
+
+
+
+
+