Performance of Standard Configurations

Global Atmosphere 7.0/Global Land 7.0

Suite id	Description	UM Version	Date Run	Platform	Decomposition*	OpenMP Threads	Length of run	Dump Frequency	Wallclock	Data output vol	Comment	Cost/model-yr
u-ab878	GA7.0 N96 AMIP	10.3	10.02.17	ARCHER	16 x 12	2	30 days	10 days	01:07:53
u-ag689/n216	GA7.0 N216 AMIP	10.5	22.06.17	ARCHER	42 x 36	1	10 days	10 days	00:29:27
u-ak155	GC3.1 N216 ORCA025	10.6.1	07.06.17	ARCHER	A:42x36 O:24x26 X:12 (6 nodes)	1	30 days	10 days	01:24:29
u-ai862	GC3.0 N96 ORCA025	10.6	20.07.17	ARCHER	A:16x18 O:18x26 X:8 (4 nodes)	1	10 days	10 days	00:27:03
u-aq711	piControl N216-ORCA025	10.7	17.10.17	ARCHER	A:36x24x2 O:30x30 X:8 nodes (16/2)	2 (atm)	1 mnth	10 days	1:45:00		25% variation in run times - expensive because multi-threaded (necessary?) working copy only
u-as037 (r59528)	GC3.1 N96 ORCA1 PI Control CMIP6	10.7	11.11.17	ARCHER	A:12x8 O:9x8 X:6 (3 nodes)	2	1m	10 days	00:56:59	162Gb (History_Data - 2month run)
u-au173	GA7.0 N96 UM10.9 AMIP Climate Development (1988 - 2008)	10.9	17.01.18	ARCHER	A:16x24	1	30 days	10 days	00:36:17
u-av321	GA7.0 N96 UM10.7 AMIP Climate Development (1988 - 2008)	10.7	23.02.18	ARCHER	A:16x24	2	30 days	10 days	00:30:47
u-ba767	GA7.0 N96 UM11.1 AMIP Climate Development (1988 - 2008)	11.1	31.08.18	ARCHER	A:14x18	2	30 days	10 days	00:33:05		Note-average time over 1st 3M
u-ba767	GA7.0 N96 UM11.1 AMIP Climate Development (1988 - 2008)	11.1	04.09.18	ARCHER	A:24x24	2	30 days	10 days	00:26:25		Note-average time over 1st 3M- (00:29:35, 00:24:34, 00:25:06)
u-bd431	GC3.0 N96 ORCA025 Present Day	11.2	28.11.18	ARCHER	A: 12x16 O:18x26 X:8 (4 nodes)	2 (atm)	1 month	30 days	01:11:55	39 Gb (History_Data)

UKESM

Suite id	Description	UM Version	Date Run	Platform	Decomposition*	OpenMP Threads	Length of run	Dump Frequency	Wallclock	Data output vol	Comment	Cost/model-yr
u-an561	UKESM0.6-CN PI, new oxidants file, lana, StratTrop	10.6	06.29.17 — 07.17.17	ARCHER	A:48x24 O:9x12	1	1yr (12 x 1 mnth cycles)	10 days	12 x 52 mins	111GB (archived (no dumps, pp+compressed netcdf))
u-q606	UKESM0.8-CN N216 ORCA025 (some improvements)	10.7	04.10.17	ARCHER	A:36x24 O:30x30 X:4 nodes (8/2)	1	1 mnth	10 days	2:00:00		working copy only
u-az760_8.3.7-GL	UKESM1 alpha release	11.0	16-09-18	ARCHER	A:48x24x1 O:9x8 X:3 nodes (6/2)	1 (atm)	1 mnth	30 days	00:49:00	~60GB	working copy only - not for production runs	190 kAU
u-bc108	UKESM1.0-beta historical	11.1		ARCHER	A:48x24x1 O:12x8 X:3 nodes (6/2)	1 (atm)	1 mnth	30 days			not yet for production runs	kAU
u-bc409	UKESM1.0-beta pre-industrial control	11.1	06.11.18	ARCHER	A:48x24x1 O:12x8 X:3 nodes (6/2)	1 (atm)	3 mnth	3 mnth	01:53:00	~86GB	not yet for production runs cce8.3.7	158 kAU
u-bd204@99131	UKESM1.0-beta pre-industrial control	11.1	04.12.18 to 19.12.18	ARCHER	A:48x24x1 O:12x8 X:3 nodes (6/2)	1 (atm)	30 years	3 mnth	251 hours	5.6TB	-	165.7 kAU

*Decomposition: A = Atmos (EW x NS), O = Ocean (EW x NS), X = XIOS processes (nodes)

Scaling & Cost Plots - ARCHER

GC3.1 N96 ORCA1 u-as037 (r 61036)

This looks at the performance of HadGEM3-GC3.1 N96ORCA1 PI Control for CMIP6 over a wide range of atmosphere decompositions with a fixed number of IO servers (6). The model was run on ARCHER in December, 2017. The ocean decomposition was 9x8 and the XIOS used 6 processes. The atmosphere was run with two threads, except for one run at 36x28 where three threads were used (3,186 processes in the graph below).

The following graphs show the model rate (in model years per day) against cost and total number of processes. Each point in the graph is the result of running the coupled suite for a model month. The cost axis was obtained by rescaling the processors axis in the right-hand graph. The cost of computing the graph was 440 kAU.

GA7.0 (u-ba767)

Similar calculations are performed for GA7.0. Real costs are used. The performance appears badly affected in the high processor limit.

UKESM AMIP (u-bh142)

The scaling graph as a function of the number of CPUs and the cost per model year is shown. Orange (1) and blue (2) are the number of OMP threads respectively. Further details, such as the North-South and East-West decomposition are revealed by holding the mouse above a point. Data from ARCHER, with a cost 0.015 kAU per core hour.

Years/Day vs nCPUs Years/Day vs Cost/Year