Opened 2 years ago

Closed 23 months ago

#2295 closed help (fixed)

Download ECMWF Data For Monsoon

Reported by: pliojop Owned by: willie
Component: Data Keywords: ECMWF, GLM, Monsoon
Cc: Platform: Monsoon2
UM Version: 10.6

Description

Afternoon,

I will be running the UM on Monsoon in the nestign suite with a global model (GLM) run at N320 resolution and a 8 km nested region. The GLM will be forced by ERA-Interim global fields, however I need to download these fields before porting them to Monsoon.

I wanted to ask if there is a specific route to go through to gain access to this data or if anyone has already undertaken such a download and has the data on Monsoon already. I was also wondering if there was a guide to which fields where needed to use in this situation.

Many thanks

James

Change History (22)

comment:1 Changed 2 years ago by willie

  • Component changed from UM Model to Data
  • Owner changed from um_support to willie
  • Status changed from new to accepted

Hi James,

We can get start data for you - just let us know the date/times you need - see http://cms.ncas.ac.uk/wiki/UM/GettingInitialData. Do you specifically need ECMWF data or would Met Office start dumps work as well?

Regards
Willie

comment:2 Changed 2 years ago by pliojop

Morning Willie,

Either works well, probably I would favour the easier to set the job up, I guess that would be UM?

I'm looking for data for the period 01/1989 through to 12/2016.

Thanks,

James

comment:3 Changed 2 years ago by willie

Hi James,

We can only get UM data for the past 6-7 years due to MASS storage limitations. The ERA-I start dumps only contain a minimal set of fields, necessary to get the UM going and so need to be reconfigured for whatever version of the UM you're using. They are available for the whole of that period I think. Let us know the specific date/times you need.

Regards
Willie

comment:4 Changed 2 years ago by pliojop

Thanks Willie,

I will be running time slices 1st November through to 30th April for the years
1989 to 2016 (ie 1st Nov 1989 - 30 Apr 1990, 1st Nov 1990 - 30 Apr 1991, etc), so I would need the daily data for that period.

James

comment:5 Changed 2 years ago by willie

Hi James,
This is an unusually vast amount of data. If we assume you only want 0Z each day, then 180 days per year x 28 years x 730MB per ECMWF dump = 3.5TB. Do you really need this?

Regards
Willie

comment:6 Changed 2 years ago by pliojop

Hi Willie,

The GLM phase of my nested simulation will be initialised each day from the ECMWF data, so as I understand it, I do need all those files.

James

comment:7 Changed 2 years ago by willie

Hi James,
Ok - this'll take a bit of organising. I'll get back to you.

Regards
Willie

comment:8 Changed 2 years ago by pliojop

Thanks, much appreciated.

comment:9 Changed 2 years ago by willie

Hi James,

I've put the 1991/2 data on Monsoon2 in /projects/nexcs-n02/frmy/data_xfer. Please copy it to your own workspace as soon as possible.

The ERA-I files are much smaller than I expected, 54MB each rather than 730MB, so the whole will be about 270GB. Let me know if these are OK.

Regards,
Willie

comment:10 Changed 2 years ago by pliojop

Thanks Willie,

I have coppied that data accross. Will test it tomorrow (Friday) and let you know.

Cheers

JP

comment:11 Changed 2 years ago by pliojop

Morning Willie,

I tried running my job (u-ar589) to read in the grib data you sent me, however the job fails in the recon stage with an error, a snip of which is below.

I was wondering if this was a usual problem or something to do with me?

Thanks

James

[snip]
tcmalloc: large alloc 4195985114795802624 bytes == (nil)

lib-4205 : UNRECOVERABLE library error

The program was unable to request more memory space.

lib-4205 : UNRECOVERABLE library error

The program was unable to request more memory space.

tcmalloc: large alloc 4195985114795802624 bytes == (nil)

ATP Stack walkback for Rank 1 done
Process died with signal 6: 'Aborted'
Forcing core dump of rank 1
View application merged backtrace tree with: stat-view atpMergedBT.dot
You may need to: module load stat

_pmiu_daemon(SIGCHLD): [NID 00288] [c1-0c1s8n0] [Tue Oct 31 02:45:17 2017] PE RANK 5 exit signal Killed
[NID 00288] 2017-10-31 02:45:17 Apid 5941169: initiated application termination
[FAIL] um-recon # return-code=137
2017-10-31T02:45:19Z CRITICAL - Task job script received signal EXIT

[end snip]

comment:12 Changed 2 years ago by willie

Hi James,

You need to check the GRIB button in um > namelist > Reconf and Anc Ctrl > General technical options.

Regards
Willie

comment:13 Changed 2 years ago by pliojop

Morning Willie,

Thanks, that resolved the issue above, however I am now getting the error message:

[error]
? Error code: 30? Error from processor: 4

? Error from routine: Rcf_Set_Data_Source
Rank 9 [Fri Nov 3 00:25:18 2017] [c9-1c2s0n0] application called MPI_Abort(MPI_COMM_WORLD, 9) - process 9

? Error from routine: Rcf_Set_Data_Source? Error number: 2

? Error from processor: 10? Error message: Section 0 Item 9 : Required field is not in input dump!

? Error message: Section 0 Item 9 : Required field is not in input dump!????????????????????????????????????????????????????????????????????????????????Rank 3 [Fri Nov 3 00:25:18 2017] [c9-1c2s0n0] application called MPI_Abort(MPI_COMM_WORLD, 9) - process 3
? Error number: 2Rank 11 [Fri Nov 3 00:25:18 2017] [c9-1c2s0n0] application called MPI_Abort(MPI_COMM_WORLD, 9) - process 11

_pmiu_daemon(SIGCHLD): [NID 04544] [c9-1c2s0n0] [Fri Nov 3 00:25:24 2017] PE RANK 8 exit signal Killed
[FAIL] um-recon # return-code=137
2017-11-03T00:25:27Z CRITICAL - Task job script received signal EXIT
[end error]

I'm presuming there is something missing in my set up? I have committed the job as it ran in this version.

Cheers

James

comment:14 Changed 2 years ago by willie

Hi James,

Your set up is designed for use with a UM input file: these have eighty or more fields. You are driving with a GRIB file which has only about eight fields. So you most likely need to specify where the expected fields are to come from. This is most often the start dump but it could equally be an ancillary file. If you switch off the science you don't need this will reduce the number of fields it is expecting in the input dump.

Regards
Willie

comment:15 Changed 2 years ago by pliojop

Thanks Willie.

Do you know where I would be able to get applicable start dumps? I presume I would need a 01/11/1991 job for the example files you sent me, to create dumps for the rest of a 1991/92 job, before needing a 01/11/1992 for the next year (and so on).

Also, from the GRIB into Dumps documentation file, it mentions that there are ENDgame required fields in the GRIB data as well. My job is version 10.6, however I am not sure if that is ENDgame or not?

Thanks,

James

comment:16 Changed 2 years ago by willie

Hi James,

I now have successfully reconfigured the GRIB start dump. See my suite u-as029.

The method is tedious: reconfigure, check the error, add the required field. Repeat six times in this case.

The following ancillary fields were added,

  1. 0,9 to Smow anc
  2. 0,31 to Seaice
  3. 0,32 to seaice
  4. 0,24 to SST
  5. 0,20 to SLT
  6. 0.23 to Smow

This is done in glm_um > Reconfig and Anc control > Config ancils and initialise dump. The table needs to be reindexed each time.

Regards
Willie

Last edited 2 years ago by willie (previous) (diff)

comment:17 Changed 2 years ago by pliojop

Hi Willie,

That is fantastic. I have been able to add in the fields as above, I must confess it is far easier to see how to do it, once you can see an example, the index numbers do make it look rather confusing at first!

However, I ran into an error when I added in these fields to u-ar589, so I copied your job u-as029 and ran my own version of it (u-as149), changing the charging code. I ran the "make ancils only" stage, and then I ran it to build the executable as it was the first time that suite ran. It also hit the same error as u-ar589. The error occurs in the stage

glm_um_fcst_000

the glm recon stage has completed successfully. For the glm_um_fcst_000 stage, the std.err is:

???!!!???!!!???!!!???!!!???!!! ERROR ???!!!???!!!???!!!???!!!???!!!
? Error code: 101
? Error from routine: UP_ANCIL
? Error message: REPLANCA :ICE FIELD DATA MISSING
? Error from processor: 0
? Error number: 21
????????????????????????????????????????????????????????????????????????????????

In the output, the more detailed error appears to be:

PPCTL_INIT: Opening new file /home/d02/jpope/cylc-run/u-as149/share/cycle/19911101T0000Z/glm/um/umglaa_cb000 on unit 12
OPEN: File /home/d02/jpope/cylc-run/u-as149/share/cycle/19911101T0000Z/glm/um/umglaa_cb000 to be Opened on Unit 12 does not Exist
OPEN: File /home/d02/jpope/cylc-run/u-as149/share/cycle/19911101T0000Z/glm/um/umglaa_cb000 Created on Unit 12
OPEN: Claimed 4194304 Bytes (524288 Words) for Buffering
OPEN: Buffer Address is 0x100067d0240
IO: Open: /home/d02/jpope/cylc-run/u-as149/share/cycle/19911101T0000Z/glm/um/umglaa_cb000 on unit 12
REPLANCA: UPDATE REQUIRED FOR FIELD 1
REPLANCA - time interpolation for ancillary field, stashcode 24
targ_time,time1,time2 360., 336., 1056.
hours,int,period -32768, 1, 12
Information used in checking ancillary data set: position of lookup table in dataset: 11
Position of first lookup table referring to data type 1
Interval between lookup tables referring to data type 1 Number of steps 10
STASH code in dataset 24 STASH code requested 24
'start' position of lookup tables for dataset 1 in overall lookup array 1
UP_ANCIL : Error in REPLANCA.

The error message makes me think that the error is sea ice relatd, however the STASH code refers to the SST field. Apologies for this dragging on.

Thanks

James

comment:18 Changed 2 years ago by willie

Hi James,

My mistake. You need to change the update_anc for STASH 0,24 to false. It should go then. I got as far as the IcelandGreenland_8p0_protoRA1M reconfiguration where it ran out of time after an hour. This is a lot of time for reconfiguration so if the domain is large you may need to increase the number of processors.

Regards
Willie

comment:19 Changed 2 years ago by pliojop

Brilliant.

Thanks Willie.

Would it be possible (going back to very start!) to request the grib files for the other years in my range

"I will be running time slices 1st November through to 30th April for the years
1989 to 2016 (ie 1st Nov 1989 - 30 Apr 1990, 1st Nov 1990 - 30 Apr 1991, etc), so I would need the daily data for that period. "

Thanks very much

James

comment:20 Changed 2 years ago by willie

Hi James,

Your data is now ready on Monsoon2 in /projects/nexcs-n02/frmy/data_xfer. Please copy it to your own work space as soon as possible.

Statistics

5075 files, 268GB
Data transfer from ECMWF to ARCHER was very slow at around 5MB/s - taking 16 hours.

Regards
Willie

comment:21 Changed 23 months ago by pliojop

Afternoon Willie,

Just to confirm, I have copied all the data over to my own section of Monsoon.

Many thanks

James

comment:22 Changed 23 months ago by willie

  • Resolution set to fixed
  • Status changed from accepted to closed
Note: See TracTickets for help on using tickets.