Changes between Version 26 and Version 27 of CDDS


Ignore:
Timestamp:
02/10/19 15:27:37 (8 months ago)
Author:
simon
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • CDDS

    v26 v27  
    8989  "experiment_id": "piClim-NTCF", 
    9090  "institution_id": "MOHC", 
    91   "license": "CMIP6 model data produced by the Met Office Hadley Centre is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and at https://ukesm.ac.uk/cmip6. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law.\n\n ", 
     91  "license": "CMIP6 model data produced by the Met Office Hadley Centre is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgement. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and at https://ukesm.ac.uk/cmip6. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law.\n\n ", 
    9292  "mip": "AerChemMIP", 
    9393  "mip_era": "CMIP6", 
     
    121121Several directories will be created (their structure will be based on information in the json request file.) It is worth familiarizing yourself with the data structure and its relation to entries in the json request file. 
    122122 
    123 `cdds-data` will hold spaces for input and output data 
     123`cdds_data` will hold spaces for input and output data 
    124124{{{ 
    125 ls /group_workspaces/jasmin4/ncas_cms/glister/AerChemMIP-piClim-NTCF/cdds_data/CMIP6/AerChemMIP/UKESM1-0-LL/piClim-NTCF/r1i1p1f2/round-1-monthly 
     125ls $CDDS_DIR/cdds_data/CMIP6/AerChemMIP/UKESM1-0-LL/piClim-NTCF/r1i1p1f2/round-1-monthly 
    126126input/  output/ 
    127127}}} 
     
    129129`cdds_proc` will contain various configuration files and logging output 
    130130{{{ 
    131 ls /group_workspaces/jasmin4/ncas_cms/glister/AerChemMIP-piClim-NTCF/cdds_proc/CMIP6/AerChemMIP/UKESM1-0-LL_piClim-NTCF_r1i1p1f2/round-1-monthly 
     131ls $CDDS_DIR/cdds_proc/CMIP6/AerChemMIP/UKESM1-0-LL_piClim-NTCF_r1i1p1f2/round-1-monthly 
    132132archive/  configure/  convert/  extract/  prepare/  qualitycheck/ 
    133133}}} 
     134 
     135 
     136 
     1379. Tell CDDS where the input data resides by specifying its location (through soft links in this case) in the cdds_data directory. In our example, the raw data is in the aerchemmip group workspace 
     138{{{ 
     139cd $CDDS_DIR/cdds_data/CMIP6/AerChemMIP/UKESM1-0-LL/piClim-NTCF/r1i1p1f2/round-1-monthly/input 
     140ln -s /gws/nopw/j04/aerchemmip_vol1/data/u-bh543_4archive u-bh543 
     141}}} 
     142 
     14310. Enable the data conversion process - simply un-comment the final {{{cdds_convert}}} command in {{{cdds_workflow_for_user.sh}}} and {{{source cdds_workflow_for_user.sh}}} again. A small amount of work will be repeated, but much of the structure already configured (in particular {{{cdds_data}}}) will persist. [ Note: this should be handled more elegantly through arguments to {{{cdds_workflow_for_user.sh}}} ] 
     144 
     145Several Rose suites will be created - monitor progress on jasmin-cylc with {{{cylc gscan}}}. Suite logging is in cylc-run as usual. 
    134146 
    135147Take a look in `convert` 
     
    140152`u-ak283-JSON` is the Rose suite that will run to perform the data conversions. 
    141153 
    142  
    143 9. Tell CDDS where the input data resides by specifying its location (through soft links in this case) in the cdds_data directory. In our example, the raw data is in the aerchemmip group workspace 
    144 {{{ 
    145 cd /group_workspaces/jasmin4/ncas_cms/glister/AerChemMIP-piClim-NTCF/cdds_data/CMIP6/AerChemMIP/UKESM1-0-LL/piClim-NTCF/r1i1p1f2/round-1-monthly/input 
    146 ln -s /gws/nopw/j04/aerchemmip_vol1/data/u-bh543_4archive u-bh543 
    147 }}} 
    148  
    149 10. Enable the data conversion process - simply un-comment the final {{{cdds_convert}}} command in {{{cdds_workflow_for_user.sh}}} and {{{source cdds_workflow_for_user.sh}}} again. A small amount of work will be repeated, but much of the structure already configured (in particular {{{cdds_data}}}) will persist. [ Note: this should be handled more elegantly through arguments to {{{cdds_workflow_for_user.sh}}} ] 
    150  
    151 Several Rose suites will be created - monitor progress on jasmin-cylc with {{{cylc-gscan}}}. Suite logging is in cylc-run as usual. 
    152  
    15315411. Converted data will be written to the `output` directory in cdds_data 
    154155{{{ 
    155 ls /group_workspaces/jasmin4/ncas_cms/glister/AerChemMIP-piClim-NTCF/cdds_data/CMIP6/AerChemMIP/UKESM1-0-LL/piClim-NTCF/r1i1p1f2/round-1-monthly/output 
     156ls $CDDS_DIR/cdds_data/CMIP6/AerChemMIP/UKESM1-0-LL/piClim-NTCF/r1i1p1f2/round-1-monthly/output 
    156157ap4/  ap4_concat/  ap4_mip_convert/  ap5/  ap5_concat/  ap5_mip_convert/ 
    157158}}}