| 1 |
Translating CMIP3 to CMIP5 |
| 2 |
========================== |
| 3 |
|
| 4 |
The script ``translate_cmip3`` converts the CMIP3 archive into a form |
| 5 |
as close to the DRS specification as possible. This transformation |
| 6 |
involves both filename and directory structure changes. From the |
| 7 |
command's help message:: |
| 8 |
|
| 9 |
Usage: translate_cmip3 [options] cmip3_root cmip5_root |
| 10 |
|
| 11 |
Options: |
| 12 |
-h, --help show this help message and exit |
| 13 |
-i INCLUDE, --include=INCLUDE |
| 14 |
Include paths matching INCLUDE regular expression |
| 15 |
-e EXCLUDE, --exclude=EXCLUDE |
| 16 |
Exclude paths matching EXCLUDE regular expression |
| 17 |
-c, --copy Copy rather than move files |
| 18 |
-d, --dryrun Emit log messages but don't translate anything |
| 19 |
-l LOGLEVEL, --loglevel=LOGLEVEL |
| 20 |
Set logging level |
| 21 |
|
| 22 |
|
| 23 |
Example |
| 24 |
------- |
| 25 |
|
| 26 |
The drslib.cmip3 module implements a similar API to |
| 27 |
drslib.cmip5 thus allowing CMIP3 paths to be converted to DRS |
| 28 |
instances then converted into CMIP5 DRS format. |
| 29 |
|
| 30 |
>>> from drslib import cmip3 |
| 31 |
>>> cmip3_trans = cmip3.make_translator('cmip3') |
| 32 |
>>> drs3 = cmip3_trans.filepath_to_drs('cmip3/20c3m/atm/da/rsus/gfdl_cm2_0/run1/rsus_A2.19610101-19651231.nc') |
| 33 |
>>> drs3 |
| 34 |
<DRS activity='cmip3', product='output', institute='GFDL', model='CM2', experiment='20c3m', frequency='day', realm='atmos', variable='rsus', table='A2', ensemble=(1, None, None), version=1, subset=None, extended='19610101-19651231'> |
| 35 |
>>> cmip5_trans.drs_to_filepath(drs3) |
| 36 |
'http://example.com/cmip5/output/GFDL/CM2/20c3m/day/atmos/v1/rsus/r1/rsus_A2_CM2_20c3m_r1_19610101-19651231.nc' |
| 37 |
|
| 38 |
|
| 39 |
CMIP3 DRS components |
| 40 |
-------------------- |
| 41 |
|
| 42 |
The CMIP3 activity is ``cmip3`` and all datasets are given the product |
| 43 |
``output``. The version component is always ``v1``. Translation of |
| 44 |
the other DRS components for CMIP3 are described below. |
| 45 |
|
| 46 |
|
| 47 |
Institute & Model |
| 48 |
''''''''''''''''' |
| 49 |
|
| 50 |
Institutes and models given in capital letters and underscores are |
| 51 |
converted to dash characters. Capitalisation is chosen to be |
| 52 |
consistent with the examples given in sections 3.2 and 3.3 of the DRS |
| 53 |
specification and dashes are used to avoid ambiguity in DRS filenames |
| 54 |
that use underscores as the component separator. |
| 55 |
|
| 56 |
Where the exact encoding is not trivial the syntax used by the IPCC |
| 57 |
Data Distribution Centre [DDC]_ is used. |
| 58 |
|
| 59 |
.. [DDC] http://www.ipcc-data.org |
| 60 |
|
| 61 |
================= ========= ===== |
| 62 |
CMIP3 directory Institute Model |
| 63 |
----------------- --------- ----- |
| 64 |
bcc_cm1 CMA BCC-CM1 |
| 65 |
bccr_bcm2_0 BCCR BCM2 |
| 66 |
cccma_cgcm3_1 CCCMA CGCM3-1-T47 |
| 67 |
cccma_cgcm3_1_t63 CCCMA GCM3-1-T63 |
| 68 |
cnrm_cm3 CNRM M3 |
| 69 |
miub_echo_g MIUB-KMA CHO-G |
| 70 |
csiro_mk3_0 CSIRO K3 |
| 71 |
csiro_mk3_5 CSIRO K3-5 |
| 72 |
gfdl_cm2_0 GFDL M2 |
| 73 |
gfdl_cm2_1 GFDL M2-1 |
| 74 |
inmcm3_0 INM M3 |
| 75 |
ipsl_cm4 IPSL M4 |
| 76 |
iap_fgoals1_0_g LASG GOALS-G1-0 |
| 77 |
mpi_echam5 MPIM CHAM5 |
| 78 |
mri_cgcm2_3_2a MRI GCM2-3-2 |
| 79 |
giss_aom NASA ISS-AOM |
| 80 |
giss_model_e_h NASA ISS-EH |
| 81 |
giss_model_e_r NASA ISS-ER |
| 82 |
ncar_ccsm3_0 NCAR CSM3 |
| 83 |
ncar_pcm1 NCAR CM |
| 84 |
miroc3_2_hires NIES IROC3-2-HI |
| 85 |
miroc3_2_medres NIES IROC3-2-MED |
| 86 |
ukmo_hadcm3 UKMO ADCM3 |
| 87 |
ukmo_hadgem1 UKMO ADGEM1 |
| 88 |
ingv_echam4 INGV CHAM4 |
| 89 |
================= ========= ===== |
| 90 |
|
| 91 |
|
| 92 |
|
| 93 |
Experiment |
| 94 |
'''''''''' |
| 95 |
|
| 96 |
The experiment component remains unchanged from the CMIP3 archive |
| 97 |
structure except that it's position in the tree changes to match the DRS specification. |
| 98 |
|
| 99 |
Frequency |
| 100 |
''''''''' |
| 101 |
|
| 102 |
The CMIP3 frequency specifiers are translated into those described in the DRS specification as follows: |
| 103 |
|
| 104 |
===== === |
| 105 |
CMIP3 DRS |
| 106 |
----- --- |
| 107 |
yr yr |
| 108 |
mo mon |
| 109 |
da day |
| 110 |
3h 3hr |
| 111 |
fixed fx |
| 112 |
===== === |
| 113 |
|
| 114 |
Modelling-realm |
| 115 |
''''''''''''''' |
| 116 |
|
| 117 |
We map CMIP3 realms onto equivilent CMIP5 realms. In some cases this mapping also depends on the variable. This mapping is defined in the table below: |
| 118 |
|
| 119 |
=========== ======== ========= |
| 120 |
CMIP3 realm Variable DRS realm |
| 121 |
----------- -------- --------- |
| 122 |
atm mrsos land |
| 123 |
atm trsult aerosol |
| 124 |
atm trsul aerosol |
| 125 |
atm tro3 atmosChem |
| 126 |
atm ``*`` atmos |
| 127 |
ice ``*`` seaIce |
| 128 |
land sftgif landIce |
| 129 |
land ``*`` land |
| 130 |
ocn ``*`` ocean |
| 131 |
=========== ======== ========= |
| 132 |
|
| 133 |
Variable name |
| 134 |
''''''''''''' |
| 135 |
|
| 136 |
Variable names are left unchanged. |
| 137 |
|
| 138 |
Ensemble member |
| 139 |
''''''''''''''' |
| 140 |
|
| 141 |
The encoding ``run<N>`` is translated into ``r<N>``. |
| 142 |
|
| 143 |
Subset and extended path |
| 144 |
'''''''''''''''''''''''' |
| 145 |
|
| 146 |
Although most filenames in the CMIP3 archive follow a consistent |
| 147 |
syntax there are enough exceptions to make complete adherence to the |
| 148 |
DRS specification impractical. Instead ``translate_cmip3`` attempts to extract the variable, MIP table name from the CMIP3 path and constructs an approximate DRS filename of the form:: |
| 149 |
|
| 150 |
<variable>_<mip-table>_<model>_<experiment>_<ensemble-member>_<extended>.nc |
| 151 |
|
| 152 |
where ``<extended>`` is the unparsed portion of the filename that may |
| 153 |
contain a temporal subset or may be irregular. Some examples are given below::: |
| 154 |
|
| 155 |
/20c3m/atm/da/rsus/gfdl_cm2_0/run1/rsus_A2.19610101-19651231.nc --> rsus_A2_CM2_20c3m_r1_19610101-19651231.nc |
| 156 |
/1pctto2x/atm/mo/rlftoaa_co2/ipsl_cm4/run1/rlftoaa_co2_A5_1860-1869.nc --> rlftoaa_co2_A5_CM4_1pctto2x_r1_1860-1869.nc |
| 157 |
/2xco2/land/fixed/orog/miroc3_2_hires/run1/orog_A1.nc --> orog_A1_MIROC3-2-HI_2xco2_r1.nc |
| 158 |
/sresa1b/atm/mo/rlut/cccma_cgcm3_1/run4/rlut_a1_sresa1b_4_cgcm3.1_t47_2001_2100.nc --> rlut_a1_CGCM3-1-T47_sresa1b_r4_sresa1b_4_cgcm3.1_t47_2001_2100.nc |
| 159 |
|