Debian Codegraph @

doc/p_cmip5/p_cmip5.rst

raw
1 The p_cmip5 module will decide whether data should be assigned to DRS
2 product=output1 or output2. Background and the algorithm steps are
3 given in `requested_subset_decision_tree_v0_5.pdf`__.
4
5 __ doc/requested_subset_decision_tree_v0_5.pdf
6
7 Set-up
8 ......
9
10 There is a set-up step performed by the code in p_cmip5/init.py, using
11 the "init(shelve_dir)" function. This modules takes information from
12 the spreadsheets CMIP5_archive_size_template.xls and
13 standard_output_17Sep2010_mod.xls and stores it in python shelves used
14 by the main code. The shelves are placed in "shelve_dir".
15
16 The cmip5_product class
17 .......................
18
19 The :mod:`drslib.p_cmip5.product` module provides a ``cmip5_product`` class with a
20 "find_product" method. At instantiation, the class picks up
21 configuration tables The configuration file (sample in
22 ini/sample_1.ini, ini/smaple_2.ini) needs to contain information about
23 each model. This information is used for a small number of cases and
24 the format is not yet stable:
25
26 .. code-block:: python
27
28 class cmip5_product:
29
30 def __init__(self,mip_table_shelve='sh/standard_output_mip',
31 template='sh/template',
32 stdo='sh/standard_output',
33 config='ini/sample_1.ini',
34 override_product_change_warning=False,
35 policy_opt1='all_rel',not_ok_excpt=False):
36
37 Optional arguments
38 ''''''''''''''''''
39 :mip_table_shelve: shelve containing information about MIP tables;
40 :template: shelve containing information mapping experiment names to labels used in the standard_output spreadsheet;
41 :stdo: shelve containing information from the standard_output spreadsheet;
42 :config: configuration file;
43 :override_product_change_warning: in some cases it is possible that adding new data to previously published data can change the product
44 designation of the previously published data. The default behaviour is to givw an error return at this point. This can be over-ridden,
45 the code will then provide the product of the file to be added and lists of changes that need to be made to previously published data.
46 It is not expected, however, that the ESG publisher will support such updates: the user should instead compile a new set of files to
47 submit using all the previously published files and the new files.
48 :policy_opt1: this controls two options for treatment of data blocks in which time slices are requested. The default is that, if the time
49 slices are specified using relative dates (e.g. relative to start of experiment) and the number of years submitted is les than the number
50 of years requested, all years submitted will be assigned to output1 without examining the dates. There is an option (depricated) to extend
51 this catch-all approach to time slices specified with absolute dates.
52 :not_ok_excpt: if True, raise an exception if product can not be designated as output1 or output2.
53
54 The find_product method
55 .......................
56
57 The principal interface to the cmip5_product class is the find_product method:
58
59 .. code-block:: python
60
61 def find_product(self,var,table,expt,model,path,startyear=None,endyear=None,verbose=False,
62 path_output1=None, path_output2=None,selective_ads_scan=True):
63
64 Required arguments
65 ''''''''''''''''''
66 :var: DRS variable name
67 :table: MIP table
68 :expt: DRS experiment name
69 :model: Model name
70 :path: Path to directory containing all the files of one atomic dataset.
71
72 Optional arguments
73 ''''''''''''''''''
74 :startyear: first year of the file to be assessed
75 :endyear: last year of file to be assessed (not currently used)
76 :verbose: if True, provide additional comments to logger
77 :path_output1: path to last published output1 data for this atomic dataset, if new data is to be considered as an addition;
78 :path_output2: path to last published output1 data for this atomic dataset, if new data is to be considered as an addition;
79 :selective_ads_scan: when scanning the atomic dataset directory, look only at files matching the variable, table, experiment and model.
80 The False option is provided to facilitate testing using dummy data files, and should not otherwise be used.
81 :return: if "not_ok_excp=True", the method will return True if it can assign output1 or output2, otherwise an exception will be raised.
82 if "not_ok_excpt=False", the method will return False when it cannot assign output1 or output2, with a message in pc.reason (where pc
83 is an instance of the cmip5_product class).
84
85 Attributes containing results
86 '''''''''''''''''''''''''''''
87 After successful completion of the find_product method, the followin attibutes of the instance contain information:
88
89 :product: the product to be assigned to the file;
90 :reason: a short summary of the reasons for the assigment;
91 :rc: a return code, 'OKnnn' if successful, 'ERRnnn' if not.
92
93
94 Usage
95 .....
96
97 The following code fragment illustrates usage of the module::
98
99 ## import module
100 import p_cmip5_v5 as p
101
102 ## create and instance of the cmip5_product class, specifying a configuration file
103 ##
104 pc2 = p.cmip5_product( config='ini/sample_2.ini')
105
106 ## test a file: using the variable, mip table, experiment id, model and specifying the path of the atomic dataset directory containing all
107 ## the submitted files.
108 ## In some cases the decision as to which product the file belongs in will depend on the contents of this directory.
109 ## verbose=True results in additional messages being printed to standard out.
110
111 if pc2.find_product( var, mip, expt,model,path,startyear=startyear, verbose=verbose):
112 print 'product is: ', pc2.product
113
114 ## A True return means the method has identified the product
115 else:
116 ## A False return means the product could not be identified
117 print 'Dont know what to do with this data:: ',pc2.reason
118
119
120 Testing the p_cmip5 module: test_p_cmip5.py
121 ...........................................
122
123 The test_p_cmip5 module can be used to test the p_cmip5 module. E.g. run the following from the directory containing the "test" subdirectory:
124
125 .. code-block:: bash
126
127 $ nosetests --tests=test/test_p_cmip5.py