Debian Codegraph @

README

raw
1 NAME
2
3 Dancer::Plugin::Catmandu::OAI - OAI-PMH provider backed by a searchable
4 Catmandu::Store
5
6 SYNOPSIS
7
8 #!/usr/bin/env perl
9
10 use Dancer;
11 use Catmandu;
12 use Dancer::Plugin::Catmandu::OAI;
13
14 Catmandu->load;
15 Catmandu->config;
16
17 my $options = {};
18
19 oai_provider '/oai' , %$options;
20
21 dance;
22
23 DESCRIPTION
24
25 Dancer::Plugin::Catmandu::OAI is a Dancer plugin to provide OAI-PMH
26 services for Catmandu::Store-s that support CQL (such as
27 Catmandu::Store::ElasticSearch). Follow the installation steps below to
28 setup your own OAI-PMH server.
29
30 REQUIREMENTS
31
32 In the examples below an ElasticSearch 1.7.2
33 https://www.elastic.co/downloads/past-releases/elasticsearch-1-7-2
34 server will be used.
35
36 Follow the instructions below for a demonstration installation:
37
38 $ cpanm Dancer Catmandu::OAI Catmandu::Store::ElasticSearch
39
40 $ wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.2.zip
41 $ unzip elasticsearch-1.7.2.zip
42 $ cd elasticsearch-1.7.2
43 $ bin/elasticsearch
44
45 RECORDS
46
47 Records stored in the Catmandu::Store can be in any format. Preferably
48 the format should be easy to convert into the mandatory OAI-DC format.
49 At a minimum each record contains an identifier '_id' and a field
50 containing a datestamp.
51
52 $ cat sample.yml
53 ---
54 _id: oai:my.server.org:123456
55 datestamp: 2016-05-17T13:37:18Z
56 creator:
57 - Musterman, Max
58 - Jansen, Jan
59 - Svenson, Sven
60 title:
61 - Test record
62 ...
63
64 CATMANDU CONFIGURATION
65
66 ElasticSearch requires a configuration file to map record fields to CQL
67 terms. Below is a minimal configuration required to query for
68 identifiers and datastamps in the ElasticSearch collection:
69
70 $ cat catmandu.yml
71 ---
72 store:
73 oai:
74 package: ElasticSearch
75 options:
76 index_name: oai
77 bags:
78 data:
79 cql_mapping:
80 default_index: basic
81 indexes:
82 _id:
83 op:
84 'any': true
85 'all': true
86 '=': true
87 'exact': true
88 field: '_id'
89 datestamp:
90 op:
91 '=': true
92 '<': true
93 '<=': true
94 '>=': true
95 '>': true
96 'exact': true
97 field: 'datestamp'
98 index_mappings:
99 publication:
100 properties:
101 datestamp: {type: date, format: date_time_no_millis}
102
103 IMPORT RECORDS
104
105 With the Catmandu configuration files in place records can be imported
106 with the catmandu command:
107
108 # Drop the existing ElasticSearch 'oai' collection
109 $ catmandu drop oai
110
111 # Import the sample record
112 $ catmandu import YAML to oai < sample.yml
113
114 # Test if the records are available in the 'oai' collection
115 $ catmandu export oai
116
117 DANCER CONFIGURATION
118
119 The Dancer configuration file 'config.yml' contains basic information
120 for the OAI-PMH plugin to work:
121
122 * store - In which Catmandu::Store are the metadata records stored
123 * bag - In which Catmandu::Bag are the records of this 'store' (use: 'data' as default)
124 * datestamp_field - Which field in the record contains a datestamp ('datestamp' in our example above)
125 * datestamp_index - Which CQL index should be used to find records within a specified date range (if not specified, the value from the 'datestamp_field' setting is used)
126 * repositoryName - The name of the repository
127 * uri_base - The full base url of the OAI controller. To be used when behind a proxy server. When not set, this module relies on the Dancer request to provide its full url. Use middleware like 'ReverseProxy' or 'Dancer::Middleware::Rebase' in that case.
128 * adminEmail - An administrative email. Can be string or array of strings. This will be included in the Identify response.
129 * compression - a compression encoding supported by the repository. Can be string or array of strings. This will be included in the Identify response.
130 * description - XML container that describes your repository. Can be string or array of strings. This will be included in the Identify response. Note that this module will try to validate the XML data.
131 * earliestDatestamp - The earliest datestamp available in the dataset as YYYY-MM-DDTHH:MM:SSZ. This will be determined dynamically if no static value is given.
132 * deletedRecord - The policy for deleted records. See also: L<https://www.openarchives.org/OAI/openarchivesprotocol.html#DeletedRecords>
133 * repositoryIdentifier - A prefix to use in OAI-PMH identifiers
134 * cql_filter - A CQL query to find all records in the database that should be made available to OAI-PMH
135 * default_search_params - set default arguments that get passed to every call to the bag's search method
136 * search_strategy - default is C<paginate>, set to C<es.scroll> to avoid deep paging (Elasticsearch only)
137 * limit - The maximum number of records to be returned in each OAI-PMH request
138 * delimiter - Delimiters used in prefixing a record identifier with a repositoryIdentifier (use: ':' as default)
139 * sampleIdentifier - A sample identifier
140 * metadata_formats - An array of metadataFormats that are supported
141 * metadataPrefix - A short string for the name of the format
142 * schema - An URL to the XSD schema of this format
143 * metadataNamespace - A XML namespace for this format
144 * template - The path to a Template Toolkit file to transform your records into this format
145 * fix - Optionally an array of one or more L<Catmandu::Fix>-es or Fix files
146 * sets - Optional an array of OAI-PMH sets and the CQL query to retrieve records in this set from the Catmandu::Store
147 * setSpec - A short string for the same of the set
148 * setName - A longer description of the set
149 * setDescription - an optional and repeatable container that may hold community-specific XML-encoded data about the set. Should be string or array of strings.
150 * cql - The CQL command to find records in this set in the L<Catmandu::Store>
151 * xsl_stylesheet - Optional path to an xsl stylesheet
152 * template_options - An optional hash of configuration options that will be passed to L<Catmandu::Exporter::Template> or L<Template>.
153
154 Below is a sample minimal configuration for the 'sample.yml' demo
155 above:
156
157 $ cat config.yml
158 charset: "UTF-8"
159 plugins:
160 'Catmandu::OAI':
161 store: oai
162 bag: data
163 datestamp_field: datestamp
164 repositoryName: "My OAI DataProvider"
165 uri_base: "http://oai.service.com/oai"
166 adminEmail: [email protected]
167 earliestDatestamp: "1970-01-01T00:00:01Z"
168 cql_filter: "datestamp>1970-01-01T00:00:01Z"
169 deletedRecord: persistent
170 repositoryIdentifier: oai.service.com
171 limit: 200
172 delimiter: ":"
173 sampleIdentifier: "oai:oai.service.com:1585315"
174 metadata_formats:
175 -
176 metadataPrefix: oai_dc
177 schema: "http://www.openarchives.org/OAI/2.0/oai_dc.xsd"
178 metadataNamespace: "http://www.openarchives.org/OAI/2.0/oai_dc/"
179 template: oai_dc.tt
180
181 METADATAPREFIX TEMPLATE
182
183 For each metadataPrefix a Template Toolkit file needs to exist which
184 translate Catmandu::Store records into XML records. At least one
185 Template Toolkit file should be made available to transform stored
186 records into Dublin Core. The example below contains an example file to
187 transform 'sample.yml' type records into Dublin Core:
188
189 $ cat oai_dc.tt
190 <oai_dc:dc xmlns="http://www.openarchives.org/OAI/2.0/oai_dc/"
191 xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/"
192 xmlns:dc="http://purl.org/dc/elements/1.1/"
193 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
194 xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
195 [%- FOREACH var IN ['title' 'creator' 'subject' 'description' 'publisher' 'contributor' 'date' 'type' 'format' 'identifier' 'source' 'language' 'relation' 'coverage' 'rights'] %]
196 [%- FOREACH val IN $var %]
197 <dc:[% var %]>[% val | html %]</dc:[% var %]>
198 [%- END %]
199 [%- END %]
200 </oai_dc:dc>
201
202 START DANCER
203
204 If all the required files are available, then a Dancer application can
205 be started. See the 'demo' directory of this distribution for a
206 complete example:
207
208 $ ls
209 app.pl catmandu.yml config.yml oai_dc.tt
210 $ cat app.pl
211 #!/usr/bin/env perl
212
213 use Dancer;
214 use Catmandu;
215 use Dancer::Plugin::Catmandu::OAI;
216
217 Catmandu->load;
218 Catmandu->config;
219
220 my $options = {};
221
222 oai_provider '/oai' , %$options;
223
224 dance;
225
226 # Start Dancer
227 $ perl ./app.pl
228
229 # Test queries:
230
231 $ curl "http://localhost:3000/oai?verb=Identify"
232 $ curl "http://localhost:3000/oai?verb=ListSets"
233 $ curl "http://localhost:3000/oai?verb=ListMetadataFormats"
234 $ curl "http://localhost:3000/oai?verb=ListIdentifiers&metadataPrefix=oai_dc"
235 $ curl "http://localhost:3000/oai?verb=ListRecords&metadataPrefix=oai_dc"
236
237 SEE ALSO
238
239 Dancer, Catmandu, Catmandu::Store
240
241 AUTHOR
242
243 Nicolas Steenlant, <nicolas.steenlant at ugent.be>
244
245 CONTRIBUTORS
246
247 Nicolas Franck, <nicolas.franck at ugent.be>
248
249 Vitali Peil, <vitali.peil at uni-bielefeld.de>
250
251 Patrick Hochstenbach, <patric.hochstenbach at ugent.be>
252
253 LICENSE
254
255 This library is free software; you can redistribute it and/or modify it
256 under the same terms as Perl itself.
257