| 1 |
<HTML> |
| 2 |
<HEAD> |
| 3 |
<TITLE>stag-ir</TITLE> |
| 4 |
<LINK REV="made" HREF="mailto:[email protected]"> |
| 5 |
</HEAD> |
| 6 |
|
| 7 |
<BODY> |
| 8 |
|
| 9 |
<A NAME="__index__"></A> |
| 10 |
<!-- INDEX BEGIN --> |
| 11 |
|
| 12 |
<UL> |
| 13 |
|
| 14 |
<LI><A HREF="#name">NAME</A></LI> |
| 15 |
<LI><A HREF="#synopsis">SYNOPSIS</A></LI> |
| 16 |
<LI><A HREF="#description">DESCRIPTION</A></LI> |
| 17 |
<UL> |
| 18 |
|
| 19 |
<LI><A HREF="#arguments">ARGUMENTS</A></LI> |
| 20 |
<UL> |
| 21 |
|
| 22 |
<LI><A HREF="#d db_name">-d DB_NAME</A></LI> |
| 23 |
<LI><A HREF="#clear">-clear</A></LI> |
| 24 |
<LI><A HREF="#insertonly">-insertonly</A></LI> |
| 25 |
<LI><A HREF="#newonly">-newonly</A></LI> |
| 26 |
<LI><A HREF="#transaction_size">-transaction_size</A></LI> |
| 27 |
<LI><A HREF="#r relationname">-r RELATION-NAME</A></LI> |
| 28 |
<LI><A HREF="#k uniquekey">-k UNIQUE-KEY</A></LI> |
| 29 |
<LI><A HREF="#u uniquekey">-u UNIQUE-KEY</A></LI> |
| 30 |
<LI><A HREF="#create">-create</A></LI> |
| 31 |
<LI><A HREF="#idtype type">-idtype TYPE</A></LI> |
| 32 |
<LI><A HREF="#p parser">-p PARSER</A></LI> |
| 33 |
<LI><A HREF="#q queryid">-q QUERY-ID</A></LI> |
| 34 |
<LI><A HREF="#top nodename">-top NODE-NAME</A></LI> |
| 35 |
<LI><A HREF="#qf queryfile">-qf QUERY-FILE</A></LI> |
| 36 |
<LI><A HREF="#keys">-keys</A></LI> |
| 37 |
</UL> |
| 38 |
|
| 39 |
</UL> |
| 40 |
|
| 41 |
<LI><A HREF="#see also">SEE ALSO</A></LI> |
| 42 |
</UL> |
| 43 |
<!-- INDEX END --> |
| 44 |
|
| 45 |
<HR> |
| 46 |
<P> |
| 47 |
<H1><A NAME="name">NAME</A></H1> |
| 48 |
<P>stag-ir.pl - information retrieval using a simple relational index</P> |
| 49 |
<P> |
| 50 |
<HR> |
| 51 |
<H1><A NAME="synopsis">SYNOPSIS</A></H1> |
| 52 |
<PRE> |
| 53 |
stag-ir.pl -r person -k social_security_no -d Pg:mydb myrecords.xml |
| 54 |
stag-ir.pl -d Pg:mydb -q 999-9999-9999 -q 888-8888-8888</PRE> |
| 55 |
<P> |
| 56 |
<HR> |
| 57 |
<H1><A NAME="description">DESCRIPTION</A></H1> |
| 58 |
<P>Indexes stag nodes (XML Elements) in a simple relational db structure |
| 59 |
- keyed by ID with an XML Blob as a value</P> |
| 60 |
<P>Imagine you have a very large file of data, in a stag compatible |
| 61 |
format such as XML. You want to index all the elements of type |
| 62 |
<STRONG>person</STRONG>; each person can be uniquely identified by |
| 63 |
<STRONG>social_security_no</STRONG>, which is a direct subnode of <STRONG>person</STRONG></P> |
| 64 |
<P>The first thing to do is to build the index file, which will be stored |
| 65 |
in the database mydb</P> |
| 66 |
<PRE> |
| 67 |
stag-ir.pl -r person -k social_security_no -d Pg:mydb myrecords.xml</PRE> |
| 68 |
<P>You can then use the index ``person-idx'' to retrieve <STRONG>person</STRONG> nodes by |
| 69 |
their social security number</P> |
| 70 |
<PRE> |
| 71 |
stag-ir.pl -d Pg:mydb -q 999-9999-9999 > some-person.xml</PRE> |
| 72 |
<P>You can export using different stag formats</P> |
| 73 |
<PRE> |
| 74 |
stag-ir.pl -d Pg:mydb -q 999-9999-9999 -w sxpr > some-person.xml</PRE> |
| 75 |
<P>You can retrieve multiple nodes (although these need to be rooted to |
| 76 |
make a valid file)</P> |
| 77 |
<PRE> |
| 78 |
stag-ir.pl -d Pg:mydb -q 999-9999-9999 -q 888-8888-8888 -top personset</PRE> |
| 79 |
<P>Or you can use a list of IDs from a file (newline delimited)</P> |
| 80 |
<PRE> |
| 81 |
stag-ir.pl -d Pg:mydb -qf my_ss_nmbrs.txt -top personset</PRE> |
| 82 |
<P> |
| 83 |
<H2><A NAME="arguments">ARGUMENTS</A></H2> |
| 84 |
<P> |
| 85 |
<H3><A NAME="d db_name">-d DB_NAME</A></H3> |
| 86 |
<P>This database will be used for storing the stag nodes</P> |
| 87 |
<P>The name can be a logical name or DBI locator or DBStag shorthand - |
| 88 |
see <A HREF="./DBIx/DBStag.html">the DBIx::DBStag manpage</A></P> |
| 89 |
<P>The database must already exist</P> |
| 90 |
<P> |
| 91 |
<H3><A NAME="clear">-clear</A></H3> |
| 92 |
<P>Deletes all data from the relation type (specified with <STRONG>-r</STRONG>) before loading</P> |
| 93 |
<P> |
| 94 |
<H3><A NAME="insertonly">-insertonly</A></H3> |
| 95 |
<P>Does not check if the ID in the file exists in the db - will always |
| 96 |
attempt an INSERT (and will fail if ID already exists)</P> |
| 97 |
<P>This is the fastest way to load data (only one SQL operation per node |
| 98 |
rather than two) but is only safe if there is no existing data</P> |
| 99 |
<P>(Default is clobber mode - existing data with same ID will be replaced)</P> |
| 100 |
<P> |
| 101 |
<H3><A NAME="newonly">-newonly</A></H3> |
| 102 |
<P>If there is already data in the specified relation in the db, and the |
| 103 |
XML being loaded specifies an ID that is already in the db, then this |
| 104 |
node will be ignored</P> |
| 105 |
<P>(Default is clobber mode - existing data with same ID will be replaced)</P> |
| 106 |
<P> |
| 107 |
<H3><A NAME="transaction_size">-transaction_size</A></H3> |
| 108 |
<P>A commit will be performed every n UPDATEs/COMMITs (and at the end)</P> |
| 109 |
<P>Default is autocommit</P> |
| 110 |
<P>note that if you are using -insertonly, and you are using |
| 111 |
transactions, and the input file contains an ID already in the |
| 112 |
database, then the transaction will fail because this script will try |
| 113 |
and insert a duplicate ID</P> |
| 114 |
<P> |
| 115 |
<H3><A NAME="r relationname">-r RELATION-NAME</A></H3> |
| 116 |
<P>This is the name of the stag node (XML element) that will be stored in |
| 117 |
the index; for example, with the XML below you may want to use the |
| 118 |
node name <STRONG>person</STRONG> and the unique key <STRONG>id</STRONG></P> |
| 119 |
<PRE> |
| 120 |
<person_set> |
| 121 |
<person> |
| 122 |
<id>...</id> |
| 123 |
</person> |
| 124 |
<person> |
| 125 |
<id>...</id> |
| 126 |
</person> |
| 127 |
... |
| 128 |
</person_set></PRE> |
| 129 |
<P>This flag should only be used when you want to store data</P> |
| 130 |
<P> |
| 131 |
<H3><A NAME="k uniquekey">-k UNIQUE-KEY</A></H3> |
| 132 |
<P>This node will be used as the unique/primary key for the data</P> |
| 133 |
<P>This node should be nested directly below the node that is being |
| 134 |
stored in the index - if it is more that one below, specify a path</P> |
| 135 |
<P>This flag should only be used when you want to store data</P> |
| 136 |
<P> |
| 137 |
<H3><A NAME="u uniquekey">-u UNIQUE-KEY</A></H3> |
| 138 |
<P>Synonym for <STRONG>-k</STRONG></P> |
| 139 |
<P> |
| 140 |
<H3><A NAME="create">-create</A></H3> |
| 141 |
<P>If specified, this will create a table for the relation name specified |
| 142 |
below; you should use this the first time you index a relation</P> |
| 143 |
<P> |
| 144 |
<H3><A NAME="idtype type">-idtype TYPE</A></H3> |
| 145 |
<P>(optional)</P> |
| 146 |
<P>This is the SQL datatype for the unique key; it defaults to <CODE>VARCHAR(255)</CODE></P> |
| 147 |
<P>If you know that your id is an integer, you can specify INTEGER here</P> |
| 148 |
<P>If your id is always a 8-character field you can do this</P> |
| 149 |
<PRE> |
| 150 |
-idtype 'CHAR(8)'</PRE> |
| 151 |
<P>This option only makes sense when combined with the <STRONG>-c</STRONG> option</P> |
| 152 |
<P> |
| 153 |
<H3><A NAME="p parser">-p PARSER</A></H3> |
| 154 |
<P>This can be the name of a stag supported format (xml, sxpr, itext) - |
| 155 |
XML is assumed by default</P> |
| 156 |
<P>It can also be a module name - this module is used to parse the input |
| 157 |
file into a stag stream; see <A HREF="./Data/Stag/BaseGenerator.html">the Data::Stag::BaseGenerator manpage</A> for details |
| 158 |
on writing your own parsers/event generators</P> |
| 159 |
<P>This flag should only be used when you want to store data</P> |
| 160 |
<P> |
| 161 |
<H3><A NAME="q queryid">-q QUERY-ID</A></H3> |
| 162 |
<P>Fetches the relation/node with unique key value equal to query-id</P> |
| 163 |
<P>Multiple arguments can be passed by specifying -q multple times</P> |
| 164 |
<P>This flag should only be used when you want to query data</P> |
| 165 |
<P> |
| 166 |
<H3><A NAME="top nodename">-top NODE-NAME</A></H3> |
| 167 |
<P>If this is specified in conjunction with <STRONG>-q</STRONG> or <STRONG>-qf</STRONG> then all the |
| 168 |
query result nodes will be nested inside a node with this name (ie |
| 169 |
this provides a root for the resulting document tree)</P> |
| 170 |
<P> |
| 171 |
<H3><A NAME="qf queryfile">-qf QUERY-FILE</A></H3> |
| 172 |
<P>This is a file of newline-seperated IDs; this is useful for querying |
| 173 |
the index in batch</P> |
| 174 |
<P> |
| 175 |
<H3><A NAME="keys">-keys</A></H3> |
| 176 |
<P>This will write a list of all primary keys in the index</P> |
| 177 |
<P> |
| 178 |
<HR> |
| 179 |
<H1><A NAME="see also">SEE ALSO</A></H1> |
| 180 |
<P><A HREF="./Data/Stag.html">the Data::Stag manpage</A></P> |
| 181 |
<P>For more complex stag to database mapping, see <A HREF="./DBIx/DBStag.html">the DBIx::DBStag manpage</A> and the |
| 182 |
scripts</P> |
| 183 |
<P><EM>stag-db.pl</EM> use file DBM indexes</P> |
| 184 |
<P><EM>stag-storenode.pl</EM> is for storing fully normalised stag trees</P> |
| 185 |
<P><EM>selectall_xml</EM></P> |
| 186 |
|
| 187 |
</BODY> |
| 188 |
|
| 189 |
</HTML> |