Debian Codegraph @

docs/topics/db/search.txt

raw
1 ======
2 Search
3 ======
4
5 A common task for web applications is to search some data in the database with
6 user input. In a simple case, this could be filtering a list of objects by a
7 category. A more complex use case might require searching with weighting,
8 categorization, highlighting, multiple languages, and so on. This document
9 explains some of the possible use cases and the tools you can use.
10
11 We'll refer to the same models used in :doc:`/topics/db/queries`.
12
13 Use Cases
14 =========
15
16 Standard textual queries
17 ------------------------
18
19 Text-based fields have a selection of matching operations. For example, you may
20 wish to allow lookup up an author like so:
21
22 .. code-block:: pycon
23
24 >>> Author.objects.filter(name__contains="Terry")
25 [<Author: Terry Gilliam>, <Author: Terry Jones>]
26
27 This is a very fragile solution as it requires the user to know an exact
28 substring of the author's name. A better approach could be a case-insensitive
29 match (:lookup:`icontains`), but this is only marginally better.
30
31 A database's more advanced comparison functions
32 -----------------------------------------------
33
34 If you're using PostgreSQL, Django provides :doc:`a selection of database
35 specific tools </ref/contrib/postgres/search>` to allow you to leverage more
36 complex querying options. Other databases have different selections of tools,
37 possibly via plugins or user-defined functions. Django doesn't include any
38 support for them at this time. We'll use some examples from PostgreSQL to
39 demonstrate the kind of functionality databases may have.
40
41 .. admonition:: Searching in other databases
42
43 All of the searching tools provided by :mod:`django.contrib.postgres` are
44 constructed entirely on public APIs such as :doc:`custom lookups
45 </ref/models/lookups>` and :doc:`database functions
46 </ref/models/database-functions>`. Depending on your database, you should
47 be able to construct queries to allow similar APIs. If there are specific
48 things which cannot be achieved this way, please open a ticket.
49
50 In the above example, we determined that a case insensitive lookup would be
51 more useful. When dealing with non-English names, a further improvement is to
52 use :lookup:`unaccented comparison <unaccent>`:
53
54 .. code-block:: pycon
55
56 >>> Author.objects.filter(name__unaccent__icontains="Helen")
57 [<Author: Helen Mirren>, <Author: Helena Bonham Carter>, <Author: Hélène Joy>]
58
59 This shows another issue, where we are matching against a different spelling of
60 the name. In this case we have an asymmetry though - a search for ``Helen``
61 will pick up ``Helena`` or ``Hélène``, but not the reverse. Another option
62 would be to use a :lookup:`trigram_similar` comparison, which compares
63 sequences of letters.
64
65 For example:
66
67 .. code-block:: pycon
68
69 >>> Author.objects.filter(name__unaccent__lower__trigram_similar="Hélène")
70 [<Author: Helen Mirren>, <Author: Hélène Joy>]
71
72 Now we have a different problem - the longer name of "Helena Bonham Carter"
73 doesn't show up as it is much longer. Trigram searches consider all
74 combinations of three letters, and compares how many appear in both search and
75 source strings. For the longer name, there are more combinations that don't
76 appear in the source string, so it is no longer considered a close match.
77
78 The correct choice of comparison functions here depends on your particular data
79 set, for example the language(s) used and the type of text being searched. All
80 of the examples we've seen are on short strings where the user is likely to
81 enter something close (by varying definitions) to the source data.
82
83 Document-based search
84 ---------------------
85
86 Standard database operations stop being a useful approach when you start
87 considering large blocks of text. Whereas the examples above can be thought of
88 as operations on a string of characters, full text search looks at the actual
89 words. Depending on the system used, it's likely to use some of the following
90 ideas:
91
92 - Ignoring "stop words" such as "a", "the", "and".
93 - Stemming words, so that "pony" and "ponies" are considered similar.
94 - Weighting words based on different criteria such as how frequently they
95 appear in the text, or the importance of the fields, such as the title or
96 keywords, that they appear in.
97
98 There are many alternatives for using searching software, some of the most
99 prominent are Elastic_ and Solr_. These are full document-based search
100 solutions. To use them with data from Django models, you'll need a layer which
101 translates your data into a textual document, including back-references to the
102 database ids. When a search using the engine returns a certain document, you
103 can then look it up in the database. There are a variety of third-party
104 libraries which are designed to help with this process.
105
106 .. _Elastic: https://www.elastic.co/
107 .. _Solr: https://solr.apache.org/
108
109 PostgreSQL support
110 ~~~~~~~~~~~~~~~~~~
111
112 PostgreSQL has its own full text search implementation built-in. While not as
113 powerful as some other search engines, it has the advantage of being inside
114 your database and so can easily be combined with other relational queries such
115 as categorization.
116
117 The :mod:`django.contrib.postgres` module provides some helpers to make these
118 queries. For example, a query might select all the blog entries which mention
119 "cheese":
120
121 .. code-block:: pycon
122
123 >>> Entry.objects.filter(body_text__search="cheese")
124 [<Entry: Cheese on Toast recipes>, <Entry: Pizza recipes>]
125
126 You can also filter on a combination of fields and on related models:
127
128 .. code-block:: pycon
129
130 >>> Entry.objects.annotate(
131 ... search=SearchVector("blog__tagline", "body_text"),
132 ... ).filter(search="cheese")
133 [
134 <Entry: Cheese on Toast recipes>,
135 <Entry: Pizza Recipes>,
136 <Entry: Dairy farming in Argentina>,
137 ]
138
139 See the ``contrib.postgres`` :doc:`/ref/contrib/postgres/search` document for
140 complete details.