Debian Codegraph @

docs/supervised-tutorial.md

raw
1 ---
2 id: supervised-tutorial
3 title: Text classification
4 ---
5
6 Text classification is a core problem to many applications, like spam detection, sentiment analysis or smart replies. In this tutorial, we describe how to build a text classifier with the fastText tool.
7
8 ## What is text classification?
9
10 The goal of text classification is to assign documents (such as emails, posts, text messages, product reviews, etc...) to one or multiple categories. Such categories can be review scores, spam v.s. non-spam, or the language in which the document was typed. Nowadays, the dominant approach to build such classifiers is machine learning, that is learning classification rules from examples. In order to build such classifiers, we need labeled data, which consists of documents and their corresponding categories (or tags, or labels).
11
12 As an example, we build a classifier which automatically classifies stackexchange questions about cooking into one of several possible tags, such as `pot`, `bowl` or `baking`.
13
14 ## Installing fastText
15
16 The first step of this tutorial is to install and build fastText. It only requires a c++ compiler with good support of c++11.
17
18 Let us start by downloading the [most recent release](https://github.com/facebookresearch/fastText/releases):
19
20 ```bash
21 $ wget https://github.com/facebookresearch/fastText/archive/v0.9.2.zip
22 $ unzip v0.9.2.zip
23 ```
24
25 Move to the fastText directory and build it:
26
27 ```bash
28 $ cd fastText-0.9.2
29 # for command line tool :
30 $ make
31 # for python bindings :
32 $ pip install .
33 ```
34
35 <!--DOCUSAURUS_CODE_TABS-->
36 <!--Command line-->
37 <br />
38 Running the binary without any argument will print the high level documentation, showing the different use cases supported by fastText:
39
40 ```bash
41 >> ./fasttext
42 usage: fasttext <command> <args>
43
44 The commands supported by fasttext are:
45
46 supervised train a supervised classifier
47 quantize quantize a model to reduce the memory usage
48 test evaluate a supervised classifier
49 predict predict most likely labels
50 predict-prob predict most likely labels with probabilities
51 skipgram train a skipgram model
52 cbow train a cbow model
53 print-word-vectors print word vectors given a trained model
54 print-sentence-vectors print sentence vectors given a trained model
55 nn query for nearest neighbors
56 analogies query for analogies
57
58 ```
59
60 In this tutorial, we mainly use the `supervised`, `test` and `predict` subcommands, which corresponds to learning (and using) text classifier. For an introduction to the other functionalities of fastText, please see the [tutorial about learning word vectors](https://fasttext.cc/docs/en/unsupervised-tutorial.html).
61
62 <!--Python-->
63 <br />
64 Calling the help function will show high level documentation of the library:
65 ```py
66 >>> import fasttext
67 >>> help(fasttext.FastText)
68 Help on module fasttext.FastText in fasttext:
69
70 NAME
71 fasttext.FastText
72
73 DESCRIPTION
74 # Copyright (c) 2017-present, Facebook, Inc.
75 # All rights reserved.
76 #
77 # This source code is licensed under the MIT license found in the
78 # LICENSE file in the root directory of this source tree.
79
80 FUNCTIONS
81 load_model(path)
82 Load a model given a filepath and return a model object.
83
84 read_args(arg_list, arg_dict, arg_names, default_values)
85
86 tokenize(text)
87 Given a string of text, tokenize it and return a list of tokens
88
89 train_supervised(*kargs, **kwargs)
90 Train a supervised model and return a model object.
91
92 input must be a filepath. The input text does not need to be tokenized
93 as per the tokenize function, but it must be preprocessed and encoded
94 as UTF-8. You might want to consult standard preprocessing scripts such
95 as tokenizer.perl mentioned here: http://www.statmt.org/wmt07/baseline.html
96
97 The input file must must contain at least one label per line. For an
98 example consult the example datasets which are part of the fastText
99 repository such as the dataset pulled by classification-example.sh.
100
101 train_unsupervised(*kargs, **kwargs)
102 Train an unsupervised model and return a model object.
103
104 input must be a filepath. The input text does not need to be tokenized
105 as per the tokenize function, but it must be preprocessed and encoded
106 as UTF-8. You might want to consult standard preprocessing scripts such
107 as tokenizer.perl mentioned here: http://www.statmt.org/wmt07/baseline.html
108
109 The input field must not contain any labels or use the specified label prefix
110 unless it is ok for those words to be ignored. For an example consult the
111 dataset pulled by the example script word-vector-example.sh, which is
112 part of the fastText repository.
113 ```
114
115 In this tutorial, we mainly use the `train_supervised`, which returns a model object, and call `test` and `predict` on this object. That corresponds to learning (and using) text classifier. For an introduction to the other functionalities of fastText, please see the [tutorial about learning word vectors](https://fasttext.cc/docs/en/unsupervised-tutorial.html).
116 <!--END_DOCUSAURUS_CODE_TABS-->
117
118 ## Getting and preparing the data
119
120 As mentioned in the introduction, we need labeled data to train our supervised classifier. In this tutorial, we are interested in building a classifier to automatically recognize the topic of a stackexchange question about cooking. Let's download examples of questions from [the cooking section of Stackexchange](http://cooking.stackexchange.com/), and their associated tags:
121
122 ```bash
123 >> wget https://dl.fbaipublicfiles.com/fasttext/data/cooking.stackexchange.tar.gz && tar xvzf cooking.stackexchange.tar.gz
124 >> head cooking.stackexchange.txt
125 ```
126
127 Each line of the text file contains a list of labels, followed by the corresponding document. All the labels start by the `__label__` prefix, which is how fastText recognize what is a label or what is a word. The model is then trained to predict the labels given the word in the document.
128
129 Before training our first classifier, we need to split the data into train and validation. We will use the validation set to evaluate how good the learned classifier is on new data.
130
131 ```bash
132 >> wc cooking.stackexchange.txt
133 15404 169582 1401900 cooking.stackexchange.txt
134 ```
135
136 Our full dataset contains 15404 examples. Let's split it into a training set of 12404 examples and a validation set of 3000 examples:
137
138 ```bash
139 >> head -n 12404 cooking.stackexchange.txt > cooking.train
140 >> tail -n 3000 cooking.stackexchange.txt > cooking.valid
141 ```
142
143 ## Our first classifier
144
145 We are now ready to train our first classifier:
146
147 <!--DOCUSAURUS_CODE_TABS-->
148 <!--Command line-->
149 ```bash
150 >> ./fasttext supervised -input cooking.train -output model_cooking
151 Read 0M words
152 Number of words: 14598
153 Number of labels: 734
154 Progress: 100.0% words/sec/thread: 75109 lr: 0.000000 loss: 5.708354 eta: 0h0m
155 ```
156
157 The `-input` command line option indicates the file containing the training examples, while the `-output` option indicates where to save the model. At the end of training, a file `model_cooking.bin`, containing the trained classifier, is created in the current directory.
158
159 <!--Python-->
160 ```py
161 >>> import fasttext
162 >>> model = fasttext.train_supervised(input="cooking.train")
163 Read 0M words
164 Number of words: 14598
165 Number of labels: 734
166 Progress: 100.0% words/sec/thread: 75109 lr: 0.000000 loss: 5.708354 eta: 0h0m
167 ```
168 The `input` argument indicates the file containing the training examples. We can now use the `model` variable to access information on the trained model.
169
170 We can also call `save_model` to save it as a file and load it later with `load_model` function.
171 ```py
172 >>> model.save_model("model_cooking.bin")
173 ```
174 <!--END_DOCUSAURUS_CODE_TABS-->
175
176
177 Now, we can test our classifier, by :
178 <!--DOCUSAURUS_CODE_TABS-->
179 <!--Command line-->
180 ```bash
181 >> ./fasttext predict model_cooking.bin -
182 ```
183
184 and then typing a sentence. Let's first try the sentence:
185
186 *Which baking dish is best to bake a banana bread ?*
187
188 The predicted tag is `baking` which fits well to this question. Let us now try a second example:
189
190 *Why not put knives in the dishwasher?*
191
192 <!--Python-->
193 ```py
194 >>> model.predict("Which baking dish is best to bake a banana bread ?")
195 ((u'__label__baking',), array([0.15613931]))
196 ```
197 The predicted tag is `baking` which fits well to this question. Let us now try a second example:
198
199 ```py
200 >>> model.predict("Why not put knives in the dishwasher?")
201 ((u'__label__food-safety',), array([0.08686075]))
202 ```
203
204 <!--END_DOCUSAURUS_CODE_TABS-->
205
206
207 The label predicted by the model is `food-safety`, which is not relevant. Somehow, the model seems to fail on simple examples.
208
209 To get a better sense of its quality, let's test it on the validation data by running:
210
211 <!--DOCUSAURUS_CODE_TABS-->
212 <!--Command line-->
213 ```bash
214 >> ./fasttext test model_cooking.bin cooking.valid
215 N 3000
216 P@1 0.124
217 R@1 0.0541
218 Number of examples: 3000
219 ```
220 The output of fastText are the precision at one (`P@1`) and the recall at one (`R@1`).
221
222 <!--Python-->
223 ```py
224 >>> model.test("cooking.valid")
225 (3000L, 0.124, 0.0541)
226 ```
227 The output are the number of samples (here `3000`), the precision at one (`0.124`) and the recall at one (`0.0541`).
228 <!--END_DOCUSAURUS_CODE_TABS-->
229
230 We can also compute the precision at five and recall at five with:
231
232 <!--DOCUSAURUS_CODE_TABS-->
233 <!--Command line-->
234 ```bash
235 >> ./fasttext test model_cooking.bin cooking.valid 5
236 N 3000
237 P@5 0.0668
238 R@5 0.146
239 Number of examples: 3000
240 ```
241 <!--Python-->
242 ```py
243 >>> model.test("cooking.valid", k=5)
244 (3000L, 0.0668, 0.146)
245 ```
246 <!--END_DOCUSAURUS_CODE_TABS-->
247
248
249 ## Advanced readers: precision and recall
250
251 The precision is the number of correct labels among the labels predicted by fastText. The recall is the number of labels that successfully were predicted, among all the real labels. Let's take an example to make this more clear:
252
253 *Why not put knives in the dishwasher?*
254
255 On Stack Exchange, this sentence is labeled with three tags: `equipment`, `cleaning` and `knives`. The top five labels predicted by the model can be obtained with:
256
257 <!--DOCUSAURUS_CODE_TABS-->
258 <!--Command line-->
259 ```bash
260 >> ./fasttext predict model_cooking.bin - 5
261 ```
262 <!--Python-->
263 ```py
264 >>> model.predict("Why not put knives in the dishwasher?", k=5)
265 ((u'__label__food-safety', u'__label__baking', u'__label__equipment', u'__label__substitutions', u'__label__bread'), array([0.0857 , 0.0657, 0.0454, 0.0333, 0.0333]))
266 ```
267 <!--END_DOCUSAURUS_CODE_TABS-->
268
269 are `food-safety`, `baking`, `equipment`, `substitutions` and `bread`.
270
271 Thus, one out of five labels predicted by the model is correct, giving a precision of 0.20. Out of the three real labels, only one is predicted by the model, giving a recall of 0.33.
272
273 For more details, see [the related Wikipedia page](https://en.wikipedia.org/wiki/Precision_and_recall).
274
275 ## Making the model better
276
277 The model obtained by running fastText with the default arguments is pretty bad at classifying new questions. Let's try to improve the performance, by changing the default parameters.
278
279 ### preprocessing the data
280
281 Looking at the data, we observe that some words contain uppercase letter or punctuation. One of the first step to improve the performance of our model is to apply some simple pre-processing. A crude normalization can be obtained using command line tools such as `sed` and `tr`:
282
283 ```bash
284 >> cat cooking.stackexchange.txt | sed -e "s/\([.\!?,'/()]\)/ \1 /g" | tr "[:upper:]" "[:lower:]" > cooking.preprocessed.txt
285 >> head -n 12404 cooking.preprocessed.txt > cooking.train
286 >> tail -n 3000 cooking.preprocessed.txt > cooking.valid
287 ```
288
289 Let's train a new model on the pre-processed data:
290
291 <!--DOCUSAURUS_CODE_TABS-->
292 <!--Command line-->
293 ```bash
294 >> ./fasttext supervised -input cooking.train -output model_cooking
295 Read 0M words
296 Number of words: 9012
297 Number of labels: 734
298 Progress: 100.0% words/sec/thread: 82041 lr: 0.000000 loss: 5.671649 eta: 0h0m
299
300 >> ./fasttext test model_cooking.bin cooking.valid
301 N 3000
302 P@1 0.164
303 R@1 0.0717
304 Number of examples: 3000
305 ```
306 <!--Python-->
307 ```py
308 >>> import fasttext
309 >>> model = fasttext.train_supervised(input="cooking.train")
310 Read 0M words
311 Number of words: 9012
312 Number of labels: 734
313 Progress: 100.0% words/sec/thread: 82041 lr: 0.000000 loss: 5.671649 eta: 0h0m
314
315 >>> model.test("cooking.valid")
316 (3000L, 0.164, 0.0717)
317 ```
318 <!--END_DOCUSAURUS_CODE_TABS-->
319
320 We observe that thanks to the pre-processing, the vocabulary is smaller (from 14k words to 9k). The precision is also starting to go up by 4%!
321
322 ### more epochs and larger learning rate
323
324 By default, fastText sees each training example only five times during training, which is pretty small, given that our training set only have 12k training examples. The number of times each examples is seen (also known as the number of epochs), can be increased using the `-epoch` option:
325
326 <!--DOCUSAURUS_CODE_TABS-->
327 <!--Command line-->
328 ```bash
329 >> ./fasttext supervised -input cooking.train -output model_cooking -epoch 25
330 Read 0M words
331 Number of words: 9012
332 Number of labels: 734
333 Progress: 100.0% words/sec/thread: 77633 lr: 0.000000 loss: 7.147976 eta: 0h0m
334 ```
335 <!--Python-->
336 ```py
337 >>> import fasttext
338 >>> model = fasttext.train_supervised(input="cooking.train", epoch=25)
339 Read 0M words
340 Number of words: 9012
341 Number of labels: 734
342 Progress: 100.0% words/sec/thread: 77633 lr: 0.000000 loss: 7.147976 eta: 0h0m
343 ```
344 <!--END_DOCUSAURUS_CODE_TABS-->
345
346 Let's test the new model:
347
348 <!--DOCUSAURUS_CODE_TABS-->
349 <!--Command line-->
350 ```bash
351 >> ./fasttext test model_cooking.bin cooking.valid
352 N 3000
353 P@1 0.501
354 R@1 0.218
355 Number of examples: 3000
356 ```
357 <!--Python-->
358 ```py
359 >>> model.test("cooking.valid")
360 (3000L, 0.501, 0.218)
361 ```
362 <!--END_DOCUSAURUS_CODE_TABS-->
363
364 This is much better! Another way to change the learning speed of our model is to increase (or decrease) the learning rate of the algorithm. This corresponds to how much the model changes after processing each example. A learning rate of 0 would mean that the model does not change at all, and thus, does not learn anything. Good values of the learning rate are in the range `0.1 - 1.0`.
365
366 <!--DOCUSAURUS_CODE_TABS-->
367 <!--Command line-->
368 ```bash
369 >> ./fasttext supervised -input cooking.train -output model_cooking -lr 1.0
370 Read 0M words
371 Number of words: 9012
372 Number of labels: 734
373 Progress: 100.0% words/sec/thread: 81469 lr: 0.000000 loss: 6.405640 eta: 0h0m
374
375 >> ./fasttext test model_cooking.bin cooking.valid
376 N 3000
377 P@1 0.563
378 R@1 0.245
379 Number of examples: 3000
380 ```
381 <!--Python-->
382 ```py
383 >>> model = fasttext.train_supervised(input="cooking.train", lr=1.0)
384 Read 0M words
385 Number of words: 9012
386 Number of labels: 734
387 Progress: 100.0% words/sec/thread: 81469 lr: 0.000000 loss: 6.405640 eta: 0h0m
388
389 >>> model.test("cooking.valid")
390 (3000L, 0.563, 0.245)
391 ```
392 <!--END_DOCUSAURUS_CODE_TABS-->
393
394 Even better! Let's try both together:
395
396 <!--DOCUSAURUS_CODE_TABS-->
397 <!--Command line-->
398 ```bash
399 >> ./fasttext supervised -input cooking.train -output model_cooking -lr 1.0 -epoch 25
400 Read 0M words
401 Number of words: 9012
402 Number of labels: 734
403 Progress: 100.0% words/sec/thread: 76394 lr: 0.000000 loss: 4.350277 eta: 0h0m
404
405 >> ./fasttext test model_cooking.bin cooking.valid
406 N 3000
407 P@1 0.585
408 R@1 0.255
409 Number of examples: 3000
410 ```
411 <!--Python-->
412 ```py
413 >>> model = fasttext.train_supervised(input="cooking.train", lr=1.0, epoch=25)
414 Read 0M words
415 Number of words: 9012
416 Number of labels: 734
417 Progress: 100.0% words/sec/thread: 76394 lr: 0.000000 loss: 4.350277 eta: 0h0m
418
419 >>> model.test("cooking.valid")
420 (3000L, 0.585, 0.255)
421 ```
422 <!--END_DOCUSAURUS_CODE_TABS-->
423
424 Let us now add a few more features to improve even further our performance!
425
426 ### word n-grams
427
428 Finally, we can improve the performance of a model by using word bigrams, instead of just unigrams. This is especially important for classification problems where word order is important, such as sentiment analysis.
429
430 <!--DOCUSAURUS_CODE_TABS-->
431 <!--Command line-->
432 ```bash
433 >> ./fasttext supervised -input cooking.train -output model_cooking -lr 1.0 -epoch 25 -wordNgrams 2
434 Read 0M words
435 Number of words: 9012
436 Number of labels: 734
437 Progress: 100.0% words/sec/thread: 75366 lr: 0.000000 loss: 3.226064 eta: 0h0m
438
439 >> ./fasttext test model_cooking.bin cooking.valid
440 N 3000
441 P@1 0.599
442 R@1 0.261
443 Number of examples: 3000
444 ```
445 <!--Python-->
446 ```py
447 >>> model = fasttext.train_supervised(input="cooking.train", lr=1.0, epoch=25, wordNgrams=2)
448 Read 0M words
449 Number of words: 9012
450 Number of labels: 734
451 Progress: 100.0% words/sec/thread: 75366 lr: 0.000000 loss: 3.226064 eta: 0h0m
452
453 >>> model.test("cooking.valid")
454 (3000L, 0.599, 0.261)
455 ```
456 <!--END_DOCUSAURUS_CODE_TABS-->
457
458 With a few steps, we were able to go from a precision at one of 12.4% to 59.9%. Important steps included:
459
460 * preprocessing the data ;
461 * changing the number of epochs (using the option `-epoch`, standard range `[5 - 50]`) ;
462 * changing the learning rate (using the option `-lr`, standard range `[0.1 - 1.0]`) ;
463 * using word n-grams (using the option `-wordNgrams`, standard range `[1 - 5]`).
464
465 ## Advanced readers: What is a Bigram?
466
467 A 'unigram' refers to a single undividing unit, or token, usually used as an input to a model. For example a unigram can be a word or a letter depending on the model. In fastText, we work at the word level and thus unigrams are words.
468
469 Similarly we denote by 'bigram' the concatenation of 2 consecutive tokens or words. Similarly we often talk about n-gram to refer to the concatenation any n consecutive tokens.
470
471 For example, in the sentence, 'Last donut of the night', the unigrams are 'last', 'donut', 'of', 'the' and 'night'. The bigrams are: 'Last donut', 'donut of', 'of the' and 'the night'.
472
473 Bigrams are particularly interesting because, for most sentences, you can reconstruct the order of the words just by looking at a bag of n-grams.
474
475 Let us illustrate this by a simple exercise, given the following bigrams, try to reconstruct the original sentence: 'all out', 'I am', 'of bubblegum', 'out of' and 'am all'.
476 It is common to refer to a word as a unigram.
477
478 ## Scaling things up
479
480 Since we are training our model on a few thousands of examples, the training only takes a few seconds. But training models on larger datasets, with more labels can start to be too slow. A potential solution to make the training faster is to use the [hierarchical softmax](#advanced-readers-hierarchical-softmax), instead of the regular softmax. This can be done with the option `-loss hs`:
481
482 <!--DOCUSAURUS_CODE_TABS-->
483 <!--Command line-->
484 ```bash
485 >> ./fasttext supervised -input cooking.train -output model_cooking -lr 1.0 -epoch 25 -wordNgrams 2 -bucket 200000 -dim 50 -loss hs
486 Read 0M words
487 Number of words: 9012
488 Number of labels: 734
489 Progress: 100.0% words/sec/thread: 2199406 lr: 0.000000 loss: 1.718807 eta: 0h0m
490 ```
491 <!--Python-->
492 ```py
493 >>> model = fasttext.train_supervised(input="cooking.train", lr=1.0, epoch=25, wordNgrams=2, bucket=200000, dim=50, loss='hs')
494 Read 0M words
495 Number of words: 9012
496 Number of labels: 734
497 Progress: 100.0% words/sec/thread: 2199406 lr: 0.000000 loss: 1.718807 eta: 0h0m
498 ```
499 <!--END_DOCUSAURUS_CODE_TABS-->
500
501 Training should now take less than a second.
502
503
504 ## Advanced readers: hierarchical softmax
505
506 The hierarchical softmax is a loss function that approximates the softmax with a much faster computation.
507
508 The idea is to build a binary tree whose leaves correspond to the labels. Each intermediate node has a binary decision activation (e.g. sigmoid) that is trained, and predicts if we should go to the left or to the right. The probability of the output unit is then given by the product of the probabilities of intermediate nodes along the path from the root to the output unit leave.
509
510 For a detailed explanation, you can have a look on [this video](https://www.youtube.com/watch?v=B95LTf2rVWM).
511
512 In fastText, we use a Huffman tree, so that the lookup time is faster for more frequent outputs and thus the average lookup time for the output is optimal.
513
514 ## Multi-label classification
515
516 When we want to assign a document to multiple labels, we can still use the softmax loss and play with the parameters for prediction, namely the number of labels to predict and the threshold for the predicted probability. However playing with these arguments can be tricky and unintuitive since the probabilities must sum to 1.
517
518 A convenient way to handle multiple labels is to use independent binary classifiers for each label. This can be done with `-loss one-vs-all` or `-loss ova`.
519
520 <!--DOCUSAURUS_CODE_TABS-->
521 <!--Command line-->
522 ```bash
523 >> ./fasttext supervised -input cooking.train -output model_cooking -lr 0.5 -epoch 25 -wordNgrams 2 -bucket 200000 -dim 50 -loss one-vs-all
524 Read 0M words
525 Number of words: 14543
526 Number of labels: 735
527 Progress: 100.0% words/sec/thread: 72104 lr: 0.000000 loss: 4.340807 ETA: 0h 0m
528 ```
529 <!--Python-->
530 ```py
531 >>> import fasttext
532 >>> model = fasttext.train_supervised(input="cooking.train", lr=0.5, epoch=25, wordNgrams=2, bucket=200000, dim=50, loss='ova')
533 Read 0M words
534 Number of words: 14543
535 Number of labels: 735
536 Progress: 100.0% words/sec/thread: 72104 lr: 0.000000 loss: 4.340807 ETA: 0h 0m
537 ```
538 <!--END_DOCUSAURUS_CODE_TABS-->
539
540 It is a good idea to decrease the learning rate compared to other loss functions.
541
542 Now let's have a look on our predictions, we want as many prediction as possible (argument `-1`) and we want only labels with probability higher or equal to `0.5` :
543 <!--DOCUSAURUS_CODE_TABS-->
544 <!--Command line-->
545 ```bash
546 >> ./fasttext predict-prob model_cooking.bin - -1 0.5
547 ```
548 and then type the sentence:
549
550 *Which baking dish is best to bake a banana bread ?*
551
552 we get:
553 ```
554 __label__baking 1.00000 __label__bananas 0.939923 __label__bread 0.592677
555 ```
556 <!--Python-->
557 ```py
558 >>> model.predict("Which baking dish is best to bake a banana bread ?", k=-1, threshold=0.5)
559 ((u''__label__baking, u'__label__bananas', u'__label__bread'), array([1.00000, 0.939923, 0.592677]))
560 ```
561 <!--END_DOCUSAURUS_CODE_TABS-->
562
563
564
565 <!--DOCUSAURUS_CODE_TABS-->
566 <!--Command line-->
567 <br />
568 We can also evaluate our results with the `test` command :
569 ```bash
570 >> ./fasttext test model_cooking.bin cooking.valid -1 0.5
571 N 3000
572 P@-1 0.702
573 R@-1 0.2
574 Number of examples: 3000
575 ```
576 and play with the threshold to obtain desired precision/recall metrics :
577
578 ```bash
579 >> ./fasttext test model_cooking.bin cooking.valid -1 0.1
580 N 3000
581 P@-1 0.591
582 R@-1 0.272
583 Number of examples: 3000
584 ```
585 <!--Python-->
586 <br />
587 We can also evaluate our results with the `test` function:
588 ```py
589 >>> model.test("cooking.valid", k=-1)
590 (3000L, 0.702, 0.2)
591 ```
592 <!--END_DOCUSAURUS_CODE_TABS-->
593
594
595 ## Conclusion
596
597 In this tutorial, we gave a brief overview of how to use fastText to train powerful text classifiers. We had a light overview of some of the most important options to tune.