Debian Codegraph @

website/blog/2019-06-25-blog-post.md

raw
1 ---
2 title: New release of python module
3 author: Onur Çelebi
4 authorURL: https://research.fb.com/people/celebi-onur/
5 authorFBID: 663146146
6 ---
7
8 Today, we are happy to release a new version of the fastText python library. The main goal of this release is to merge two existing python modules: the official `fastText` module which was available on our github repository and the unofficial `fasttext` module which was available on pypi.org. We hope that this new version will address the confusion due to the previous existence of two similar, but different, python modules.
9
10 The new version of our library is now available on [pypi.org](https://pypi.org/project/fasttext/) as well as on our github repository, and you can find [an overview of its API here](/docs/en/python-module.html).
11
12
13
14 fastText vs fasttext: what happened?
15 ----------------------------------
16 There was an ongoing confusion among our user community about the existence of both `fastText` and `fasttext` modules.
17
18 When fastText was first released in 2016, it was a command line only utility. Very soon, people wanted to use fastText's capabilities from python without having to call a binary for each action. In August 2016, [Bayu Aldi Yansyah](https://github.com/pyk), a developer outside of Facebook, published a python wrapper of fastText. His work was very helpful to a lot of people in our community and he published his unofficial python library on pypi with the pretty straighforward module name `fasttext` (note the lowercase `t`).
19
20 Later, our team began to work on an official python binding of fastText, that was published under the same github repository as the C++ source code. However, the module name for this official library was `fastText` (note the uppercase `T`).
21
22 Last year, Bayu Aldi Yansyah gave us admin access to the pypi project so that we could merge the two libraries.
23
24 To sum up, we ended up with two libraries that had:
25
26 - almost the same name
27 - different APIs
28 - different versions
29 - different ways to install
30
31 That was a very confusing situation for the community.
32
33 What actions did we take?
34 --------------------------
35 Today we are merging the two python libraries. We decided to keep the official API and top level functions such as `train_unsupervised` and `train_supervised` as well as returning numpy objects. We remove `cbow`, `skipgram` and `supervised` functions from the unofficial API. However, [we bring nice ideas](#wordvectormodel-and-supervisedmodel-objects) from the unofficial API to the official one. In particular, we liked the pythonic approach of `WordVectorModel`. This new python module is named `fasttext`, and is available on both [pypi](https://pypi.org/project/fasttext/) and our [github](https://github.com/facebookresearch/fastText) repository.
36
37 From now, we will refer to the tool as "fastText", however the name of the python module is `fasttext`.
38
39
40
41 What is the right way to do now?
42 --------------------------------
43
44 Before, you would either use `fastText` (uppercase `T`):
45 ```python
46 import fastText
47 # and call:
48 fastText.train_supervised
49 fastText.train_unsupervised
50 ```
51
52 or use `fasttext` (lowercase `t`):
53 ```python
54 import fasttext
55 # and call:
56 fasttext.cbow
57 fasttext.skipgram
58 fasttext.supervised
59 ```
60
61
62 Now, the right way to do is to
63 `import fasttext` (lowercase `t`)
64 and use
65 ```python
66 import fasttext
67 # and call:
68 fasttext.train_supervised
69 fasttext.train_unsupervised
70 ```
71
72 We are keeping the lowercase `fasttext` module name, while we keep the `fastText` API.
73
74 This is because:
75
76 - the standard way to name python modules is all lowercases
77 - the API from `fastText` is exposing numpy arrays, which is widely used by the machine learning community.
78
79
80 You can find a more comprehensive overview of our python API [here](/docs/en/python-module.html).
81
82 Should I modify my existing code?
83 ---------------------------------
84 Depending on the version of the python module you were using, you might need to do some little modifications on your existing code.
85
86 ### 1) You were using the official `fastText` module:
87
88 You don't have to do much. Just replace your `import fastText` lines by `import fasttext` and everything should work as usual.
89
90 ### 2) You were using the unofficial `fasttext` module:
91
92 If you were using the functions `cbow`, `skipgram`, `supervised` and/or `WordVectorModel`, `SupervisedModel` objects, you were using the unofficial `fasttext` module.
93
94 Updating your code should be pretty straightforward, but it still implies some little changes.
95
96 #### `cbow` function: use `train_unsupervised` instead.
97 For example, replace:
98
99 ```
100 fasttext.cbow("train.txt", "model_file", lr=0.05, dim=100, ws=5, epoch=5)
101 ```
102 with
103 ```
104 model = fasttext.train_unsupervised("train.txt", model='cbow', lr=0.05, dim=100, ws=5, epoch=5)
105 model.save_model("model_file.bin")
106 ```
107
108 #### `skipgram` function: use `train_unsupervised` instead.
109 For example, replace:
110
111 ```
112 fasttext.skipgram("train.txt", "model_file", lr=0.05, dim=100, ws=5, epoch=5)
113 ```
114 with
115 ```
116 model = fasttext.train_unsupervised("train.txt", model='skipgram', lr=0.05, dim=100, ws=5, epoch=5)
117 model.save_model("model_file.bin")
118 ```
119
120
121 #### `supervised` function: use `train_supervised` instead
122 For example, replace:
123 ```
124 fasttext.supervised("train.txt", "model_file", lr=0.1, dim=100, epoch=5, word_ngrams=2, loss='softmax')
125 ```
126 with
127 ```
128 model = fasttext.train_supervised("train.txt", lr=0.1, dim=100, epoch=5, , word_ngrams=2, loss='softmax')
129 model.save_model("model_file.bin")
130 ```
131
132 #### Parameters
133
134 - As you can see, you can use either `word_ngrams` or `wordNgrams` as parameter name. Because the parameter names from the unofficial API are mapped to the official ones: `min_count` to `minCount`, `word_ngrams` to `wordNgrams`, `lr_update_rate` to `lrUpdateRate`, `label_prefix` to `label` and `pretrained_vectors` to `pretrainedVectors`.
135 - `silent` parameter is not supported. Use `verbose` parameter instead.
136 - `encoding` parameter is not supported, every input should be encoded in `utf-8`.
137
138
139 ### `WordVectorModel` and `SupervisedModel` objects
140
141 Instead of `WordVectorModel` and `SupervisedModel` objects, we return a model object that mimics some nice ideas from the unofficial API.
142
143 ```python
144 model = fasttext.train_unsupervised("train.txt", model='skipgram')
145 print(model.words) # list of words in dictionary
146 print(model['king']) # get the vector of the word 'king'
147 print('king' in model) # check if a word is in dictionary
148 ```
149
150
151
152 ```python
153 model = fasttext.train_supervised("train.txt")
154 print(model.words) # list of words in dictionary
155 print(model.labels) # list of labels
156 ```
157
158 The model object also contains the arguments of the training:
159
160 ```python
161 print(model.epoch)
162 print(model.loss)
163 print(model.wordNgrams)
164 ```
165
166 Thank you!
167 ------------
168 We want to thank our incredible community. We truly appreciate your feedback, a big thank you to everyone reporting issues and contributing to the project. In particular we want to express how grateful we are to [Bayu Aldi Yansyah](https://github.com/pyk) who did a great job with his python library and for giving us the ownership of the pypi `fasttext` project.