Debian Codegraph @

docs/webassembly-module.md

raw
1 ---
2 id: webassembly-module
3 title: WebAssembly module
4 ---
5
6 In this document we present how to use fastText in javascript with WebAssembly.
7
8 ## Table of contents
9
10 * [Requirements](#requirements)
11 * [Building WebAssembly binaries](#building-webassembly-binaries)
12 * [Build a webpage that uses fastText](#build-a-webpage-that-uses-fasttext)
13 * [Load a model](#load-a-model)
14 * [Train a model](#train-a-model)
15 * [Disclaimer](#disclaimer)
16 * [Text classification](#text-classification)
17 * [Word representations](#word-representations)
18 * [Quantized models](#quantized-models)
19 * [API](#api)
20 * [`model` object](#model-object)
21 * [`loadModel`](#loadmodel)
22 * [`trainSupervised`](#trainsupervised)
23 * [`trainUnsupervised`](#trainunsupervised)
24
25 # Requirements
26
27 For building [fastText](https://fasttext.cc/) with WebAssembly bindings, we will need:
28 - a compiler with good C++11 support, since it uses C\++11 features,
29 - [emscripten](https://emscripten.org/),
30 - a [browser that supports WebAssembly](https://caniuse.com/#feat=wasm).
31
32
33 # Building WebAssembly binaries
34
35 First, download and install emscripten sdk as [described here](https://emscripten.org/docs/getting_started/downloads.html#installation-instructions).
36
37
38 We need to make sure we activated the PATH for emscripten:
39 ```bash
40 $ source /path/to/emsdk/emsdk_env.sh
41 ```
42
43 Clone [fastText repository](https://github.com/facebookresearch/fastText/):
44
45 ```bash
46 $ git clone [email protected]:facebookresearch/fastText.git
47 ```
48
49 Build WebAssembly binaries:
50 ```bash
51 $ cd fastText
52 $ make wasm
53 ```
54
55 This will create `fasttext_wasm.wasm` and `fasttext_wasm.js` in the `webassembly` folder.
56
57 - `fasttext_wasm.wasm` is the binary file that will be loaded in the webassembly's virtual machine.
58 - `fasttext_wasm.js` is a javascript file built by emscripten, that helps to load `fasttext_wasm.wasm` file in the virtual machine and provides some helper functions.
59 - `fasttext.js` is the wrapper that provides a nice API for fastText.
60
61 As the user of the library, we will interact with classes and methods defined in `fasttext.js`. We won't deal with `fasttext_wasm.*` files, but they are necessary to run fastText in the javascript's VM.
62
63 # Build a webpage that uses fastText
64
65 In this section we are going to build a minimal HTML page that loads fastText WebAssembly module.
66
67 At the root of the repository, create a folder `webassembly-test`, and copy the files mentioned in the previous section:
68
69 ```bash
70 $ mkdir webassembly-test
71 $ cp webassembly/fasttext_wasm.wasm webassembly-test/
72 $ cp webassembly/fasttext_wasm.js webassembly-test/
73 $ cp webassembly/fasttext.js webassembly-test/
74 ```
75
76 Inside that folder, create `test.html` file containing:
77 ```html
78 <!DOCTYPE html>
79 <html>
80 <head>
81 <meta charset="UTF-8">
82 <meta name="viewport" content="width=device-width, initial-scale=1, minimum-scale=1.0, maximum-scale=1.0, user-scalable=no">
83 </head>
84 <body>
85 <script type="module">
86 import {FastText, addOnPostRun} from "./fasttext.js";
87
88 addOnPostRun(() => {
89 let ft = new FastText();
90 console.log(ft);
91 });
92
93 </script>
94 </body>
95 </html>
96 ```
97
98 It is important to add the attribute `type="module"` to the script tag, because we use ES6 style imports. `addOnPostRun` is a function that helps to provide a handler that is called when fastText is successfully loaded in the virtual machine. Once we are called inside that function, we can create an instance of `FastText`, that we will use to access the api.
99
100
101 Let's test it.
102
103 Opening `test.html` directly in the browser won't work since we are dynamically loading webassembly resources. The `test.html` file must be served from a webserver. The easiest way to achieve this is to use python's simple http server module:
104
105 ```bash
106 $ cd webassembly-test
107 $ python -m SimpleHTTPServer
108 ```
109
110 Then browse `http://localhost:8000/test.html` in your browser. If everything worked as expected, you should see `FastText {f: FastText}` in the javascript console.
111
112
113 # Load a model
114
115 In order to load a fastText model that was already trained, we can use `loadModel` function. In the example below we use `lid.176.ftz` that you can download from [here](/docs/en/language-identification.html).
116
117 Place the model file you want to load inside the same directory than the HTML file, and inside the script part:
118 ```javascript
119 import {FastText, addOnPostRun} from "./fasttext.js";
120
121 const printVector = function(predictions) {
122 for (let i=0; i<predictions.size(); i++){
123 let prediction = predictions.get(i);
124 console.log(predictions.get(i));
125 }
126 }
127
128 addOnPostRun(() => {
129 let ft = new FastText();
130
131 const url = "lid.176.ftz";
132 ft.loadModel(url).then(model => {
133
134 console.log("Model loaded.")
135
136 let text = "Bonjour à tous. Ceci est du français";
137 console.log(text);
138 printVector(model.predict(text, 5, 0.0));
139
140 text = "Hello, world. This is english";
141 console.log(text);
142 printVector(model.predict(text, 5, 0.0));
143
144 text = "Merhaba dünya. Bu da türkçe"
145 console.log(text);
146 printVector(model.predict(text, 5, 0.0));
147 });
148 });
149 ```
150
151 `loadModel` function returns a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves to a `model` object. We can then use [`model` object](#model-object) to call various methods, such as `predict`.
152
153 We define `printVector` function that loops through a representation of `std::vector` in javascript, and displays the items. Here, we use it to display prediction results.
154
155 You can also refer to `webassembly/doc/examples/predict.html` in the source code.
156
157 # Calling other methods
158
159 Once the model is loaded, you can call any method like `model.getDimension()` or `model.getSubwords(word)`. You can refer to [this](#api) section of the document for a complete API. You can also have a look to `webassembly/doc/examples/misc.html` file in the source code for further examples.
160
161
162 # Train a model
163
164 ### Disclaimer
165
166 It is also possible to train a model inside the browser with fastText's WebAssembly API. The training can be slow because at the time of writing, it is not possible to use multithreading in WebAssembly (along with dynamic memory growth). So most of the time, we would train a model with the python or command line tool, eventually quantize it, and load it in the WebAssembly module. However, training a model inside the browser can be useful for creating animations or educational tools.
167
168 ### Text classification
169
170 Place the `cooking.train` file (as described [here](/docs/en/supervised-tutorial.html)) inside the same directory:
171
172 ```javascript
173 import {FastText, addOnPostRun} from "./fasttext.js";
174
175 const trainCallback = (progress, loss, wst, lr, eta) => {
176 console.log([progress, loss, wst, lr, eta]);
177 };
178
179 addOnPostRun(() => {
180 let ft = new FastText();
181
182 ft.trainSupervised("cooking.train", {
183 'lr':1.0,
184 'epoch':10,
185 'loss':'hs',
186 'wordNgrams':2,
187 'dim':50,
188 'bucket':200000
189 }, trainCallback).then(model => {
190 console.log('Trained.');
191 });
192 });
193 ```
194
195 `trainCallback` function is called by the module to show progress, average training cost, number of words per second (per thread, but there is only one thread), learning rate, estimated remaining time.
196
197
198 ### Word representations
199
200 Place the `fil9` file (as described [here](/docs/en/unsupervised-tutorial.html)) inside the same directory:
201
202 ```javascript
203 import {FastText, addOnPostRun} from "./fasttext.js";
204
205 const trainCallback = (progress, loss, wst, lr, eta) => {
206 console.log([progress, loss, wst, lr, eta]);
207 };
208
209 addOnPostRun(() => {
210 let ft = new FastText();
211
212 ft.trainUnsupervised("fil9", 'skipgram', {
213 'lr':0.1,
214 'epoch':1,
215 'loss':'ns',
216 'wordNgrams':2,
217 'dim':50,
218 'bucket':200000
219 }, trainCallback).then(model => {
220 console.log('Trained.');
221 });
222 });
223 ```
224
225 # Quantized models
226
227 Quantization is a technique that reduces the size of your models. You can quantize your model as [described here](/docs/en/faqs.html#how-can-i-reduce-the-size-of-my-fasttext-models).
228
229 You can load a quantized model in fastText's WebAssembly module, as we did in ["Load a model" section](#load-a-model).
230
231
232 In the context of web, it is particularly useful to have smaller models since they can be downloaded much faster. You can use our autotune feature as [described here](/docs/en/autotune.html#constrain-model-size) in order to find the best trade-off between accuracy and model size that fits your needs.
233
234
235 # API
236
237 ## `model` object
238
239 `trainSupervised`, `trainUnsupervised` and `loadModel` functions return a Promise that resolves to an instance of `FastTextModel` class, that we generaly name `model` object.
240
241 This object exposes several functions:
242
243 ```javascript
244 isQuant // true if the model is quantized.
245 getDimension // the dimension (size) of a lookup vector (hidden layer).
246 getWordVector(word) // the vector representation of `word`.
247 getSentenceVector(text) // the vector representation of `text`.
248 getNearestNeighbors(word, k=10) // nearest `k` neighbors of `word`.
249 getAnalogies(wordA, wordB, wordC, k) // nearest `k` neighbors of the operation `wordA - wordB + wordC`.
250 getWordId(word) // get the word id within the dictionary.
251 getSubwordId(subword) // the index (within input matrix) a subword hashes to.
252 getSubwords(word) // the subwords and their indicies.
253 getInputVector(ind) // given an index, get the corresponding vector of the Input Matrix.
254 predict(text, k = 1, threshold = 0.0) // Given a string, get a list of labels and a list of corresponding
255 // probabilities. k controls the number of returned labels.
256 getInputMatrix() // get a reference to the full input matrix of a (non-quantized) Model.
257 getOutputMatrix() // get a reference to the full output matrix of a (non-quantized) Model.
258 getWords() // get the entire list of words of the dictionary including the frequency
259 // of the individual words. This does not include any subwords. For that
260 // please consult the function get_subwords.
261 getLabels() // get the entire list of labels of the dictionary including the frequency
262 getLine(text) // split a line of text into words and labels.
263 saveModel() // saves the model file in WebAssembly's in-memory FS and returns a blob
264 test(url, k, threshold) // downloads the test file from the specified url, evaluates the supervised model with it.
265 ```
266
267 You can also have a look to `webassembly/doc/examples/misc.html` file in the source code for further examples.
268
269 ## `loadModel`
270
271 You can load a model as follows:
272
273 `ft.loadModel(url);`
274
275 `loadModel` returns a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves to a [`model` object](#model-object).
276
277
278 ## `trainSupervised`
279
280 You can train a text classification model with fastText's WebAssembly API as follows:
281
282 `ft.trainSupervised(trainFile, args, trainCallback);`
283
284 - `trainFile`: the url of the input file
285 - `args`: a dictionary with following keys:
286 ```javascript
287 lr # learning rate [0.1]
288 dim # size of word vectors [100]
289 ws # size of the context window [5]
290 epoch # number of epochs [5]
291 minCount # minimal number of word occurences [1]
292 minCountLabel # minimal number of label occurences [1]
293 minn # min length of char ngram [0]
294 maxn # max length of char ngram [0]
295 neg # number of negatives sampled [5]
296 wordNgrams # max length of word ngram [1]
297 loss # loss function {ns, hs, softmax, ova} [softmax]
298 bucket # number of buckets [2000000]
299 thread # number of threads [number of cpus]
300 lrUpdateRate # change the rate of updates for the learning rate [100]
301 t # sampling threshold [0.0001]
302 label # label prefix ['__label__']
303 ```
304 - `trainCallback` is the name of the function that will be called during training to provide various information. Set this argument to `null` if you don't need a callback, or provide a function that has the following signature: `function myCallback(progress, loss, wst, lr, eta){ ... }`
305
306 `trainSupervised` returns a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves to a [`model` object](#model-object).
307
308 ## `trainUnsupervised`
309
310 You can train a word representation model with fastText's WebAssembly API as follows:
311
312 `ft.trainUnsupervised(trainFile, modelname, args, trainCallback);`
313
314 - `trainFile`: the url of the input file
315 - `modelName`: must be `"cbow"` or `"skipgram"`
316 - `args`: a dictionary with following keys:
317 ```javascript
318 lr # learning rate [0.05]
319 dim # size of word vectors [100]
320 ws # size of the context window [5]
321 epoch # number of epochs [5]
322 minCount # minimal number of word occurences [5]
323 minn # min length of char ngram [3]
324 maxn # max length of char ngram [6]
325 neg # number of negatives sampled [5]
326 wordNgrams # max length of word ngram [1]
327 loss # loss function {ns, hs, softmax, ova} [ns]
328 bucket # number of buckets [2000000]
329 thread # number of threads [number of cpus]
330 lrUpdateRate # change the rate of updates for the learning rate [100]
331 t # sampling threshold [0.0001]
332 ```
333 - `trainCallback` is the name of the function that will be called during training to provide various information. Set this argument to `null` if you don't need a callback, or provide a function that has the following signature: `function myCallback(progress, loss, wst, lr, eta){ ... }`
334
335 `trainUnsupervised` returns a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves to a [`model` object](#model-object).
336
337
338