Debian Codegraph @

site/content/languages/tinygo.md

raw
1 +++
2 title = "TinyGo"
3 +++
4
5 ## Introduction
6
7 [TinyGo][1] is an alternative compiler for Go source code. It can generate
8 `%.wasm` files instead of architecture-specific binaries through two targets:
9
10 * `wasm`: for browser (JavaScript) use.
11 * `wasi`: for use outside the browser.
12
13 This document is maintained by wazero, which is a WebAssembly runtime that
14 embeds in Go applications. Hence, all notes below will be about TinyGo's
15 `wasi` target.
16
17 ## Overview
18
19 When TinyGo compiles a `%.go` file with its `wasi` target, the output `%.wasm`
20 depends on a subset of features in the [WebAssembly 2.0 Core specification]
21 ({{< ref "/specs#core" >}}) and [WASI]({{< ref "/specs#wasi" >}}) host
22 functions.
23
24 Unlike some compilers, TinyGo also supports importing custom host functions and
25 exporting functions back to the host.
26
27 Here's a basic example of source in TinyGo:
28
29 ```go
30 package main
31
32 //export add
33 func add(x, y uint32) uint32 {
34 return x + y
35 }
36
37 // main is required for the `wasi` target, even if it isn't used.
38 func main() {}
39 ```
40
41 The following is the minimal command to build a `%.wasm` binary.
42 ```bash
43 tinygo build -o main.wasm -target=wasi main.go
44 ```
45
46 The resulting wasm exports the `add` function so that the embedding host can
47 call it, regardless of if the host is written in Go or not.
48
49 ## Disclaimer
50
51 This document includes notes contributed by the wazero community. While wazero
52 includes TinyGo examples, and maintainers often contribute to TinyGo, this
53 isn't a TinyGo official document. For more help, consider the [TinyGo Using
54 WebAssembly Guide][4] or joining the [#TinyGo channel on the Gophers Slack][5].
55
56 Meanwhile, please help us [maintain][6] this document and [star our GitHub
57 repository][7], if it is helpful. Together, we can make WebAssembly easier on
58 the next person.
59
60 ## Constraints
61
62 Please read our overview of WebAssembly and
63 [constraints]({{< ref "_index.md#constraints" >}}). In short, expect
64 limitations in both language features and library choices when developing your
65 software.
66
67 ### Unsupported standard libraries
68
69 TinyGo does not completely implement the Go standard library when targeting
70 `wasi`. What is missing is documented [here][26].
71
72 The first constraint people notice is that `encoding/json` usage compiles, but
73 panics at runtime.
74 ```go
75 package main
76
77 import "encoding/json"
78
79 type response struct {
80 Ok bool `json:"ok"`
81 }
82
83 func main() {
84 var res response
85 if err := json.Unmarshal([]byte(`{"ok": true}`), &res); err != nil {
86 println(err)
87 }
88 }
89 ```
90 This is due to limited support for reflection, and effects other [serialization
91 tools][18] also. See [Frequently Asked Questions](#frequently-asked-questions)
92 for some workarounds.
93
94 ### Unsupported System Calls
95
96 You may also notice some other features not yet work. For example, the below
97 will compile, but print "readdir unimplemented : errno 54" at runtime.
98
99 ```go
100 package main
101
102 import "os"
103
104 func main() {
105 if _, err := os.ReadDir("."); err != nil {
106 println(err)
107 }
108 }
109 ```
110
111 The underlying error is often, but not always `syscall.ENOSYS` which is the
112 standard way to stub a syscall until it is implemented. If you are interested
113 in more, see [System Calls](#system-calls).
114
115 ## Memory
116
117 When TinyGo compiles go into wasm, it configures the WebAssembly linear memory
118 to an initial size of 2 pages (128KB), and marks a position in that memory as
119 the heap base. All memory beyond that is used for the Go heap.
120
121 Allocations within Go (compiled to `%.wasm`) are managed as one would expect.
122 The allocator can [grow][20] until `memory.grow` on the host returns -1.
123
124 ### Host Allocations
125
126 Sometimes a host function needs to allocate memory directly. For example, to
127 write JSON of a given length before invoking an exported function to parse it.
128
129 The below snippet is a realistic example of a function exported to the host,
130 who needs to allocate memory first.
131 ```go
132 //export configure
133 func configure(ptr uintptr, size uint32) {
134 json := ptrToString(ptr, size)
135 }
136 ```
137 Note: WebAssembly uses 32-bit memory addressing, so a `uintptr` is 32-bits.
138
139 The general flow is that the host allocates memory by calling an allocation
140 function with the size needed. Then, it writes data, in this case JSON, to the
141 memory offset (`ptr`). At that point, it can call a host function, ex
142 `configure`, passing the `ptr` and `size` allocated. The guest wasm (compiled
143 from Go) will be able to read the data. To ensure no memory leaks, the host
144 calls a free function, with the same `ptr`, afterwards and unconditionally.
145
146 Note: wazero includes an [example project][8] that shows this.
147
148 The general call patterns are the following. Host is the process embedding the
149 WebAssembly runtime, such as wazero. Guest is the TinyGo source compiled to
150 target wasi.
151
152 * Host allocates a string to call an exported Guest function
153 * Host calls the built-in export `malloc` to get the memory offset to write
154 the string, which is passed as a parameter to the exported Guest function.
155 The host owns that allocation, so must call the built-in export `free` when
156 done. The Guest uses `ptrToString` to retrieve the string from the Wasm
157 parameters.
158 * Guest passes a string to an imported Host function
159 * Guest uses `stringToPtr` to get the memory offset needed by the Host
160 function. The host reads that string directly from Wasm memory. The
161 original string is subject to garbage collection on the Guest, so the Host
162 shouldn't call the built-in export `free` on it.
163 * Guest returns a string from an exported function
164 * Guest uses `ptrToLeakedString` to get the memory offset needed by the Host,
165 and returns it and the length. This is a transfer of ownership, so the
166 string won't be garbage collected on the Guest. The host reads that string
167 directly from Wasm memory and must call the built-in export `free` when
168 complete.
169
170 The built-in `malloc` and `free` functions the Host calls like this in the
171 WebAssembly text format.
172 ```webassembly
173 (func (export "malloc") (param $size i32) (result (;$ptr;) i32))
174 (func (export "free") (param $ptr i32))
175 ```
176
177 The other Guest function, such as `ptrToString` are too much code to inline
178 into this document, If you need these, you can copy them from the
179 [example project][8] or add a dependency on [tinymem][9].
180
181 ## System Calls
182
183 Please read our overview of WebAssembly and
184 [System Calls]({{< ref "_index.md#system-calls" >}}). In short, WebAssembly is
185 a stack-based virtual machine specification, so operates at a lower level than
186 an operating system.
187
188 For functionality the operating system would otherwise provide, TinyGo imports
189 host functions defined in [WASI]({{< ref "/specs#wasi" >}}).
190
191 For example, `tinygo build -o main.wasm -target=wasi main.go` compiles the
192 below `main` function into a WASI function exported as `_start`.
193
194 When the WebAssembly runtime calls `_start`, you'll see the effective
195 `GOARCH=wasm` and `GOOS=linux`.
196
197 ```go
198 package main
199
200 import (
201 "fmt"
202 "runtime"
203 )
204
205 func main() {
206 fmt.Println(runtime.GOARCH, runtime.GOOS)
207 }
208 ```
209
210 Note: wazero includes an [example WASI project][21] including [source code][22]
211 that implements `cat` without any WebAssembly-specific code.
212
213 ### WASI Internals
214
215 While developing WASI in TinyGo is outside the scope of this document, the
216 below pointers will help you understand the underlying architecture of the
217 `wasi` target. Ideally, these notes can help you frame support or feature
218 requests with the TinyGo team.
219
220 A close look at the [wasi target][11] reveals how things work. Underneath,
221 TinyGo leverages the `wasm32-unknown-wasi` LLVM target for the system call
222 layer (libc), which is eventually implemented by the [wasi-libc][12] library.
223
224 Similar to normal code, TinyGo decides which abstraction to use with GOOS and
225 GOARCH specific suffixes and build flags.
226
227 For example, `os.Args` is implemented directly using WebAssembly host functions
228 in [runtime_wasm_wasi.go][13]. `syscall.Chdir` is implemented with the same
229 [syscall_libc.go][14] used for other architectures, while `syscall.ReadDirent`
230 is stubbed (returns `syscall.ENOSYS`), in [syscall_libc_wasi.go][15].
231
232 ## Concurrency
233
234 Please read our overview of WebAssembly and
235 [concurrency]({{< ref "_index.md#concurrency" >}}). In short, the current
236 WebAssembly specification does not support parallel processing.
237
238 Tinygo uses only one core/thread regardless of target. This happens to be a
239 good match for Wasm's current lack of support for (multiple) threads. Tinygo's
240 goroutine scheduler on Wasm currently uses Binaryen's [Asyncify][23], a Wasm
241 postprocessor also used by other languages targeting Wasm to provide similar
242 concurrency.
243
244 In summary, TinyGo supports goroutines by default and acts like `GOMAXPROCS=1`.
245 Since [goroutines are not threads][24], the following code will run with the
246 expected output, despite goroutines defined in opposite dependency order.
247 ```go
248 package main
249
250 import "fmt"
251
252 func main() {
253 msg := make(chan int)
254 finished := make(chan int)
255 go func() {
256 <-msg
257 fmt.Println("consumer")
258 finished <- 1
259 }()
260 go func() {
261 fmt.Println("producer")
262 msg <- 1
263 }()
264 <-finished
265 }
266 ```
267
268 There are some glitches to this. For example, if that same function was
269 exported (`//export notMain`), and called while main wasn't running, the line
270 that creates a goroutine currently [panics at runtime][25].
271
272 Given problems like this, some choose a compile-time failure instead, via
273 `-scheduler=none`. Since code often needs to be custom in order to work with
274 wasm anyway, there may be limited impact to removing goroutine support.
275
276 ## Optimizations
277
278 Below are some commonly used configurations that allow optimizing for size or
279 performance vs defaults. Note that sometimes one sacrifices the other.
280
281 ### Binary size
282
283 Those with `%.wasm` binary size constraints can set `tinygo` flags to reduce
284 it. For example, a simple `cat` program can reduce from default of 260KB to
285 60KB using both flags below.
286
287 * `-scheduler=none`: Reduces size, but fails at compile time on goroutines.
288 * `--no-debug`: Strips DWARF, but retains the WebAssembly name section.
289
290 ### Performance
291
292 Those with runtime performance constraints can set `tinygo` flags to improve
293 it.
294
295 * `-gc=leaking`: Avoids GC which improves performance for short-lived programs.
296 * `-opt=2`: Enable additional optimizations, frequently at the expense of binary
297 size.
298
299 ## Frequently Asked Questions
300
301 ### Why do I have to define main?
302
303 If you are using TinyGo's `wasi` target, you should define at least a no-op
304 `func main() {}` in your source.
305
306 If you don't, instantiation of the WebAssembly will fail unless you've exported
307 the following from the host:
308 ```webassembly
309 (func (import "env" "main.main") (param i32) (result i32))
310 ```
311
312 ### How do I use json?
313 TinyGo doesn't yet implement [reflection APIs][16] needed by `encoding/json`.
314 Meanwhile, most users resort to non-reflective parsers, such as [gjson][17].
315
316 ### Why does my wasm import WASI functions even when I don't use it?
317 TinyGo has a `wasm` target (for browsers) and a `wasi` target for runtimes that
318 support [WASI]({{< ref "/specs#wasi" >}}). This document is written only about
319 the `wasi` target.
320
321 Some users are surprised to see imports from WASI (`wasi_snapshot_preview1`),
322 when their neither has a main function nor uses memory. At least implementing
323 `panic` requires writing to the console, and `fd_write` is used for this.
324
325 A bare or standalone WebAssembly target doesn't yet exist, but if interested,
326 you can follow [this issue][19].
327
328 ### Why is my `%.wasm` binary so big?
329 TinyGo defaults can be overridden for those who can sacrifice features or
330 performance for a [smaller binary](#binary-size). After that, tuning your
331 source code may reduce binary size further.
332
333 TinyGo minimally needs to implement garbage collection and `panic`, and the
334 wasm to implement that is often not considered big (~4KB). What's often
335 surprising to users are APIs that seem simple, but require a lot of supporting
336 functions, such as `fmt.Println`, which can require 100KB of wasm.
337
338 [1]: https://tinygo.org/
339 [4]: https://tinygo.org/docs/guides/webassembly/
340 [5]: https://github.com/tinygo-org/tinygo#getting-help
341 [6]: https://github.com/tetratelabs/wazero/tree/main/site/content/languages/tinygo.md
342 [7]: https://github.com/tetratelabs/wazero/stargazers
343 [8]: https://github.com/tetratelabs/wazero/tree/main/examples/allocation/tinygo
344 [9]: https://github.com/tetratelabs/tinymem
345 [11]: https://github.com/tinygo-org/tinygo/blob/v0.25.0/targets/wasi.json
346 [12]: https://github.com/WebAssembly/wasi-libc
347 [13]: https://github.com/tinygo-org/tinygo/blob/v0.25.0/src/runtime/runtime_wasm_wasi.go#L34-L62
348 [14]: https://github.com/tinygo-org/tinygo/blob/v0.25.0/src/syscall/syscall_libc.go#L85-L92
349 [15]: https://github.com/tinygo-org/tinygo/blob/v0.25.0/src/syscall/syscall_libc_wasi.go#L263-L265
350 [16]: https://github.com/tinygo-org/tinygo/issues/2660
351 [17]: https://github.com/tidwall/gjson
352 [18]: https://github.com/tinygo-org/tinygo/issues/447
353 [19]: https://github.com/tinygo-org/tinygo/issues/3068
354 [20]: https://github.com/tinygo-org/tinygo/blob/v0.25.0/src/runtime/arch_tinygowasm.go#L47-L62
355 [21]: https://github.com/tetratelabs/wazero/tree/main/imports/wasi_snapshot_preview1/example
356 [22]: https://github.com/tetratelabs/wazero/tree/main/imports/wasi_snapshot_preview1/example/testdata/tinygo
357 [23]: https://github.com/WebAssembly/binaryen/blob/main/src/passes/Asyncify.cpp
358 [24]: http://tleyden.github.io/blog/2014/10/30/goroutines-vs-threads/
359 [25]: https://github.com/tinygo-org/tinygo/issues/3095
360 [26]: https://tinygo.org/docs/reference/lang-support/stdlib/