| 1 |
# Linking psx and C code without cgo |
| 2 |
|
| 3 |
## Overview |
| 4 |
|
| 5 |
In some embedded situations, there is a desire to compile Go binaries |
| 6 |
to include some C code, but not `libc` etc. For a long time, I had |
| 7 |
assumed this was not possible, since using `cgo` *requires* `libc` and |
| 8 |
`libpthread` linkage. |
| 9 |
|
| 10 |
This _embedded compilation_ need was referenced in a [bug |
| 11 |
filed](https://bugzilla.kernel.org/show_bug.cgi?id=216610) against the |
| 12 |
[`"psx"`](https://pkg.go.dev/kernel.org/pub/linux/libs/security/libcap/psx) |
| 13 |
package. The bug-filer was seeking an alternative to `CGO_ENABLED=1` |
| 14 |
compilation _requiring_ the `cgo` variant of `psx` build. However, the |
| 15 |
go `"runtime"` package will always |
| 16 |
[`panic()`](https://cs.opensource.google/go/go/+/refs/tags/go1.19.2:src/runtime/os_linux.go;l=717-720) |
| 17 |
if you try this because it needs `libpthread` and `[g]libc` to work. |
| 18 |
|
| 19 |
In researching that bug report, however, I have learned there is a |
| 20 |
trick to combining a non-CGO built binary with compiled C code. I |
| 21 |
learned about it from a brief reference in the [Go Programming |
| 22 |
Language |
| 23 |
Wiki](https://zchee.github.io/golang-wiki/GcToolchainTricks/). |
| 24 |
|
| 25 |
This present directory evolved from my attempt to understand and |
| 26 |
hopefully resolve what was going on as reported in that bug into an |
| 27 |
example of this _trick_. I was unable to resolve the problem as |
| 28 |
reported because of the aformentioned `panic()` in the Go |
| 29 |
runtime. However, I was able to demonstrate embedding C code in a Go |
| 30 |
binary _without_ use of cgo. In such a binary, the Go-native version |
| 31 |
of `"psx"` is thus achievable. This is what the example in this |
| 32 |
present directory demonstrates. |
| 33 |
|
| 34 |
*Caveat Emptor*: this example is very fragile. The Go team only |
| 35 |
supports `cgo` linking against C. That being said, I'd certainly like |
| 36 |
to receive bug fixes, etc for this directory if you find you need to |
| 37 |
evolve it to make it work for your use case. |
| 38 |
|
| 39 |
## Content |
| 40 |
|
| 41 |
In this example we have: |
| 42 |
|
| 43 |
- Some C code for the functions `fib_init()` and `fib_next()` that |
| 44 |
combine to implement a _compute engine_ to determine [Fibonacci |
| 45 |
Numbers](https://en.wikipedia.org/wiki/Fibonacci_number). The source |
| 46 |
for this is in the sub directory `c/fib.c`. |
| 47 |
|
| 48 |
- Some Go code, in the directory `go/fibber` that uses this C compiled |
| 49 |
compute kernel. |
| 50 |
|
| 51 |
- `c/gcc.sh` which is a wrapper for `gcc` that adjusts the compilation |
| 52 |
to be digestible by Go's (internal) linker (the one that gets invoked |
| 53 |
when compiling `CGO_ENABLED=0`. Using `gcc` directly instead of this |
| 54 |
wrapper generates an incomplete binary - which miscomputes the |
| 55 |
expected answers. See the discussion below for what seems to be going |
| 56 |
on. |
| 57 |
|
| 58 |
- A top level `Makefile` to build it all. |
| 59 |
|
| 60 |
## Building and running the built binary |
| 61 |
|
| 62 |
Set things up with: |
| 63 |
``` |
| 64 |
$ git clone git://git.kernel.org/pub/scm/libs/libcap/libcap.git |
| 65 |
$ cd libcap |
| 66 |
$ make all |
| 67 |
$ cd contrib/bug216610 |
| 68 |
$ make clean all |
| 69 |
``` |
| 70 |
When you run `./go/fib` it should generate the following output: |
| 71 |
``` |
| 72 |
$ ./go/fib |
| 73 |
psx syscall result: PID=<nnnnn> |
| 74 |
fib: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ... |
| 75 |
$ |
| 76 |
``` |
| 77 |
Where `<nnnnn>` is the PID of the program at runtime and will be |
| 78 |
different each time the program is invoked. |
| 79 |
|
| 80 |
## Discussion |
| 81 |
|
| 82 |
The Fibonacci detail of what is going on is mostly uninteresting. The |
| 83 |
reason for developing this example was to explore the build issues in |
| 84 |
the reported [Bug |
| 85 |
216610](https://bugzilla.kernel.org/show_bug.cgi?id=216610). Ultimately, |
| 86 |
this example offers an alternative path to building a `nocgo` program |
| 87 |
that links to compute kernel of C code. |
| 88 |
|
| 89 |
The reason we have added the `c/gcc.sh` wrapper for `gcc` is that |
| 90 |
we've found the Go linker has a hard time digesting the |
| 91 |
cross-sectional `%rip` based data addressing that various optimization |
| 92 |
modes of gcc like to use. Specifically, in the x86_64/amd64 |
| 93 |
architecture, if a `R_X86_64_PC32` relocation entry made in a `.text` |
| 94 |
section refers to an `.rodata.cst8` section in a generated `.syso` |
| 95 |
file, the Go linker seems to [replace this reference with a `0` offset |
| 96 |
to |
| 97 |
`(%rip)`](https://github.com/golang/go/issues/24321#issuecomment-1296084103). What |
| 98 |
our wrapper script does is rewrite the generated assembly to store |
| 99 |
these data references to the `.text` section. The Go linker has no |
| 100 |
problem with this _same section_ relative addressing and is able to |
| 101 |
link the resulting objects without problems. |
| 102 |
|
| 103 |
If you want to cross compile, we have support for 32-bit arm |
| 104 |
compilation: what is needed for the Raspberry PI. To get this support, |
| 105 |
try: |
| 106 |
``` |
| 107 |
$ make clean all arms |
| 108 |
$ cd go |
| 109 |
$ GOARCH=arm CGO_ENABLED=0 go build |
| 110 |
``` |
| 111 |
The generated `fib` binary runs on a 32-bit Raspberry Pi. |
| 112 |
|
| 113 |
## Future thoughts |
| 114 |
|
| 115 |
At present, this example only works on Linux with `x86_64` and `arm` |
| 116 |
build architectures. (In go-speak that is `linux_amd64` and |
| 117 |
`linux_arm`). This is because I have only provided some bridging |
| 118 |
assembly for Go to C calling conventions for those architecture |
| 119 |
targets: `./go/fibber/fibs_linux_amd64.s` and |
| 120 |
`./go/fibber/fibs_linux_arm.s`. The non-native, `make arms`, cross |
| 121 |
compilation requires the `docker` command to be available. |
| 122 |
|
| 123 |
I intend to implement an `arm64` build, when I have a system on which |
| 124 |
to test it. |
| 125 |
|
| 126 |
**Note** The Fedora system on which I've been developing this has some |
| 127 |
SELINUX impediment to naively using the `docker -v ...` bind mount |
| 128 |
option. I need the `:z` suffix for bind mounting. I don't know how |
| 129 |
common an issue this is. On Fedora, building the arm variants of the |
| 130 |
.syso file can be performed as follows: |
| 131 |
``` |
| 132 |
$ docker run --rm -v $PWD/c:/shared:z -h debian -u $(id -u) -it expt shared/build.sh |
| 133 |
``` |
| 134 |
|
| 135 |
## Reporting bugs |
| 136 |
|
| 137 |
Please report issues or offer improvements to this example via the |
| 138 |
[Fully Capable `libcap`](https://sites.google.com/site/fullycapable/) |
| 139 |
website. |