| 1 |
# Logs API |
| 2 |
|
| 3 |
## Abstract |
| 4 |
|
| 5 |
`go.opentelemetry.io/otel/log` provides |
| 6 |
[Logs API](https://opentelemetry.io/docs/specs/otel/logs/api/). |
| 7 |
|
| 8 |
The prototype was created in |
| 9 |
[#4725](https://github.com/open-telemetry/opentelemetry-go/pull/4725). |
| 10 |
|
| 11 |
## Background |
| 12 |
|
| 13 |
The key challenge is to create a performant API compliant with the [specification](https://opentelemetry.io/docs/specs/otel/logs/api/) |
| 14 |
with an intuitive and user friendly design. |
| 15 |
Performance is seen as one of the most important characteristics of logging libraries in Go. |
| 16 |
|
| 17 |
## Design |
| 18 |
|
| 19 |
This proposed design aims to: |
| 20 |
|
| 21 |
- be specification compliant, |
| 22 |
- be similar to Trace and Metrics API, |
| 23 |
- take advantage of both OpenTelemetry and `slog` experience to achieve acceptable performance. |
| 24 |
|
| 25 |
### Module structure |
| 26 |
|
| 27 |
The API is published as a single `go.opentelemetry.io/otel/log` Go module. |
| 28 |
|
| 29 |
The package structure is similar to Trace API and Metrics API. |
| 30 |
The Go module consists of the following packages: |
| 31 |
|
| 32 |
- `go.opentelemetry.io/otel/log` |
| 33 |
- `go.opentelemetry.io/otel/log/embedded` |
| 34 |
- `go.opentelemetry.io/otel/log/logtest` |
| 35 |
- `go.opentelemetry.io/otel/log/noop` |
| 36 |
|
| 37 |
Rejected alternative: |
| 38 |
|
| 39 |
- [Reuse slog](#reuse-slog) |
| 40 |
|
| 41 |
### LoggerProvider |
| 42 |
|
| 43 |
The [`LoggerProvider` abstraction](https://opentelemetry.io/docs/specs/otel/logs/api/#loggerprovider) |
| 44 |
is defined as `LoggerProvider` interface in [provider.go](provider.go). |
| 45 |
|
| 46 |
The specification may add new operations to `LoggerProvider`. |
| 47 |
The interface may have methods added without a package major version bump. |
| 48 |
This embeds `embedded.LoggerProvider` to help inform an API implementation |
| 49 |
author about this non-standard API evolution. |
| 50 |
This approach is already used in Trace API and Metrics API. |
| 51 |
|
| 52 |
#### LoggerProvider.Logger |
| 53 |
|
| 54 |
The `Logger` method implements the [`Get a Logger` operation](https://opentelemetry.io/docs/specs/otel/logs/api/#get-a-logger). |
| 55 |
|
| 56 |
The required `name` parameter is accepted as a `string` method argument. |
| 57 |
|
| 58 |
The `LoggerOption` options are defined to support optional parameters. |
| 59 |
|
| 60 |
Implementation requirements: |
| 61 |
|
| 62 |
- The [specification requires](https://opentelemetry.io/docs/specs/otel/logs/api/#concurrency-requirements) |
| 63 |
the method to be safe to be called concurrently. |
| 64 |
|
| 65 |
- The method should use some default name if the passed name is empty |
| 66 |
in order to meet the [specification's SDK requirement](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logger-creation) |
| 67 |
to return a working logger when an invalid name is passed |
| 68 |
as well as to resemble the behavior of getting tracers and meters. |
| 69 |
|
| 70 |
`Logger` can be extended by adding new `LoggerOption` options |
| 71 |
and adding new exported fields to the `LoggerConfig` struct. |
| 72 |
This design is already used in Trace API for getting tracers |
| 73 |
and in Metrics API for getting meters. |
| 74 |
|
| 75 |
Rejected alternative: |
| 76 |
|
| 77 |
- [Passing struct as parameter to LoggerProvider.Logger](#passing-struct-as-parameter-to-loggerproviderlogger). |
| 78 |
|
| 79 |
### Logger |
| 80 |
|
| 81 |
The [`Logger` abstraction](https://opentelemetry.io/docs/specs/otel/logs/api/#logger) |
| 82 |
is defined as `Logger` interface in [logger.go](logger.go). |
| 83 |
|
| 84 |
The specification may add new operations to `Logger`. |
| 85 |
The interface may have methods added without a package major version bump. |
| 86 |
This embeds `embedded.Logger` to help inform an API implementation |
| 87 |
author about this non-standard API evolution. |
| 88 |
This approach is already used in Trace API and Metrics API. |
| 89 |
|
| 90 |
### Logger.Emit |
| 91 |
|
| 92 |
The `Emit` method implements the [`Emit a LogRecord` operation](https://opentelemetry.io/docs/specs/otel/logs/api/#emit-a-logrecord). |
| 93 |
|
| 94 |
[`Context` associated with the `LogRecord`](https://opentelemetry.io/docs/specs/otel/context/) |
| 95 |
is accepted as a `context.Context` method argument. |
| 96 |
|
| 97 |
Calls to `Emit` are supposed to be on the hot path. |
| 98 |
Therefore, in order to reduce the number of heap allocations, |
| 99 |
the [`LogRecord` abstraction](https://opentelemetry.io/docs/specs/otel/logs/api/#emit-a-logrecord), |
| 100 |
is defined as `Record` struct in [record.go](record.go). |
| 101 |
|
| 102 |
[`Timestamp`](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-timestamp) |
| 103 |
is accessed using following methods: |
| 104 |
|
| 105 |
```go |
| 106 |
func (r *Record) Timestamp() time.Time |
| 107 |
func (r *Record) SetTimestamp(t time.Time) |
| 108 |
``` |
| 109 |
|
| 110 |
[`ObservedTimestamp`](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-observedtimestamp) |
| 111 |
is accessed using following methods: |
| 112 |
|
| 113 |
```go |
| 114 |
func (r *Record) ObservedTimestamp() time.Time |
| 115 |
func (r *Record) SetObservedTimestamp(t time.Time) |
| 116 |
``` |
| 117 |
|
| 118 |
[`EventName`](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-eventname) |
| 119 |
is accessed using following methods: |
| 120 |
|
| 121 |
```go |
| 122 |
func (r *Record) EventName() string |
| 123 |
func (r *Record) SetEventName(s string) |
| 124 |
``` |
| 125 |
|
| 126 |
[`SeverityNumber`](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-severitynumber) |
| 127 |
is accessed using following methods: |
| 128 |
|
| 129 |
```go |
| 130 |
func (r *Record) Severity() Severity |
| 131 |
func (r *Record) SetSeverity(s Severity) |
| 132 |
``` |
| 133 |
|
| 134 |
`Severity` type is defined in [severity.go](severity.go). |
| 135 |
The constants are are based on |
| 136 |
[Displaying Severity recommendation](https://opentelemetry.io/docs/specs/otel/logs/data-model/#displaying-severity). |
| 137 |
Additionally, `Severity[Level]` constants are defined to make the API more readable and user friendly. |
| 138 |
|
| 139 |
[`SeverityText`](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-severitytext) |
| 140 |
is accessed using following methods: |
| 141 |
|
| 142 |
```go |
| 143 |
func (r *Record) SeverityText() string |
| 144 |
func (r *Record) SetSeverityText(s string) |
| 145 |
``` |
| 146 |
|
| 147 |
[`Body`](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-body) |
| 148 |
is accessed using following methods: |
| 149 |
|
| 150 |
```go |
| 151 |
func (r *Record) Body() Value |
| 152 |
func (r *Record) SetBody(v Value) |
| 153 |
``` |
| 154 |
|
| 155 |
[Log record attributes](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-attributes) |
| 156 |
are accessed using following methods: |
| 157 |
|
| 158 |
```go |
| 159 |
func (r *Record) WalkAttributes(f func(KeyValue) bool) |
| 160 |
func (r *Record) AddAttributes(attrs ...KeyValue) |
| 161 |
``` |
| 162 |
|
| 163 |
`Record` has a `AttributesLen` method that returns |
| 164 |
the number of attributes to allow slice preallocation |
| 165 |
when converting records to a different representation: |
| 166 |
|
| 167 |
```go |
| 168 |
func (r *Record) AttributesLen() int |
| 169 |
``` |
| 170 |
|
| 171 |
The records attributes design and implementation is based on |
| 172 |
[`slog.Record`](https://pkg.go.dev/log/slog#Record). |
| 173 |
It allows achieving high-performance access and manipulation of the attributes |
| 174 |
while keeping the API user friendly. |
| 175 |
It relieves the user from making his own improvements |
| 176 |
for reducing the number of allocations when passing attributes. |
| 177 |
|
| 178 |
The abstractions described in |
| 179 |
[the specification](https://opentelemetry.io/docs/specs/otel/logs/#new-first-party-application-logs) |
| 180 |
are defined in [keyvalue.go](keyvalue.go). |
| 181 |
|
| 182 |
`Value` is representing `any`. |
| 183 |
`KeyValue` is representing a key(string)-value(`any`) pair. |
| 184 |
|
| 185 |
`Kind` is an enumeration used for specifying the underlying value type. |
| 186 |
`KindEmpty` is used for an empty (zero) value. |
| 187 |
`KindBool` is used for boolean value. |
| 188 |
`KindFloat64` is used for a double precision floating point (IEEE 754-1985) value. |
| 189 |
`KindInt64` is used for a signed integer value. |
| 190 |
`KindString` is used for a string value. |
| 191 |
`KindBytes` is used for a slice of bytes (in spec: A byte array). |
| 192 |
`KindSlice` is used for a slice of values (in spec: an array (a list) of any values). |
| 193 |
`KindMap` is used for a slice of key-value pairs (in spec: `map<string, any>`). |
| 194 |
|
| 195 |
These types are defined in `go.opentelemetry.io/otel/log` package |
| 196 |
as they are tightly coupled with the API and different from common attributes. |
| 197 |
|
| 198 |
The internal implementation of `Value` is based on |
| 199 |
[`slog.Value`](https://pkg.go.dev/log/slog#Value) |
| 200 |
and the API is mostly inspired by |
| 201 |
[`attribute.Value`](https://pkg.go.dev/go.opentelemetry.io/otel/attribute#Value). |
| 202 |
The benchmarks[^1] show that the implementation is more performant than |
| 203 |
[`attribute.Value`](https://pkg.go.dev/go.opentelemetry.io/otel/attribute#Value). |
| 204 |
|
| 205 |
The value accessors (`func (v Value) As[Kind]` methods) must not panic, |
| 206 |
as it would violate the [specification](https://opentelemetry.io/docs/specs/otel/error-handling/): |
| 207 |
|
| 208 |
> API methods MUST NOT throw unhandled exceptions when used incorrectly by end |
| 209 |
> users. The API and SDK SHOULD provide safe defaults for missing or invalid |
| 210 |
> arguments. [...] Whenever the library suppresses an error that would otherwise |
| 211 |
> have been exposed to the user, the library SHOULD log the error using |
| 212 |
> language-specific conventions. |
| 213 |
|
| 214 |
Therefore, the value accessors should return a zero value |
| 215 |
and log an error when a bad accessor is called. |
| 216 |
|
| 217 |
The `Severity`, `Kind`, `Value`, `KeyValue` may implement |
| 218 |
the [`fmt.Stringer`](https://pkg.go.dev/fmt#Stringer) interface. |
| 219 |
However, it is not needed for the first stable release |
| 220 |
and the `String` methods can be added later. |
| 221 |
|
| 222 |
The caller must not subsequently mutate the record passed to `Emit`. |
| 223 |
This would allow the implementation to not clone the record, |
| 224 |
but simply retain, modify or discard it. |
| 225 |
The implementation may still choose to clone the record or copy its attributes |
| 226 |
if it needs to retain or modify it, |
| 227 |
e.g. in case of asynchronous processing to eliminate the possibility of data races, |
| 228 |
because the user can technically reuse the record and add new attributes |
| 229 |
after the call (even when the documentation says that the caller must not do it). |
| 230 |
|
| 231 |
Implementation requirements: |
| 232 |
|
| 233 |
- The [specification requires](https://opentelemetry.io/docs/specs/otel/logs/api/#concurrency-requirements) |
| 234 |
the method to be safe to be called concurrently. |
| 235 |
|
| 236 |
- The method must not interrupt the record processing if the context is canceled |
| 237 |
per ["ignoring context cancellation" guideline](../CONTRIBUTING.md#ignoring-context-cancellation). |
| 238 |
|
| 239 |
- The [specification requires](https://opentelemetry.io/docs/specs/otel/logs/api/#emit-a-logrecord) |
| 240 |
use the current time as observed timestamp if the passed is empty. |
| 241 |
|
| 242 |
- The method should handle the trace context passed via `ctx` argument in order to meet the |
| 243 |
[specification's SDK requirement](https://opentelemetry.io/docs/specs/otel/logs/sdk/#readablelogrecord) |
| 244 |
to populate the trace context fields from the resolved context. |
| 245 |
|
| 246 |
`Emit` can be extended by adding new exported fields to the `Record` struct. |
| 247 |
|
| 248 |
Rejected alternatives: |
| 249 |
|
| 250 |
- [Record as interface](#record-as-interface) |
| 251 |
- [Options as parameter to Logger.Emit](#options-as-parameter-to-loggeremit) |
| 252 |
- [Passing record as pointer to Logger.Emit](#passing-record-as-pointer-to-loggeremit) |
| 253 |
- [Logger.WithAttributes](#loggerwithattributes) |
| 254 |
- [Record attributes as slice](#record-attributes-as-slice) |
| 255 |
- [Use any instead of defining Value](#use-any-instead-of-defining-value) |
| 256 |
- [Severity type encapsulating number and text](#severity-type-encapsulating-number-and-text) |
| 257 |
- [Reuse attribute package](#reuse-attribute-package) |
| 258 |
- [Mix receiver types for Record](#mix-receiver-types-for-record) |
| 259 |
- [Add XYZ method to Logger](#add-xyz-method-to-logger) |
| 260 |
- [Rename KeyValue to Attr](#rename-keyvalue-to-attr) |
| 261 |
|
| 262 |
### Logger.Enabled |
| 263 |
|
| 264 |
The `Enabled` method implements the [`Enabled` operation](https://opentelemetry.io/docs/specs/otel/logs/api/#enabled). |
| 265 |
|
| 266 |
[`Context` associated with the `LogRecord`](https://opentelemetry.io/docs/specs/otel/context/) |
| 267 |
is accepted as a `context.Context` method argument. |
| 268 |
|
| 269 |
Calls to `Enabled` are supposed to be on the hot path and the list of arguments |
| 270 |
can be extendend in future. Therefore, in order to reduce the number of heap |
| 271 |
allocations and make it possible to handle new arguments, `Enabled` accepts |
| 272 |
a `EnabledParameters` struct, defined in [logger.go](logger.go), as the second |
| 273 |
method argument. |
| 274 |
|
| 275 |
The `EnabledParameters` uses fields, instead of getters and setters, to allow |
| 276 |
simpler usage which allows configuring the `EnabledParameters` in the same line |
| 277 |
where `Enabled` is called. |
| 278 |
|
| 279 |
### noop package |
| 280 |
|
| 281 |
The `go.opentelemetry.io/otel/log/noop` package provides |
| 282 |
[Logs API No-Op Implementation](https://opentelemetry.io/docs/specs/otel/logs/noop/). |
| 283 |
|
| 284 |
### Trace context correlation |
| 285 |
|
| 286 |
The bridge implementation should do its best to pass |
| 287 |
the `ctx` containing the trace context from the caller |
| 288 |
so it can later be passed via `Logger.Emit`. |
| 289 |
|
| 290 |
It is not expected that users (caller or bridge implementation) reconstruct |
| 291 |
a `context.Context`. Reconstructing a `context.Context` with |
| 292 |
[`trace.ContextWithSpanContext`](https://pkg.go.dev/go.opentelemetry.io/otel/trace#ContextWithSpanContext) |
| 293 |
and [`trace.NewSpanContext`](https://pkg.go.dev/go.opentelemetry.io/otel/trace#NewSpanContext) |
| 294 |
would usually involve more memory allocations. |
| 295 |
|
| 296 |
The logging libraries which have recording methods that accepts `context.Context`, |
| 297 |
such us [`slog`](https://pkg.go.dev/log/slog), |
| 298 |
[`logrus`](https://pkg.go.dev/github.com/sirupsen/logrus), |
| 299 |
[`zerolog`](https://pkg.go.dev/github.com/rs/zerolog), |
| 300 |
makes passing the trace context trivial. |
| 301 |
|
| 302 |
However, some libraries do not accept a `context.Context` in their recording methods. |
| 303 |
Structured logging libraries, |
| 304 |
such as [`logr`](https://pkg.go.dev/github.com/go-logr/logr) |
| 305 |
and [`zap`](https://pkg.go.dev/go.uber.org/zap), |
| 306 |
offer passing `any` type as a log attribute/field. |
| 307 |
Therefore, their bridge implementations can define a "special" log attributes/field |
| 308 |
that will be used to capture the trace context. |
| 309 |
|
| 310 |
[The prototype](https://github.com/open-telemetry/opentelemetry-go/pull/4725) |
| 311 |
has bridge implementations that handle trace context correlation efficiently. |
| 312 |
|
| 313 |
## Benchmarking |
| 314 |
|
| 315 |
The benchmarks take inspiration from [`slog`](https://pkg.go.dev/log/slog), |
| 316 |
because for the Go team it was also critical to create API that would be fast |
| 317 |
and interoperable with existing logging packages.[^2][^3] |
| 318 |
|
| 319 |
The benchmark results can be found in [the prototype](https://github.com/open-telemetry/opentelemetry-go/pull/4725). |
| 320 |
|
| 321 |
## Rejected alternatives |
| 322 |
|
| 323 |
### Reuse slog |
| 324 |
|
| 325 |
The API must not be coupled to [`slog`](https://pkg.go.dev/log/slog), |
| 326 |
nor any other logging library. |
| 327 |
|
| 328 |
The API needs to evolve orthogonally to `slog`. |
| 329 |
|
| 330 |
`slog` is not compliant with the [Logs API](https://opentelemetry.io/docs/specs/otel/logs/api/). |
| 331 |
and we cannot expect the Go team to make `slog` compliant with it. |
| 332 |
|
| 333 |
The interoperability can be achieved using [a log bridge](https://opentelemetry.io/docs/specs/otel/glossary/#log-appender--bridge). |
| 334 |
|
| 335 |
You can read more about OpenTelemetry Logs design on [opentelemetry.io](https://opentelemetry.io/docs/concepts/signals/logs/). |
| 336 |
|
| 337 |
### Record as interface |
| 338 |
|
| 339 |
`Record` is defined as a `struct` because of the following reasons. |
| 340 |
|
| 341 |
Log record is a value object without any behavior. |
| 342 |
It is used as data input for Logger methods. |
| 343 |
|
| 344 |
The log record resembles the instrument config structs like [metric.Float64CounterConfig](https://pkg.go.dev/go.opentelemetry.io/otel/metric#Float64CounterConfig). |
| 345 |
|
| 346 |
Using `struct` instead of `interface` improves the performance as e.g. |
| 347 |
indirect calls are less optimized, |
| 348 |
usage of interfaces tend to increase heap allocations.[^3] |
| 349 |
|
| 350 |
### Options as parameter to Logger.Emit |
| 351 |
|
| 352 |
One of the initial ideas was to have: |
| 353 |
|
| 354 |
```go |
| 355 |
type Logger interface{ |
| 356 |
embedded.Logger |
| 357 |
Emit(ctx context.Context, options ...RecordOption) |
| 358 |
} |
| 359 |
``` |
| 360 |
|
| 361 |
The main reason was that design would be similar |
| 362 |
to the [Meter API](https://pkg.go.dev/go.opentelemetry.io/otel/metric#Meter) |
| 363 |
for creating instruments. |
| 364 |
|
| 365 |
However, passing `Record` directly, instead of using options, |
| 366 |
is more performant as it reduces heap allocations.[^4] |
| 367 |
|
| 368 |
Another advantage of passing `Record` is that API would not have functions like `NewRecord(options...)`, |
| 369 |
which would be used by the SDK and not by the users. |
| 370 |
|
| 371 |
Finally, the definition would be similar to [`slog.Handler.Handle`](https://pkg.go.dev/log/slog#Handler) |
| 372 |
that was designed to provide optimization opportunities.[^2] |
| 373 |
|
| 374 |
### Passing record as pointer to Logger.Emit |
| 375 |
|
| 376 |
So far the benchmarks do not show differences that would |
| 377 |
favor passing the record via pointer (and vice versa). |
| 378 |
|
| 379 |
Passing via value feels safer because of the following reasons. |
| 380 |
|
| 381 |
The user would not be able to pass `nil`. |
| 382 |
Therefore, it reduces the possibility to have a nil pointer dereference. |
| 383 |
|
| 384 |
It should reduce the possibility of a heap allocation. |
| 385 |
|
| 386 |
It follows the design of [`slog.Handler`](https://pkg.go.dev/log/slog#Handler). |
| 387 |
|
| 388 |
If follows one of Google's Go Style Decisions |
| 389 |
to prefer [passing values](https://google.github.io/styleguide/go/decisions#pass-values). |
| 390 |
|
| 391 |
### Passing struct as parameter to LoggerProvider.Logger |
| 392 |
|
| 393 |
Similarly to `Logger.Emit`, we could have something like: |
| 394 |
|
| 395 |
```go |
| 396 |
type LoggerProvider interface{ |
| 397 |
embedded.LoggerProvider |
| 398 |
Logger(name string, config LoggerConfig) |
| 399 |
} |
| 400 |
``` |
| 401 |
|
| 402 |
The drawback of this idea would be that this would be |
| 403 |
a different design from Trace and Metrics API. |
| 404 |
|
| 405 |
The performance of acquiring a logger is not as critical |
| 406 |
as the performance of emitting a log record. While a single |
| 407 |
HTTP/RPC handler could write hundreds of logs, it should not |
| 408 |
create a new logger for each log entry. |
| 409 |
The bridge implementation should reuse loggers whenever possible. |
| 410 |
|
| 411 |
### Logger.WithAttributes |
| 412 |
|
| 413 |
We could add `WithAttributes` to the `Logger` interface. |
| 414 |
Then `Record` could be a simple struct with only exported fields. |
| 415 |
The idea was that the SDK would implement the performance improvements |
| 416 |
instead of doing it in the API. |
| 417 |
This would allow having different optimization strategies. |
| 418 |
|
| 419 |
During the analysis[^5], it occurred that the main problem of this proposal |
| 420 |
is that the variadic slice passed to an interface method is always heap allocated. |
| 421 |
|
| 422 |
Moreover, the logger returned by `WithAttribute` was allocated on the heap. |
| 423 |
|
| 424 |
Lastly, the proposal was not specification compliant. |
| 425 |
|
| 426 |
### Record attributes as slice |
| 427 |
|
| 428 |
One of the proposals[^6] was to have `Record` as a simple struct: |
| 429 |
|
| 430 |
```go |
| 431 |
type Record struct { |
| 432 |
Timestamp time.Time |
| 433 |
ObservedTimestamp time.Time |
| 434 |
EventName string |
| 435 |
Severity Severity |
| 436 |
SeverityText string |
| 437 |
Body Value |
| 438 |
Attributes []KeyValue |
| 439 |
} |
| 440 |
``` |
| 441 |
|
| 442 |
The bridge implementations could use [`sync.Pool`](https://pkg.go.dev/sync#Pool) |
| 443 |
for reducing the number of allocations when passing attributes. |
| 444 |
|
| 445 |
The benchmarks results were better. |
| 446 |
|
| 447 |
In such a design, most bridges would have a `sync.Pool` |
| 448 |
to reduce the number of heap allocations. |
| 449 |
However, the `sync.Pool` will not work correctly with API implementations |
| 450 |
that would take ownership of the record |
| 451 |
(e.g. implementations that do not copy records for asynchronous processing). |
| 452 |
The current design, even in case of improper API implementation, |
| 453 |
has lower chances of encountering a bug as most bridges would |
| 454 |
create a record, pass it, and forget about it. |
| 455 |
|
| 456 |
For reference, here is the reason why `slog` does not use `sync.Pool`[^3] |
| 457 |
as well: |
| 458 |
|
| 459 |
> We can use a sync pool for records though we decided not to. |
| 460 |
You can but it's a bad idea for us. Why? |
| 461 |
Because users have control of Records. |
| 462 |
Handler writers can get their hands on a record |
| 463 |
and we'd have to ask them to free it |
| 464 |
or try to free it magically at some some point. |
| 465 |
But either way, they could get themselves in trouble by freeing it twice |
| 466 |
or holding on to one after they free it. |
| 467 |
That's a use after free bug and that's why `zerolog` was problematic for us. |
| 468 |
`zerolog` as as part of its speed exposes a pool allocated value to users |
| 469 |
if you use `zerolog` the normal way, that you'll see in all the examples, |
| 470 |
you will never encounter a problem. |
| 471 |
But if you do something a little out of the ordinary you can get |
| 472 |
use after free bugs and we just didn't want to put that in the standard library. |
| 473 |
|
| 474 |
Therefore, we decided to not follow the proposal as it is |
| 475 |
less user friendly (users and bridges would use e.g. a `sync.Pool` to reduce |
| 476 |
the number of heap allocation), less safe (more prone to use after free bugs |
| 477 |
and race conditions), and the benchmark differences were not significant. |
| 478 |
|
| 479 |
### Use any instead of defining Value |
| 480 |
|
| 481 |
[Logs Data Model](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-body) |
| 482 |
defines Body to be `any`. |
| 483 |
One could propose to define `Body` (and attribute values) as `any` |
| 484 |
instead of a defining a new type (`Value`). |
| 485 |
|
| 486 |
First of all, [`any` type defined in the specification](https://opentelemetry.io/docs/specs/otel/logs/data-model/#type-any) |
| 487 |
is not the same as `any` (`interface{}`) in Go. |
| 488 |
|
| 489 |
Moreover, using `any` as a field would decrease the performance.[^7] |
| 490 |
|
| 491 |
Notice it will be still possible to add following kind and factories |
| 492 |
in a backwards compatible way: |
| 493 |
|
| 494 |
```go |
| 495 |
const KindMap Kind |
| 496 |
|
| 497 |
func AnyValue(value any) KeyValue |
| 498 |
|
| 499 |
func Any(key string, value any) KeyValue |
| 500 |
``` |
| 501 |
|
| 502 |
However, currently, it would not be specification compliant. |
| 503 |
|
| 504 |
### Severity type encapsulating number and text |
| 505 |
|
| 506 |
We could combine severity into a single field defining a type: |
| 507 |
|
| 508 |
```go |
| 509 |
type Severity struct { |
| 510 |
Number SeverityNumber |
| 511 |
Text string |
| 512 |
} |
| 513 |
``` |
| 514 |
|
| 515 |
However, the [Logs Data Model](https://opentelemetry.io/docs/specs/otel/logs/data-model/#log-and-event-record-definition) |
| 516 |
define it as independent fields. |
| 517 |
It should be more user friendly to have them separated. |
| 518 |
Especially when having getter and setter methods, setting one value |
| 519 |
when the other is already set would be unpleasant. |
| 520 |
|
| 521 |
### Reuse attribute package |
| 522 |
|
| 523 |
It was tempting to reuse the existing |
| 524 |
[https://pkg.go.dev/go.opentelemetry.io/otel/attribute] package |
| 525 |
for defining log attributes and body. |
| 526 |
|
| 527 |
However, this would be wrong because [the log attribute definition](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-attributes) |
| 528 |
is different from [the common attribute definition](https://opentelemetry.io/docs/specs/otel/common/#attribute). |
| 529 |
|
| 530 |
Moreover, it there is nothing telling that [the body definition](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-body) |
| 531 |
has anything in common with a common attribute value. |
| 532 |
|
| 533 |
Therefore, we define new types representing the abstract types defined |
| 534 |
in the [Logs Data Model](https://opentelemetry.io/docs/specs/otel/logs/data-model/#definitions-used-in-this-document). |
| 535 |
|
| 536 |
### Mix receiver types for Record |
| 537 |
|
| 538 |
Methods of [`slog.Record`](https://pkg.go.dev/log/slog#Record) |
| 539 |
have different receiver types. |
| 540 |
|
| 541 |
In `log/slog` GitHub issue we can only find that the reason is:[^8] |
| 542 |
|
| 543 |
>> some receiver of Record struct is by value |
| 544 |
> Passing Records by value means they incur no heap allocation. |
| 545 |
> That improves performance overall, even though they are copied. |
| 546 |
|
| 547 |
However, the benchmarks do not show any noticeable differences.[^9] |
| 548 |
|
| 549 |
The compiler is smart-enough to not make a heap allocation for any of these methods. |
| 550 |
The use of a pointer receiver does not cause any heap allocation. |
| 551 |
From Go FAQ:[^10] |
| 552 |
|
| 553 |
> In the current compilers, if a variable has its address taken, |
| 554 |
> that variable is a candidate for allocation on the heap. |
| 555 |
> However, a basic escape analysis recognizes some cases |
| 556 |
> when such variables will not live past the return from the function |
| 557 |
> and can reside on the stack. |
| 558 |
|
| 559 |
The [Understanding Allocations: the Stack and the Heap](https://www.youtube.com/watch?v=ZMZpH4yT7M0) |
| 560 |
presentation by Jacob Walker describes the escape analysis with details. |
| 561 |
|
| 562 |
Moreover, also from Go FAQ:[^10] |
| 563 |
|
| 564 |
> Also, if a local variable is very large, |
| 565 |
> it might make more sense to store it on the heap rather than the stack. |
| 566 |
|
| 567 |
Therefore, even if we use a value receiver and the value is very large |
| 568 |
it may be heap allocated. |
| 569 |
|
| 570 |
Both [Go Code Review Comments](https://go.dev/wiki/CodeReviewComments#receiver-type) |
| 571 |
and [Google's Go Style Decisions](https://google.github.io/styleguide/go/decisions#receiver-type) |
| 572 |
highly recommend making the methods for a type either all pointer methods |
| 573 |
or all value methods. Google's Go Style Decisions even goes further and says: |
| 574 |
|
| 575 |
> There is a lot of misinformation about whether passing a value or a pointer |
| 576 |
> to a function can affect performance. |
| 577 |
> The compiler can choose to pass pointers to values on the stack |
| 578 |
> as well as copying values on the stack, |
| 579 |
> but these considerations should not outweigh the readability |
| 580 |
> and correctness of the code in most circumstances. |
| 581 |
> When the performance does matter, it is important to profile both approaches |
| 582 |
> with a realistic benchmark before deciding that one approach outperforms the other. |
| 583 |
|
| 584 |
Because, the benchmarks[^9] do not proof any performance difference |
| 585 |
and the general recommendation is to not mix receiver types, |
| 586 |
we decided to use pointer receivers for all `Record` methods. |
| 587 |
|
| 588 |
### Add XYZ method to Logger |
| 589 |
|
| 590 |
The `Logger` does not have methods like `SetSeverity`, etc. |
| 591 |
as the Logs API needs to follow (be compliant with) |
| 592 |
the [specification](https://opentelemetry.io/docs/specs/otel/logs/api/) |
| 593 |
|
| 594 |
### Rename KeyValue to Attr |
| 595 |
|
| 596 |
There was a proposal to rename `KeyValue` to `Attr` (or `Attribute`).[^11] |
| 597 |
New developers may not intuitively know that `log.KeyValue` is an attribute in |
| 598 |
the OpenTelemetry parlance. |
| 599 |
|
| 600 |
During the discussion we agreed to keep the `KeyValue` name. |
| 601 |
|
| 602 |
The type is used in multiple semantics: |
| 603 |
|
| 604 |
- as a log attribute, |
| 605 |
- as a map item, |
| 606 |
- as a log record Body. |
| 607 |
|
| 608 |
As for map item semantics, this type is a key-value pair, not an attribute. |
| 609 |
Naming the type as `Attr` would convey semantical meaning |
| 610 |
that would not be correct for a map. |
| 611 |
|
| 612 |
We expect that most of the Logs API users will be OpenTelemetry contributors. |
| 613 |
We plan to implement bridges for the most popular logging libraries ourselves. |
| 614 |
Given we will all have the context needed to disambiguate these overlapping |
| 615 |
names, developers' confusion should not be an issue. |
| 616 |
|
| 617 |
For bridges not developed by us, |
| 618 |
developers will likely look at our existing bridges for inspiration. |
| 619 |
Our correct use of these types will be a reference to them. |
| 620 |
|
| 621 |
At last, we provide `ValueFromAttribute` and `KeyValueFromAttribute` |
| 622 |
to offer reuse of `attribute.Value` and `attribute.KeyValue`. |
| 623 |
|
| 624 |
[^1]: [Handle structured body and attributes](https://github.com/pellared/opentelemetry-go/pull/7) |
| 625 |
[^2]: Jonathan Amsterdam, [The Go Blog: Structured Logging with slog](https://go.dev/blog/slog) |
| 626 |
[^3]: Jonathan Amsterdam, [GopherCon Europe 2023: A Fast Structured Logging Package](https://www.youtube.com/watch?v=tC4Jt3i62ns) |
| 627 |
[^4]: [Emit definition discussion with benchmarks](https://github.com/open-telemetry/opentelemetry-go/pull/4725#discussion_r1400869566) |
| 628 |
[^5]: [Logger.WithAttributes analysis](https://github.com/pellared/opentelemetry-go/pull/3) |
| 629 |
[^6]: [Record attributes as field and use sync.Pool for reducing allocations](https://github.com/pellared/opentelemetry-go/pull/4) and [Record attributes based on slog.Record](https://github.com/pellared/opentelemetry-go/pull/6) |
| 630 |
[^7]: [Record.Body as any](https://github.com/pellared/opentelemetry-go/pull/5) |
| 631 |
[^8]: [log/slog: structured, leveled logging](https://github.com/golang/go/issues/56345#issuecomment-1302563756) |
| 632 |
[^9]: [Record with pointer receivers only](https://github.com/pellared/opentelemetry-go/pull/8) |
| 633 |
[^10]: [Go FAQ: Stack or heap](https://go.dev/doc/faq#stack_or_heap) |
| 634 |
[^11]: [Rename KeyValue to Attr discussion](https://github.com/open-telemetry/opentelemetry-go/pull/4809#discussion_r1476080093) |