Send telemetry
Ottertrail receives traces, logs, and metrics in the OpenTelemetry protocol (OTLP) over HTTP. Any OpenTelemetry SDK or the OpenTelemetry Collector can send to it. Exceptions in the data join the project’s error list, beside the browser’s errors.
The endpoint and the key
Section titled “The endpoint and the key”An OTLP exporter adds the signal’s path to a base address. Give it your instance’s address:
| Setting | Value |
|---|---|
| Base address | https://analytics.example.com |
| Paths | /v1/traces, /v1/logs, and /v1/metrics |
| Authentication | The header Authorization: Bearer sk_…, with the project’s secret key |
| Protocol | HTTP, with protobuf or JSON, compressed with gzip or not. gRPC is not supported. |
Most SDKs read these standard environment variables:
OTEL_EXPORTER_OTLP_ENDPOINT=https://analytics.example.comOTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer%20sk_…OTEL_EXPORTER_OTLP_PROTOCOL=http/protobufThe space after Bearer is written %20 in OTEL_EXPORTER_OTLP_HEADERS.
Use the secret key, never the public key. A request with the public key or with no key answers 401. The secret key is in the project’s Settings, on the Your server tab, which the owner and admins can open.
Name the service, environment, and release
Section titled “Name the service, environment, and release”Ottertrail reads these resource attributes:
| Attribute | Use |
|---|---|
service.name |
The service. Set it with OTEL_SERVICE_NAME. |
service.version |
The release, as data-release is for the tracker. |
deployment.environment.name |
The environment, such as staging. The older deployment.environment also works. Without it, the data counts as production. |
Set them with the standard variables:
OTEL_SERVICE_NAME=checkoutOTEL_RESOURCE_ATTRIBUTES=service.version=1.4.2,deployment.environment.name=productionUse the same environment and release names as your pages. Then a release’s errors, traces, and logs line up.
To link an exception to one of your users, set the attribute user.id on the span or log record, with the same ID that the tracker sends. The older enduser.id also works. A server’s user ID is not signed: the secret key already authenticates it.
Node.js
Section titled “Node.js”-
Install the SDK and the automatic instrumentations:
Terminal window npm install @opentelemetry/api @opentelemetry/auto-instrumentations-node -
Set the variables, and start your app with the instrumentations loaded:
Terminal window export OTEL_SERVICE_NAME=checkoutexport OTEL_RESOURCE_ATTRIBUTES=service.version=1.4.2,deployment.environment.name=productionexport OTEL_EXPORTER_OTLP_ENDPOINT=https://analytics.example.comexport OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20${OTTERTRAIL_SECRET_KEY}"export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobufexport OTEL_TRACES_EXPORTER=otlpexport OTEL_METRICS_EXPORTER=otlpexport OTEL_LOGS_EXPORTER=otlpnode --require @opentelemetry/auto-instrumentations-node/register server.js--requireloads the instrumentations before your code. For an app in ES modules, read the instrumentations’ own documentation for the loader it needs. -
To report an exception that your code catches, record it on the active span:
import { SpanStatusCode, trace } from "@opentelemetry/api";try {await chargeCard(order);} catch (error) {const span = trace.getActiveSpan();span?.recordException(error);span?.setStatus({ code: SpanStatusCode.ERROR });throw error;}
Python
Section titled “Python”-
Install the distribution and the OTLP exporter, then the instrumentations for the libraries your app uses:
Terminal window pip install opentelemetry-distro opentelemetry-exporter-otlpopentelemetry-bootstrap -a install -
Set the variables, and start your app with
opentelemetry-instrument:Terminal window export OTEL_SERVICE_NAME=checkoutexport OTEL_RESOURCE_ATTRIBUTES=service.version=1.4.2,deployment.environment.name=productionexport OTEL_EXPORTER_OTLP_ENDPOINT=https://analytics.example.comexport OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20${OTTERTRAIL_SECRET_KEY}"export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobufexport OTEL_TRACES_EXPORTER=otlpexport OTEL_METRICS_EXPORTER=otlpexport OTEL_LOGS_EXPORTER=otlpexport OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=trueopentelemetry-instrument python app.pySet
OTEL_EXPORTER_OTLP_PROTOCOLtohttp/protobuf. Ottertrail does not accept gRPC. -
An exception that leaves a span is recorded on it. To report one that your code catches, record it yourself:
from opentelemetry import tracefrom opentelemetry.trace import Status, StatusCodetry:charge_card(order)except Exception as error:span = trace.get_current_span()span.record_exception(error)span.set_status(Status(StatusCode.ERROR))raise
Go has no automatic setup. Make the providers in your main function. The OTLP HTTP exporters read the variables above.
-
Add the modules:
Terminal window go get go.opentelemetry.io/otel \go.opentelemetry.io/otel/sdk \go.opentelemetry.io/otel/sdk/metric \go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp \go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp -
Set up traces and metrics:
package mainimport ("context""log""go.opentelemetry.io/otel""go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp""go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"sdkmetric "go.opentelemetry.io/otel/sdk/metric"sdktrace "go.opentelemetry.io/otel/sdk/trace")func main() {ctx := context.Background()traceExporter, err := otlptracehttp.New(ctx)if err != nil {log.Fatal(err)}tracerProvider := sdktrace.NewTracerProvider(sdktrace.WithBatcher(traceExporter))defer tracerProvider.Shutdown(ctx)otel.SetTracerProvider(tracerProvider)metricExporter, err := otlpmetrichttp.New(ctx)if err != nil {log.Fatal(err)}meterProvider := sdkmetric.NewMeterProvider(sdkmetric.WithReader(sdkmetric.NewPeriodicReader(metricExporter)),)defer meterProvider.Shutdown(ctx)otel.SetMeterProvider(meterProvider)// Start your server here.}The default resource reads
OTEL_SERVICE_NAMEandOTEL_RESOURCE_ATTRIBUTES. -
For logs, add the OTLP log exporter (
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp) and a bridge for your logging library, such asotelslogforlog/slog. -
Add instrumentation for your HTTP server and database client from the OpenTelemetry Go contrib modules, such as
otelhttp. -
To report an error, record it on the span, and set the span’s status:
if err != nil {span := trace.SpanFromContext(ctx)span.RecordError(err, trace.WithStackTrace(true))span.SetStatus(codes.Error, err.Error())}This needs the packages
go.opentelemetry.io/otel/traceandgo.opentelemetry.io/otel/codes.
How exceptions become errors
Section titled “How exceptions become errors”Ottertrail makes an error from:
- A span event named
exceptionwithexception.typeorexception.message, up to 10 for each span. - A log record with
exception.typeorexception.messagein its attributes.
The SDKs’ recordException, record_exception, and RecordError make the span event.
- An exception on a span counts as handled, unless the span’s status is Error.
- An exception in a log record counts as handled below the Error severity, and as not handled from Error up, or without a severity.
- The error’s path is the span’s
http.route, or the log record’s ownhttp.routewhen it has one. - The error’s page links to the trace, on the span where the exception happened. See Server errors.
What Ottertrail keeps
Section titled “What Ottertrail keeps”- Traces, logs, and metrics are kept for
TELEMETRY_RETENTION_DAYS, 30 days by default. See the configuration reference. - Errors made from exceptions are kept as long as the project’s events.
- Exponential histograms and summaries are not supported. They are refused and counted in the answer’s
partial_success. So are histogram points with more than 64 bucket bounds, and records with a time outside the retention or more than an hour ahead.
The limits page lists the request sizes and budgets.
Answers to a request
Section titled “Answers to a request”| Answer | Meaning |
|---|---|
| 200 | Stored. A partial_success in the answer counts the records that were refused, and why. Exporters do not send these again. |
| 400 | The body is not OTLP, or not the encoding it says. |
| 401 | No key, or a key that is not a project’s secret key. |
| 413 | The request is too large: send smaller batches. |
| 415 | Another content type, such as gRPC, or a compression other than gzip. |
| 429 | Over the project’s or the instance’s budget, or too many requests at once. Exporters try again after Retry-After. |
| 503 | The instance cannot store it now. Exporters try again after Retry-After, and keep the data in their queue meanwhile. |
Ottertrail answers 200 only after it has written the data.
Check that data arrives
Section titled “Check that data arrives”- Start your app, and send it a few requests.
- Open the project’s Traces, Logs, or Metrics page. The SDKs send in batches, so allow a few seconds. Until the first data arrives, each page shows the settings above to copy, with this instance’s address.
- If nothing shows, read your app’s log for the exporter’s errors. A 401 means the key is wrong. A 415 means the protocol is gRPC or the encoding is not supported.