Skip to content

ateom exports only through the relay or its own endpoint, with its pod identity and its own exporter selection - #2189

Open
Quentin Bisson (QuentinBisson) wants to merge 5 commits into
agent-substrate:mainfrom
QuentinBisson:fix/otel-exporter-followups
Open

Quentin Bisson (QuentinBisson) wants to merge 5 commits into
agent-substrate:mainfrom
QuentinBisson:fix/otel-exporter-followups

Conversation

@QuentinBisson

@QuentinBisson Quentin Bisson (QuentinBisson) commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #1878 (#1877). Stacked on #2190: the first two commits are that PR and drop out once it merges.

  • ateom exports only when it has somewhere to send: atelet's relay or an OTLP endpoint of its own. With neither, it used to export to localhost:4317, where no collector runs. Now it skips the trace, metric and log exporters. ateom decides this at startup because only ateom can see whether its node's relay exists. A relay is enough on its own, since atelet only serves the socket when it has a collector.

  • ate-controller now always sets ateom's pod identity (OTEL_RESOURCE_ATTRIBUTES and the Downward API refs it substitutes), the metric export tuning and the sampler. It still sets the endpoint only when one is configured. Before, an install whose atelet had a collector while ate-controller had none got ateom telemetry through the relay with no pod identity attached. The endpoint now only configures ateom's direct path, so the old "empty endpoint disables ateom telemetry" contract no longer holds. On such installs, WorkerPool Deployments roll once when the controller is upgraded. Default installs set the endpoint for every component through ate-otel-config, so their worker pods don't change.

  • ate-controller now sets OTEL_TRACES_EXPORTER and OTEL_METRICS_EXPORTER on ateom, from new worker-only ATEOM_OTEL_TRACES_EXPORTER and ATEOM_OTEL_METRICS_EXPORTER variables in ate-otel-config (flags --otel-traces-exporter and --otel-metrics-exporter). The relay forwards ateom's telemetry as it is, so there was no way to turn the workers' exporters off. The controller does not pass on its own values, since OTEL_METRICS_EXPORTER=none there means the metrics are scraped from /metrics, which ateom does not serve.

  • Tests pass (a span reaches a fake collector through the relay connection without an endpoint; a relay-capable component with neither a relay nor an endpoint has no OTLP destination and builds no log provider; the worker Deployment carries the identity and tuning env with and without an endpoint, and the endpoint only with one; each exporter variable reaches the worker only when set)

  • Appropriate changes to documentation are included in the PR

@QuentinBisson Quentin Bisson (QuentinBisson) changed the title Components honor OTEL_TRACES_EXPORTER=none and treat OTEL_METRICS_EXPORTER=prometheus as none Components honor OTEL_TRACES_EXPORTER=none and OTEL_METRICS_EXPORTER=prometheus; ateom exports only through the relay or its own endpoint Oct 5, 2026
@QuentinBisson
Quentin Bisson (QuentinBisson) force-pushed the fix/otel-exporter-followups branch 2 times, most recently from 401be34 to 9a7133f Compare October 5, 2026 08:40
@QuentinBisson Quentin Bisson (QuentinBisson) changed the title Components honor OTEL_TRACES_EXPORTER=none and OTEL_METRICS_EXPORTER=prometheus; ateom exports only through the relay or its own endpoint ateom exports only through the relay or its own endpoint, and always carries its pod identity Oct 5, 2026
… /metrics is served

prometheus is the specification's pull exporter. Components that serve
/metrics already provide it, so the value now drops the OTLP push there
instead of warning and pushing every series a second time. ateom serves
no /metrics, so there the value stays unsupported: it warns and keeps the
OTLP export.
…oint

A relay-capable component with neither a relay connection nor an OTLP
endpoint would export to the SDK default, localhost:4317, where a worker
pod runs no collector. Traces, metrics and logs now skip the exporter in
that case. A relay connection is enough on its own: atelet only serves
the socket when it has a collector.
ateom exports through atelet's relay whenever atelet has a collector,
whether or not ate-controller has an endpoint to propagate. The pod
identity in OTEL_RESOURCE_ATTRIBUTES, the metric export tuning and the
sampler now reach the worker pods either way; the endpoint is still set
only when configured. Worker pools on installs without an endpoint roll
once when the controller is upgraded.
@QuentinBisson
Quentin Bisson (QuentinBisson) marked this pull request as ready for review October 5, 2026 09:08
ate-controller propagated the endpoint, sampler and metric tuning to the
ateom worker pods but not the exporter selection, so turning traces or
metrics off for the workers was not possible: the relay forwards ateom's
telemetry as it is, and atelet's own values do not apply to it.

The worker values come from ATEOM_OTEL_TRACES_EXPORTER and
ATEOM_OTEL_METRICS_EXPORTER in ate-otel-config, the defaults of the new
--otel-traces-exporter and --otel-metrics-exporter flags, rather than the
controller's own variables. On the control plane OTEL_METRICS_EXPORTER=none
means the metrics are scraped from /metrics instead of pushed, and ateom
serves no /metrics, so passing that value on would drop ateom's metrics.
Traces follow the same rule so every worker exporter setting is set the
same way. Unset, ateom keeps its OTLP defaults.
@QuentinBisson Quentin Bisson (QuentinBisson) changed the title ateom exports only through the relay or its own endpoint, and always carries its pod identity ateom exports only through the relay or its own endpoint, with its pod identity and its own exporter selection Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant