How to use OpenTelemetry sink ?

[zyy17]

Notice that Vector already has OpenTelemetry Sink and try the following config:

sources:
  otel_traces:
    type: "opentelemetry"
    http:
      address: "0.0.0.0:4318"
    grpc:
      address: "0.0.0.0:4317"

sinks:
  store_traces:
    type: opentelemetry
    inputs:
      - otel_traces.traces
    # My OpenTelemetry traces the backend.
    # It will accept the protobuf encode traces data through HTTP.
    protocol:
      type: "http"
      uri: "http://localhost:4000/v1/otlp/v1/traces"
      method: post
      encoding:
        codec: "native"

Unfortunately, my trace backend can't parse the data. It seems the data have been transformed to another format in Vector.

My vector version is:

./vector --version       
vector 0.44.0 (aarch64-apple-darwin 3cdc7c3 2025-01-13 21:26:04.735691656)

Also, I can't find any documents that describe the example of opentelemetry. How can I use OpenTelemetry sink?

[pront]

Hey @zyy17, if I understand correctly you are doing OTEL traces -> OTEL traces via Vector.

Edit: Please refer to #22231 (comment)

Older DYI solution (not recommended):

• Reduce/Remap to publish batches: [sink OTLP] cannot ingest log batches  #22188 (comment) (credits to @someStrangerFromTheAbyss)
• VRL mappings: Provide OOTB mapping (with OTLP) for incoming Opentelemetry logs to outgoing Opentelemetry logs #22696 (comment)

[rauanmayemir]

@pront I know that, but do I need to specify protocol.encoding.protobuf.desc_file? I'm guessing I do.

[pront]

Yes, you need to provide that.

[rauanmayemir]

That's what I thought, thanks! Was really looking forward to it, will wait until native protobuf is supported.
(sending json payloads was already possible with a http sink)

[pront]

Can you help me understand your use case better? It might be a while since that is supported. And when it is, it should be trivial to migrate.

IMO main problem we have now is batching and lack of auto-grouping, which can be worked around with the reduce transform.

Ref: #22188

[rauanmayemir]

My use case is that certain tools allow ingesting logs in opentelemetry format, but only support binary protobuf payloads.

[pront]

My use case is that certain tools allow ingesting logs in opentelemetry format, but only support binary protobuf payloads

Moving to a new thread @rauanmayemir

Edit: Here is a full example OTEL to OTEL example.

References:

• https://vector.dev/docs/reference/configuration/sources/opentelemetry/#otlp-decoding-usage
• https://opentelemetry.io/docs/specs/otlp/#otlphttp-request
• https://github.com/open-telemetry/opentelemetry-proto/tree/v1.5.0/examples

[diktamxx]

Does this sink support the JSON format?

[pront]

Hi @diktamxx, this discussion contained outdated information. I updated the links in my response above.

[kmatata]

facing a similar issue, I think...

tried looking into the network traffic; sudo tcpdump -i lo port 5318 -A

found vector sending an array instead of json, even after setting the appropriate fields to use json:

{"code":3,"message":"ReadObjectCB: expect { or n, but found [, error found in #1 byte of ...|[{\"composit|..., bigger context ...|[{\"composite_sort_key\":\"20250217212324_0002\",\"level|..."}
03:43:14.916651 IP localhost.39072 > localhost.5318: Flags [.], ack 325, win 510, options [nop,nop,TS val 2844527821 ecr 2844527821], length 0
E..4..@.@.%.................c........(.....

---

suggestion by @pront was very useful; https://vector.dev/docs/reference/configuration/sinks/opentelemetry/#how-it-works

also helped to dealing with the array issue by handling it at the encoding level:
("at least for my case":)
vector sends data as,:

[{
  "resourceLogs": [{...}]
}]

and OLTP expects:

{
  "resourceLogs": [{...}]
}

hence adding this to transforms worked out well

# First create a new transform
transforms:
  format_for_otlp:
    type: "remap"
    inputs: ["prev_parsed_log"]
    source: |
    # Remove wrapping
    ...

im not sure why using the endpoint http:localhost/port/v1/traces doesnt work only /v1/logs....curious

Hope this helps in some small way