feat: Add FFI for trace exporter#1952
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 15 commits intoMay 26, 2026
Merged
Conversation
Contributor
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
lloeki
force-pushed
the
lloeki/ffi-for-trace-exporter
branch
from
96342f8 to
303d914
Compare
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 693a1ca | Docs | Datadog PR Page | Give us feedback! |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1952 +/- ##
==========================================
+ Coverage 72.76% 72.79% +0.03%
==========================================
Files 458 459 +1
Lines 75789 76106 +317
==========================================
+ Hits 55147 55405 +258
- Misses 20642 20701 +59
🚀 New features to boost your workflow:
|
lloeki
force-pushed
the
lloeki/ffi-for-trace-exporter
branch
2 times, most recently
from
2b39efc to
2701231
Compare
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
yannham
reviewed
May 7, 2026
hoolioh
requested changes
May 7, 2026
Member
Author
|
Review comments addressed. If acceptable, I suggest we merge this one as a first iteration of the FFI API. A new stacked PR based on this one is brewing, which has more contentious API changes coming from the dd-trace-rb side review. |
hoolioh
approved these changes
May 14, 2026
Contributor
hoolioh
left a comment
hoolioh
left a comment
There was a problem hiding this comment.
I just left a comment requesting a change, other than that LGTM overall for a first version.
yannham
approved these changes
May 18, 2026
lloeki
force-pushed
the
lloeki/ffi-for-trace-exporter
branch
from
9d7998b to
5e2089f
Compare
Introduce an opaque `TracerSpan` handle wrapping `Span<BytesData>` and expose it to C callers via four FFI functions: - `ddog_tracer_span_new`: create a span with all scalar fields - `ddog_tracer_span_free`: release the span - `ddog_tracer_span_set_meta`: add a string tag - `ddog_tracer_span_set_metric`: add a numeric tag This enables language tracers (e.g. Ruby) to build spans field-by-field through the C API, bypassing msgpack serialization on the caller side.
Introduce an opaque `TracerTraceChunks` handle wrapping `Vec<Vec<SpanBytes>>` and expose it to C callers via four FFI functions: - `ddog_tracer_trace_chunks_new`: create a container with optional capacity hint - `ddog_tracer_trace_chunks_free`: release the container - `ddog_tracer_trace_chunks_begin_chunk`: start a new trace chunk - `ddog_tracer_trace_chunks_push_span`: move a `TracerSpan` into the current chunk, consuming it Spans are consumed on push to enforce single-ownership and prevent double-use from C callers.
Wire `TracerTraceChunks` to the existing `TraceExporter::send_trace_chunks` method through a new C-callable function. The chunks are consumed on call and the agent response is optionally written to an out-parameter. This completes the span-building FFI surface: callers can now construct spans via `ddog_tracer_span_*`, group them via `ddog_tracer_trace_chunks_*`, and send them via `ddog_trace_exporter_send_trace_chunks` — all without serializing to msgpack on the caller side. The `TraceExporter` type alias in `trace_exporter.rs` is widened to `pub(crate)` to allow cross-module access.
Replace the `match` with identity `Ok` arm with `map_err`, which is the idiomatic Rust combinator for transforming only the error side of a `Result`.
The inner fields are only accessed within the `tracer` module itself, so `pub(crate)` is unnecessarily broad. Default (private) visibility is sufficient since the module owns all access.
`Vec::with_capacity(0)` and `Vec::new()` are identical — both create an empty vec with no heap allocation — so the branch is unnecessary.
…race_chunks`
Replace the `match` with `let Some(...) = ... else { return ... }`,
which is the idiomatic Rust pattern for early-return on a refutable
binding (stabilized in Rust 1.65).
`make_minimal_span` and `make_chunks` have no safety preconditions for their callers — all unsafety is self-contained. Move the `unsafe` from the function signature into internal blocks so the functions present a safe interface.
Extract the span-building logic into an inner closure that returns `Result`, allowing the `?` operator to propagate conversion errors. The outer body calls `inner().err()` to map `Ok(()) -> None` and `Err(e) -> Some(e)`, matching the FFI return type.
Replace `NonNull<Box<T>>` with `&mut MaybeUninit<Box<T>>` for out-parameters in `ddog_tracer_span_new` and `ddog_tracer_trace_chunks_new`. This makes the uninitialized nature of the slot explicit in the type system and allows using `MaybeUninit::write` instead of raw `ptr::write`. Note that `&mut Box<T>` cannot be used here: assignment through `&mut Box<T>` drops the old value first, which is undefined behavior when the slot is uninitialized. `MaybeUninit::write` overwrites without dropping, which is the correct semantics for out-parameters. cbindgen produces identical C signatures (`T **out_handle`) for all three representations.
Restore the established `NonNull<Box<T>>` convention used throughout `trace_exporter.rs` for FFI out-parameters. While `&mut MaybeUninit` is technically more precise, it is less idiomatic for FFI and inconsistent with the rest of the module. cbindgen produces identical C headers for both representations.
Replace 11 positional parameters with a `#[repr(C)]` struct passed by reference. This avoids holding all fields on the stack per call and allows adding or changing fields without breaking the function signature. Follows the same pattern as `TelemetryClientConfig` elsewhere in the codebase.
Accept a capacity hint for pre-allocating the inner span vector, avoiding reallocations when the number of spans is known beforehand.
…egin_chunk` Change the return type from void to `Option<Box<ExporterError>>` and wrap the body in `catch_panic!` with a null handle check, making the function consistent with every other mutating function in the tracer FFI API.
lloeki
force-pushed
the
lloeki/ffi-for-trace-exporter
branch
from
5e2089f to
b9ea727
Compare
Member
Author
|
9d7998b had all CI OK and now post-rebase there's no CI running? EDIT: GitHub Actions incident 
|
lloeki
force-pushed
the
lloeki/ffi-for-trace-exporter
branch
from
b1c81ea to
693a1ca
Compare
Member
Author
|
/merge |
|
View all feedbacks in Devflow UI.
This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
The expected merge time in
Tests failed on this commit b9ba4bb: What to do next?
|
gh-worker-dd-devflow-36fce6
Bot
added
mergequeue-status: done
and removed
mergequeue-status: in_progress
labels
gh-worker-dd-mergequeue-cf854d
Bot
deleted the
lloeki/ffi-for-trace-exporter
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Add span-building and trace-sending FFI functions to
libdd-data-pipeline-ffi, enabling language tracers to construct spans field-by-field through the C API instead of passing pre-serialized msgpack.New types:
TracerSpan: opaque handle wrappingSpan<BytesData>TracerTraceChunks: opaque handle wrappingVec<Vec<SpanBytes>>New functions:
ddog_tracer_span_new,_free,_set_meta,_set_metricddog_tracer_trace_chunks_new,_free,_begin_chunk,_push_spanddog_trace_exporter_send_trace_chunksMotivation
The Ruby tracer needs a native trace export path that bypasses Ruby-side msgpack serialization. This provides the FFI surface for it; the corresponding C extension in
dd-trace-rbcalls these functions.Continuation of the prototype in #1661, reworked to stay close to
main: the genericTraceExporter<H>andSharedRuntimeare untouched; this is purely additive.Additional Notes
Spans are consumed on push, chunks are consumed on send — single-ownership enforced at the API level.
libdd-trace-utilsis promoted from dev-dependency to regular dependency since the FFI crate now needsSpanBytesat build time.Sibling PR for Ruby is at: DataDog/dd-trace-rb#5690
How to test the change?
cargo test -p libdd-data-pipeline-ffi --lib— 39 tests, all passing. The existingtrace_exportertests are unaffected.