feat(tracebuffer): flush on drop, drain on shutdown, and support nested tokio runtimes#2005
Open
Aaalibaba42 wants to merge 8 commits into
Open
feat(tracebuffer): flush on drop, drain on shutdown, and support nested tokio runtimes#2005Aaalibaba42 wants to merge 8 commits into
Aaalibaba42 wants to merge 8 commits into
Conversation
Contributor
📚 Documentation Check Results📦
|
Contributor
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
Contributor
🔒 Cargo Deny Results📦
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2005 +/- ##
==========================================
+ Coverage 72.83% 72.92% +0.09%
==========================================
Files 458 458
Lines 75789 76036 +247
==========================================
+ Hits 55201 55453 +252
+ Misses 20588 20583 -5
🚀 New features to boost your workflow:
|
Contributor
|
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
Aaalibaba42
force-pushed
the
jwiriath/tracebuffer-sharedruntime-rusttracer
branch
from
b5a716f to
d55d383
Compare
Aaalibaba42
force-pushed
the
jwiriath/tracebuffer-sharedruntime-rusttracer
branch
2 times, most recently
from
d4d669c to
463495c
Compare
Aaalibaba42
changed the title
yannham
requested changes
May 20, 2026
Contributor
yannham
left a comment
yannham
left a comment
There was a problem hiding this comment.
We've discussed this offline, and the consensus seemed to be that the actual issue of the runtime-in-the-runtime problem is that some functions of our API pretend to be async but actually use block_on under the hood. It seems one possible solution is to actually make them async, and to offer a customary blocking API at the top-level only. Then, from Rust we could only use the purely async API and not care about the shared runtime at all.
On the other hand, the current solution secretly spawns a thread and run a second runtime on the side, while blocking the original executor's thread waiting for the answer. This is a bit fishy, and we never want to block an async executor thread, as this will block a whole bunch of unrelated async tasks queued on this thread.
Aaalibaba42
force-pushed
the
jwiriath/tracebuffer-sharedruntime-rusttracer
branch
from
13fe812 to
f8d6d54
Compare
…kio-inside-tokio problems
… have tokio-inside-tokio problems" This reverts commit a9fceca.
Aaalibaba42
force-pushed
the
jwiriath/tracebuffer-sharedruntime-rusttracer
branch
from
f8d6d54 to
fd24165
Compare
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What?
Make the
TraceBufferandSharedRuntimesafe to use from within a tokio runtime, add a synchronousflush_and_waitAPI, ensure pending spans are drained on shutdown, and fire a best-effort flush onDrop.Why?
SharedRuntime::block_onandshutdownpanic with "Cannot start a runtime from within a runtime" when called from inside a tokio context (e.g. the Rust tracer embedded in an async application).TraceBufferwithout an explicit flush also lost any buffered spans.How?
SharedRuntime(libdd-shared-runtime):block_onandshutdownnow detect a running tokioHandleviaHandle::try_current()and, when found, drive the blocking work on a scoped OS thread (std::thread::scope) instead of callingRuntime::block_ondirectly. TheArc<Runtime>is explicitly dropped on that same thread so its join-on-drop doesn't panic either. AddedSendbounds on the future and its output to support this.TraceBuffer(libdd-data-pipeline): Addedflush_and_wait(timeout)— triggers a flush, captures the batch generation, and blocks until the worker has processed it. AddedReceiver::drain()to synchronously pull remaining chunks without waiting for a flush trigger;TraceExporterWorker::shutdownnow calls it to export any leftover spans. ImplementedDrop for TraceBuffer<T>that fires a non-blocking flush notify.TraceExporter: AddedSendbounds onT::TextandT::Bytesforsend_trace_chunksto satisfy the newblock_onsignature.Additional Notes
Dropimpl is intentionally best-effort (errors are swallowed) — if the runtime is already gone, there's nothing useful to do.flush_and_waitshort-circuits and returns immediately when the batch is empty.block_on,shutdown, anddropfrom inside#[tokio::test].