[Bug]: `cargo build --release` fails on `log_service` bin with "queries overflow the depth limit"

[Momix-77]

Hallo Guys !

This report is published with the support of my Composer AI system!

What happened?

Running a full workspace release build (cargo build --release from the repo root) fails while compiling the log_service binary in chroma-log-service. The build aborts with a rustc query-depth error on a large async block inside LogServerWrapper::run.

Expected: cargo build --release completes successfully for the entire workspace (as required for production Docker images in rust/Dockerfile and local release builds).

Actual: Compilation fails on chroma-log-service (bin log_service) with exit code 101:

error: queries overflow the depth limit!
  |
  = help: consider increasing the recursion limit by adding a `#![recursion_limit = "256"]`
          attribute to your crate (`log_service`)
  = note: query depth increased by 130 when computing layout of
          `{async block@chroma_log_service::LogServerWrapper::run::{closure#0}::{closure#0}}`

error: could not compile `chroma-log-service` (bin "log_service") due to 1 previous error

Root cause: rust/log-service/src/lib.rs already sets #![recursion_limit = "256"], but rust/log-service/src/bin/log.rs is a separate crate target (see [[bin]] in rust/log-service/Cargo.toml) and does not inherit that attribute. In --release mode, layout computation for the deeply nested async closure in LogServerWrapper::run exceeds the default compiler query depth.

Scope / impact:

• cargo build --release (full workspace) — fails
• cargo build --release -p chroma-log-service --bin log_service — fails (minimal repro)
• cargo build --release --bin chroma — succeeds (does not build log_service)
• cargo build (debug profile) — succeeds for the same targets tested locally

This affects anyone building the distributed stack from source in release mode, including Docker image builds that compile the full workspace.

Versions

• Chroma: 1.5.9 (1.5.9-155-gccf5968d8)
• Commit: ccf5968d82947a629262b19a10fb74adc2a63148 (main)
• Rust: 1.96.0 (ac68faa20 2026-05-25)
• Cargo: 1.96.0 (30a34c682 2026-05-25)
• Python: 3.13.13
• OS: Linux 6.18.36-1-longterm, x86_64
• Dockerfile builder: rust:1.92.0 (rust/Dockerfile)

Reproduced on rustc 1.96.0. Failure is specific to the log_service binary crate in release profile.

Relevant log output

$ cargo build --release
   Compiling chromadb_rust_bindings v0.1.0 (.../rust/python_bindings)
error: queries overflow the depth limit!
  |
  = help: consider increasing the recursion limit by adding a `#![recursion_limit = "256"]` attribute to your crate (`log_service`)
  = note: query depth increased by 130 when computing layout of `{async block@chroma_log_service::LogServerWrapper::run::{closure#0}::{closure#0}}`

error: could not compile `chroma-log-service` (bin "log_service") due to 1 previous error
# exit code: 101

Minimal repro:

cargo build --release -p chroma-log-service --bin log_service

Control (succeeds):

cargo build --release --bin chroma

Steps to reproduce

1. Clone https://github.com/chroma-core/chroma and checkout main (tested at ccf5968d8).
2. Install Rust toolchain (tested with 1.96.0).
3. From repo root: cargo build --release
4. Observe failure when compiling chroma-log-service bin log_service.

Suggested fix

Add the same crate-level attribute used in rust/log-service/src/lib.rs to the binary entrypoint rust/log-service/src/bin/log.rs:

#![recursion_limit = "256"]

fn main() {
    // ...
}

This matches the pattern already used across the workspace (rust/worker/src/lib.rs, rust/segment/src/lib.rs, rust/garbage_collector/src/lib.rs, etc.).

Verified locally after applying the one-line fix:

• cargo build --release -p chroma-log-service --bin log_service — OK
• cargo build --release (full workspace) — OK

Workaround

Build only the needed binary:

cargo build --release --bin chroma

For local single-node dev, chroma run via the Python CLI works without a full workspace release build.

Additional context

• log_service is a thin wrapper around chroma_log_service::log_entrypoint(); heavy async logic lives in the library crate where recursion_limit is already raised.
• rust/Dockerfile runs cargo build --workspace ... ${release_flag} and copies log_service among other binaries — this may surface in release Docker builds depending on toolchain version.
• CI may not catch this if full release workspace builds are not exercised on every PR.

[Auntara12]

Applied the fix exactly as suggested in this issue (#![recursion_limit = "256"] in log.rs, matching the pattern in lib.rs and the other crates) - validated independently on macOS ARM64 (rustc 1.97.0), same failure reproduced before the fix, clean build after.

Also checked whether CI would have caught this: can-build-release runs cargo check --release, which passes cleanly on the unfixed code - only cargo build --release (full codegen) hits the error. So this class of bug is currently invisible to PR CI. Kept the PR narrow to just the one-line attribute.

#7437

[Momix-77]

@Auntara12

Thank you very much for the quick response and for opening PR #7437!
This is exactly the same recursion limit issue I encountered a few weeks ago when building the Python wheel for chroma-blockstore via pip3 install . on Python 3.13 (with ROCm 7.2.1). Adding #![recursion_limit = "256"] in the respective lib.rs fixed it perfectly for me at the time.
Great to see the same pattern now being applied to log_service as well. I’ll test the PR locally as soon as possible and report back if everything builds cleanly in my environment.

Would it make sense to roll this out systematically to all affected crates (blockstore, segment, log_service, etc.) to prevent similar issues with complex async generics and PyO3 bindings in the future?

Thanks again for the excellent work on this!

Best regards,
Momix!