Add C++ full-body tracking examples#807
Conversation
|
📝 Docs preview is not auto-deployed for fork PRs. A maintainer with write access to |
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughAdds a C++ full-body pose printer and a C++ MCAP recorder. The examples create Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant FullBodyTrackerPico
participant OpenXRSession
participant DeviceIOSession
participant MCAPWriter
participant replay_full_body.py
FullBodyTrackerPico->>OpenXRSession: provide required extensions
OpenXRSession->>DeviceIOSession: start session with full_body recording config
DeviceIOSession->>MCAPWriter: write full_body topics
DeviceIOSession->>DeviceIOSession: update until duration elapses
MCAPWriter-->>replay_full_body.py: provide recorded full_body topics
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/schemaio/full_body_printer.cpp`:
- Around line 91-105: Bound the sampling loop in the full-body printer so it
cannot run indefinitely when tracked.data remains null. Add a wall-clock
deadline or maximum update count around the while loop, exit when the bound is
reached, and report that no body-tracking samples arrived while preserving the
existing sample-count termination and printing behavior.
- Around line 50-57: Update the sample output around the pelvis and head pose
retrieval to check each joint’s own is_valid() status before printing
coordinates. Print an unavailable marker for an invalid pelvis or head, while
preserving coordinate output independently for each valid joint; do not rely on
the aggregate valid_count or skeleton-level tracking flag.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 95415ff7-5082-44cc-b921-bd680e6ab51d
📒 Files selected for processing (8)
docs/source/device/body_tracking.rstdocs/source/device/trackers.rstdocs/source/references/mcap_record_replay.rstexamples/mcap_record_replay/CMakeLists.txtexamples/mcap_record_replay/cpp/CMakeLists.txtexamples/mcap_record_replay/cpp/record_full_body.cppexamples/schemaio/CMakeLists.txtexamples/schemaio/full_body_printer.cpp
| // Joint poses are unspecified while their tracking is lost — gate on is_valid per joint | ||
| // (all_joint_poses_tracked is a whole-skeleton quality flag only). | ||
| const auto& pelvis = joints[core::BodyJointPico_PELVIS]->pose().position(); | ||
| const auto& head = joints[core::BodyJointPico_HEAD]->pose().position(); | ||
|
|
||
| std::cout << "Sample " << sample_count << std::fixed << std::setprecision(3) << " valid=" << valid_count << "/" | ||
| << core::FullBodyTrackerPico::JOINT_COUNT << " pelvis=[" << pelvis.x() << ", " << pelvis.y() << ", " | ||
| << pelvis.z() << "] head=[" << head.x() << ", " << head.y() << ", " << head.z() << "]" << std::endl; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Do not print invalid pelvis or head poses.
The code counts validity but always prints pelvis/head coordinates. Those poses are explicitly unspecified when either joint is invalid; gate each position on its own is_valid() result and print an unavailable marker otherwise.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@examples/schemaio/full_body_printer.cpp` around lines 50 - 57, Update the
sample output around the pelvis and head pose retrieval to check each joint’s
own is_valid() status before printing coordinates. Print an unavailable marker
for an invalid pelvis or head, while preserving coordinate output independently
for each valid joint; do not rely on the aggregate valid_count or skeleton-level
tracking flag.
| while (received_count < MAX_SAMPLES) | ||
| { | ||
| // Update session (this calls update on all trackers). | ||
| session->update(); | ||
|
|
||
| // Print current data if available. tracked.data stays null while body tracking is | ||
| // inactive (limp mode or the runtime has not started delivering joints yet). | ||
| const auto& tracked = tracker->get_body_pose(*session); | ||
| if (tracked.data) | ||
| { | ||
| print_body_pose(*tracked.data, received_count++); | ||
| } | ||
|
|
||
| // Tick at ~30 Hz. | ||
| std::this_thread::sleep_for(std::chrono::milliseconds(33)); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Bound the no-sample path.
In limp mode, tracked.data remains null, so received_count never advances and this loop never exits. Add a wall-clock timeout or maximum update count and report that no body-tracking samples arrived.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@examples/schemaio/full_body_printer.cpp` around lines 91 - 105, Bound the
sampling loop in the full-body printer so it cannot run indefinitely when
tracked.data remains null. Add a wall-clock deadline or maximum update count
around the while loop, exit when the bound is reached, and report that no
body-tracking samples arrived while preserving the existing sample-count
termination and printing behavior.
- examples/schemaio/full_body_printer.cpp: minimal C++ reader for FullBodyTrackerPico (XR_BD_body_tracking, 24 joints), following the se3_printer pattern. - examples/mcap_record_replay/cpp/record_full_body.cpp: C++ counterpart of record_full_body.py; records the full_body channel by passing a McapRecordingConfig to DeviceIOSession::run(), producing files that replay unchanged with replay_full_body.py. - Reference both examples from the body-tracking, trackers, and MCAP record/replay docs. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
Wire the full-body tracking C++ examples into the rig launcher so one command brings up the whole session instead of a hand-started runtime: - rigs/full_body.yaml: first consumer-only rig — no producer plugin and no collection_id; the printer (validity gate) and the MCAP recorder read XR_BD_body_tracking joints directly from the runtime. No params: the recorder keeps its own defaults, and a fresh timestamped .mcap is written per Enter-rerun. - full_body_printer: print '[body tracking inactive]' once per second while body tracking is unavailable (limp mode), sharing the literal record_full_body already uses, so the rig's validity-gate pane is never silently mute. - rig_tests: load + footgun tests for the shipped rig, mirroring the se3_tracker pair. - docs: rig launch pointers in body_tracking.rst and mcap_record_replay.rst, a limp-mode troubleshooting entry, and rig.rst now documents the consumer-only rig shape (collection_id rendezvous scoped to producer rigs). Build green (full_body_printer, record_full_body), rig tests 30/30, pre-commit and Sphinx docs clean. Signed-off-by: Jiwen Cai <jiwenc@nvidia.com>
19ffc03 to
9868fd6
Compare
1 participant
Summary
Full-body tracking (
FullBodyTrackerPico,XR_BD_body_tracking) was fully supported in the C++ DeviceIO API but only demonstrated from Python. This adds the missing C++ examples and wires them into the docs.examples/schemaio/full_body_printer.cpp— minimal C++ reader for the 24-joint body skeleton, following these3_printerpattern (DeviceIOSession+ tracker, ~30 Hz print loop with per-jointis_validgating).examples/mcap_record_replay/cpp/record_full_body.cpp— C++ counterpart ofrecord_full_body.py: records thefull_bodychannel by passing acore::McapRecordingConfigtoDeviceIOSession::run(). It uses the same channel base name asFullBodySource, so recordings replay unchanged withreplay_full_body.py(record in C++, replay/visualize in Python).device/body_tracking.rst,device/trackers.rst, andreferences/mcap_record_replay.rstnow reference the C++ examples, including a note on C++-side MCAP recording.Testing
full_body_printer,record_full_body).clang-format-14clean;SKIP=check-copyright-year pre-commit run --all-filespasses.NV_CXR_RUNTIME_DIRguidance, matching the sibling examples.<source name>/<sub_channel>).Summary by CodeRabbit