Add DDP gradient-sync (all_reduce) timing via comm hook (TRA-16)

[Pendu]

What this adds

TraceML currently times dataloader, H2D, forward, backward, and optimizer, but is blind to the
biggest distributed-specific cost: DDP gradient synchronization. This PR adds per-bucket timing
of every DDP all_reduce during loss.backward(), emitted as a new step-scoped event
_traceml_comm:ddp_grad_sync, with zero changes to user training code.

Why it matters: on a 2-node run (validation below) the per-step summary attributes the
all_reduce time to compute/wait, so a genuinely comm-bound run is indistinguishable from a
compute-bound one. This PR makes the comm signal exist as first-class data.

How it works

• instrumentation/hooks/ddp_comm_hook.py: a comm-hook factory that wraps a base hook
  (default: PyTorch's allreduce_hook) with a CUDA-event pair around each bucket's collective.
  gpu_start is recorded on the compute stream at hook entry; gpu_end in the future's
  .then() callback after NCCL completes. Events resolve asynchronously via the existing
  sampler path (no syncs on the hot path). The reduced bucket tensor passes through unchanged.
• instrumentation/patches/ddp_comm_patch.py: a class-level patch on
  DistributedDataParallel.forward, installed once at traceml.init(), that lazily installs
  the comm hook on each DDP instance's first forward. This mirrors the existing patch_h2d
  pattern and makes installation structural rather than dependent on which object an
  integration happens to pass around (the failure class from feat: add callback-based Hugging Face integration #135 / Harden instrumentation-state: make integration telemetry completeness self-verifying #141).
• Config: new patch_ddp_comm flag on TraceMLInitConfig, wired exactly like patch_h2d
  (auto=on, manual=off, selective=opt-in). Explicit API: traceml.wrap_ddp(model, base_hook=None).
• Fail-open everywhere: if the user already registered their own comm hook, or install fails
  for any reason, TraceML warns on stderr and training proceeds untouched.

Scope: data-only (deliberate)

Events flow through the existing name-agnostic pipeline (sampler -> aggregator -> SQLite
step_time_samples.events_json) and are queryable today. They are NOT yet surfaced in the
terminal/dashboard renderer or the diagnosis rules; that is a follow-up (internally TRA-30)
because the render layer is a closed allowlist and a comm bucket needs overlap-correction
(early buckets overlap backward compute) plus a comm-bound/comm-straggler rule.

Validation

• Unit tests: tests/test_ddp_comm_hook.py (install/idempotency/fail-open, per-bucket emission,
  base_hook composition, CUDA event pool cleanup on error paths, value pass-through regression
  guard, config parity, order-independent global-state handling). Two gated 2-rank gloo tests
  (TRACEML_RUN_DIST_TESTS=1): gradient numerical identity vs a no-hook reference, and the
  real auto-install chain (init(mode="auto") -> first forward -> hook installed -> events).
• Real hardware (2x AWS g4dn.xlarge, 1x T4 each, 2-node NCCL over TCP):
  • Full DDP test suite passed on T4.
  • Auto-install chain verified with no explicit wrap_ddp call: hook installed on both ranks,
    per-bucket events on every step, gradients bit-identical across ranks.
  • Full-pipeline traceml run (multi-node, aggregator on node 0): all 60/60 step rows
    (30 steps x 2 ranks) contain _traceml_comm:ddp_grad_sync with GPU-resolved durations in
    SQLite, alongside the five existing internal streams. final_summary.json contains no comm
    key, confirming the data-only boundary behaves as intended.
• examples/ddp_comm_hook_demo.py (also listed in the examples README): runnable single- or
  multi-node; prints a per-step table of all step phases (data/h2d/fwd/bwd/opt/comm/total) from
  in-process data under plain torchrun, or read back from the session DB under traceml run.

Overhead notes

• Timing is two pooled CUDA events per bucket plus one Future.then callback; no
  synchronization is added to the training path.
• Installing any comm hook makes PyTorch's reducer take the unfused copy path instead of the
  fused copy+divide (documented in reducer.cpp); gradients remain numerically identical
  (asserted in the gated test against a no-hook reference).

Known limitations (intentional, called out in docstrings)

• Stateful comm hooks (e.g. PowerSGD) are not supported yet: the hook registers with
  state=None. fp16_compress_hook works (verified).
• HF Trainer gets comm timing automatically (its init uses mode="auto"). Lightning and Ray
  integrations do not enable patch_ddp_comm yet: TraceMLRayConfig cannot express the flag,
  and enabling it for Lightning alone would break re-init for the documented Ray+Lightning
  selective recipe. Enabling both together is a small follow-up.
• During no_sync() accumulation steps DDP suppresses comm hooks, so those steps correctly
  produce zero comm events.

Notes for review

• Commit history is incremental (the branch evolved through review rounds); squash-merge
  recommended, matching how recent PRs landed.
• Branch is rebased on current main.