Skip to content

Expose --engine flag on the layup orbitfit CLI#327

Open
matthewholman wants to merge 11 commits into
mainfrom
feat/cli-engine-flag
Open

Expose --engine flag on the layup orbitfit CLI#327
matthewholman wants to merge 11 commits into
mainfrom
feat/cli-engine-flag

Conversation

@matthewholman
Copy link
Copy Markdown
Collaborator

This branch was developed with Claude Code, on top of an initial
implementation I had started.

Wires the engine choice end-to-end through layup orbitfit's
command-line entry point.

Why

PR #326 adds the universal Bernstein–Khushalani fitter as
engine="bk_native" on do_fit, but the layup-orbitfit CLI
entry point doesn't forward that argument — so even after #326
lands, running

layup orbitfit input.csv ADES_csv

always uses the Cartesian engine. This PR closes that gap so the
new engine is actually reachable from the command line:

layup orbitfit input.csv ADES_csv --engine bk_native

What changed

End-to-end plumbing:

CLI argparse --engine {cartesian, bk_native}, default 'cartesian'
  └─► orbitfit_cli reads cli_args.engine
       └─► orbitfit() takes engine= parameter
            └─► process_data_by_id forwards through **kwargs
                 └─► _orbitfit() receives engine and passes through
                      └─► do_fit(..., engine=...) (already engine-aware in #326)
                           └─► _run_fit() dispatches to the right LM

argparse's choices=["cartesian", "bk_native"] gives users an
early error on a typo'd engine name; the existing _run_fit
dispatch also raises ValueError for unknown engines if a Python
caller bypasses argparse.

Tests

  • test_orbit_fit_cli_raises_with_unknown_engine: parallel to the
    existing unknown-iod test. A FakeCliArgs with engine="not_an_engine"
    triggers ValueError from _run_fit's dispatch.
  • test_orbit_fit_cli_raises_with_unknown_iod: the existing test
    gets a FakeCliArgs.engine = "cartesian" field so it continues
    passing after orbitfit_cli starts reading that attribute.

Confirmed manually: layup orbitfit --help now shows

  --engine {cartesian,bk_native}
                        LM fitter to use after IOD: 'cartesian' (default;
                        classic barycentric-Cartesian LM) or 'bk_native'
                        (universal Bernstein-Khushalani fit with energy prior,
                        better-conditioned for distant short-arc targets and
                        at least as good elsewhere).

Dependencies

Stacks on feat/bk-everywhere (PR #326 — adds the engine parameter to
do_fit and the bk_native engine itself). This PR is purely the CLI-
plumbing follow-up. Once #326 lands, this PR collapses to its one new
commit.

Review Checklist for Source Code Changes

  • Does pip install still work?
  • Have you written a unit test for any new functions? — test_orbit_fit_cli_raises_with_unknown_engine
  • Do all the units tests run successfully? — 43 BK + CLI validation tests pass locally
  • Does Layup run successfully on a test set of input files/databases? — layup orbitfit --help shows the new flag; the existing test_orbit_fit_cli_raises_with_unknown_iod continues to pass
  • Have you used black on the files you have updated?

matthewholman and others added 11 commits May 15, 2026 18:03
… Python

Without these, accessing rho_hat / a_vec / d_vec from Python raises
"Unable to convert function return value to a Python type" because the
binding can't see the Eigen and std::array conversions.

This is a minimal enabler (no behavior change); the A/D-vector
correctness fix on fix/tangent-basis-vectors is independent and lives
on its own branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure C++/Eigen math layer for the Bernstein-Khushalani parameterization,
with the barycenter as the BK coordinate origin and gnomonic projection
defining the tangent plane at a fiducial direction n0.  No ASSIST,
REBOUND, or pybind11 dependencies, so this translation unit can be
tested in isolation.

New files in src/lib/orbit_fit/:
  bk_basis.h    -- types (BKState, BKFiducial) and function declarations
  bk_basis.cpp  -- implementations: choose_fiducial, bk_to_cartesian,
                   cartesian_to_bk, dcart_dbk (full 6x6 including the
                   bottom-left cross-term block), sigma_gdot_sq

The 6x6 Jacobian dcart_dbk has the block structure expected from the
math:
  [ d r / d (alpha,beta,gamma)     0                          ]
  [ d v / d (alpha,beta,gamma)     d v / d (adot,bdot,gdot)   ]
with the top-left and bottom-right 3x3 blocks identical (both built
from the (1/gamma)-scaled tangent vectors), and the bottom-left block
holding the cross-term contributions through the second derivatives
d^2 rho_hat / d (alpha, beta)^2.

sigma_gdot_sq returns the bound-orbit energy-prior variance,
gamma^2 (2 mu gamma^3 - adot^2 - bdot^2), or +infinity when the
right-hand side is non-positive (tangential rates already exceed
escape).  Returning +infinity yields zero precision in the prior
matrix used by the LM step, which is the desired "no prior" behavior.

orbit_fit.cpp adds a single line to its unity-build chain:
  #include "bk_basis.cpp"

so the new translation unit compiles into the existing _core module.
No pybind11 binding yet -- that comes in a follow-up commit alongside
Python-side unit tests for the math primitives.

The math derivation, design decisions (barycenter origin, fixed
gdot prior, eigendecomp+energy-prior solver, file layout, layered
test plan) live in the project memory file bk_everywhere_design.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a bk_basis_bindings(py::module&) entry point that exposes BKState,
BKFiducial, bk_choose_fiducial, bk_to_cartesian, cartesian_to_bk,
dcart_dbk, and sigma_gdot_sq to Python via pybind11 / pybind11/eigen.h,
and wires the binding into main.cpp's _core module alongside the
existing detection_bindings etc.

tests/layup/test_bk_basis.py covers the pure-math invariants
(25 tests, all passing):

  * Round-trip Cartesian <-> BK across mainbelt / NEO / TNO regimes
    (rtol 1e-12).
  * Analytic dcart_dbk vs central-difference, per-element relative
    error < 1e-5 with parameter-scaled epsilon.
  * Mixed-partial symmetry of the second-derivative cross-terms
    appearing in the bottom-left block of dcart_dbk
    (d^2 r / d alpha d beta == d^2 r / d beta d alpha to FD tolerance).
  * Fiducial-direction gauge invariance: two valid n0 choices recover
    the same Cartesian orbit through round-trip.
  * Special-case forms at the fiducial direction alpha = beta = 0:
    position is (1/gamma) n0, top-left and bottom-right Jacobian
    blocks are [(1/gamma) a, (1/gamma) b, -(1/gamma^2) n0] as columns,
    bottom-left block vanishes when rates are zero.
  * sigma_gdot_sq agreement with the Cartesian energy bound at the
    parabolic boundary, and +inf return when tangential rates already
    exceed escape.

The energy-bound test caught a real bug in the first cut of
sigma_gdot_sq: the formula gamma^2 (2 mu gamma^3 - adot^2 - bdot^2)
is only exact at the fiducial direction.  Off-fiducial, the gnomonic
tangent vectors rho_hat_alpha, rho_hat_beta have magnitudes
sqrt((1+beta^2))/s^2 and sqrt((1+alpha^2))/s^2 respectively, and an
inner product -alpha*beta/s^4, so the true tangential-velocity term is

  |adot rho_hat_alpha + bdot rho_hat_beta|^2 =
    [adot^2 (1+beta^2) - 2 adot bdot alpha beta + bdot^2 (1+alpha^2)] / s^4

which reduces to adot^2 + bdot^2 only at alpha = beta = 0.  Fixed in
sigma_gdot_sq (and bk_basis.h documentation) to use the exact form,
which reproduces the parabolic-boundary condition |v|^2 = 2 mu / |r|
to machine precision in the test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bk_fit.cpp contains the LM driver that performs an orbit fit in the
universal Bernstein-Khushalani parameterization on top of layup's
existing Cartesian variational machinery.  Included from orbit_fit.cpp
inside the `namespace orbit_fit` block, after the Cartesian helpers
(compute_residuals, create_sequences, get_weight_matrix, converged)
and the Observation/FitResult types are in scope, so no forward
declarations are needed.

The driver structure mirrors run_from_vector_with_initial_guess but
operates in BK basis throughout:

  1. Pick a fiducial direction from the observations' rho_hat vectors
     (mean direction, Gram-Schmidt for the orthonormal a, b).
  2. Convert the Cartesian seed to BK via cartesian_to_bk.
  3. Compute a fixed bound-orbit energy prior precision on gdot from
     the BK seed (1 / sigma_gdot_sq), zero precision otherwise.
  4. LM loop:
       - convert current BK state to Cartesian -> reb_particle
       - call compute_residuals to get tangent-plane residuals and
         Cartesian 6-element partials per observation
       - chain-rule: B_bk = B_cart * dcart_dbk(current BK, fiducial)
       - assemble C = B_bk^T W B_bk + lambda I + P_prior,
                 grad = B_bk^T W r + P_prior * p_bk
       - solve, Marquardt rho-ratio accept/reject, update BK state,
         check convergence (using the existing `converged` predicate).
  5. On convergence:
       - cov_bk = (B^T W B + P_prior)^-1 (Hessian without lambda)
       - cov_cart = J cov_bk J^T (J = dcart_dbk at converged BK state)
       - return FitResult with state = bk_to_cartesian(BK_final) and
         cov flattened from cov_cart.  method = "bk_native".

Initial lambda and Marquardt accept threshold match the Cartesian
fit at orbit_fit.cpp:553.  Early-exit guard: returns a non-success
FitResult (flag = 1) without crashing when detections.size() < 3.

main.cpp gains an orbit_fit::bk_fit_bindings(m) call alongside the
existing orbit_fit bindings, exposing run_bk_native_fit to Python.

tests/layup/test_bk_fit.py covers the Layer 2 smoke tests:
  * binding loads and run_bk_native_fit returns a FitResult
  * empty-obs path returns flag != 0 without crashing
  * <3 obs path triggers the early-exit guard

Layer 2 convergence tests against synthetic observations from a
known orbit (and the Cartesian/BK agreement test on well-arced
mainbelt) are next steps -- they need either the predict-path
output piped back in or the diagnostic/scan dataset wired up to
this branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Generate synthetic observations from a known Cartesian state via
layup's predict_sequence path (a fixed barycenter observer, so the
only dynamical content is the orbit itself), then feed those
observations back into both run_bk_native_fit and the existing
Cartesian fit.

Three new test categories on top of the smoke tests:

  * test_bk_native_fit_recovers_known_state: with the truth state as
    the seed, BK converges in essentially one iteration to a fit
    state matching the truth to rtol=1e-6 and chi2 < 1e-12.
    Parameterized over a 3 AU mainbelt 60-day arc and a 40 AU TNO
    300-day arc.

  * test_bk_native_fit_recovers_from_perturbed_seed: with the seed
    perturbed by 0.1% in each component, the LM loop still converges
    to the truth state (rtol=1e-6) -- exercising the chain-rule
    Jacobian + Marquardt damping on a non-trivial number of
    iterations.  Same two orbital regimes.

  * test_bk_and_cartesian_fits_agree: for the well-constrained
    mainbelt case, run_bk_native_fit and run_from_vector_with_initial_guess
    converge to states that agree at rtol=1e-6.  Establishes the
    "no regression on the easy case" baseline.

All seven tests in tests/layup/test_bk_fit.py pass with ASSIST
ephemeris available; skip cleanly without it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the universal BK fitter (run_bk_native_fit) into layup's Python
do_fit pipeline alongside the existing Cartesian fit, so callers can
choose the engine per call rather than via the C++ entry point.

Changes:

  - src/layup/orbitfit.py
      * Import run_bk_native_fit from layup.routines.
      * Add a module-level _MU_SUN constant (heliocentric GM in
        AU^3 / day^2) used to construct the BK fixed energy prior.
      * Add _run_fit(assist_ephem, initial_guess, obs, engine) helper
        that dispatches to:
          - run_from_vector_with_initial_guess  for engine='cartesian'
          - run_bk_native_fit (with _MU_SUN)    for engine='bk_native'
          - ValueError otherwise
      * do_fit gains an `engine='cartesian'` parameter (default
        preserves the existing behavior).  All five
        run_from_vector_with_initial_guess call sites inside do_fit
        are now routed through _run_fit so the engine choice
        propagates uniformly.

  - tests/layup/test_bk_fit.py
      * test_run_fit_dispatch_cartesian: _run_fit(..., 'cartesian')
        matches direct run_from_vector_with_initial_guess on
        synthetic mainbelt observations.
      * test_run_fit_dispatch_bk_native: _run_fit(..., 'bk_native')
        matches direct run_bk_native_fit(ephem, ig, obs, MU_SUN).
      * test_run_fit_dispatch_unknown_engine_raises: ValueError on
        an unknown engine name.

The 'auto' (distance-dispatched) engine from PR 323 is intentionally
not wired up here; when 323 lands first this branch rebases and
gains both options.  Likewise the 'bk' (liborbfit-backed) engine
from PR 323 is independent of this work.

All 10 tests in test_bk_fit.py and 25 tests in test_bk_basis.py
continue to pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tests/layup/test_bk_everywhere.py drives both engine='cartesian' and
engine='bk_native' against the diagnostic/scan dataset at
~/Dropbox/claude_layup/diagnostic/scan/truth/ -- the same 7-population,
14-arc-length scan that PR 323's auto-dispatch was validated against.
Skips when either the ASSIST ephemeris or the diagnostic scan is
unavailable, so CI is unaffected.

Two test groups:

  * test_engine_sweep_well_arced_cases: on long-arc cases (30-60d
    mainbelt + 60d classical TNO), both engines converge near truth
    (drift < 1% of heliocentric distance) and agree with each other.

  * test_bk_beats_cartesian_on_short_arc_distant: on distant short-arc
    cases (70 AU scattered / 42 AU classical at 10-14 day arcs), BK
    drifts no more than Cartesian from truth AND uses fewer LM
    iterations.  This is the regime BK was designed for, and the
    diagnostic data shows it strongly: on scattered_70AU_arc_014.00d
    BK stays 0.02 AU from truth in 6 iterations while Cartesian
    wanders 4.5 AU over 58 iterations.

The module also exposes a sweep_cases_from_diagnostic() helper for
ad-hoc engine-sweep harness scripts.

All 6 Layer 3 tests pass (in addition to the 25 Layer 1 + 10 Layer 2
tests, for 41 total BK tests on this branch).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A runnable CLI script that drives both engine='cartesian' and
engine='bk_native' across an entire diagnostic-scan directory, writes
per-case metrics to CSV, and prints a population-level summary
(BK wins / Cartesian wins / per-engine failures / mean iteration
counts, plus median+mean drift and iteration ratios).

Usage:
    python tools/bk_engine_sweep.py --scan-dir <dir> --output <csv>

Defaults discover the project's diagnostic scan at
~/Dropbox/claude_layup/diagnostic/scan/truth and the layup ephemeris
cache at ~/Library/Caches/layup.  Both are overrideable via flags,
so anyone with a compatible truth dataset can reproduce.

Running on the 98-case scan (truth state as LM seed, sigma_arcsec=0.1):

  Population              n   BK win   Cart win   cart fail   bk fail   both fail
  -------------------------------------------------------------------------------
  centaur_15AU           14       13          0           0         0           1
  centaur_25AU           14        9          2           2         0           1
  classical_42AU         14       10          3           0         0           1
  mainbelt_2.5AU         14       10          3           0         0           1
  mainbelt_3.5AU         14       12          1           0         0           1
  scattered_70AU         14        7          2           4         0           1
  sednoid_80AU           14        6          1           6         0           1
  -------------------------------------------------------------------------------
  TOTAL                  98       67         12          12         0           7

  Across 79 cases where both engines succeed:
    drift ratio  (BK / Cart):  median=0.560, mean=187.199
    iter ratio   (BK / Cart):  median=0.386, mean=0.524

Headline: BK never fails when Cartesian succeeds (0 / 98), succeeds in
12 cases where Cartesian flag=2's out, and on the typical case is ~2x
closer to truth in ~40% the iterations.  The mean drift ratio of 187 is
inflated by the 70-80 AU short-arc cases where Cartesian wanders 5-13 AU
while BK stays put.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure whitespace -- two blank-line adjustments black wants for PEP 8
spacing.  No test changes; 25 tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR 326 added the engine= parameter to do_fit, but the layup-orbitfit
command-line entry point didn't forward it -- so end users running
`layup orbitfit input.csv ADES_csv` always got the Cartesian engine.
This wires --engine end-to-end:

  CLI argparse (--engine {cartesian, bk_native}, default 'cartesian')
    -> orbitfit_cli reads cli_args.engine
       -> orbitfit() forwards engine through to process_data_by_id's
          kwargs
          -> _orbitfit() receives engine and passes to do_fit()
             -> do_fit (already engine-aware from PR 326) dispatches
                via _run_fit() to either run_from_vector_with_initial_guess
                or run_bk_native_fit.

argparse's choices=["cartesian", "bk_native"] gives users an early
error on a typo'd engine name; the Python-level _run_fit dispatch
also raises ValueError if a caller passes an unknown engine
directly (existing behavior).

Tests:
  - test_orbit_fit_cli_raises_with_unknown_engine: parallel to the
    existing unknown-iod test; FakeCliArgs.engine='not_an_engine'
    triggers a ValueError from the dispatch.
  - The existing test_orbit_fit_cli_raises_with_unknown_iod gets a
    FakeCliArgs.engine='cartesian' field so it continues to pass
    after orbitfit_cli starts reading that attribute.

Confirmed `layup orbitfit --help` shows the new flag with usage
`--engine {cartesian,bk_native}` and a description of the tradeoff.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@matthewholman matthewholman requested a review from mschwamb May 17, 2026 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant