quote-ceiling-estimator

Quote-as-ceiling billing for AI agent APIs that charge by the step. Pre-estimate the cost, treat the estimate as a hard maximum, and if a job overruns the estimate the platform eats the difference — the subscriber never sees it on their invoice.

---

AI agent APIs bill by the step. Before your job dispatches, you don't know how many steps it will take — and neither does your user. This library enforces a simple contract:

The estimate shown before dispatch is the maximum ever charged. Overruns are absorbed by the platform, not passed to the user.

It's a correctness guarantee for usage-metered APIs — not a suggestion, not a dashboard metric. A CI-enforced ceiling.

---

Table of contents

• The three invariants
• Quickstart
• Architecture
• When to use this
• Three-phase rollout
• ASC 606 compliance
• How this repo got built: the extraction + TestSprite loop
• Use in your product
• Attribution

---

The three invariants

Every call to apply_quote_ceiling proves, in code:

1. Double-entry      billed_credits + platform_absorbed_credits == actual_credits
2. Ceiling           billed_credits <= quoted_credits
3. Idempotency       retries on the same run_id are a no-op

In plain English: every credit is accounted for (nothing vanishes), the subscriber is never billed more than the quote, and if a retry or duplicate webhook fires the run is recorded exactly once.

These are asserted at runtime (not just in tests), so a bug in the billing slice crashes loud in CI instead of silently overcharging a subscriber. 59 tests pass on every push; two full rounds of TestSprite AI test generation explored the API surface for missed edge cases.

Quickstart

git clone https://github.com/thebizfixer/quote-ceiling-estimator.git
cd quote-ceiling-estimator
pip install -e ".[dev]"
pytest                          # 59 passed
uvicorn app.main:app --reload   # http://127.0.0.1:8000/docs

Estimate a run, then record actuals against the ceiling:

# 1. Get a quote (the ceiling)
curl -s -X POST http://127.0.0.1:8000/estimate \
  -H "Content-Type: application/json" \
  -d '{"target_population_size": 25, "channels": [{"slug":"wsj","category":"news"}], "tier_name": "Investigator"}'
# => {"credits": 5.0, "cost_usd": 0.0625, ...}

# 2. Record a run that overruns 3x
curl -s -X POST http://127.0.0.1:8000/run/record \
  -H "Content-Type: application/json" \
  -d '{"run_id":"demo-1","quote_credits":5.0,"actual_credits":15.0,"sale_usd_per_credit":0.0125}'
# => {"inserted": true, "entry": {
#      "billed_credits": 5.0,                  # capped at quote
#      "platform_absorbed_credits": 10.0,      # overrun absorbed
#      "platform_absorbed_usd": 0.125,         # accrued to platform
#      "enforced": true
#    }}

Architecture

flowchart LR
    User[Subscriber] -->|1. target, channels, tier| Estimator["estimator.py<br/>estimate_snapshot_pipeline"]
    Estimator -->|2. SnapshotEstimate<br/>credits = CEILING| Quote[(Persisted<br/>quote)]
    Quote -->|3. run_id, quote| Agent[AI agent API<br/>step-billed]
    Agent -->|4. actual_credits| Ceiling["billing.py<br/>apply_quote_ceiling"]
    Ceiling -->|"billed_credits<br/>≤ quoted_credits"| Subscriber[(Invoice)]
    Ceiling -->|"platform_absorbed_credits<br/>(overrun)"| Platform[(Platform<br/>absorbs overrun)]

Loading

Modules:

File	Role
src/estimator_pkg/estimator.py	estimate_snapshot_pipeline — deterministic pre-run quote
src/estimator_pkg/billing.py	apply_quote_ceiling — the three-invariant contract
src/estimator_pkg/schemas.py	SnapshotEstimate, RunHistoryEntry (Pydantic v2)
src/estimator_pkg/sale_usd.py	Subscriber's effective sales-price $/credit
src/estimator_pkg/runtime_stats.py	Thread-safe in-memory runtime stats (drop-in for a real DB)
src/estimator_pkg/constants.py	Tier configs, dispatch caps, step-cost rates
src/estimator_pkg/flags.py	Phase-gated rollout flags
app/main.py	FastAPI demo — /estimate, /run/record, /health

When to use this

Ordered most-specific-first — if any of these sound like you, drop this module in:

1. AI agent APIs that charge by the step. TinyFish, OpenAI Agents SDK, Anthropic tool use, Gemini function calls — any provider where you dispatch a job and pay for however many steps the model takes. You cannot give your user a predictable price without a ceiling on top.
2. Usage-metered SaaS. Any product where a single request from your user triggers a bunch of downstream calls you have to pay for — and the count isn't predictable in advance.
3. Per-run billing with SLA budgets. Anywhere you've written "this run cost ~$X" in a product UI and don't want to be wrong.

Three-phase rollout

The library ships safe by default (Phase 0, shadow mode) so you can flip it on production without breaking existing billing:

Phase	Flag	Behavior
0 — Shadow (default)	all flags off	Quote is computed and persisted; nothing enforces it yet. Compare in logs.
1 — Deadline	estimator_drive_deadline=True	Orchestrator uses pipeline_seconds × 1.05 as the wall-clock ceiling.
2 — Billing	estimator_enforce_quote_ceiling=True + estimator_absorb_variance=True	billed_credits capped at quoted_credits; overruns accrue to platform_absorbed_credits and platform_absorbed_usd.

ASC 606 compliance

Absorbed overruns aren't "lost revenue" — they're a cost the platform takes on deliberately, so the subscriber gets a predictable bill. The library records them that way: the subscriber's payment is counted as revenue (at the lower of quote or actual), and any overrun is logged as a cost the platform paid to deliver the run. That lines up with the standard US revenue-recognition rule for usage-metered products (ASC 606, or IFRS 15 outside the US).

The number your finance team actually needs — platform_absorbed_usd — is saved on every run, ready to reconcile. No scraping Stripe exports at quarter-end. Full memo: docs/BILLING_CONTROLS.md.

How this repo got built: the extraction + TestSprite loop

This library is an extraction from a wired-up product (Sentimentary), not a clean-room implementation. That matters: the moment custom code leaves its home platform, the contracts that were implicit inside the monorepo become someone else's problem. Error strings that changed shape with every run. Schemas that were "understood" by the consumer. Edge cases that existed in production but were never tested because the platform never triggered them.

I ran TestSprite post-extraction to find those gaps systematically. Round 1 passed 3 of 8 AI-generated tests. The five failures told a story:

• Two were contract stability issues — my Unknown tier_name error included the variable-order Known tiers: [...] list in the detail string, which no sane client could assert against. Moved the list to an X-Known-Tiers response header; the detail became "Unknown tier_name" (stable).
• One was a schema-doc bug — my TestSprite code summary claimed SnapshotEstimate.target_samples existed; it doesn't (the real field is per_agent[].target_size). Correction shipped.
• One was semantic — TestSprite inferred from my contract that quote_credits == 0, actual_credits > 0 should return an error. It shouldn't — that's the platform absorbing 100% of the overrun, which is the point of the pattern. I hadn't documented the semantics anywhere, so the AI agent inferred incorrectly. Documentation fix; added explicit tests.

Fixes shipped, Round 2 passed 10/10.

The unexpected upside: the same process surfaced real gaps back in the source platform's production code — edge cases that had shipped because nothing in the existing test suite exercised them. Extraction + AI test generation turned out to be a way to audit the source platform, not just validate the extracted library. Those findings are now on the source platform's backport backlog.

Use in your product

This library is extracted from the production billing path of Sentimentary — sentiment research in minutes, not weeks. If you're shipping an AI-agent product and need predictable subscriber billing, either:

• Lift the pattern from this repo (Apache-2.0, commercial use OK, NOTICE attribution required), or
• Talk to me at thebizfixer.com — I built the full orchestrator (estimator, ceiling, accrual, investor reconciliation) and will help you ship yours.

Attribution

Originally built for Sentimentary by Topher Ross — sentiment research in minutes, not weeks.

Licensed under Apache License 2.0. See NOTICE.

---

Subfolder note. This directory lives inside the Sentimentary monorepo and is published standalone to GitHub via git subtree push --prefix=quote-ceiling-estimator public main.