diff --git a/CHANGELOG.md b/CHANGELOG.md index b6204d0..dfa01b7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ This project follows [Semantic Versioning](https://semver.org/). From **v1.0.0** ### Added +- **`flightdeck demo`** — runs the packaged **examples/quickstart** workflow (init → pricing → policy → register → ingest → diff → promote → history) in a **temp workspace**, with no **`sed`** or repo paths; wheels ship fixtures under **`flightdeck/_bundled_quickstart`** via Hatch **`force-include`**. **`FLIGHTDECK_QUICKSTART_ROOT`** overrides fixture resolution for CI or forks. - **Web UI (`flightdeck serve`):** **`/#/settings`** for appearance (Light / Dark / System, **`flightdeck-theme`**); collapsible sidebar (**`flightdeck-sidebar-collapsed`**); **offline system font stack** (no remote font CSS); sidebar + favicon use **bundled** **`/assets/flightdeck-icon-*.png`** with stable **`GET /flightdeck-icon.png`** fallback; **`html[data-theme="dark"]`** tokens and Playwright **`web/e2e/`** (`smoke` icon checks, `theme.spec.ts`, `sidebar.spec.ts`). - **`flightdeck pricing check`** — reports **`flightdeck-bundled-*`** snapshot age vs **`--max-age-days`** (default **90**); **`--fail`** for CI. **`release diff`** / **`POST /v1/diff`** append **`pricing.warnings`** when bundled snapshots exceed the same age threshold. - **`flightdeck.integrations.telemetry.configure_otel_tracing()`** — optional OTLP HTTP **`TracerProvider`** wiring when the **`telemetry`** extra is installed (see **`docs/sdk-integrations.md`**). diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index f1d2c5c..60f02a0 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -73,9 +73,12 @@ python -m ruff check src tests python -m pytest flightdeck --help flightdeck doctor +flightdeck demo flightdeck-quickstart-verify ``` +**Fast path for contributors:** **`flightdeck demo`** runs the same core ledger steps as below in a **temp workspace** (fixtures from **`examples/quickstart`**, or **`flightdeck/_bundled_quickstart`** inside an installed wheel). **`flightdeck-quickstart-verify`** adds **`release verify`** + **`doctor`**. + Match **CI**’s CLI smoke: **`flightdeck --help`** must run successfully after changes to the CLI surface. Full command flags and exit codes: [README.md](https://github.com/flightdeckdev/flightdeck/blob/main/README.md). Cross-platform quickstart parity: **`flightdeck-quickstart-verify`** / **`python -m flightdeck.quickstart_smoke`** (also run in CI). HTTP API reference: **[docs/http-api.md](docs/http-api.md)**. Python SDK: **[docs/sdk.md](docs/sdk.md)**. @@ -151,6 +154,14 @@ If **PyPI** rejects **attestations** for your project, set **`attestations: fals ## Local Demo +**One command** (uses bundled **`examples/quickstart`** fixtures; no **`sed`**): + +```bash +flightdeck demo +``` + +**Manual** (same story as **`flightdeck demo`**, in your cwd): + ```bash flightdeck init flightdeck pricing import examples/quickstart/pricing-baseline.yaml diff --git a/README.md b/README.md index 8501379..6121b69 100644 --- a/README.md +++ b/README.md @@ -67,11 +67,33 @@ flowchart LR --- +## Fast start + +After **`pip install flightdeck-ai`** (or **`uv tool install flightdeck-ai`**): + +```bash +flightdeck demo +``` + +**`flightdeck demo`** runs the full quickstart ledger flow in a disposable temp workspace—no **`sed`**, no fixture paths—using **`examples/quickstart`** from your checkout or packaged **`flightdeck/_bundled_quickstart`** from PyPI. + +**Web UI** (needs a workspace in the current directory): + +```bash +flightdeck init +flightdeck serve +``` + +Open **http://127.0.0.1:8765/**. Same end-to-end checks CI uses: **`flightdeck-quickstart-verify`** (contributors: **`uv run flightdeck-quickstart-verify`**). + +--- + ## Install and smoke-test ```bash uv sync --extra dev uv run flightdeck --help +uv run flightdeck demo uv run flightdeck-quickstart-verify ``` @@ -138,6 +160,7 @@ Bundled pricing from `init` is a **convenience snapshot**—`flightdeck pricing uv sync --frozen --extra dev uv run python -m ruff check src tests uv run python -m pytest +uv run flightdeck demo uv run flightdeck-quickstart-verify uv run flightdeck --help ``` diff --git a/docs/cli.md b/docs/cli.md index 5591085..7177af9 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -15,10 +15,12 @@ serve` see [http-api.md](http-api.md). | `--version` | Print the installed version and exit | | `--help` | Print help for any command or subcommand | -All commands require a `flightdeck.yaml` in the working directory (or the default path +Most commands require `flightdeck.yaml` in the working directory (or the default path `./flightdeck.yaml`). Run `flightdeck init` to create one. **`flightdeck init`** writes the config, then loads it to migrate the ledger and (by default) import bundled pricing. +**`flightdeck demo`** is an exception: it creates a **temporary** workspace and does not read `./flightdeck.yaml` from your shell cwd. + ## Actor resolution Several commands that write to the audit ledger (`release promote`, `release rollback`, @@ -76,6 +78,27 @@ set **`database_url`** to a `postgresql://…` (or `postgres://…`) DSN and ins --- +## `flightdeck demo` + +Run the **examples/quickstart** workflow end-to-end in a **disposable temp directory**: **`init`** → custom **`pricing import`** (both YAMLs) → **`policy set`** → **`release register`** (both bundles) → substitute **`release_id`** placeholders in JSONL → **`runs ingest`** → **`release diff`** → **`release promote`** (baseline under policy) → **`release history`**. + +Does **not** require **`flightdeck.yaml`** in the current directory. Fixtures resolve in order: **`--quickstart-root`**, **`FLIGHTDECK_QUICKSTART_ROOT`**, **`examples/quickstart`** relative to a git checkout, then **`flightdeck/_bundled_quickstart`** packaged in the wheel (PyPI installs). + +```bash +flightdeck demo [--quickstart-root DIR] [--verify / --no-verify] [--doctor / --no-doctor] [--keep-workspace] +``` + +| Option | Default | Description | +|--------|---------|-------------| +| `--quickstart-root` | (see above) | Directory containing `policy.yaml`, pricing YAMLs, `*-events.jsonl`, and `baseline-release` / `candidate-release` | +| `--verify` | off | Also run **`release verify`** on the baseline bundle (parity with **`flightdeck-quickstart-verify`**) | +| `--doctor` | off | Also run **`flightdeck doctor`** | +| `--keep-workspace` | off | Keep the temp workspace and print its path | + +On success, prints a short confirmation. Exit **0** on success, **1** on failure (same as subprocess failures from underlying CLI steps). + +--- + ## `flightdeck doctor` Run read-only health checks on the workspace ledger (SQLite file or PostgreSQL when diff --git a/examples/quickstart/README.md b/examples/quickstart/README.md index c2a361c..77a1206 100644 --- a/examples/quickstart/README.md +++ b/examples/quickstart/README.md @@ -7,7 +7,13 @@ These files are meant to be copied or substituted locally: - `policy.yaml` is an example active policy used by `release diff` and `release promote`. - `*-events.jsonl` contain placeholder `release_id` values (`__BASELINE_RELEASE_ID__`, `__CANDIDATE_RELEASE_ID__`). -Fastest path (from **repository root**, with **uv**): +Fastest path after **`pip install flightdeck-ai`**: + +```bash +flightdeck demo +``` + +Full CI parity (verify + doctor; from **repository root** with **uv**): ```bash uv run flightdeck-quickstart-verify diff --git a/pyproject.toml b/pyproject.toml index 5faa96e..17cfad1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -78,6 +78,7 @@ flightdeck-quickstart-verify = "flightdeck.quickstart_smoke:quickstart_verify_ma [tool.hatch.build.targets.wheel] packages = ["src/flightdeck"] +force-include = { "examples/quickstart" = "src/flightdeck/_bundled_quickstart" } [tool.uv] # Contributor installs: `uv sync --extra dev` (see DEVELOPMENT.md). After changing diff --git a/src/flightdeck/_bundled_quickstart/__init__.py b/src/flightdeck/_bundled_quickstart/__init__.py new file mode 100644 index 0000000..1af0f92 --- /dev/null +++ b/src/flightdeck/_bundled_quickstart/__init__.py @@ -0,0 +1 @@ +"""Bundled quickstart fixtures (wheel: see pyproject hatch force-include).""" diff --git a/src/flightdeck/cli/main.py b/src/flightdeck/cli/main.py index a1e832b..7238ea6 100644 --- a/src/flightdeck/cli/main.py +++ b/src/flightdeck/cli/main.py @@ -14,6 +14,7 @@ from flightdeck import __version__ from flightdeck.bundle import bundle_checksum +from flightdeck.demo_flow import demo_session from flightdeck.bundled_pricing_bootstrap import ( BUNDLED_PRICING_VERSION, DEFAULT_CATALOG_RELATIVE_PATH, @@ -106,6 +107,65 @@ def init(path_: str, no_bundled_pricing: bool) -> None: ) +@cli.command() +@click.option( + "--quickstart-root", + "quickstart_root_opt", + type=click.Path(exists=True, file_okay=False, path_type=Path), + default=None, + help="Directory with quickstart YAML/JSONL fixtures (default: repo examples/ or bundled wheel copy).", +) +@click.option( + "--verify/--no-verify", + default=False, + show_default=True, + help="Also run release verify on the baseline bundle (matches flightdeck-quickstart-verify).", +) +@click.option( + "--doctor/--no-doctor", + default=False, + show_default=True, + help="Also run flightdeck doctor after the workflow.", +) +@click.option( + "--keep-workspace", + is_flag=True, + default=False, + help="Keep the temp workspace and print its path (for inspection).", +) +def demo( + quickstart_root_opt: Path | None, + verify: bool, + doctor: bool, + keep_workspace: bool, +) -> None: + """Run the bundled quickstart end-to-end in a disposable workspace (no manual sed). + + Typical install: ``pip install flightdeck-ai`` then ``flightdeck demo``. Next: ``flightdeck init`` + in your project and wire ``runs ingest`` / ``release diff`` from real agents. + """ + ws = demo_session( + verify=verify, + doctor=doctor, + qs_dir=str(quickstart_root_opt) if quickstart_root_opt is not None else None, + promote_reason="demo", + keep_workspace=keep_workspace, + ) + click.echo( + "Demo OK — workspace initialized, releases registered, runs ingested, " + "diff computed, baseline promoted under policy." + ) + extras = [] + if verify: + extras.append("verify") + if doctor: + extras.append("doctor") + if extras: + click.echo(f"(also ran: {', '.join(extras)})") + if keep_workspace and ws is not None: + click.echo(f"Workspace: {ws}") + + @cli.command("doctor") @click.option( "--backup", diff --git a/src/flightdeck/demo_flow.py b/src/flightdeck/demo_flow.py new file mode 100644 index 0000000..f540303 --- /dev/null +++ b/src/flightdeck/demo_flow.py @@ -0,0 +1,181 @@ +"""Shared quickstart demo / CI verification (fixtures + subprocess CLI calls).""" + +from __future__ import annotations + +import os +import shutil +import subprocess +import sys +import tempfile +from pathlib import Path + +BASELINE_PH = "__BASELINE_RELEASE_ID__" +CANDIDATE_PH = "__CANDIDATE_RELEASE_ID__" + + +def flightdeck_argv() -> list[str]: + exe = shutil.which("flightdeck") + if exe: + return [exe] + return [sys.executable, "-m", "flightdeck.cli.main"] + + +def quickstart_root(*, env_dir: str | None = None) -> Path: + """Resolve the directory containing quickstart YAML/JSONL fixtures. + + Order: explicit ``env_dir``, ``FLIGHTDECK_QUICKSTART_ROOT``, repo + ``examples/quickstart``, then wheel-bundled ``_bundled_quickstart``. + """ + if env_dir: + p = Path(env_dir).expanduser() + if not p.is_dir(): + msg = f"Not a directory: {p}" + raise FileNotFoundError(msg) + return p.resolve() + + env = os.environ.get("FLIGHTDECK_QUICKSTART_ROOT") + if env: + p = Path(env).expanduser() + if not p.is_dir(): + msg = ( + f"FLIGHTDECK_QUICKSTART_ROOT is not a directory: {p}. " + "Unset it or point it at examples/quickstart." + ) + raise FileNotFoundError(msg) + return p.resolve() + + repo = Path(__file__).resolve().parents[2] + examples = repo / "examples" / "quickstart" + if examples.is_dir(): + return examples + + bundled = Path(__file__).resolve().parent / "_bundled_quickstart" + if bundled.is_dir() and (bundled / "policy.yaml").is_file(): + return bundled + + msg = ( + "Quickstart fixtures not found. Clone the repo or set FLIGHTDECK_QUICKSTART_ROOT " + "to a copy of examples/quickstart." + ) + raise FileNotFoundError(msg) + + +def _run(fd: list[str], *args: str, cwd: Path) -> subprocess.CompletedProcess[str]: + return subprocess.run( + [*fd, *args], + cwd=cwd, + check=True, + text=True, + capture_output=True, + ) + + +def run_quickstart_verify( + workspace: Path, + qs: Path, + fd: list[str] | None = None, + *, + verify: bool = True, + doctor: bool = True, + promote_reason: str = "quickstart smoke", +) -> None: + """Run the full quickstart workflow used by CI (temp workspace must exist).""" + fd = fd or flightdeck_argv() + baseline_events = workspace / "baseline-events.jsonl" + candidate_events = workspace / "candidate-events.jsonl" + + _run(fd, "init", cwd=workspace) + _run(fd, "pricing", "import", str(qs / "pricing-baseline.yaml"), cwd=workspace) + _run(fd, "pricing", "import", str(qs / "pricing-candidate.yaml"), cwd=workspace) + _run(fd, "policy", "set", str(qs / "policy.yaml"), cwd=workspace) + + reg_b = _run(fd, "release", "register", str(qs / "baseline-release"), cwd=workspace) + baseline_id = reg_b.stdout.strip() + reg_c = _run(fd, "release", "register", str(qs / "candidate-release"), cwd=workspace) + candidate_id = reg_c.stdout.strip() + + baseline_events.write_text( + (qs / "baseline-events.jsonl").read_text(encoding="utf-8").replace(BASELINE_PH, baseline_id), + encoding="utf-8", + ) + candidate_events.write_text( + (qs / "candidate-events.jsonl").read_text(encoding="utf-8").replace(CANDIDATE_PH, candidate_id), + encoding="utf-8", + ) + + _run(fd, "runs", "ingest", str(baseline_events), cwd=workspace) + _run(fd, "runs", "ingest", str(candidate_events), cwd=workspace) + _run(fd, "release", "diff", baseline_id, candidate_id, "--window", "7d", cwd=workspace) + _run( + fd, + "release", + "promote", + baseline_id, + "--env", + "local", + "--window", + "7d", + "--reason", + promote_reason, + cwd=workspace, + ) + _run(fd, "release", "history", "--agent", "agent_support", "--env", "local", cwd=workspace) + if verify: + _run(fd, "release", "verify", baseline_id, "--path", str(qs / "baseline-release"), cwd=workspace) + if doctor: + _run(fd, "doctor", cwd=workspace) + + +def run_demo_happy_path( + workspace: Path, + qs: Path, + fd: list[str] | None = None, + *, + verify: bool = False, + doctor: bool = False, + promote_reason: str = "demo", +) -> None: + """Minimal demo: same ledger steps as CI verify, optional verify/doctor.""" + run_quickstart_verify( + workspace, + qs, + fd, + verify=verify, + doctor=doctor, + promote_reason=promote_reason, + ) + + +def demo_session( + *, + verify: bool, + doctor: bool, + qs_dir: str | None, + promote_reason: str, + keep_workspace: bool, +) -> Path | None: + """Create a temp workspace, run the demo. + + Removes the workspace when ``keep_workspace`` is false (unless setup fails). + Returns the workspace path when ``keep_workspace`` is true; otherwise ``None``. + """ + qs = quickstart_root(env_dir=qs_dir) + fd = flightdeck_argv() + tmp_s = tempfile.mkdtemp(prefix="flightdeck_demo_") + workspace = Path(tmp_s) + try: + run_demo_happy_path( + workspace, + qs, + fd, + verify=verify, + doctor=doctor, + promote_reason=promote_reason, + ) + except Exception: + shutil.rmtree(workspace, ignore_errors=True) + raise + if keep_workspace: + return workspace + shutil.rmtree(workspace, ignore_errors=True) + return None diff --git a/src/flightdeck/quickstart_smoke.py b/src/flightdeck/quickstart_smoke.py index deb03fd..c1c9b80 100644 --- a/src/flightdeck/quickstart_smoke.py +++ b/src/flightdeck/quickstart_smoke.py @@ -2,80 +2,20 @@ from __future__ import annotations -import shutil import subprocess import sys import tempfile from pathlib import Path -REPO = Path(__file__).resolve().parents[2] -QS = REPO / "examples" / "quickstart" -BASELINE_PH = "__BASELINE_RELEASE_ID__" -CANDIDATE_PH = "__CANDIDATE_RELEASE_ID__" - - -def _flightdeck_cmd() -> list[str]: - exe = shutil.which("flightdeck") - if exe: - return [exe] - return [sys.executable, "-m", "flightdeck.cli.main"] - - -def _run(fd: list[str], *args: str, cwd: Path) -> subprocess.CompletedProcess[str]: - return subprocess.run( - [*fd, *args], - cwd=cwd, - check=True, - text=True, - capture_output=True, - ) +from flightdeck.demo_flow import flightdeck_argv, quickstart_root, run_quickstart_verify def main() -> None: - fd = _flightdeck_cmd() + fd = flightdeck_argv() + qs = quickstart_root() with tempfile.TemporaryDirectory(prefix="fd_qs_", ignore_cleanup_errors=True) as tmp_s: tmp = Path(tmp_s) - baseline_events = tmp / "baseline-events.jsonl" - candidate_events = tmp / "candidate-events.jsonl" - - _run(fd, "init", cwd=tmp) - _run(fd, "pricing", "import", str(QS / "pricing-baseline.yaml"), cwd=tmp) - _run(fd, "pricing", "import", str(QS / "pricing-candidate.yaml"), cwd=tmp) - _run(fd, "policy", "set", str(QS / "policy.yaml"), cwd=tmp) - - reg_b = _run(fd, "release", "register", str(QS / "baseline-release"), cwd=tmp) - baseline_id = reg_b.stdout.strip() - reg_c = _run(fd, "release", "register", str(QS / "candidate-release"), cwd=tmp) - candidate_id = reg_c.stdout.strip() - - baseline_events.write_text( - (QS / "baseline-events.jsonl").read_text(encoding="utf-8").replace(BASELINE_PH, baseline_id), - encoding="utf-8", - ) - candidate_events.write_text( - (QS / "candidate-events.jsonl").read_text(encoding="utf-8").replace(CANDIDATE_PH, candidate_id), - encoding="utf-8", - ) - - _run(fd, "runs", "ingest", str(baseline_events), cwd=tmp) - _run(fd, "runs", "ingest", str(candidate_events), cwd=tmp) - _run(fd, "release", "diff", baseline_id, candidate_id, "--window", "7d", cwd=tmp) - _run( - fd, - "release", - "promote", - baseline_id, - "--env", - "local", - "--window", - "7d", - "--reason", - "quickstart smoke", - cwd=tmp, - ) - _run(fd, "release", "history", "--agent", "agent_support", "--env", "local", cwd=tmp) - _run(fd, "release", "verify", baseline_id, "--path", str(QS / "baseline-release"), cwd=tmp) - _run(fd, "doctor", cwd=tmp) + run_quickstart_verify(tmp, qs, fd) print("quickstart_smoke: OK") diff --git a/tests/test_demo_flow.py b/tests/test_demo_flow.py new file mode 100644 index 0000000..6d8dce9 --- /dev/null +++ b/tests/test_demo_flow.py @@ -0,0 +1,76 @@ +"""Tests for bundled quickstart resolution and demo flow helpers.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +from click.testing import CliRunner + +from flightdeck import demo_flow +from flightdeck.cli.main import cli + + +def test_quickstart_root_prefers_repo_examples(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.delenv("FLIGHTDECK_QUICKSTART_ROOT", raising=False) + repo_root = Path(demo_flow.__file__).resolve().parents[2] + examples = repo_root / "examples" / "quickstart" + if not examples.is_dir(): + pytest.skip("examples/quickstart not present in this checkout") + + assert demo_flow.quickstart_root() == examples.resolve() + + +def test_quickstart_root_env_override(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + qs = tmp_path / "qs" + qs.mkdir() + (qs / "policy.yaml").write_text("policy_id: x\n", encoding="utf-8") + monkeypatch.setenv("FLIGHTDECK_QUICKSTART_ROOT", str(qs)) + + assert demo_flow.quickstart_root() == qs.resolve() + + +def test_demo_session_keep_workspace(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + repo_root = Path(demo_flow.__file__).resolve().parents[2] + examples = repo_root / "examples" / "quickstart" + if not examples.is_dir(): + pytest.skip("examples/quickstart not present") + + monkeypatch.delenv("FLIGHTDECK_QUICKSTART_ROOT", raising=False) + ws = demo_flow.demo_session( + verify=False, + doctor=False, + qs_dir=None, + promote_reason="pytest demo", + keep_workspace=True, + ) + assert ws is not None + cfg = ws / "flightdeck.yaml" + assert cfg.is_file() + + +def test_demo_cli_exits_zero(monkeypatch: pytest.MonkeyPatch) -> None: + repo_root = Path(__file__).resolve().parents[1] + monkeypatch.chdir(repo_root) + runner = CliRunner() + res = runner.invoke(cli, ["demo"]) + assert res.exit_code == 0, res.output + assert "Demo OK" in res.output + + +def test_demo_session_cleanup(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + repo_root = Path(demo_flow.__file__).resolve().parents[2] + examples = repo_root / "examples" / "quickstart" + if not examples.is_dir(): + pytest.skip("examples/quickstart not present") + + monkeypatch.delenv("FLIGHTDECK_QUICKSTART_ROOT", raising=False) + ws = demo_flow.demo_session( + verify=False, + doctor=False, + qs_dir=None, + promote_reason="pytest demo", + keep_workspace=False, + ) + assert ws is None +