Skip to content

Add audit-ci read-only CI right-sizing report#299

Merged
EffortlessSteven merged 1 commit into
mainfrom
feat/audit-ci
Jun 6, 2026
Merged

Add audit-ci read-only CI right-sizing report#299
EffortlessSteven merged 1 commit into
mainfrom
feat/audit-ci

Conversation

@EffortlessSteven

Copy link
Copy Markdown
Member

What

Roadmap #25 (ADR 0002): ub-review audit-ci — the read-only adoption wedge. Deterministic-only v0; every recommendation is judgment: deterministic.

  • Artifacts per the contract doc under <out>/ci-audit/: inventory.json, history.json, costs.json, correlation.json, recommendations.json, audit-report.md (schemas ub-review.ci_{inventory,history,costs,correlation,recommendations}.v1).
  • Evidence from the GitHub API only (workflows/runs/jobs, GETs, pagination capped and the cap recorded); no YAML crate and no new dependencies — triggers/paths/timeouts come from a targeted line-scan, with unextracted fields recorded as explicit evidence gaps.
  • Independent merge-decision signal: a job counts an independent failure only when every cheaper job (lower p50, same run) passed; the rule text ships inside correlation.json and the report.
  • Tier rules: security patterns (codeql/secret/scan/sign/provenance/attest/deploy/release/publish/permission/push/apply/terraform/sarif/compliance/docker) are flag-for-human always — overmatch is deliberate, ambiguity resolves to human; survivorship caps confidence at low when absence-of-failures is the only signal; thin history (<20 runs) never right-sizes; nothing right-sizes below adaptive.
  • Report grouped by decision relevance (adaptive candidates first), each number stated once per line; tokenless mode degrades to inventory-only with the gap stated exactly once; degradation gaps mirrored into every artifact so each is honest standalone.
  • Ambient GITHUB_TOKEN pickup is intentional (adoption wedge); dropped API items are counted as evidence gaps, never silently shrunk.

Validation

  • 274 tests pass (18 audit-ci unit tests: tier matrix, security escapes incl. docker-push/tf-apply/upload-sarif/compliance-check, correlation rule, survivorship cap, tokenless degradation, report receipts/no-boilerplate, repo-slug parsing); fmt/check/clippy/doc/policy-check clean after rebase onto Add gate verdict surface with fail-on-gate contract #298.
  • Live smoke against this repo, both modes: tokenless (7 jobs, inventory-only, conservative flags) and token (10 jobs, 1000 PR runs across 6 workflows; all receipt pointers resolved to real artifact fragments; release jobs auto-flagged).
  • 3 adversarial review lenses; all 3 high findings fixed pre-PR (security-pattern escapes, flat report ordering, triple-stated receipts).

Known gaps

  • Branch protection/rulesets not queried in v0: required_check is null/unknown, recorded as a gap.
  • permissions/uses_secrets/matrix_size not extracted from YAML (line-scan scope), recorded as a gap; security flagging is name/uses-based, run: step contents are not scanned.
  • History/costs are PR-runs-only; workflows list is single-page (>100 workflows truncates with the unknown-workflow marker).
  • A token-present API failure on the workflows/runs listing aborts instead of degrading to inventory-only.
  • run_curl_json_get parallels the existing POST helper (~55 shared lines); dedup deferred to the modularization phase.

@gemini-code-assist

Copy link
Copy Markdown

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Jun 6, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@EffortlessSteven, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 30 minutes and 14 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 4ccf4e5b-33d6-4c55-9d22-85c7898d4d30

📥 Commits

Reviewing files that changed from the base of the PR and between ef4a338 and f5f1e8d.

📒 Files selected for processing (2)
  • src/cli.rs
  • src/main.rs
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/audit-ci

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented Jun 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.74390% with 283 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/main.rs 82.74% 283 Missing ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant