Skip to content

Commit c8b3c6e

Browse files
authored
Tighten Dependabot auto-fix prompt: truthful counts, minimal diffs (#1562)
Observed on the first run (PR #1559): - The PR body claimed "33+ open Dependabot alerts (32 moderate, 1 high)" but the Dependabot API only had ONE open alert (uuid, #172). The inflated count appears to have come from counting transitive `node_modules/**/uuid` occurrences in package-lock.json rather than distinct alert records. - The agent ran `npm audit fix`, which dragged along unrelated version bumps: brace-expansion and path-to-regexp (whose alerts had been auto-dismissed weeks earlier) and ~20 @fluidframework/* packages from 2.91.0 to 2.93.0. The diff ballooned well beyond what the single open alert required. Prompt changes: 1. Alert enumeration is the single source of truth. The agent must run a specific `gh api` query, persist the output to alerts.json, and build the PR title/body one-to-one from that list. It must not count transitive-path occurrences as separate alerts. 2. Preference order for fixes. (a) overrides in package.json, (b) direct-dependency bump, (c) parent-package bump only if (a)/(b) are infeasible, with justification required. `npm audit fix` is explicitly forbidden. 3. Diff discipline. After regenerating the lockfile, run `git diff --stat origin/main` and abort if package.json touches deps not on the alert list (other than adding overrides), or if any bump crosses a MAJOR version unnecessarily. 4. Structured PR body. Title is `chore(deps): fix N open Dependabot alert(s)` where N matches the alert count. Body has fixed sections: Summary, Alerts addressed (one bullet per alert with strategy noted), Collateral changes (or "None"), Verification. 5. Reaffirm no Co-authored-by trailers (these caused CLA friction on the first run).
1 parent 1e66d90 commit c8b3c6e

1 file changed

Lines changed: 88 additions & 21 deletions

File tree

.github/workflows/dependabot-autofix.yml

Lines changed: 88 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ jobs:
109109
110110
PROMPT=$(cat <<PROMPT_EOF
111111
You are running inside a GitHub Actions job. Follow the skill
112-
below to fix ALL currently open Dependabot alerts for the
112+
below to fix all currently open Dependabot alerts for the
113113
repository \`${REPO}\` in a SINGLE consolidated pull request.
114114
115115
Environment already set up for you:
@@ -124,28 +124,95 @@ jobs:
124124
\`gh pr create\`.
125125
126126
Mandatory rules:
127-
1. Use \`gh api repos/${REPO}/dependabot/alerts --paginate --jq '.[] | select(.state=="open")'\`
128-
to enumerate open alerts. Authenticate that call with
129-
\`GH_TOKEN=\$COPILOT_GITHUB_TOKEN gh api ...\` since the App token
130-
typically cannot read Dependabot alerts.
131-
2. If there are ZERO open alerts, do not commit, do not push,
132-
do not open a PR. Print "NO_OPEN_ALERTS" and exit.
133-
3. Apply fixes for ALL open alerts on branch \`${BRANCH}\`
134-
following Steps 1-6 of the skill. Produce one logical commit
135-
(or several, your choice) on that branch only.
136-
4. Run \`npm run build\` AND \`npm test\`. If either fails after
137-
your fixes, DO NOT push and DO NOT open a PR. Print
127+
128+
1. ALERT ENUMERATION — source of truth.
129+
Run exactly:
130+
GH_TOKEN=\$COPILOT_GITHUB_TOKEN gh api \\\\
131+
"repos/${REPO}/dependabot/alerts?state=open&per_page=100" \\\\
132+
--paginate \\\\
133+
--jq '.[] | {number, ghsa: .security_advisory.ghsa_id, pkg: .dependency.package.name, ecosystem: .dependency.package.ecosystem, severity: .security_vulnerability.severity, scope: .dependency.scope, manifest: .dependency.manifest_path, summary: .security_advisory.summary, vulnerable: .security_vulnerability.vulnerable_version_range, patched: .security_vulnerability.first_patched_version.identifier}'
134+
Save the raw output to \`alerts.json\`. The PR description
135+
and the count of "alerts fixed" MUST be built one-to-one
136+
from this list. Do NOT count transitive-path occurrences
137+
of the same alert number as separate alerts. Do NOT
138+
inflate counts based on how many \`node_modules/**\`
139+
entries appear in \`package-lock.json\`.
140+
141+
2. ZERO ALERTS — exit cleanly.
142+
If step 1 returns an empty list, do not commit, do not
143+
push, do not open a PR. Print "NO_OPEN_ALERTS" and exit.
144+
145+
3. MINIMUM VIABLE FIX — prefer the smallest blast radius.
146+
For each alert, pick the least-invasive strategy in this
147+
strict preference order and record your choice in the PR
148+
body:
149+
a. Transitive dependency → add/extend an entry in the
150+
\`overrides\` block of \`package.json\` pinning the
151+
vulnerable package to the first patched version.
152+
This is the preferred fix.
153+
b. Direct dependency → bump that specific dependency in
154+
\`package.json\` to the first patched version.
155+
c. Only if (a) and (b) are infeasible (e.g. override
156+
breaks peer-dep resolution, or a parent package
157+
pins the vulnerable version), escalate to bumping a
158+
parent package. In that case, justify the escalation
159+
in the PR body and keep the parent bump as narrow
160+
as possible (patch > minor > major).
161+
Never run \`npm audit fix\` or \`npm audit fix --force\` as
162+
a shortcut — they produce sprawling diffs with unrelated
163+
upgrades. Use \`npm install\` / \`npm update\` targeted at
164+
the specific package(s) you are fixing.
165+
166+
4. DIFF DISCIPLINE.
167+
After the lockfile is regenerated, run:
168+
git diff --stat origin/main
169+
If any of the following are true, STOP, revert with
170+
\`git checkout -- package.json package-lock.json\`, and
171+
try a narrower strategy from rule 3:
172+
- package.json changed a dependency that is NOT on the
173+
alert list (except for adding entries to \`overrides\`).
174+
- Any dependency bumped a MAJOR version that is not
175+
strictly required to reach the first patched version.
176+
177+
5. VERIFY.
178+
Run \`npm run build\` AND \`npm test\`. If either fails,
179+
do NOT push and do NOT open a PR. Print
138180
"BUILD_OR_TEST_FAILED" and exit non-zero.
139-
5. When build and tests pass:
140-
- \`git push -u origin ${BRANCH}\` (uses \$GH_TOKEN).
141-
- \`gh pr create --base main --head ${BRANCH} \\\\
142-
--title "chore(deps): auto-fix Dependabot alerts" \\\\
143-
--body "<summary of alerts addressed, per-package old→new versions, CVE IDs, and any overrides added>"\`.
144-
- Print the PR URL.
145-
6. Never manually edit integrity hashes in package-lock.json
181+
182+
6. OPEN THE PR (verification passed).
183+
- \`git push -u origin ${BRANCH}\` (uses \$GH_TOKEN).
184+
- Build the PR title and body strictly from \`alerts.json\`:
185+
* Title: \`chore(deps): fix N open Dependabot alert(s)\`
186+
where N = number of distinct entries in \`alerts.json\`.
187+
* Body format:
188+
## Summary
189+
Fixes N open Dependabot alert(s) as enumerated by
190+
\`gh api repos/${REPO}/dependabot/alerts?state=open\`.
191+
## Alerts addressed
192+
For each entry in \`alerts.json\`, one bullet:
193+
- \`<pkg>\` (#<number>, <severity>, <ghsa>) —
194+
vulnerable \`<vulnerable>\` → patched \`<patched>\`.
195+
Strategy: override | direct-bump | parent-bump.
196+
Rationale if parent-bump: <one line>.
197+
## Collateral changes
198+
List any other dep bumps that \`npm install\` cascaded
199+
(e.g. peer updates) and explain why they were
200+
necessary. If there are none, say "None".
201+
## Verification
202+
- \`npm run build\`: PASS
203+
- \`npm test\`: PASS
204+
- Open the PR with \`gh pr create --base main --head ${BRANCH}\`
205+
using that title and body.
206+
- Print the PR URL.
207+
208+
7. COMMIT / AUTHORSHIP HYGIENE.
209+
Never manually edit integrity hashes in package-lock.json
146210
(see skill). Use \`npm install\`, \`npm update\`, or a clean
147-
reinstall to let npm compute them.
148-
7. Never use \`--allow-all\` semantics; stick to the tools you
211+
reinstall to let npm compute them. Do not add any
212+
\`Co-authored-by:\` trailer.
213+
214+
8. PERMISSIONS.
215+
Never use \`--allow-all\` semantics; stick to the tools you
149216
already have permission to use.
150217
151218
----- SKILL START -----

0 commit comments

Comments
 (0)