From 731d00c3f6157411f572911a99dc616775d39cb5 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Wed, 13 May 2026 15:03:01 -0600 Subject: [PATCH 01/17] Add independent peer review validation workflow --- .../validateIndependentPeerReview.yml | 51 ++ README.md | 7 + package-lock.json | 803 ++++++++++++++++++ package.json | 17 + scripts/validateIndependentPeerReview.ts | 202 +++++ 5 files changed, 1080 insertions(+) create mode 100644 .github/workflows/validateIndependentPeerReview.yml create mode 100644 package-lock.json create mode 100644 package.json create mode 100644 scripts/validateIndependentPeerReview.ts diff --git a/.github/workflows/validateIndependentPeerReview.yml b/.github/workflows/validateIndependentPeerReview.yml new file mode 100644 index 0000000..e962c05 --- /dev/null +++ b/.github/workflows/validateIndependentPeerReview.yml @@ -0,0 +1,51 @@ +# Note: This workflow is configured to run on all pull requests throughout the Expensify org, not just this repo. +# That has a few consequences: +# - We need to checkout the repo it's running on, and not just the GitHub-Actions repo +# - branch and path matching does not work in the workflow layer. From the docs: https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#supported-event-triggers +# > Any filters you specify for the supported events are ignored - for example, branches, branches-ignore, paths, types and so on. The workflow is only triggered, and is always triggered, by the default activity types of the supported events +name: Validate independent peer review + +on: pull_request + +permissions: + contents: read + pull-requests: read + +jobs: + validateIndependentPeerReview: + runs-on: blacksmith-2vcpu-ubuntu-2404 + steps: + # v3.1.1 + - name: Generate a GitHub App token + id: generateAppToken + uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 + with: + app-id: ${{ secrets.OS_BOTIFY_APP_ID }} + private-key: ${{ secrets.OS_BOTIFY_PRIVATE_KEY }} + owner: ${{ github.repository_owner }} + repositories: | + ${{ github.event.repository.name }} + GitHub-Actions + permission-administration: read + permission-contents: read + permission-members: read + permission-metadata: read + permission-pull-requests: read + + - name: Checkout repos + id: repo + uses: Expensify/GitHub-Actions/checkoutRepoAndGitHubActions@main + + # v4.3.0 + - name: Setup Node + uses: actions/setup-node@cdca7365b2dadb8aad0a33bc7601856ffabcc48e + + - name: Install npm packages + run: npm ci + working-directory: GitHub-Actions + + - name: Validate independent peer review + run: npm run validate-independent-peer-review + working-directory: GitHub-Actions + env: + GITHUB_TOKEN: ${{ steps.generateAppToken.outputs.token }} diff --git a/README.md b/README.md index fd85a3d..fd1491b 100644 --- a/README.md +++ b/README.md @@ -37,6 +37,12 @@ jobs: secrets: inherit ``` +### `validateIndependentPeerReview.yml` + +Used as an org-level ruleset workflow to block pull requests that do not have enough independent Expensify employee approvals. The check only reads GitHub pull request metadata; it does not checkout or execute code from the pull request branch. + +This workflow requires a GitHub App token with read access for repository metadata, pull requests, branch protection administration, and organization members. It uses `OS_BOTIFY_APP_ID` and `OS_BOTIFY_PRIVATE_KEY` to generate that token. If GitHub does not return a branch-protection review count, the workflow defaults to requiring one independent approval, so the ruleset should target only the intended protected branches. + ## Rulesets GitHub [org-level rulesets](https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#require-workflows-to-pass-before-merging) can be configured to run a workflow check against pull requests in all repos in the org. This is a very powerful feature, but there are some caveats and best practices to be aware of when enabling a ruleset. @@ -47,3 +53,4 @@ GitHub [org-level rulesets](https://docs.github.com/en/enterprise-cloud@latest/r - If you need to target or exclude specific paths, that must be implemented manually in the workflow itself. - Due to a GitHub :bug:, PRs that are open when the rule is enabled will get stuck with a pending check that will never get picked up. The easiest way to fix that is to close and reopen the PR. Consider writing a script to close and reopen all open PRs across the org after the check is enabled. - It is less disruptive to [configure the ruleset to `Evaluate` first](https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#using-evaluate-mode-for-ruleset-workflows), then `Active` once the kinks are worked out. +- For `validateIndependentPeerReview.yml`, start with a ruleset targeting only a test branch, then test the workflow from a GitHub-Actions branch, then from `main`, and only then enable it for the intended repositories and branches. diff --git a/package-lock.json b/package-lock.json new file mode 100644 index 0000000..95df49d --- /dev/null +++ b/package-lock.json @@ -0,0 +1,803 @@ +{ + "name": "@expensify/github-actions", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "@expensify/github-actions", + "dependencies": { + "@octokit/request-error": "^7.1.0", + "@octokit/rest": "^22.0.1" + }, + "devDependencies": { + "@octokit/webhooks-types": "^7.6.1", + "@types/node": "^25.7.0", + "tsx": "^4.21.0", + "typescript": "^6.0.3" + } + }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.27.7.tgz", + "integrity": "sha512-EKX3Qwmhz1eMdEJokhALr0YiD0lhQNwDqkPYyPhiSwKrh7/4KRjQc04sZ8db+5DVVnZ1LmbNDI1uAMPEUBnQPg==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.27.7.tgz", + "integrity": "sha512-jbPXvB4Yj2yBV7HUfE2KHe4GJX51QplCN1pGbYjvsyCZbQmies29EoJbkEc+vYuU5o45AfQn37vZlyXy4YJ8RQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.27.7.tgz", + "integrity": "sha512-62dPZHpIXzvChfvfLJow3q5dDtiNMkwiRzPylSCfriLvZeq0a1bWChrGx/BbUbPwOrsWKMn8idSllklzBy+dgQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/android-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.27.7.tgz", + "integrity": "sha512-x5VpMODneVDb70PYV2VQOmIUUiBtY3D3mPBG8NxVk5CogneYhkR7MmM3yR/uMdITLrC1ml/NV1rj4bMJuy9MCg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.27.7.tgz", + "integrity": "sha512-5lckdqeuBPlKUwvoCXIgI2D9/ABmPq3Rdp7IfL70393YgaASt7tbju3Ac+ePVi3KDH6N2RqePfHnXkaDtY9fkw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/darwin-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.27.7.tgz", + "integrity": "sha512-rYnXrKcXuT7Z+WL5K980jVFdvVKhCHhUwid+dDYQpH+qu+TefcomiMAJpIiC2EM3Rjtq0sO3StMV/+3w3MyyqQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.27.7.tgz", + "integrity": "sha512-B48PqeCsEgOtzME2GbNM2roU29AMTuOIN91dsMO30t+Ydis3z/3Ngoj5hhnsOSSwNzS+6JppqWsuhTp6E82l2w==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/freebsd-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.27.7.tgz", + "integrity": "sha512-jOBDK5XEjA4m5IJK3bpAQF9/Lelu/Z9ZcdhTRLf4cajlB+8VEhFFRjWgfy3M1O4rO2GQ/b2dLwCUGpiF/eATNQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.27.7.tgz", + "integrity": "sha512-RkT/YXYBTSULo3+af8Ib0ykH8u2MBh57o7q/DAs3lTJlyVQkgQvlrPTnjIzzRPQyavxtPtfg0EopvDyIt0j1rA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.27.7.tgz", + "integrity": "sha512-RZPHBoxXuNnPQO9rvjh5jdkRmVizktkT7TCDkDmQ0W2SwHInKCAV95GRuvdSvA7w4VMwfCjUiPwDi0ZO6Nfe9A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ia32": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.27.7.tgz", + "integrity": "sha512-GA48aKNkyQDbd3KtkplYWT102C5sn/EZTY4XROkxONgruHPU72l+gW+FfF8tf2cFjeHaRbWpOYa/uRBz/Xq1Pg==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-loong64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.27.7.tgz", + "integrity": "sha512-a4POruNM2oWsD4WKvBSEKGIiWQF8fZOAsycHOt6JBpZ+JN2n2JH9WAv56SOyu9X5IqAjqSIPTaJkqN8F7XOQ5Q==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-mips64el": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.27.7.tgz", + "integrity": "sha512-KabT5I6StirGfIz0FMgl1I+R1H73Gp0ofL9A3nG3i/cYFJzKHhouBV5VWK1CSgKvVaG4q1RNpCTR2LuTVB3fIw==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-ppc64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.27.7.tgz", + "integrity": "sha512-gRsL4x6wsGHGRqhtI+ifpN/vpOFTQtnbsupUF5R5YTAg+y/lKelYR1hXbnBdzDjGbMYjVJLJTd2OFmMewAgwlQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-riscv64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.27.7.tgz", + "integrity": "sha512-hL25LbxO1QOngGzu2U5xeXtxXcW+/GvMN3ejANqXkxZ/opySAZMrc+9LY/WyjAan41unrR3YrmtTsUpwT66InQ==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-s390x": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.27.7.tgz", + "integrity": "sha512-2k8go8Ycu1Kb46vEelhu1vqEP+UeRVj2zY1pSuPdgvbd5ykAw82Lrro28vXUrRmzEsUV0NzCf54yARIK8r0fdw==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/linux-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.27.7.tgz", + "integrity": "sha512-hzznmADPt+OmsYzw1EE33ccA+HPdIqiCRq7cQeL1Jlq2gb1+OyWBkMCrYGBJ+sxVzve2ZJEVeePbLM2iEIZSxA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.27.7.tgz", + "integrity": "sha512-b6pqtrQdigZBwZxAn1UpazEisvwaIDvdbMbmrly7cDTMFnw/+3lVxxCTGOrkPVnsYIosJJXAsILG9XcQS+Yu6w==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/netbsd-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.27.7.tgz", + "integrity": "sha512-OfatkLojr6U+WN5EDYuoQhtM+1xco+/6FSzJJnuWiUw5eVcicbyK3dq5EeV/QHT1uy6GoDhGbFpprUiHUYggrw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.27.7.tgz", + "integrity": "sha512-AFuojMQTxAz75Fo8idVcqoQWEHIXFRbOc1TrVcFSgCZtQfSdc1RXgB3tjOn/krRHENUB4j00bfGjyl2mJrU37A==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openbsd-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.27.7.tgz", + "integrity": "sha512-+A1NJmfM8WNDv5CLVQYJ5PshuRm/4cI6WMZRg1by1GwPIQPCTs1GLEUHwiiQGT5zDdyLiRM/l1G0Pv54gvtKIg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.27.7.tgz", + "integrity": "sha512-+KrvYb/C8zA9CU/g0sR6w2RBw7IGc5J2BPnc3dYc5VJxHCSF1yNMxTV5LQ7GuKteQXZtspjFbiuW5/dOj7H4Yw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.27.7.tgz", + "integrity": "sha512-ikktIhFBzQNt/QDyOL580ti9+5mL/YZeUPKU2ivGtGjdTYoqz6jObj6nOMfhASpS4GU4Q/Clh1QtxWAvcYKamA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.27.7.tgz", + "integrity": "sha512-7yRhbHvPqSpRUV7Q20VuDwbjW5kIMwTHpptuUzV+AA46kiPze5Z7qgt6CLCK3pWFrHeNfDd1VKgyP4O+ng17CA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.27.7.tgz", + "integrity": "sha512-SmwKXe6VHIyZYbBLJrhOoCJRB/Z1tckzmgTLfFYOfpMAx63BJEaL9ExI8x7v0oAO3Zh6D/Oi1gVxEYr5oUCFhw==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.27.7.tgz", + "integrity": "sha512-56hiAJPhwQ1R4i+21FVF7V8kSD5zZTdHcVuRFMW0hn753vVfQN8xlx4uOPT4xoGH0Z/oVATuR82AiqSTDIpaHg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@octokit/auth-token": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/@octokit/auth-token/-/auth-token-6.0.0.tgz", + "integrity": "sha512-P4YJBPdPSpWTQ1NU4XYdvHvXJJDxM6YwpS0FZHRgP7YFkdVxsWcpWGy/NVqlAA7PcPCnMacXlRm1y2PFZRWL/w==", + "license": "MIT", + "engines": { + "node": ">= 20" + } + }, + "node_modules/@octokit/core": { + "version": "7.0.6", + "resolved": "https://registry.npmjs.org/@octokit/core/-/core-7.0.6.tgz", + "integrity": "sha512-DhGl4xMVFGVIyMwswXeyzdL4uXD5OGILGX5N8Y+f6W7LhC1Ze2poSNrkF/fedpVDHEEZ+PHFW0vL14I+mm8K3Q==", + "license": "MIT", + "dependencies": { + "@octokit/auth-token": "^6.0.0", + "@octokit/graphql": "^9.0.3", + "@octokit/request": "^10.0.6", + "@octokit/request-error": "^7.0.2", + "@octokit/types": "^16.0.0", + "before-after-hook": "^4.0.0", + "universal-user-agent": "^7.0.0" + }, + "engines": { + "node": ">= 20" + } + }, + "node_modules/@octokit/endpoint": { + "version": "11.0.3", + "resolved": "https://registry.npmjs.org/@octokit/endpoint/-/endpoint-11.0.3.tgz", + "integrity": "sha512-FWFlNxghg4HrXkD3ifYbS/IdL/mDHjh9QcsNyhQjN8dplUoZbejsdpmuqdA76nxj2xoWPs7p8uX2SNr9rYu0Ag==", + "license": "MIT", + "dependencies": { + "@octokit/types": "^16.0.0", + "universal-user-agent": "^7.0.2" + }, + "engines": { + "node": ">= 20" + } + }, + "node_modules/@octokit/graphql": { + "version": "9.0.3", + "resolved": "https://registry.npmjs.org/@octokit/graphql/-/graphql-9.0.3.tgz", + "integrity": "sha512-grAEuupr/C1rALFnXTv6ZQhFuL1D8G5y8CN04RgrO4FIPMrtm+mcZzFG7dcBm+nq+1ppNixu+Jd78aeJOYxlGA==", + "license": "MIT", + "dependencies": { + "@octokit/request": "^10.0.6", + "@octokit/types": "^16.0.0", + "universal-user-agent": "^7.0.0" + }, + "engines": { + "node": ">= 20" + } + }, + "node_modules/@octokit/openapi-types": { + "version": "27.0.0", + "resolved": "https://registry.npmjs.org/@octokit/openapi-types/-/openapi-types-27.0.0.tgz", + "integrity": "sha512-whrdktVs1h6gtR+09+QsNk2+FO+49j6ga1c55YZudfEG+oKJVvJLQi3zkOm5JjiUXAagWK2tI2kTGKJ2Ys7MGA==", + "license": "MIT" + }, + "node_modules/@octokit/plugin-paginate-rest": { + "version": "14.0.0", + "resolved": "https://registry.npmjs.org/@octokit/plugin-paginate-rest/-/plugin-paginate-rest-14.0.0.tgz", + "integrity": "sha512-fNVRE7ufJiAA3XUrha2omTA39M6IXIc6GIZLvlbsm8QOQCYvpq/LkMNGyFlB1d8hTDzsAXa3OKtybdMAYsV/fw==", + "license": "MIT", + "dependencies": { + "@octokit/types": "^16.0.0" + }, + "engines": { + "node": ">= 20" + }, + "peerDependencies": { + "@octokit/core": ">=6" + } + }, + "node_modules/@octokit/plugin-request-log": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/@octokit/plugin-request-log/-/plugin-request-log-6.0.0.tgz", + "integrity": "sha512-UkOzeEN3W91/eBq9sPZNQ7sUBvYCqYbrrD8gTbBuGtHEuycE4/awMXcYvx6sVYo7LypPhmQwwpUe4Yyu4QZN5Q==", + "license": "MIT", + "engines": { + "node": ">= 20" + }, + "peerDependencies": { + "@octokit/core": ">=6" + } + }, + "node_modules/@octokit/plugin-rest-endpoint-methods": { + "version": "17.0.0", + "resolved": "https://registry.npmjs.org/@octokit/plugin-rest-endpoint-methods/-/plugin-rest-endpoint-methods-17.0.0.tgz", + "integrity": "sha512-B5yCyIlOJFPqUUeiD0cnBJwWJO8lkJs5d8+ze9QDP6SvfiXSz1BF+91+0MeI1d2yxgOhU/O+CvtiZ9jSkHhFAw==", + "license": "MIT", + "dependencies": { + "@octokit/types": "^16.0.0" + }, + "engines": { + "node": ">= 20" + }, + "peerDependencies": { + "@octokit/core": ">=6" + } + }, + "node_modules/@octokit/request": { + "version": "10.0.9", + "resolved": "https://registry.npmjs.org/@octokit/request/-/request-10.0.9.tgz", + "integrity": "sha512-o8Bi3f608eyM+7BmBiUWxFsdjLb3/ym1cQek5LZOv9KkZcxRrHCPhhRzm6xjO6HVZ85ItD6+sTsjxo821SVa/A==", + "license": "MIT", + "dependencies": { + "@octokit/endpoint": "^11.0.3", + "@octokit/request-error": "^7.0.2", + "@octokit/types": "^16.0.0", + "content-type": "^2.0.0", + "fast-content-type-parse": "^3.0.0", + "json-with-bigint": "^3.5.3", + "universal-user-agent": "^7.0.2" + }, + "engines": { + "node": ">= 20" + } + }, + "node_modules/@octokit/request-error": { + "version": "7.1.0", + "resolved": "https://registry.npmjs.org/@octokit/request-error/-/request-error-7.1.0.tgz", + "integrity": "sha512-KMQIfq5sOPpkQYajXHwnhjCC0slzCNScLHs9JafXc4RAJI+9f+jNDlBNaIMTvazOPLgb4BnlhGJOTbnN0wIjPw==", + "license": "MIT", + "dependencies": { + "@octokit/types": "^16.0.0" + }, + "engines": { + "node": ">= 20" + } + }, + "node_modules/@octokit/rest": { + "version": "22.0.1", + "resolved": "https://registry.npmjs.org/@octokit/rest/-/rest-22.0.1.tgz", + "integrity": "sha512-Jzbhzl3CEexhnivb1iQ0KJ7s5vvjMWcmRtq5aUsKmKDrRW6z3r84ngmiFKFvpZjpiU/9/S6ITPFRpn5s/3uQJw==", + "license": "MIT", + "dependencies": { + "@octokit/core": "^7.0.6", + "@octokit/plugin-paginate-rest": "^14.0.0", + "@octokit/plugin-request-log": "^6.0.0", + "@octokit/plugin-rest-endpoint-methods": "^17.0.0" + }, + "engines": { + "node": ">= 20" + } + }, + "node_modules/@octokit/types": { + "version": "16.0.0", + "resolved": "https://registry.npmjs.org/@octokit/types/-/types-16.0.0.tgz", + "integrity": "sha512-sKq+9r1Mm4efXW1FCk7hFSeJo4QKreL/tTbR0rz/qx/r1Oa2VV83LTA/H/MuCOX7uCIJmQVRKBcbmWoySjAnSg==", + "license": "MIT", + "dependencies": { + "@octokit/openapi-types": "^27.0.0" + } + }, + "node_modules/@octokit/webhooks-types": { + "version": "7.6.1", + "resolved": "https://registry.npmjs.org/@octokit/webhooks-types/-/webhooks-types-7.6.1.tgz", + "integrity": "sha512-S8u2cJzklBC0FgTwWVLaM8tMrDuDMVE4xiTK4EYXM9GntyvrdbSoxqDQa+Fh57CCNApyIpyeqPhhFEmHPfrXgw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/node": { + "version": "25.7.0", + "resolved": "https://registry.npmjs.org/@types/node/-/node-25.7.0.tgz", + "integrity": "sha512-z+pdZyxE+RTQE9AcboAZCb4otwcrvgHD+GlBpPgn0emDVt0ohrTMhAwlr2Wd9nZ+nihhYFxO2pThz3C5qSu2Eg==", + "dev": true, + "license": "MIT", + "dependencies": { + "undici-types": "~7.21.0" + } + }, + "node_modules/before-after-hook": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/before-after-hook/-/before-after-hook-4.0.0.tgz", + "integrity": "sha512-q6tR3RPqIB1pMiTRMFcZwuG5T8vwp+vUvEG0vuI6B+Rikh5BfPp2fQ82c925FOs+b0lcFQ8CFrL+KbilfZFhOQ==", + "license": "Apache-2.0" + }, + "node_modules/content-type": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz", + "integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==", + "license": "MIT", + "engines": { + "node": ">=18" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/express" + } + }, + "node_modules/esbuild": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.7.tgz", + "integrity": "sha512-IxpibTjyVnmrIQo5aqNpCgoACA/dTKLTlhMHihVHhdkxKyPO1uBBthumT0rdHmcsk9uMonIWS0m4FljWzILh3w==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.27.7", + "@esbuild/android-arm": "0.27.7", + "@esbuild/android-arm64": "0.27.7", + "@esbuild/android-x64": "0.27.7", + "@esbuild/darwin-arm64": "0.27.7", + "@esbuild/darwin-x64": "0.27.7", + "@esbuild/freebsd-arm64": "0.27.7", + "@esbuild/freebsd-x64": "0.27.7", + "@esbuild/linux-arm": "0.27.7", + "@esbuild/linux-arm64": "0.27.7", + "@esbuild/linux-ia32": "0.27.7", + "@esbuild/linux-loong64": "0.27.7", + "@esbuild/linux-mips64el": "0.27.7", + "@esbuild/linux-ppc64": "0.27.7", + "@esbuild/linux-riscv64": "0.27.7", + "@esbuild/linux-s390x": "0.27.7", + "@esbuild/linux-x64": "0.27.7", + "@esbuild/netbsd-arm64": "0.27.7", + "@esbuild/netbsd-x64": "0.27.7", + "@esbuild/openbsd-arm64": "0.27.7", + "@esbuild/openbsd-x64": "0.27.7", + "@esbuild/openharmony-arm64": "0.27.7", + "@esbuild/sunos-x64": "0.27.7", + "@esbuild/win32-arm64": "0.27.7", + "@esbuild/win32-ia32": "0.27.7", + "@esbuild/win32-x64": "0.27.7" + } + }, + "node_modules/fast-content-type-parse": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/fast-content-type-parse/-/fast-content-type-parse-3.0.0.tgz", + "integrity": "sha512-ZvLdcY8P+N8mGQJahJV5G4U88CSvT1rP8ApL6uETe88MBXrBHAkZlSEySdUlyztF7ccb+Znos3TFqaepHxdhBg==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/fastify" + }, + { + "type": "opencollective", + "url": "https://opencollective.com/fastify" + } + ], + "license": "MIT" + }, + "node_modules/fsevents": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz", + "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/get-tsconfig": { + "version": "4.14.0", + "resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.14.0.tgz", + "integrity": "sha512-yTb+8DXzDREzgvYmh6s9vHsSVCHeC0G3PI5bEXNBHtmshPnO+S5O7qgLEOn0I5QvMy6kpZN8K1NKGyilLb93wA==", + "dev": true, + "license": "MIT", + "dependencies": { + "resolve-pkg-maps": "^1.0.0" + }, + "funding": { + "url": "https://github.com/privatenumber/get-tsconfig?sponsor=1" + } + }, + "node_modules/json-with-bigint": { + "version": "3.5.8", + "resolved": "https://registry.npmjs.org/json-with-bigint/-/json-with-bigint-3.5.8.tgz", + "integrity": "sha512-eq/4KP6K34kwa7TcFdtvnftvHCD9KvHOGGICWwMFc4dOOKF5t4iYqnfLK8otCRCRv06FXOzGGyqE8h8ElMvvdw==", + "license": "MIT" + }, + "node_modules/resolve-pkg-maps": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/resolve-pkg-maps/-/resolve-pkg-maps-1.0.0.tgz", + "integrity": "sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/privatenumber/resolve-pkg-maps?sponsor=1" + } + }, + "node_modules/tsx": { + "version": "4.21.0", + "resolved": "https://registry.npmjs.org/tsx/-/tsx-4.21.0.tgz", + "integrity": "sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==", + "dev": true, + "license": "MIT", + "dependencies": { + "esbuild": "~0.27.0", + "get-tsconfig": "^4.7.5" + }, + "bin": { + "tsx": "dist/cli.mjs" + }, + "engines": { + "node": ">=18.0.0" + }, + "optionalDependencies": { + "fsevents": "~2.3.3" + } + }, + "node_modules/typescript": { + "version": "6.0.3", + "resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.3.tgz", + "integrity": "sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw==", + "dev": true, + "license": "Apache-2.0", + "bin": { + "tsc": "bin/tsc", + "tsserver": "bin/tsserver" + }, + "engines": { + "node": ">=14.17" + } + }, + "node_modules/undici-types": { + "version": "7.21.0", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.21.0.tgz", + "integrity": "sha512-w9IMgQrz4O0YN1LtB7K5P63vhlIOvC7opSmouCJ+ZywlPAlO9gIkJ+otk6LvGpAs2wg4econaCz3TvQ9xPoyuQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/universal-user-agent": { + "version": "7.0.3", + "resolved": "https://registry.npmjs.org/universal-user-agent/-/universal-user-agent-7.0.3.tgz", + "integrity": "sha512-TmnEAEAsBJVZM/AADELsK76llnwcf9vMKuPz8JflO1frO8Lchitr0fNaN9d+Ap0BjKtqWqd/J17qeDnXh8CL2A==", + "license": "ISC" + } + } +} diff --git a/package.json b/package.json new file mode 100644 index 0000000..5331a3f --- /dev/null +++ b/package.json @@ -0,0 +1,17 @@ +{ + "name": "@expensify/github-actions", + "private": true, + "scripts": { + "validate-independent-peer-review": "tsx scripts/validateIndependentPeerReview.ts" + }, + "devDependencies": { + "@octokit/webhooks-types": "^7.6.1", + "@types/node": "^25.7.0", + "tsx": "^4.21.0", + "typescript": "^6.0.3" + }, + "dependencies": { + "@octokit/request-error": "^7.1.0", + "@octokit/rest": "^22.0.1" + } +} diff --git a/scripts/validateIndependentPeerReview.ts b/scripts/validateIndependentPeerReview.ts new file mode 100644 index 0000000..ec90da2 --- /dev/null +++ b/scripts/validateIndependentPeerReview.ts @@ -0,0 +1,202 @@ +import {readFileSync} from 'node:fs'; +import {RequestError} from '@octokit/request-error'; +import {Octokit, type RestEndpointMethodTypes} from '@octokit/rest'; +import type {PullRequestEvent} from '@octokit/webhooks-types'; + +type Commit = RestEndpointMethodTypes['pulls']['listCommits']['response']['data'][number]; +type Review = RestEndpointMethodTypes['pulls']['listReviews']['response']['data'][number]; + +type PullRequestContext = { + owner: string; + repo: string; + number: number; + baseRef: string; +}; + +const botUsers = new Set(['botify', 'MelvinBot', 'exfy-zapier']); +const defaultRequiredApprovingReviewCount = 1; +const githubToken = process.env.GITHUB_TOKEN ?? process.env.GH_TOKEN; +if (!githubToken) { + throw new Error('GITHUB_TOKEN or GH_TOKEN is required'); +} + +const octokit = new Octokit({ + auth: githubToken, +}); + +function formatUsers(users: string[]): string { + return users.length > 0 ? users.join(', ') : '(none)'; +} + +function unique(values: string[]): string[] { + return [...new Set(values)].sort((a, b) => a.localeCompare(b)); +} + +function getPullRequestContext(): PullRequestContext { + const eventPath = process.env.GITHUB_EVENT_PATH; + if (!eventPath) { + throw new Error('GITHUB_EVENT_PATH is required'); + } + + const event = JSON.parse(readFileSync(eventPath, 'utf8')) as PullRequestEvent; + return { + owner: event.repository.owner.login, + repo: event.repository.name, + number: event.pull_request.number, + baseRef: event.pull_request.base.ref, + }; +} + +async function getRequiredApprovingReviewCount({owner, repo, baseRef}: PullRequestContext): Promise { + try { + const {data} = await octokit.rest.repos.getPullRequestReviewProtection({ + owner, + repo, + branch: baseRef, + }); + return data.required_approving_review_count ?? 0; + } catch (error: unknown) { + if (error instanceof RequestError && error.status === 404) { + console.log(`${owner}/${repo}@${baseRef} did not return a branch protection review count; requiring ${defaultRequiredApprovingReviewCount} independent approval(s).`); + return defaultRequiredApprovingReviewCount; + } + throw error; + } +} + +function getLatestApprovers(reviews: Review[]): string[] { + const latestOpinionatedReviewByUser = new Map(); + const opinionatedStates = new Set(['APPROVED', 'CHANGES_REQUESTED', 'DISMISSED']); + + for (const review of reviews) { + const login = review.user?.login; + if (login && opinionatedStates.has(review.state)) { + latestOpinionatedReviewByUser.set(login, review.state); + } + } + + return unique([...latestOpinionatedReviewByUser.entries()] + .filter(([, state]) => state === 'APPROVED') + .map(([login]) => login)); +} + +function coAuthorEmails(message: string): string[] { + return [...message.matchAll(/^Co-authored-by:\s+.+<(.+)>$/gim)].map((match) => match[1].trim()); +} + +function resolveCoAuthorLogin(email: string): string | null { + return email.match(/^(?:\d+\+)?(.+)@users\.noreply\.github\.com$/i)?.[1] ?? null; +} + +function getCommitAuthors(commits: Commit[]): {authors: string[]; unresolvedExpensifyCoAuthors: string[]} { + const authors = new Set(); + const unresolvedExpensifyCoAuthors = new Set(); + + for (const commit of commits) { + const canonicalAuthor = commit.author?.login ?? ''; + if (canonicalAuthor) { + authors.add(canonicalAuthor); + } + if (canonicalAuthor && !botUsers.has(canonicalAuthor)) { + continue; + } + + for (const email of coAuthorEmails(commit.commit.message)) { + const login = resolveCoAuthorLogin(email); + if (login) { + authors.add(login); + } else if (email.trim().toLowerCase().endsWith('@expensify.com')) { + unresolvedExpensifyCoAuthors.add(email.trim()); + } + } + } + + return { + authors: unique([...authors]), + unresolvedExpensifyCoAuthors: unique([...unresolvedExpensifyCoAuthors]), + }; +} + +async function isEmployee(username: string): Promise { + try { + await octokit.rest.orgs.checkMembershipForUser({ + org: 'Expensify', + username, + }); + return true; + } catch (error: unknown) { + if (error instanceof RequestError && error.status === 404) { + return false; + } + throw error; + } +} + +async function isRepoWriter({owner, repo}: PullRequestContext, username: string): Promise { + const {data} = await octokit.rest.repos.getCollaboratorPermissionLevel({ + owner, + repo, + username, + }); + return ['admin', 'maintain', 'write'].includes(data.permission); +} + +async function getIndependentEmployeeApprovers(context: PullRequestContext, approvers: string[], authors: string[]): Promise { + const authorSet = new Set(authors); + const independentEmployeeApprovers: string[] = []; + for (const approver of approvers) { + if (!authorSet.has(approver) && await isEmployee(approver) && await isRepoWriter(context, approver)) { + independentEmployeeApprovers.push(approver); + } + } + return independentEmployeeApprovers; +} + +async function main(): Promise { + const context = getPullRequestContext(); + const {owner, repo, number} = context; + const pullRequestParams = { + owner, + repo, + pull_number: number, + per_page: 100, + }; + const [requiredApprovingReviewCount, reviews, commits] = await Promise.all([ + getRequiredApprovingReviewCount(context), + octokit.paginate(octokit.rest.pulls.listReviews, pullRequestParams), + octokit.paginate(octokit.rest.pulls.listCommits, pullRequestParams), + ]); + const approvers = getLatestApprovers(reviews); + + if (requiredApprovingReviewCount === 0) { + console.log(`${owner}/${repo}#${number} targets ${context.baseRef}, which does not require approving reviews.`); + return; + } + + const {authors, unresolvedExpensifyCoAuthors} = getCommitAuthors(commits); + if (unresolvedExpensifyCoAuthors.length > 0) { + throw new Error(`Unable to resolve Expensify co-author emails to GitHub users: ${formatUsers(unresolvedExpensifyCoAuthors)}`); + } + if (authors.every((author) => botUsers.has(author))) { + throw new Error(`Unable to verify independent peer review because ${owner}/${repo}#${number} has no human commit authors or co-authors.`); + } + + const independentEmployeeApprovers = await getIndependentEmployeeApprovers(context, approvers, authors); + if (independentEmployeeApprovers.length < requiredApprovingReviewCount) { + throw new Error([ + `${owner}/${repo}#${number} does not have enough independent Expensify employee approvals.`, + `Required independent approvals: ${requiredApprovingReviewCount}`, + `Commit authors/co-authors: ${formatUsers(authors)}`, + `Approvers: ${formatUsers(approvers)}`, + `Independent employee approvers: ${formatUsers(independentEmployeeApprovers)}`, + ].join('\n')); + } + + console.log(`${owner}/${repo}#${number} has ${independentEmployeeApprovers.length} independent Expensify employee approval(s).`); +} + +main().catch((error: unknown) => { + const message = error instanceof Error ? error.message : String(error); + console.error(`::error::${message.replace(/%/g, '%25').replace(/\r/g, '%0D').replace(/\n/g, '%0A')}`); + process.exit(1); +}); From aba7b19a50b52458f29cd40fd8836871d8aa09e4 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Wed, 13 May 2026 15:57:20 -0600 Subject: [PATCH 02/17] Use GitHub App client ID for peer review check --- .github/workflows/validateIndependentPeerReview.yml | 2 +- README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/validateIndependentPeerReview.yml b/.github/workflows/validateIndependentPeerReview.yml index e962c05..ff3aaef 100644 --- a/.github/workflows/validateIndependentPeerReview.yml +++ b/.github/workflows/validateIndependentPeerReview.yml @@ -20,7 +20,7 @@ jobs: id: generateAppToken uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 with: - app-id: ${{ secrets.OS_BOTIFY_APP_ID }} + client-id: ${{ secrets.OS_BOTIFY_CLIENT_ID }} private-key: ${{ secrets.OS_BOTIFY_PRIVATE_KEY }} owner: ${{ github.repository_owner }} repositories: | diff --git a/README.md b/README.md index fd1491b..f630196 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ jobs: Used as an org-level ruleset workflow to block pull requests that do not have enough independent Expensify employee approvals. The check only reads GitHub pull request metadata; it does not checkout or execute code from the pull request branch. -This workflow requires a GitHub App token with read access for repository metadata, pull requests, branch protection administration, and organization members. It uses `OS_BOTIFY_APP_ID` and `OS_BOTIFY_PRIVATE_KEY` to generate that token. If GitHub does not return a branch-protection review count, the workflow defaults to requiring one independent approval, so the ruleset should target only the intended protected branches. +This workflow requires a GitHub App token with read access for repository metadata, pull requests, branch protection administration, and organization members. It uses `OS_BOTIFY_CLIENT_ID` and `OS_BOTIFY_PRIVATE_KEY` to generate that token. If GitHub does not return a branch-protection review count, the workflow defaults to requiring one independent approval, so the ruleset should target only the intended protected branches. ## Rulesets GitHub [org-level rulesets](https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#require-workflows-to-pass-before-merging) can be configured to run a workflow check against pull requests in all repos in the org. This is a very powerful feature, but there are some caveats and best practices to be aware of when enabling a ruleset. From 2d2a6c9324c7695ef2ee3866f9a2c87541da2afa Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 16:36:27 -0600 Subject: [PATCH 03/17] Use GraphQL for peer review validation --- package-lock.json | 2 +- package.json | 2 +- scripts/validateIndependentPeerReview.ts | 185 ++++++++++++++++------- 3 files changed, 133 insertions(+), 56 deletions(-) diff --git a/package-lock.json b/package-lock.json index 95df49d..814f29d 100644 --- a/package-lock.json +++ b/package-lock.json @@ -6,7 +6,7 @@ "": { "name": "@expensify/github-actions", "dependencies": { - "@octokit/request-error": "^7.1.0", + "@octokit/graphql": "^9.0.3", "@octokit/rest": "^22.0.1" }, "devDependencies": { diff --git a/package.json b/package.json index 5331a3f..c9fcd31 100644 --- a/package.json +++ b/package.json @@ -11,7 +11,7 @@ "typescript": "^6.0.3" }, "dependencies": { - "@octokit/request-error": "^7.1.0", + "@octokit/graphql": "^9.0.3", "@octokit/rest": "^22.0.1" } } diff --git a/scripts/validateIndependentPeerReview.ts b/scripts/validateIndependentPeerReview.ts index ec90da2..0e00179 100644 --- a/scripts/validateIndependentPeerReview.ts +++ b/scripts/validateIndependentPeerReview.ts @@ -1,10 +1,9 @@ import {readFileSync} from 'node:fs'; -import {RequestError} from '@octokit/request-error'; +import {graphql} from '@octokit/graphql'; import {Octokit, type RestEndpointMethodTypes} from '@octokit/rest'; import type {PullRequestEvent} from '@octokit/webhooks-types'; type Commit = RestEndpointMethodTypes['pulls']['listCommits']['response']['data'][number]; -type Review = RestEndpointMethodTypes['pulls']['listReviews']['response']['data'][number]; type PullRequestContext = { owner: string; @@ -13,8 +12,52 @@ type PullRequestContext = { baseRef: string; }; +type BranchProtectionResponse = { + repository: { + ref: { + branchProtectionRule: { + requiredApprovingReviewCount: number; + } | null; + } | null; + } | null; +}; + +type LatestOpinionatedReviewsResponse = { + repository: { + pullRequest: { + latestOpinionatedReviews: { + nodes: Array<{ + state: string; + author: { + login: string; + } | null; + }>; + }; + } | null; + } | null; +}; + +type TeamMembersResponse = { + organization: { + team: { + members: { + pageInfo: { + hasNextPage: boolean; + endCursor: string | null; + }; + nodes: Array<{ + login: string; + }>; + }; + } | null; + } | null; +}; +type TeamResponse = NonNullable['team']; + const botUsers = new Set(['botify', 'MelvinBot', 'exfy-zapier']); const defaultRequiredApprovingReviewCount = 1; +const expensifyOrganization = 'Expensify'; +const expensifyEmployeeTeamSlug = 'expensify-expensify'; const githubToken = process.env.GITHUB_TOKEN ?? process.env.GH_TOKEN; if (!githubToken) { throw new Error('GITHUB_TOKEN or GH_TOKEN is required'); @@ -23,6 +66,11 @@ if (!githubToken) { const octokit = new Octokit({ auth: githubToken, }); +const githubGraphql = graphql.defaults({ + headers: { + authorization: `token ${githubToken}`, + }, +}); function formatUsers(users: string[]): string { return users.length > 0 ? users.join(', ') : '(none)'; @@ -49,35 +97,55 @@ function getPullRequestContext(): PullRequestContext { async function getRequiredApprovingReviewCount({owner, repo, baseRef}: PullRequestContext): Promise { try { - const {data} = await octokit.rest.repos.getPullRequestReviewProtection({ + const response = await githubGraphql(` + query RequiredApprovingReviewCount($owner: String!, $repo: String!, $branchRef: String!) { + repository(owner: $owner, name: $repo) { + ref(qualifiedName: $branchRef) { + branchProtectionRule { + requiredApprovingReviewCount + } + } + } + } + `, { owner, repo, - branch: baseRef, + branchRef: `refs/heads/${baseRef}`, }); - return data.required_approving_review_count ?? 0; + return response.repository?.ref?.branchProtectionRule?.requiredApprovingReviewCount ?? 0; } catch (error: unknown) { - if (error instanceof RequestError && error.status === 404) { - console.log(`${owner}/${repo}@${baseRef} did not return a branch protection review count; requiring ${defaultRequiredApprovingReviewCount} independent approval(s).`); - return defaultRequiredApprovingReviewCount; - } - throw error; + const message = error instanceof Error ? error.message : String(error); + console.log(`${owner}/${repo}@${baseRef} did not return a branch protection review count (${message}); requiring ${defaultRequiredApprovingReviewCount} independent approval(s).`); + return defaultRequiredApprovingReviewCount; } } -function getLatestApprovers(reviews: Review[]): string[] { - const latestOpinionatedReviewByUser = new Map(); - const opinionatedStates = new Set(['APPROVED', 'CHANGES_REQUESTED', 'DISMISSED']); - - for (const review of reviews) { - const login = review.user?.login; - if (login && opinionatedStates.has(review.state)) { - latestOpinionatedReviewByUser.set(login, review.state); +async function getLatestApprovers({owner, repo, number}: PullRequestContext): Promise { + const response = await githubGraphql(` + query LatestOpinionatedReviews($owner: String!, $repo: String!, $prNumber: Int!) { + repository(owner: $owner, name: $repo) { + pullRequest(number: $prNumber) { + latestOpinionatedReviews(last: 100, writersOnly: true) { + nodes { + state + author { + login + } + } + } + } + } } - } + `, { + owner, + repo, + prNumber: number, + }); - return unique([...latestOpinionatedReviewByUser.entries()] - .filter(([, state]) => state === 'APPROVED') - .map(([login]) => login)); + return unique(response.repository?.pullRequest?.latestOpinionatedReviews.nodes + .filter((review) => review.state === 'APPROVED') + .map((review) => review.author?.login ?? '') + .filter((login) => login !== '') ?? []); } function coAuthorEmails(message: string): string[] { @@ -117,39 +185,48 @@ function getCommitAuthors(commits: Commit[]): {authors: string[]; unresolvedExpe }; } -async function isEmployee(username: string): Promise { - try { - await octokit.rest.orgs.checkMembershipForUser({ - org: 'Expensify', - username, +async function getEmployeeLogins(): Promise> { + const employeeLogins = new Set(); + let cursor: string | null = null; + do { + const response: TeamMembersResponse = await githubGraphql(` + query TeamMembers($organization: String!, $teamSlug: String!, $cursor: String) { + organization(login: $organization) { + team(slug: $teamSlug) { + members(first: 100, after: $cursor) { + pageInfo { + hasNextPage + endCursor + } + nodes { + login + } + } + } + } + } + `, { + organization: expensifyOrganization, + teamSlug: expensifyEmployeeTeamSlug, + cursor, }); - return true; - } catch (error: unknown) { - if (error instanceof RequestError && error.status === 404) { - return false; + const team: TeamResponse = response.organization?.team ?? null; + if (!team) { + throw new Error(`${expensifyOrganization}/${expensifyEmployeeTeamSlug} team could not be found.`); } - throw error; - } -} - -async function isRepoWriter({owner, repo}: PullRequestContext, username: string): Promise { - const {data} = await octokit.rest.repos.getCollaboratorPermissionLevel({ - owner, - repo, - username, - }); - return ['admin', 'maintain', 'write'].includes(data.permission); + for (const member of team.members.nodes) { + employeeLogins.add(member.login); + } + cursor = team.members.pageInfo.hasNextPage ? team.members.pageInfo.endCursor : null; + } while (cursor); + return employeeLogins; } -async function getIndependentEmployeeApprovers(context: PullRequestContext, approvers: string[], authors: string[]): Promise { +function getIndependentEmployeeApprovers(approvers: string[], authors: string[], employeeLogins: Set): string[] { const authorSet = new Set(authors); - const independentEmployeeApprovers: string[] = []; - for (const approver of approvers) { - if (!authorSet.has(approver) && await isEmployee(approver) && await isRepoWriter(context, approver)) { - independentEmployeeApprovers.push(approver); - } - } - return independentEmployeeApprovers; + return approvers.filter((approver) => { + return !authorSet.has(approver) && employeeLogins.has(approver); + }); } async function main(): Promise { @@ -161,12 +238,11 @@ async function main(): Promise { pull_number: number, per_page: 100, }; - const [requiredApprovingReviewCount, reviews, commits] = await Promise.all([ + const [requiredApprovingReviewCount, approvers, commits] = await Promise.all([ getRequiredApprovingReviewCount(context), - octokit.paginate(octokit.rest.pulls.listReviews, pullRequestParams), + getLatestApprovers(context), octokit.paginate(octokit.rest.pulls.listCommits, pullRequestParams), ]); - const approvers = getLatestApprovers(reviews); if (requiredApprovingReviewCount === 0) { console.log(`${owner}/${repo}#${number} targets ${context.baseRef}, which does not require approving reviews.`); @@ -181,7 +257,8 @@ async function main(): Promise { throw new Error(`Unable to verify independent peer review because ${owner}/${repo}#${number} has no human commit authors or co-authors.`); } - const independentEmployeeApprovers = await getIndependentEmployeeApprovers(context, approvers, authors); + const employeeLogins = await getEmployeeLogins(); + const independentEmployeeApprovers = getIndependentEmployeeApprovers(approvers, authors, employeeLogins); if (independentEmployeeApprovers.length < requiredApprovingReviewCount) { throw new Error([ `${owner}/${repo}#${number} does not have enough independent Expensify employee approvals.`, From 89bebf1686834247398e1f602922bdd019fe15a8 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 16:43:45 -0600 Subject: [PATCH 04/17] Simplify peer review validation helpers --- scripts/validateIndependentPeerReview.ts | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/scripts/validateIndependentPeerReview.ts b/scripts/validateIndependentPeerReview.ts index 0e00179..0644a59 100644 --- a/scripts/validateIndependentPeerReview.ts +++ b/scripts/validateIndependentPeerReview.ts @@ -52,7 +52,6 @@ type TeamMembersResponse = { } | null; } | null; }; -type TeamResponse = NonNullable['team']; const botUsers = new Set(['botify', 'MelvinBot', 'exfy-zapier']); const defaultRequiredApprovingReviewCount = 1; @@ -210,23 +209,21 @@ async function getEmployeeLogins(): Promise> { teamSlug: expensifyEmployeeTeamSlug, cursor, }); - const team: TeamResponse = response.organization?.team ?? null; - if (!team) { + const members = response.organization?.team?.members; + if (!members) { throw new Error(`${expensifyOrganization}/${expensifyEmployeeTeamSlug} team could not be found.`); } - for (const member of team.members.nodes) { + for (const member of members.nodes) { employeeLogins.add(member.login); } - cursor = team.members.pageInfo.hasNextPage ? team.members.pageInfo.endCursor : null; + cursor = members.pageInfo.hasNextPage ? members.pageInfo.endCursor : null; } while (cursor); return employeeLogins; } function getIndependentEmployeeApprovers(approvers: string[], authors: string[], employeeLogins: Set): string[] { const authorSet = new Set(authors); - return approvers.filter((approver) => { - return !authorSet.has(approver) && employeeLogins.has(approver); - }); + return approvers.filter((approver) => !authorSet.has(approver) && employeeLogins.has(approver)); } async function main(): Promise { From e72fba83d00ce78061e8c6da6fa9d82fdde9dafa Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 16:45:30 -0600 Subject: [PATCH 05/17] Add newline --- scripts/validateIndependentPeerReview.ts | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/validateIndependentPeerReview.ts b/scripts/validateIndependentPeerReview.ts index 0644a59..462818c 100644 --- a/scripts/validateIndependentPeerReview.ts +++ b/scripts/validateIndependentPeerReview.ts @@ -65,6 +65,7 @@ if (!githubToken) { const octokit = new Octokit({ auth: githubToken, }); + const githubGraphql = graphql.defaults({ headers: { authorization: `token ${githubToken}`, From f473b3b040e1b6af1d28a9858c58d71f114469d7 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 16:49:14 -0600 Subject: [PATCH 06/17] Hard-code OS Botify app ID --- .github/workflows/validateIndependentPeerReview.yml | 2 +- README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/validateIndependentPeerReview.yml b/.github/workflows/validateIndependentPeerReview.yml index ff3aaef..b88cd9e 100644 --- a/.github/workflows/validateIndependentPeerReview.yml +++ b/.github/workflows/validateIndependentPeerReview.yml @@ -20,7 +20,7 @@ jobs: id: generateAppToken uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 with: - client-id: ${{ secrets.OS_BOTIFY_CLIENT_ID }} + app-id: 365978 private-key: ${{ secrets.OS_BOTIFY_PRIVATE_KEY }} owner: ${{ github.repository_owner }} repositories: | diff --git a/README.md b/README.md index f630196..d23d77a 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ jobs: Used as an org-level ruleset workflow to block pull requests that do not have enough independent Expensify employee approvals. The check only reads GitHub pull request metadata; it does not checkout or execute code from the pull request branch. -This workflow requires a GitHub App token with read access for repository metadata, pull requests, branch protection administration, and organization members. It uses `OS_BOTIFY_CLIENT_ID` and `OS_BOTIFY_PRIVATE_KEY` to generate that token. If GitHub does not return a branch-protection review count, the workflow defaults to requiring one independent approval, so the ruleset should target only the intended protected branches. +This workflow requires a GitHub App token with read access for repository metadata, pull requests, branch protection administration, and organization members. It uses the OS Botify app ID `365978` and `OS_BOTIFY_PRIVATE_KEY` to generate that token. If GitHub does not return a branch-protection review count, the workflow defaults to requiring one independent approval, so the ruleset should target only the intended protected branches. ## Rulesets GitHub [org-level rulesets](https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#require-workflows-to-pass-before-merging) can be configured to run a workflow check against pull requests in all repos in the org. This is a very powerful feature, but there are some caveats and best practices to be aware of when enabling a ruleset. From 23b2a2c6552e7ed9c6822c4de9f320cede3dca29 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 16:49:48 -0600 Subject: [PATCH 07/17] use client-id --- .github/workflows/validateIndependentPeerReview.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/validateIndependentPeerReview.yml b/.github/workflows/validateIndependentPeerReview.yml index b88cd9e..556fcc0 100644 --- a/.github/workflows/validateIndependentPeerReview.yml +++ b/.github/workflows/validateIndependentPeerReview.yml @@ -20,7 +20,7 @@ jobs: id: generateAppToken uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 with: - app-id: 365978 + client-id: 365978 private-key: ${{ secrets.OS_BOTIFY_PRIVATE_KEY }} owner: ${{ github.repository_owner }} repositories: | From 61d554cd0e52a8dbe81385de6d8f8e3f9cab969d Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 16:53:29 -0600 Subject: [PATCH 08/17] Fix OS Botify token permissions --- .github/workflows/validateIndependentPeerReview.yml | 3 +-- README.md | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/.github/workflows/validateIndependentPeerReview.yml b/.github/workflows/validateIndependentPeerReview.yml index 556fcc0..8fe02d4 100644 --- a/.github/workflows/validateIndependentPeerReview.yml +++ b/.github/workflows/validateIndependentPeerReview.yml @@ -20,13 +20,12 @@ jobs: id: generateAppToken uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 with: - client-id: 365978 + client-id: Iv1.26553ed1d375dd3c private-key: ${{ secrets.OS_BOTIFY_PRIVATE_KEY }} owner: ${{ github.repository_owner }} repositories: | ${{ github.event.repository.name }} GitHub-Actions - permission-administration: read permission-contents: read permission-members: read permission-metadata: read diff --git a/README.md b/README.md index d23d77a..acdf705 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ jobs: Used as an org-level ruleset workflow to block pull requests that do not have enough independent Expensify employee approvals. The check only reads GitHub pull request metadata; it does not checkout or execute code from the pull request branch. -This workflow requires a GitHub App token with read access for repository metadata, pull requests, branch protection administration, and organization members. It uses the OS Botify app ID `365978` and `OS_BOTIFY_PRIVATE_KEY` to generate that token. If GitHub does not return a branch-protection review count, the workflow defaults to requiring one independent approval, so the ruleset should target only the intended protected branches. +This workflow requires a GitHub App token with read access for repository metadata, pull requests, and organization members. It uses the OS Botify client ID `Iv1.26553ed1d375dd3c` and `OS_BOTIFY_PRIVATE_KEY` to generate that token. If GitHub does not return a branch-protection review count, the workflow defaults to requiring one independent approval, so the ruleset should target only the intended protected branches. ## Rulesets GitHub [org-level rulesets](https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#require-workflows-to-pass-before-merging) can be configured to run a workflow check against pull requests in all repos in the org. This is a very powerful feature, but there are some caveats and best practices to be aware of when enabling a ruleset. From 0b48d7423665b2857c748876e96fe03936a71643 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 16:57:24 -0600 Subject: [PATCH 09/17] Rename peer review workflow --- ...alidateIndependentPeerReview.yml => verifyPeerReviews.yml} | 2 +- README.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) rename .github/workflows/{validateIndependentPeerReview.yml => verifyPeerReviews.yml} (98%) diff --git a/.github/workflows/validateIndependentPeerReview.yml b/.github/workflows/verifyPeerReviews.yml similarity index 98% rename from .github/workflows/validateIndependentPeerReview.yml rename to .github/workflows/verifyPeerReviews.yml index 8fe02d4..957454d 100644 --- a/.github/workflows/validateIndependentPeerReview.yml +++ b/.github/workflows/verifyPeerReviews.yml @@ -3,7 +3,7 @@ # - We need to checkout the repo it's running on, and not just the GitHub-Actions repo # - branch and path matching does not work in the workflow layer. From the docs: https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#supported-event-triggers # > Any filters you specify for the supported events are ignored - for example, branches, branches-ignore, paths, types and so on. The workflow is only triggered, and is always triggered, by the default activity types of the supported events -name: Validate independent peer review +name: Verify peer reviews on: pull_request diff --git a/README.md b/README.md index acdf705..4cbc0e6 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ jobs: secrets: inherit ``` -### `validateIndependentPeerReview.yml` +### `verifyPeerReviews.yml` Used as an org-level ruleset workflow to block pull requests that do not have enough independent Expensify employee approvals. The check only reads GitHub pull request metadata; it does not checkout or execute code from the pull request branch. @@ -53,4 +53,4 @@ GitHub [org-level rulesets](https://docs.github.com/en/enterprise-cloud@latest/r - If you need to target or exclude specific paths, that must be implemented manually in the workflow itself. - Due to a GitHub :bug:, PRs that are open when the rule is enabled will get stuck with a pending check that will never get picked up. The easiest way to fix that is to close and reopen the PR. Consider writing a script to close and reopen all open PRs across the org after the check is enabled. - It is less disruptive to [configure the ruleset to `Evaluate` first](https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#using-evaluate-mode-for-ruleset-workflows), then `Active` once the kinks are worked out. -- For `validateIndependentPeerReview.yml`, start with a ruleset targeting only a test branch, then test the workflow from a GitHub-Actions branch, then from `main`, and only then enable it for the intended repositories and branches. +- For `verifyPeerReviews.yml`, start with a ruleset targeting only a test branch, then test the workflow from a GitHub-Actions branch, then from `main`, and only then enable it for the intended repositories and branches. From e57f967c9d203a9b28c79fd519a1bc953adc8e25 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 17:01:48 -0600 Subject: [PATCH 10/17] Rename peer review verifier --- .../{verifyPeerReviews.yml => verifyPeerReview.yml} | 8 ++++---- README.md | 4 ++-- package.json | 2 +- ...lidateIndependentPeerReview.ts => verifyPeerReview.ts} | 0 4 files changed, 7 insertions(+), 7 deletions(-) rename .github/workflows/{verifyPeerReviews.yml => verifyPeerReview.yml} (92%) rename scripts/{validateIndependentPeerReview.ts => verifyPeerReview.ts} (100%) diff --git a/.github/workflows/verifyPeerReviews.yml b/.github/workflows/verifyPeerReview.yml similarity index 92% rename from .github/workflows/verifyPeerReviews.yml rename to .github/workflows/verifyPeerReview.yml index 957454d..99aa4dd 100644 --- a/.github/workflows/verifyPeerReviews.yml +++ b/.github/workflows/verifyPeerReview.yml @@ -3,7 +3,7 @@ # - We need to checkout the repo it's running on, and not just the GitHub-Actions repo # - branch and path matching does not work in the workflow layer. From the docs: https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#supported-event-triggers # > Any filters you specify for the supported events are ignored - for example, branches, branches-ignore, paths, types and so on. The workflow is only triggered, and is always triggered, by the default activity types of the supported events -name: Verify peer reviews +name: Verify peer review on: pull_request @@ -12,7 +12,7 @@ permissions: pull-requests: read jobs: - validateIndependentPeerReview: + verifyPeerReview: runs-on: blacksmith-2vcpu-ubuntu-2404 steps: # v3.1.1 @@ -43,8 +43,8 @@ jobs: run: npm ci working-directory: GitHub-Actions - - name: Validate independent peer review - run: npm run validate-independent-peer-review + - name: Verify peer review + run: npm run verify-peer-review working-directory: GitHub-Actions env: GITHUB_TOKEN: ${{ steps.generateAppToken.outputs.token }} diff --git a/README.md b/README.md index 4cbc0e6..1dbb753 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ jobs: secrets: inherit ``` -### `verifyPeerReviews.yml` +### `verifyPeerReview.yml` Used as an org-level ruleset workflow to block pull requests that do not have enough independent Expensify employee approvals. The check only reads GitHub pull request metadata; it does not checkout or execute code from the pull request branch. @@ -53,4 +53,4 @@ GitHub [org-level rulesets](https://docs.github.com/en/enterprise-cloud@latest/r - If you need to target or exclude specific paths, that must be implemented manually in the workflow itself. - Due to a GitHub :bug:, PRs that are open when the rule is enabled will get stuck with a pending check that will never get picked up. The easiest way to fix that is to close and reopen the PR. Consider writing a script to close and reopen all open PRs across the org after the check is enabled. - It is less disruptive to [configure the ruleset to `Evaluate` first](https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#using-evaluate-mode-for-ruleset-workflows), then `Active` once the kinks are worked out. -- For `verifyPeerReviews.yml`, start with a ruleset targeting only a test branch, then test the workflow from a GitHub-Actions branch, then from `main`, and only then enable it for the intended repositories and branches. +- For `verifyPeerReview.yml`, start with a ruleset targeting only a test branch, then test the workflow from a GitHub-Actions branch, then from `main`, and only then enable it for the intended repositories and branches. diff --git a/package.json b/package.json index c9fcd31..e5eed16 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "@expensify/github-actions", "private": true, "scripts": { - "validate-independent-peer-review": "tsx scripts/validateIndependentPeerReview.ts" + "verify-peer-review": "tsx scripts/verifyPeerReview.ts" }, "devDependencies": { "@octokit/webhooks-types": "^7.6.1", diff --git a/scripts/validateIndependentPeerReview.ts b/scripts/verifyPeerReview.ts similarity index 100% rename from scripts/validateIndependentPeerReview.ts rename to scripts/verifyPeerReview.ts From b6aea8531b5a7d1adc8c2d68ac9be241752a0db1 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 17:05:54 -0600 Subject: [PATCH 11/17] Clarify peer review failures --- .github/workflows/verifyPeerReview.yml | 2 ++ scripts/verifyPeerReview.ts | 35 ++++++++++++++++++++++++-- 2 files changed, 35 insertions(+), 2 deletions(-) diff --git a/.github/workflows/verifyPeerReview.yml b/.github/workflows/verifyPeerReview.yml index 99aa4dd..dd43067 100644 --- a/.github/workflows/verifyPeerReview.yml +++ b/.github/workflows/verifyPeerReview.yml @@ -4,6 +4,7 @@ # - branch and path matching does not work in the workflow layer. From the docs: https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#supported-event-triggers # > Any filters you specify for the supported events are ignored - for example, branches, branches-ignore, paths, types and so on. The workflow is only triggered, and is always triggered, by the default activity types of the supported events name: Verify peer review +run-name: Verify peer review for ${{ github.repository }}#${{ github.event.pull_request.number }} on: pull_request @@ -13,6 +14,7 @@ permissions: jobs: verifyPeerReview: + name: Check independent approval runs-on: blacksmith-2vcpu-ubuntu-2404 steps: # v3.1.1 diff --git a/scripts/verifyPeerReview.ts b/scripts/verifyPeerReview.ts index 462818c..8c9c0f0 100644 --- a/scripts/verifyPeerReview.ts +++ b/scripts/verifyPeerReview.ts @@ -1,4 +1,4 @@ -import {readFileSync} from 'node:fs'; +import {appendFileSync, readFileSync} from 'node:fs'; import {graphql} from '@octokit/graphql'; import {Octokit, type RestEndpointMethodTypes} from '@octokit/rest'; import type {PullRequestEvent} from '@octokit/webhooks-types'; @@ -80,6 +80,35 @@ function unique(values: string[]): string[] { return [...new Set(values)].sort((a, b) => a.localeCompare(b)); } +function escapeWorkflowCommandValue(value: string): string { + return value.replace(/%/g, '%25').replace(/\r/g, '%0D').replace(/\n/g, '%0A'); +} + +function escapeWorkflowCommandProperty(value: string): string { + return escapeWorkflowCommandValue(value).replace(/:/g, '%3A').replace(/,/g, '%2C'); +} + +function getFailureTitle(message: string): string { + if (message.includes('does not have enough independent Expensify employee approvals')) { + return 'Missing independent peer review'; + } + if (message.includes('Unable to resolve Expensify co-author emails')) { + return 'Unresolved Expensify co-author'; + } + if (message.includes('has no human commit authors or co-authors')) { + return 'No human commit author'; + } + return 'Peer review verification failed'; +} + +function writeStepSummary(title: string, message: string): void { + const stepSummaryPath = process.env.GITHUB_STEP_SUMMARY; + if (!stepSummaryPath) { + return; + } + appendFileSync(stepSummaryPath, `## ${title}\n\n${message.replace(/\n/g, '\n\n')}\n`); +} + function getPullRequestContext(): PullRequestContext { const eventPath = process.env.GITHUB_EVENT_PATH; if (!eventPath) { @@ -272,6 +301,8 @@ async function main(): Promise { main().catch((error: unknown) => { const message = error instanceof Error ? error.message : String(error); - console.error(`::error::${message.replace(/%/g, '%25').replace(/\r/g, '%0D').replace(/\n/g, '%0A')}`); + const title = getFailureTitle(message); + writeStepSummary(title, message); + console.error(`::error title=${escapeWorkflowCommandProperty(title)}::${escapeWorkflowCommandValue(message)}`); process.exit(1); }); From 2d8ce9c93a439767548d65e5275f0d1ceb84352e Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 17:14:22 -0600 Subject: [PATCH 12/17] Test informational peer review checks --- .github/workflows/verifyPeerReview.yml | 2 ++ scripts/verifyPeerReview.ts | 10 ++++++++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/.github/workflows/verifyPeerReview.yml b/.github/workflows/verifyPeerReview.yml index dd43067..7c9d40d 100644 --- a/.github/workflows/verifyPeerReview.yml +++ b/.github/workflows/verifyPeerReview.yml @@ -14,6 +14,7 @@ permissions: jobs: verifyPeerReview: + if: ${{ !github.event.pull_request.draft }} name: Check independent approval runs-on: blacksmith-2vcpu-ubuntu-2404 steps: @@ -50,3 +51,4 @@ jobs: working-directory: GitHub-Actions env: GITHUB_TOKEN: ${{ steps.generateAppToken.outputs.token }} + VERIFY_PEER_REVIEW_MODE: informational diff --git a/scripts/verifyPeerReview.ts b/scripts/verifyPeerReview.ts index 8c9c0f0..eaa8716 100644 --- a/scripts/verifyPeerReview.ts +++ b/scripts/verifyPeerReview.ts @@ -57,6 +57,7 @@ const botUsers = new Set(['botify', 'MelvinBot', 'exfy-zapier']); const defaultRequiredApprovingReviewCount = 1; const expensifyOrganization = 'Expensify'; const expensifyEmployeeTeamSlug = 'expensify-expensify'; +const isInformationalMode = process.env.VERIFY_PEER_REVIEW_MODE === 'informational'; const githubToken = process.env.GITHUB_TOKEN ?? process.env.GH_TOKEN; if (!githubToken) { throw new Error('GITHUB_TOKEN or GH_TOKEN is required'); @@ -302,7 +303,12 @@ async function main(): Promise { main().catch((error: unknown) => { const message = error instanceof Error ? error.message : String(error); const title = getFailureTitle(message); - writeStepSummary(title, message); - console.error(`::error title=${escapeWorkflowCommandProperty(title)}::${escapeWorkflowCommandValue(message)}`); + const annotationType = isInformationalMode ? 'warning' : 'error'; + const summary = isInformationalMode ? `${message}\n\nThis check is running in informational mode, so it did not fail the workflow.` : message; + writeStepSummary(title, summary); + console.error(`::${annotationType} title=${escapeWorkflowCommandProperty(title)}::${escapeWorkflowCommandValue(summary)}`); + if (isInformationalMode) { + process.exit(0); + } process.exit(1); }); From 4bee49de41e066f8a3f3046d0ddb2ebcd2b9fefd Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 17:16:43 -0600 Subject: [PATCH 13/17] Undo draft --- .github/workflows/verifyPeerReview.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.github/workflows/verifyPeerReview.yml b/.github/workflows/verifyPeerReview.yml index 7c9d40d..5263ac1 100644 --- a/.github/workflows/verifyPeerReview.yml +++ b/.github/workflows/verifyPeerReview.yml @@ -14,7 +14,6 @@ permissions: jobs: verifyPeerReview: - if: ${{ !github.event.pull_request.draft }} name: Check independent approval runs-on: blacksmith-2vcpu-ubuntu-2404 steps: From 1d3a55983946fecb3cbbaf06dc3bedcaafc4222f Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 17:19:47 -0600 Subject: [PATCH 14/17] Show informational peer review failures --- .github/workflows/verifyPeerReview.yml | 1 + scripts/verifyPeerReview.ts | 5 +---- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/.github/workflows/verifyPeerReview.yml b/.github/workflows/verifyPeerReview.yml index 5263ac1..661c783 100644 --- a/.github/workflows/verifyPeerReview.yml +++ b/.github/workflows/verifyPeerReview.yml @@ -46,6 +46,7 @@ jobs: working-directory: GitHub-Actions - name: Verify peer review + continue-on-error: true run: npm run verify-peer-review working-directory: GitHub-Actions env: diff --git a/scripts/verifyPeerReview.ts b/scripts/verifyPeerReview.ts index eaa8716..b26eead 100644 --- a/scripts/verifyPeerReview.ts +++ b/scripts/verifyPeerReview.ts @@ -304,11 +304,8 @@ main().catch((error: unknown) => { const message = error instanceof Error ? error.message : String(error); const title = getFailureTitle(message); const annotationType = isInformationalMode ? 'warning' : 'error'; - const summary = isInformationalMode ? `${message}\n\nThis check is running in informational mode, so it did not fail the workflow.` : message; + const summary = isInformationalMode ? `${message}\n\nThis check is running in informational mode, so the workflow continues after reporting this failure.` : message; writeStepSummary(title, summary); console.error(`::${annotationType} title=${escapeWorkflowCommandProperty(title)}::${escapeWorkflowCommandValue(summary)}`); - if (isInformationalMode) { - process.exit(0); - } process.exit(1); }); From 74415cd18562396fd201ef5f050e02509c933f60 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Fri, 15 May 2026 17:22:40 -0600 Subject: [PATCH 15/17] Restore hard peer review failure --- .github/workflows/verifyPeerReview.yml | 2 -- scripts/verifyPeerReview.ts | 7 ++----- 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/.github/workflows/verifyPeerReview.yml b/.github/workflows/verifyPeerReview.yml index 661c783..dd43067 100644 --- a/.github/workflows/verifyPeerReview.yml +++ b/.github/workflows/verifyPeerReview.yml @@ -46,9 +46,7 @@ jobs: working-directory: GitHub-Actions - name: Verify peer review - continue-on-error: true run: npm run verify-peer-review working-directory: GitHub-Actions env: GITHUB_TOKEN: ${{ steps.generateAppToken.outputs.token }} - VERIFY_PEER_REVIEW_MODE: informational diff --git a/scripts/verifyPeerReview.ts b/scripts/verifyPeerReview.ts index b26eead..8c9c0f0 100644 --- a/scripts/verifyPeerReview.ts +++ b/scripts/verifyPeerReview.ts @@ -57,7 +57,6 @@ const botUsers = new Set(['botify', 'MelvinBot', 'exfy-zapier']); const defaultRequiredApprovingReviewCount = 1; const expensifyOrganization = 'Expensify'; const expensifyEmployeeTeamSlug = 'expensify-expensify'; -const isInformationalMode = process.env.VERIFY_PEER_REVIEW_MODE === 'informational'; const githubToken = process.env.GITHUB_TOKEN ?? process.env.GH_TOKEN; if (!githubToken) { throw new Error('GITHUB_TOKEN or GH_TOKEN is required'); @@ -303,9 +302,7 @@ async function main(): Promise { main().catch((error: unknown) => { const message = error instanceof Error ? error.message : String(error); const title = getFailureTitle(message); - const annotationType = isInformationalMode ? 'warning' : 'error'; - const summary = isInformationalMode ? `${message}\n\nThis check is running in informational mode, so the workflow continues after reporting this failure.` : message; - writeStepSummary(title, summary); - console.error(`::${annotationType} title=${escapeWorkflowCommandProperty(title)}::${escapeWorkflowCommandValue(summary)}`); + writeStepSummary(title, message); + console.error(`::error title=${escapeWorkflowCommandProperty(title)}::${escapeWorkflowCommandValue(message)}`); process.exit(1); }); From 74efe1d9fadc94346d88057ace18629321069f46 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Mon, 18 May 2026 10:50:15 -0600 Subject: [PATCH 16/17] Pass peer review check before approval --- scripts/verifyPeerReview.ts | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scripts/verifyPeerReview.ts b/scripts/verifyPeerReview.ts index 8c9c0f0..9c1e4f9 100644 --- a/scripts/verifyPeerReview.ts +++ b/scripts/verifyPeerReview.ts @@ -275,6 +275,10 @@ async function main(): Promise { console.log(`${owner}/${repo}#${number} targets ${context.baseRef}, which does not require approving reviews.`); return; } + if (approvers.length === 0) { + console.log(`${owner}/${repo}#${number} has no approving reviews from writers; regular branch protection will block merge until an approval exists.`); + return; + } const {authors, unresolvedExpensifyCoAuthors} = getCommitAuthors(commits); if (unresolvedExpensifyCoAuthors.length > 0) { From 72a99fd143c5af3fe4f6758e30149e310f3741b3 Mon Sep 17 00:00:00 2001 From: Andrew Gable Date: Mon, 18 May 2026 15:37:55 -0600 Subject: [PATCH 17/17] Use Melvin app for peer review check --- .github/workflows/verifyPeerReview.yml | 4 ++-- README.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/verifyPeerReview.yml b/.github/workflows/verifyPeerReview.yml index dd43067..61a99d5 100644 --- a/.github/workflows/verifyPeerReview.yml +++ b/.github/workflows/verifyPeerReview.yml @@ -22,8 +22,8 @@ jobs: id: generateAppToken uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 with: - client-id: Iv1.26553ed1d375dd3c - private-key: ${{ secrets.OS_BOTIFY_PRIVATE_KEY }} + app-id: '179547' + private-key: ${{ secrets.MELVIN_APP_PRIVATE_KEY }} owner: ${{ github.repository_owner }} repositories: | ${{ github.event.repository.name }} diff --git a/README.md b/README.md index 1dbb753..9f9e75a 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ jobs: Used as an org-level ruleset workflow to block pull requests that do not have enough independent Expensify employee approvals. The check only reads GitHub pull request metadata; it does not checkout or execute code from the pull request branch. -This workflow requires a GitHub App token with read access for repository metadata, pull requests, and organization members. It uses the OS Botify client ID `Iv1.26553ed1d375dd3c` and `OS_BOTIFY_PRIVATE_KEY` to generate that token. If GitHub does not return a branch-protection review count, the workflow defaults to requiring one independent approval, so the ruleset should target only the intended protected branches. +This workflow requires a GitHub App token with read access for repository metadata, pull requests, and organization members. It uses the Melvin Bot app ID `179547` and `MELVIN_APP_PRIVATE_KEY` to generate that token. If GitHub does not return a branch-protection review count, the workflow defaults to requiring one independent approval, so the ruleset should target only the intended protected branches. ## Rulesets GitHub [org-level rulesets](https://docs.github.com/en/enterprise-cloud@latest/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/available-rules-for-rulesets#require-workflows-to-pass-before-merging) can be configured to run a workflow check against pull requests in all repos in the org. This is a very powerful feature, but there are some caveats and best practices to be aware of when enabling a ruleset.