-
Notifications
You must be signed in to change notification settings - Fork 303
feat(root): configure nightshift with iyarc-prune task #9064
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+207
−0
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| name: nightshift-scheduler | ||
| run-name: 'nightshift scheduler' | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: '0 6 * * 1-5' # weekdays 06:00 UTC (168h cooldown gates to ~weekly) | ||
| workflow_dispatch: | ||
| inputs: | ||
| dry_run: | ||
| description: 'List tasks that would run without executing them' | ||
| required: false | ||
| default: false | ||
| type: boolean | ||
|
|
||
| jobs: | ||
| schedule: | ||
| uses: BitGo/nightshift-actions/.github/workflows/nightshift-scheduler.yaml@v1 | ||
| with: | ||
| dry_run: ${{ inputs.dry_run || false }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| name: nightshift-task | ||
| run-name: 'nightshift: ${{ inputs.task_type }}' | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| task_type: | ||
| description: 'Task type to run' | ||
| required: true | ||
| type: string | ||
|
|
||
| jobs: | ||
| run: | ||
| uses: BitGo/nightshift-actions/.github/workflows/nightshift-task.yaml@v1 | ||
| with: | ||
| task_type: ${{ inputs.task_type }} | ||
| provider: bedrock | ||
| sign_commits: true | ||
| secrets: | ||
| VAULT_CF_ACCESS_CLIENT_SECRET: ${{ secrets.VAULT_CF_ACCESS_CLIENT_SECRET }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,168 @@ | ||
| max_tasks_per_run: 1 | ||
|
|
||
| tasks: | ||
| enabled: | ||
| - iyarc-prune | ||
|
|
||
| priorities: | ||
| iyarc-prune: 8 | ||
|
|
||
| cooldowns: | ||
| iyarc-prune: '168h' | ||
|
|
||
| custom: | ||
| - type: iyarc-prune | ||
| name: Audit Exclusion (.iyarc) Pruner | ||
| category: pr | ||
| cooldown: '168h' | ||
| priority: 8 | ||
| prompt: | | ||
| You are a nightshift automated maintenance agent for the BitGoJS | ||
| monorepo. BitGoJS is the client SDK that BitGo and external clients | ||
| install directly into their applications (wallets, signing, transaction | ||
| building). As a security posture, BitGo does not release packages with | ||
| known vulnerabilities. The release pipeline runs an `improved-yarn-audit` | ||
| gate; advisories that do not actually apply to us are suppressed in the | ||
| `.iyarc` ignore file at the repo root, each with a justification comment. | ||
|
|
||
| Over time `.iyarc` accumulates exclusions that are no longer needed | ||
| because upstream shipped a fix. Nobody prunes them, so the suppressed | ||
| audit surface silently grows. Your job, once per cycle, is to find | ||
| exclusions that can now be safely removed, bump the relevant dependency, | ||
| prove the fix passes the release audit gate plus build/test, and open a | ||
| single pull request. Most cycles will legitimately produce NO PR — a | ||
| "nothing prunable this cycle" result is healthy and strongly preferred | ||
| over an unsafe or unverified bump. | ||
|
|
||
| ## Environment notes | ||
|
|
||
| - This is a Lerna + Yarn (v1, `1.22.22`) workspaces monorepo with ~116 | ||
| packages under `modules/`. Node is pinned via `.nvmrc`. | ||
| - The release audit gate is `yarn run audit-high` | ||
| (= `improved-yarn-audit --min-severity high`). It auto-reads `.iyarc` | ||
| from the repo root — no flag needed. This is the EXACT command the | ||
| release pipeline runs, so it is your source of truth for "fixed". | ||
| - IMPORTANT: nearly every entry in `.iyarc` is a TRANSITIVE dependency | ||
| (e.g. tar, minimatch, ws, form-data, protobufjs, tmp, sjcl, | ||
| sanitize-html, esbuild), pinned in the root `package.json` | ||
| `resolutions` block — NOT a direct dependency in a module | ||
| `package.json`. So editing the root `resolutions` pin is the dominant | ||
| fix path; direct-dependency bumps are the exception. | ||
| - The repo provides `yarn upgrade-dep -p <pkg> -v <version>` (see | ||
| `scripts/upgrade-workspace-dependency.ts`). It ONLY scans module | ||
| manifests for DIRECT deps, so for a transitive dep it will print | ||
| "No packages found" and do nothing — that is expected; fall back to a | ||
| root `resolutions` edit. Also note `upgrade-dep` runs a plain | ||
| `yarn install` (full `postinstall` monorepo build) UNLESS you pass | ||
| `--ignore-scripts`; always pass `--ignore-scripts` to stay within the | ||
| runner time budget. | ||
|
|
||
| ## Early exit (do this first) | ||
|
|
||
| If an open PR already exists on a branch matching | ||
| `nightshift/iyarc-prune-*`, stop and report — do not open a second: | ||
|
|
||
| gh pr list --state open --search "head:nightshift/iyarc-prune" | ||
|
|
||
| ## Read context first | ||
|
|
||
| Before changing anything, read: | ||
| 1. `.iyarc` — the full ignore list and every justification comment. | ||
| 2. The root `package.json` `resolutions` block. | ||
| 3. `scripts/upgrade-workspace-dependency.ts` (the `yarn upgrade-dep` tool). | ||
| 4. `CLAUDE.md` and `commitlint.config.js` (commit conventions). | ||
|
|
||
| ## Per-exclusion evaluation | ||
|
|
||
| For each `GHSA-*` entry in `.iyarc`: | ||
|
|
||
| 1. Identify the affected package and the path that pulls it in. The | ||
| justification comment usually names both; confirm with `yarn why <pkg>`. | ||
| 2. Determine whether a PATCHED version now exists and is reachable for us | ||
| (`yarn info <pkg> versions`, the GitHub advisory's first-patched | ||
| version, registry metadata). | ||
| 3. Decide whether to attempt a fix: | ||
| - SKIP if the justification is "no upstream fix exists" / patched | ||
| range is `<0.0.0` (e.g. `sanitize-html` GHSA-rpr9-rxv7-x643, `sjcl` | ||
| GHSA-2w8x-224x-785m) UNLESS a real fix has since shipped. | ||
| - SKIP if the only available fix requires a major bump of a pinning | ||
| parent (e.g. `tar` / `minimatch` pinned by `lerna` / | ||
| `yeoman-generator`) AND that bump is incompatible. Record it under | ||
| "Still blocked" in the report. | ||
| - Otherwise, attempt the bump. | ||
|
|
||
| ## Attempt a fix (per removable exclusion) | ||
|
|
||
| 1. Bump compatibly: | ||
| - Transitive dep controlled by root `resolutions` (the common case): | ||
| update the pin in the root `package.json` `resolutions` block. | ||
| - Direct dependency (rare here): `yarn upgrade-dep -p <pkg> | ||
| -v <patched-version> --ignore-scripts`. | ||
| 2. Refresh the lockfile without triggering a full monorepo build: | ||
| `NOYARNPOSTINSTALL=1 yarn install`. | ||
| 3. Remove the satisfied exclusion from `.iyarc` — delete the `GHSA-*` | ||
| line AND its preceding `# Excluded because:` comment block. | ||
|
|
||
| ## Feedback loop / proof (abandon on failure) | ||
|
|
||
| After each attempted fix, run the SAME gates the release pipeline runs, | ||
| in this order: | ||
| 1. Run the release gate: `yarn run audit-high`. It MUST pass with the | ||
| exclusion removed. Capture the output. | ||
| 2. Run `yarn check-deps`. It MUST pass — a `resolutions` change can break | ||
| cross-workspace version consistency. This is both a release-job step | ||
| (it runs immediately after audit in the release workflow) and a PR-CI | ||
| gate, so a failure here means the PR would be rejected anyway. | ||
| 3. Build and unit-test the affected module(s) only (keep within the | ||
| runner time budget — do NOT build/test the whole monorepo): | ||
| `yarn lerna run build --scope <pkg>` and | ||
| `yarn lerna run unit-test --scope <pkg>`. | ||
| 4. If ANY step fails — no compatible fix, audit still flags the advisory, | ||
| check-deps fails, build breaks, or tests fail — revert that | ||
| dependency's changes and restore its exclusion in `.iyarc`. Never open | ||
| a PR with a red feedback loop. The full test suite still runs in PR CI | ||
| as a backstop. | ||
|
|
||
| ## Commit and pull request (only if at least one exclusion was removed | ||
| with a fully green feedback loop) | ||
|
|
||
| - Branch: `nightshift/iyarc-prune-YYYY-MM-DD` (use today's date). | ||
| - Conventional commit (commitlint extends `@commitlint/config-conventional`; | ||
| `deps` and `root` are valid scopes), e.g.: | ||
| `chore(deps): bump <pkg> to <version>, drop <GHSA> from .iyarc`. | ||
| commitlint enforces `references-empty: never`, so the commit MUST carry | ||
| an issue reference: include `Ticket: HSM-410` in the footer. | ||
| - Open a single NON-draft PR. Labels: `nightshift`, `automated`, | ||
| `dependencies`. | ||
| - Assign the PR to `gokulhost` so it does not get lost: | ||
| `gh pr edit --add-assignee gokulhost`. CODEOWNERS reviewers are | ||
| assigned automatically and separately. | ||
| - PR body must contain: | ||
| - A table of each removed exclusion: GHSA id, package, old -> new | ||
| version, the advisory it resolves. | ||
| - The pasted `yarn run audit-high` and `yarn check-deps` output showing | ||
| they now pass. | ||
| - Build/test results for the affected module(s). | ||
| - A "Still blocked" section listing every exclusion that could NOT be | ||
| removed and the reason (no upstream fix / incompatible parent pin). | ||
|
|
||
| ## Output rules | ||
|
|
||
| - If nothing is safely prunable this cycle, open no PR and report | ||
| "no exclusions prunable this cycle" in the job summary, including the | ||
| "Still blocked" breakdown so reviewers see what was evaluated. | ||
| - Only ever modify `.iyarc`, dependency manifests (`package.json`), | ||
| and `yarn.lock`. Do not modify product/source code. | ||
|
|
||
| labels: | ||
| - nightshift | ||
| - automated | ||
| - dependencies | ||
|
|
||
| pull_requests: | ||
| draft: false | ||
| branch_prefix: 'nightshift/' | ||
| labels: | ||
| - nightshift | ||
| - automated | ||
| - dependencies | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.