Move CI from GitHub Actions to Buildkite#116
Conversation
Add a Buildkite CI pipeline running the same matrix as the GitHub Actions workflow: pytest and pylint on Python 3.9-3.13, plus pytest>=9 on 3.13. The pipeline runs in the Open Source cluster with no secrets. Test results are dogfooded to the test-collector-python Test Engine suite using an ephemeral OIDC token instead of the long-lived BUILDKITE_ANALYTICS_TOKEN secret used by GitHub Actions. The GitHub Actions workflow stays in place until the Buildkite pipeline has proven itself; a follow-up will remove it. Amp-Thread-ID: https://ampcode.com/threads/T-019f26cf-e438-70a4-a167-1d1c45d42519 Co-authored-by: Amp <amp@ampcode.com>
CI now runs on Buildkite with the same coverage: pytest and pylint on Python 3.9-3.13, plus pytest>=9 on 3.13. Also drop the github-actions dependabot ecosystem since no workflows remain. Amp-Thread-ID: https://ampcode.com/threads/T-019f26cf-e438-70a4-a167-1d1c45d42519 Co-authored-by: Amp <amp@ampcode.com>
|
@buildsworth-bk review approve L2 |
![buildsworth-bk-app[bot]](https://avatars.githubusercontent.com/in/3010600?s=60&v=4){kind=small}
There was a problem hiding this comment.
I didn't find any code-level issues to raise. This moves CI execution and token handling into Buildkite configuration, which I read as above the default L1 approval ceiling, so I'm leaving this as a comment rather than an approval.
Want to dig deeper? The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 2683, then answer my questions about the findings.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d2cf3fa124
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| queue: hosted | ||
|
|
||
| steps: | ||
| - label: ":pytest: pytest {{matrix}}" |
There was a problem hiding this comment.
When this Buildkite pipeline is configured to build third-party fork PRs, it starts directly with command steps and never adds the block gate described for fork-triggered builds; Buildkite's block-step docs state that a block step pauses execution until a team member unblocks it (https://buildkite.com/docs/pipelines/configure/step-types/block-step). As written, forked PR code reaches .buildkite/steps/test, which mints and exports a Test Engine OIDC token before running tests, so untrusted fork builds can upload or poison results for the suite.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
The block step is in the pipeline's server-side YAML steps, managed by infra-as-code, where it's safer from change.
There was a problem hiding this comment.
I found one unanswered question around where the fork-PR gate lives. Even with the verified L2 grant, I’m leaving this as a comment rather than approving because the change is CI/OIDC configuration, which is above the L2 approval ceiling.
Want to dig deeper? The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 2684, then answer my questions about the findings.
| agents: | ||
| queue: hosted | ||
|
|
||
| steps: |
There was a problem hiding this comment.
Question: The PR description mentions a block step for fork-triggered builds, but I don’t see that gate before the runnable steps in this checked-in pipeline (and the test step mints the OIDC token). Is the block configured outside the repo, or should it be represented here so fork PRs can’t reach the token step before approval?
There was a problem hiding this comment.
It's in the static build step that uploads this YAML
There was a problem hiding this comment.
Here's the server-side steps:
steps:
# block the build for approval if triggered by a fork
- block: "Allow this build to run?"
prompt: ":rotating_light: Review the PR for concerning changes before unblocking"
if: build.pull_request.repository.fork
blocked_state: running
- label: ":pipeline:"
command: "buildkite-agent pipeline upload"
agents:
queue: hosted
|
@buildsworth-bk review approve L2 |
There was a problem hiding this comment.
The earlier fork-PR gate question is answered by the server-side static upload step you shared. I didn't find any new code-level issues; because this change moves CI and OIDC token handling, I still read it as above the approval ceiling, so I'm leaving this as a comment rather than an approval.
Want to dig deeper? The full session log is attached to this Buildkite build. Download the session file and open a new pi session with it:
Download the buildsworth logs from build 2685, then answer my questions about the findings.
1 participant
Summary
Moves CI from GitHub Actions to Buildkite, with the same coverage:
pytest>=9Runs on the new test-collector-python pipeline, gated by a block step for fork-triggered builds, and no secrets — the collector dogfoods its own test results to the
test-collector-pythonTest Engine suite using an ephemeral OIDC token instead of a long-livedBUILDKITE_ANALYTICS_TOKENsecret.The GitHub Actions workflow is removed in this PR, along with the
github-actionsdependabot ecosystem (no workflows remain). TheBUILDKITE_ANALYTICS_TOKENrepo secret can be deleted once this merges.Verified on this branch: build #2 ran all 11 jobs green and test results uploaded to the suite via OIDC.