Summary / recommendation
Use GitHub Actions to automatically capture failing CI runs into a draft “triage” PR that contains failing-test evidence, then use GitHub Copilot (Copilot Chat in the IDE) to generate and iterate candidate fixes locally. Keep commits small, require a passing CI and at least one human review before merge. This balances speed with safety.
High-level workflow (what runs automatically vs. what you do)
- Automatic: on any workflow_run that finishes with conclusion: failure, create a branch, commit the run id + logs (or link) and open a draft PR. Add labels like triage/ci-failure.
- Developer: fetch the branch, reproduce locally (use the attached logs/run id), open files in IDE, run Copilot Chat to get 2–3 candidate fixes, implement candidate(s), run unit/integration tests locally in the repo container, push a follow-up commit to the draft PR.
- CI: require full test suite, linters, static analyzers, and safety checks before PR can be merged.
Minimal GitHub Action hook (paste into .github/workflows/ci-triage.yml)
- triggers: on: workflow_run (types: [completed])
- job condition: if: github.event.workflow_run.conclusion == 'failure'
- job steps: checkout, create branch bugfix/ci-failure-${{github.event.workflow_run.id}}, add a file failing-runs/run-${{id}}.json (or a short text file with the failing job URL), commit & push, then use peter-evans/create-pull-request to open a draft PR linking the failing run.
(Use GITHUB_TOKEN; keep the action’s permissions minimal — only contents: write.)
Example Copilot Chat prompts (use in IDE Copilot Chat)
1) Reproduce test and triage prompt
"I have this failing test: . Explain the likely root causes in 3 bullets and list the exact command(s) to reproduce the failure in Docker/container/local-dev. Provide a minimal failing test snippet I can run locally."
2) Candidate fix prompt (ask for ranked fixes)
"Propose 3 ranked candidate fixes for this failing test. For each: 1) one-line summary, 2) code diff (small patch), 3) tests to add/update, 4) risk level (low/med/high), 5) runtime perf impact. Prefer minimal, easily-reviewed fixes first."
3) Commit message / PR body prompt
"Write a conventional commit message and a PR body that includes: failure reproduction steps, failing test output, proposed change summary, risk & rollback notes, tests added, and CI acceptance criteria."
Guardrails (musts)
- Never auto-merge AI-generated code. Require at least one human reviewer.
- Require new/modified tests that reproduce the bug. CI must pass full suite before merge.
- Limit auto-created draft PRs to adding evidence files, not code changes.
- Enforce static analysis/security scans on every candidate commit.
- Set patch size threshold (e.g., no more than 200 lines changed) for auto-assignment to junior reviewers.
Decision criteria: when to auto-apply vs manual
- Auto-apply small fixes (formatting, obvious null-check) only if: single-file change, <10 lines, low-risk, and passing unit tests + security scan. Otherwise require manual PR.
- Team size/skill: small teams should favor more human checks; large teams can automate more triage steps.
- Budget / quality: more automation (auto triage + suggested patches) saves time but increases review overhead. If quality is critical, keep all AI outputs review-only.
Practical checklist to enable this tomorrow
1) Add workflow_run Action that creates draft PRs with run id/logs. 2) Add PR template that includes a “Reproduction” section. 3) Teach devs the 3 Copilot prompts above and a convention for branch names. 4) Add CI gates: full-test, linters, static analysis, tests coverage check. 5) Add repo rules: require PR approval and passing CI before merge. 6) Run a dry run for a week, review false positives/extra noise, tweak thresholds.
Best-for / Avoid-if
- Best for teams that want fast triage and reproducible evidence for debugging.
- Avoid if you have strict regulatory/security requirements that forbid AI-suggested code without formal verification.
Tools mentioned: GitHub Copilot (for in-editor candidate fixes) and ChatGPT (optional for drafting PR text and high-level reasoning).
Compare GitHub Copilot and Cursor