Goal: let CI run and gate Copilot-generated tests automatically, but quarantine anything suspicious and force a human to sign off before merging to main.

Recommendation: implement a three-state pipeline (Auto-approve → Quarantine → Block) driven by test metadata added when tests are generated. Auto-run fast static checks + repeatable execution; quarantine tests that are flaky, low-quality, or access risky resources; require a human code-owner review to promote quarantined tests back to main.

How it works (high level)
- PRs that contain Copilot-generated tests must include a provenance header or label (e.g., // generated-by: GitHub Copilot). CI uses that to route checks.
- CI runs deterministic checks (linters, security scanners), then executes the new tests repeatedly and in isolation. Collect pass rate and side-effects.
- If tests meet thresholds they’re allowed to land automatically (or merge with minimal review). If they’re flaky, use network/IO, contain weak assertions, or fail mutation coverage checks, move them to a quarantine location and block merge until a human approves.

Decision criteria (set thresholds up front)
- Determinism: pass rate >= 99% over N=20 repeats = OK; pass rate < 95% = quarantine.
- Coverage & assertions: tests must assert meaningful state (not only console/log), cover relevant code paths, and survive basic mutation testing (optional; high-value repos only).
- Resource usage: any test that opens network, DB, or filesystem in non-mocked form → quarantine unless marked with approved integration-test tag.
- Security: secrets usage, spawning shells, or writing to production paths → block and require immediate manual review.

Practical checklist to implement in CI
1. Tagging: require generated tests include a single-line provenance tag in header or PR body.
2. Pre-checks: run linters, security SAST scan, and search for risky patterns (network calls, exec, env access).
3. Determinism run: run new tests 20x in a clean container; record flaky failures.
4. Heuristics: flag tests with <2 assertions, only snapshots, or randomized data without seeds.
5. Quarantine action: move flagged tests to /tests/quarantine or add @quarantine decorator; add label on PR and post bot comment with remediation steps.
6. Human gate: require codeowner + test-owner approval (branch-protection rule) to move quarantined tests back to main; reviewer must annotate why test is safe.
7. Optional hardening: run mutation testing (Stryker/Mutant), sandbox exec, and run in parallel on ephemeral infra.
8. Metrics & telemetry: track quarantine rate, root causes, and time-to-approve to tune thresholds.

Best-for / Avoid-if
- Best for: teams that frequently accept AI-generated tests and have CI infra and a test-owner workflow. Works well when you can run repeatable isolated test runs in CI.
- Avoid if: tiny teams without CI capacity, or when rapid prototyping matters more than long-term test hygiene—start with manual review until you can automate.

When the right answer depends
- Budget: mutation testing, repeat runs, and sandboxing cost compute. Lower budgets can reduce repeats and rely more on heuristics + manual review.
- Skill level & team size: small teams should start conservative (require review) and add auto-approve thresholds as confidence grows.
- Workflow stage: early-stage projects may prefer stricter quarantine; mature projects can relax thresholds and auto-merge safe tests.

If you use GitHub Copilot to generate tests, add a small provenance header and a bot that enforces the tag so CI can identify and apply these rules automatically (see GitHub Copilot docs for best practices).

Compare GitHub Copilot and Cursor