Troubleshooting Copilot hallucinations in large repos

Short answer / recommendation
Start by forcing the assistant to only see the current, authoritative API surface and then make generated edits gated by fast, targeted tests. Practically: (1) reduce Copilot’s context to relevant files and index the current API surface, (2) add automated lint/CI rules that reject use of deprecated APIs, and (3) require generated patches to run only the minimal affected tests before merge.

Why this works
Hallucinations happen when the model’s context includes stale code or when the model guesses an API instead of reading the authoritative source. If you both tighten context (so suggestions come from current files) and add fast verification (so bad suggestions are caught before merge), you greatly reduce real-world breakage.

Decision criteria (pick what fits your org)
- Small team / limited budget: prefer in-repo measures (JSDoc @deprecated, ESLint rules, pre-commit hooks, jest --findRelatedTests) — lower cost, fast win.
- Large monorepo / high change velocity: invest in code indexing (LSIF/Sourcegraph) + dependency-graph-aware CI (nx/bazel) so the assistant and CI can reason about affected packages.
- Tight latency requirement (fast CI): create a targeted test tier that runs only impacted tests on dev machines and in PR gates.

Concrete steps / checklist (implement in this order)
1) Make current APIs authoritative
- Add @deprecated annotations to old APIs (JSDoc/@Deprecated/TS deprecate tags) so IDEs and linters flag them.
- Create a small, committed “API index” (JSON/YAML) mapping current public entry points; used by linters or as an extraction target for assistants.
2) Limit assistant context to authoritative files
- When using Copilot in the editor, open the files that define the current API (and avoid opening huge old branch folders). The assistant primarily uses visible workspace context.
- If you have budget: set up a code-search/indexing tool (LSIF/Sourcegraph) so developer queries return the latest symbols.
3) Add automated prevention rules
- ESLint/TSC rule to error on imports from deprecated modules (example: ban import path pattern or specific exported names).
- Pre-commit hook (husky + lint-staged) to run the lint rule and quick unit check of changed files.
4) Require tests for generated patches
- Developer pre-check: run quick, targeted tests locally. For JS/TS use: npx jest --findRelatedTests or jest --changedSince=origin/main. For monorepos with nx: nx affected --target=test --base=origin/main --head=HEAD.
- CI: gate PRs so that the CI runs only the “affected” test set (faster) and fails PRs that introduce deprecated API usage.
5) Feedback loop
- When a Copilot suggestion uses a deprecated API, file a small codemod or add explicit lint suppression messages that cause Copilot to avoid that pattern next time (and update the API index).

Best-for / Avoid-if
- Best-for: teams with many long-lived branches and a monorepo where fast feedback is critical. Works well if you can add automated linting and targeted CI.
- Avoid-if: tiny repos where the overhead of indexing/LSIF is heavier than the problem—there, rely on simple JSDoc + test gating.

Practical examples & enforcement
- Quick local test: git add -A; FILES=$(git diff --staged --name-only); npx jest --findRelatedTests $FILES
- CI: use nx/bazel to compute affected tests, or run jest --changedSince=origin/main. Fail PRs that don’t run the required affected tests.

If you want, I can draft a sample ESLint rule and a husky pre-commit script tuned to your monorepo layout.

Tool note: If you use GitHub Copilot, keep only the current branch/workspace files open and pair the above rules with local indexing for best results.

Compare GitHub Copilot and Cursor