Short answer / recommendation
Run a dedicated GitHub Action (or CI job) that: (a) computes a minimal diff, (b) scrubs secrets locally, (c) chunks and condenses file-level changes, and (d) sends only the condensed text to an LLM via a tightly scoped/ephemeral credential or a self‑hosted model. Make the check non-blocking (draft PR description or comment) or run a very small fast model for a quick draft and queue a full summary asynchronously to avoid slowing builds.

Why this works
Sending full repos or raw diffs leaks secrets and is slow. Summarizing locally first reduces payload, protects secrets, and lets you control which text reaches the model. Using ephemeral/OIDC tokens or self-hosted LLMs eliminates long-lived keys and external data exposure.

Decision criteria (how to choose implementation)
- Security priority: If you must avoid external providers, self-host a model (requires infra/budget). If moderate risk, use a managed API with ephemeral keys + strict scrub rules.
- Budget and latency: Managed APIs are faster to set up; self-hosting costs more but avoids egress. If CI speed is critical, run small-model quick drafts in CI and full summaries offloaded to an async worker.
- Team size / workflow stage: Small teams: managed API + GitHub Actions. Large/regulated orgs: self-hosted model + VPC-restricted runner.

Practical checklist (concrete steps)
1) Trigger and gating
- Run on PR opened/updated, or a nightly job for CHANGELOG generation. Make the status check optional or non-blocking for fast iteration.
2) Compute a minimal, relevant diff inside the runner
- Use git diff origin/main...HEAD (or whatever your base is). Limit to tracked source files and skip binaries/large assets.
3) Local scrub and reduce
- Apply regex scrubs BEFORE any network call. Examples to drop: AKIA[0-9A-Z]{16}, ghp_[A-Za-z0-9_]+, any -----BEGIN (RSA|PRIVATE) KEY----- blocks.
- Drop exact paths (.env, secrets/, build folders) and skip generated code.
- If diff is large, summarize per file (e.g., file header + changed function names + changed line snippets) and then merge summaries.
4) Chunk + prompt
- Chunk per file or per 1–2k tokens. Ask the LLM to output: 1-sentence PR summary, 3-bullet changelog, risk/impact notes, and suggested PR title.
5) Secure LLM access
- Preferred: use OIDC from GitHub Actions to mint ephemeral tokens where supported, or pass a GitHub secret scoped to the action with repo:contents read-only. For maximum safety, call a self-hosted LLM behind your network.
6) Post results
- Insert draft into PR body (PATCH via GitHub REST/GraphQL) or create a comment. Mark “auto-generated” and provide an edit sanity check for authors.
7) Performance & cost optimizations
- Quick path: use a lightweight model for a draft in <10s, then enqueue a higher-quality summary. Cache summaries by commit hash to avoid redoing unchanged diffs.

Best-for / Avoid-if
- Best for: teams that want consistent, reviewable PR descriptions and automated CHANGELOGs with minimal human overhead. Good for small-to-medium diffs.
- Avoid if: your repo contains many sensitive artifacts or regulation forbids external inference; prefer self-hosting in that case.

Extras (small safety rules)
- Never send entire repo or secrets file contents. Log only hashes and filenames in CI logs. Add a human review step before marking auto-generated text as “final.”

If you want, I can provide a compact GitHub Action skeleton (diff extraction + scrub + example prompt) or a sample scrub regex list. For LLM choice, managed ChatGPT-style APIs are fastest to implement; self-hosting fits high-security/budget teams.

Compare ChatGPT and Gemini