GitHub put AI security findings in the pull request. Advisory is not a gate

THE SIGNAL IN ONE SENTENCE

A security warning can now arrive in the same place where developers argue about variable names, approve changes and press the merge button. That is the useful part of GitHub's October 15 release. AI Scan for pull requests is now generally available. When code scanning and the repository's AI Scan setting are enabled, GitHub can inspect eligible pull-request changes in languages and frameworks that CodeQL does not currently cover. The examples GitHub names include PHP, Shell and Bash, Terraform configuration, Dockerfiles, JSP for Java and Blazor for C#. The finding appears in the pull request beside ordinary CodeQL results. It carries an AI label, explains the suspected risk and, in many cases, proposes a repair. The awkward part is written just as plainly in GitHub's documentation. AI Scan findings are advisory. They do not fail the code-scanning check. They cannot satisfy a ruleset requirement that blocks a merge. The machine can point at the smoke. It does not close the fire door. GitHub released a second generally available system on the same day. Agentic autofix takes a backlog code-scanning alert on the default branch, explores related files, proposes a change, reruns CodeQL and opens a draft pull request for review. A team can assign one alert or select as many as 25 alerts for one agent session and one proposed repair pull request. Put the two releases together and a sensible loop appears. One system watches new pull requests in parts of the codebase that traditional CodeQL coverage may miss. Another can investigate eligible alerts already sitting on the default branch and prepare a repair that crosses several files. Both move security work closer to the ordinary development workflow. That is progress. It is not a gate. AI Scan runs when a qualifying pull request is created and after each new commit. It does not require CodeQL default setup, and it can continue even when CodeQL analysis fails or waits. The scanner may use code search to collect context from the repository before it raises a finding. It uses GitHub's specialized prompts rather than repository instruction files such as copilot-instructions or CLAUDE.md. The scope is deliberately narrower than the phrase AI security review might suggest. AI Scan analyzes pull requests, not an entire repository. Its findings do not become backlog alerts in the repository's security view. It does not run on pull requests from forks. It does not run on pull requests created by Dependabot. It targets selected languages and frameworks outside current CodeQL coverage, and GitHub says those supported areas and detection categories can change. The current categories include string injection, weak cryptography, broken access control, sensitive-data exposure, security misconfiguration, authentication failures, data-integrity failures, server-side request forgery and supply-chain risks. That is a broad and useful checklist. It is not proof of complete coverage. An organization can now see trends in AI Scan findings, resolutions and the availability or acceptance of suggested fixes through GitHub's security overview. The AI Scan metrics view can be filtered by time and repository. Unlike the CodeQL view, GitHub says CSV export is not yet available for the AI Scan tab. That small reporting limitation matters more than it sounds. A security team needs to move beyond counting how many warnings appeared. It needs to know which classes of defects escaped, how many findings were useful, how many were noise, which teams dismissed them, how long remediation took and whether an accepted patch later caused a regression. A dashboard inside one vendor is a view. A durable assurance program needs an exportable record. Agentic autofix solves a different problem. It starts from a code-scanning alert on the default branch. A person with the required access assigns the alert to Copilot. The agent opens an investigation, follows references into related files, creates a proposed repair and reruns analysis. It then opens a draft pull request with a summary, the reason it believes the alert is closed and the validation steps it performed. The draft status is important. It is the product admitting that a generated patch is a proposal, not a verdict. GitHub's own documentation says the validation is best effort. The agent can rerun CodeQL with the configured query suite, but it cannot confirm fixes for alerts from custom CodeQL queries, the security-extended query suite or third-party scanning tools. It may also decide that it cannot validate a fix or that the original alert might be a false positive. Even when CodeQL stops reporting the original alert, the change can still be wrong. A patch may silence the scanner while breaking behavior. It may move unsafe data into a different path. It may narrow a check so the known test passes but a neighboring input still fails. It may introduce a performance problem, change authorization semantics, mishandle an error path or create a fresh vulnerability outside the original query. Passing the detector that named the problem is necessary evidence. It is not independent evidence. This is where ordinary software engineering earns its keep. The generated pull request should run the repository's normal unit, integration and end-to-end tests. Security-sensitive changes need a regression test that fails on the vulnerable behavior and passes on the repair. A human reviewer should inspect the changed data flow, permission boundary and failure path. High-risk repositories may require a security owner, code owner or separate approver who did not request the fix. The review should also inspect the agent session log. If the agent searched several files, ignored one configuration or made an assumption about how a value reaches the vulnerable function, that reasoning belongs beside the diff. The patch is easier to trust when the search trail is visible and easier to reject when the trail skipped the dangerous branch. Organizations also need to decide what an AI Scan warning means before the warning arrives. Because AI Scan cannot directly block a merge through a ruleset, a team can build an explicit human gate around it. A critical suspected injection flaw might require a security reviewer and documented disposition. A lower-confidence warning might create a tracked issue with a deadline. Repeated false positives in one pattern might trigger a tuning review rather than a ritual dismissal. The important thing is that the response is designed, not improvised under deadline pressure. A finding should have a stable identifier, severity, affected lines, ownership, status, rationale and final disposition. If a team merges despite the warning, record who accepted the risk and why. If the finding is wrong, keep enough evidence to improve the scanner and recognize the same pattern later. Do not turn the advisory label into an excuse to ignore the result. Do not turn the AI label into an automatic stop sign either. GitHub excludes fork pull requests and Dependabot pull requests from AI Scan. Those are not obscure corners. Forks are a normal path for open-source contributions. Dependabot exists specifically to change dependencies, a part of the software supply chain where security matters a great deal. Teams need separate controls for those lanes. Existing CodeQL coverage, dependency review, secret scanning, package-policy checks, sandboxed tests and human review do not become optional because a newer scanner is absent. A dashboard should make excluded pull requests visible so nobody mistakes silence for a clean bill of health. Billing creates another failure mode. AI Scan consumes Code Security AI Credits. GitHub says that if a hard budget leaves no credits available, the system cannot produce findings until the budget changes or a new billing cycle begins. Agentic autofix consumes Code Security AI Credits and GitHub Actions minutes. A depleted budget can therefore look like a quiet week. Security owners should monitor credit use, set alerts before the hard stop and surface coverage loss inside the development workflow. If a pull request was not scanned because the budget ran dry, that absence should be visible to the reviewer. A missing result is not a passing result. Access controls matter as well. Enterprise owners or security managers must allow agentic autofix. Organization or repository administrators then enable it, and users with write access can assign eligible alerts to Copilot. This creates a reasonable chain of authority, but organizations should still ask what the agent can read, which actions it can perform, how its branch is isolated and who can request another iteration through a pull-request comment. The agent should receive enough access to investigate and propose. It should not inherit an invisible shortcut around protected branches, required checks, code owners or deployment approval. The best use of these systems is not to declare that security review has been automated. It is to move more evidence into the place where software decisions already happen. AI Scan can catch a suspicious shell interpolation or insecure Dockerfile change before it disappears into the default branch. Agentic autofix can turn an aging alert into a concrete diff instead of another red number in a backlog. Both can reduce the distance between a security team and the developer who understands the surrounding code. The practical scoreboard should be unsentimental. Measure confirmed vulnerabilities found before merge. Measure false positives and dismissed findings. Measure alerts missed by AI Scan but caught later by another tool or an incident. Measure the fraction of generated patches that pass independent tests without major human rewriting. Measure time to review, time to repair, reverts and defects introduced by the repair. Break the results down by language, framework, repository and severity. GitHub has published product behavior and explicit limitations. It has not published independent false-positive, false-negative or repair-quality results for every supported language and framework. Teams should treat the general-availability badge as permission to evaluate the tools in production, not as borrowed evidence that the tools are already reliable in their own codebase. Put the warning in the pull request. Let the agent prepare the patch. Keep the branch protections, tests, security review and human merge authority exactly where they belong. Advisory is a useful signal. A gate is an accountable decision.