Can Copilot Fix Its Own Security Findings? Testing GitHub Agentic Autofix

Can Copilot Fix Its Own Security Findings? Testing GitHub Agentic Autofix GitHub can now assign a code scanning alert directly to Copilot cloud agent. The agent explores the repository, changes code, attempts validation, and opens a draft pull request. That is more ambitious than generating a patch beside one vulnerable line. It is also not the same as proving that a vulnerability is gone. The practical answer to the title is therefore: Copilot can attempt end-to-end remediation, but the strength of the evidence depends on the scanner and your CI controls. A human still owns the security decision. This deep dive goes beyond my earlier introduction to Copilot coding agent. Rather than surveying general coding tasks, it examines one narrow DevSecOps question: how should a team test, review, and govern agentic autofix without mistaking automation for assurance? Evidence boundary: Agentic autofix is a paid public preview. I have not invented a successful run, timing result, credit charge, or vulnerability closure. The experiment below is a reproducible evaluation design for an eligible sandbox. Product behaviour is grounded in GitHub's 10 July 2026 announcement, its 16 July clarification, and linked GitHub documentation. What GitHub Announced, and What It Clarified On 10 July 2026, GitHub announced agentic autofix for code scanning alerts in public preview. The original workflow is straightforward: - Assign one or more code scanning alerts to Copilot. - Copilot cloud agent inspects relevant files across the codebase. - It proposes a fix and attempts to validate it. - It iterates when necessary. - It opens a draft pull request containing the changes, an explanation, and validation details. GitHub says generation typically takes two to four minutes, but that is product guidance, not a result measured in this article. The editor's note added on 16 July 2026 matters. It clarified that assignment works for all first-party and third-party code scanning alerts. In other words, eligibility is not limited to CodeQL findings. Alerts uploaded by integrated tools through SARIF can also be assigned. That does not make all validation equal. GitHub's detailed autofix documentation says agentic autofix re-runs CodeQL using the code-scanning query suite on a best-effort basis. It cannot confirm alerts from custom CodeQL queries or the security-extended suite through that validation path, and fix quality for third-party alerts is not guaranteed. This distinction is the centre of a sound evaluation: | Question | Answer in the preview | |---|---| | Can the alert be assigned? | Yes, for first-party and third-party code scanning alerts. | | Can Copilot edit multiple files? | Yes. The cloud agent explores the wider codebase. | | Does GitHub open a pull request? | Yes, normally a draft pull request with a summary and validation notes. | | Is the originating finding always re-proven as closed? | No. Built-in validation has scanner and query-suite limits. | | Is the pull request ready to merge without review? | No. Validation, behaviour, dependencies, and risk still require human review. | Do not extend the claim beyond code scanning. Dependabot and secret scanning are separate alert families with their own remediation workflows. The Agent Loop Is the Real Change Classic Copilot Autofix translates alert context into a single suggested change. Agentic autofix delegates a task to Copilot cloud agent, which can use repository context and tools over several iterations. The loop is roughly: alert context -> inspect related code and repository instructions -> form a remediation plan -> edit one or more files -> run available analysis and tests -> inspect failures or remaining findings -> revise the change -> open a draft pull request for review Repository and organisation custom instructions apply while the agent works. That makes existing engineering guidance part of the control plane. A repository that tells agents to run focused tests, avoid new dependencies, and preserve public APIs gives the remediation loop better constraints than a repository with no executable acceptance criteria. Validation must still be read scanner by scanner: - CodeQL code-scanning suite: Copilot can re-run CodeQL and use the result to iterate. - Custom or security-extended CodeQL queries: the documented built-in re-run does not prove those alerts are resolved. Re-run the repository's actual advanced setup in PR CI. - Third-party SARIF alerts: assignment is supported, but the scanner's own PR workflow remains the authoritative check. A clean CodeQL result does not close a Semgrep, Snyk, Checkmarx, or other third-party finding. - Functional behaviour: no static analyser proves that the application still meets its requirements. Unit, integration, contract, and negative security tests remain essential. The agent may also report that it could not validate a fix or that an alert appears to be a false positive. That is useful evidence, not an invitation to auto-dismiss the finding. Four Ways to Assign Work GitHub exposes the same remediation idea through four entry points: - Individual alert UI: open a code scanning alert and choose Assign to Copilot. - Alert backlog batch: select between 1 and 25 alerts and assign them together. Copilot works on the selected set in one pull request. - Security campaign: select between 1 and 25 campaign alerts for one pull request. This fits an organised remediation deadline, but it increases review and rollback coupling. - REST API: update one code scanning alert and set assignees to['copilot-swe-agent[bot]'] . The repository includes a safe PowerShell REST helper. Preview its target without making a request: ./code/assign-alert-to-copilot.ps1 -Owner 'acme-security' -Repository 'agentic-autofix-lab' -AlertNumber 42 -WhatIf For an authorised run, set GITHUB_TOKEN in the process environment and omit -WhatIf . The helper sends: PATCH /repos/acme-security/agentic-autofix-lab/code-scanning/alerts/42 Accept: application/vnd.github+json X-GitHub-Api-Version: 2026-03-10 {"assignees":["copilot-swe-agent[bot]"]} The Update a code scanning alert endpoint returns 200 OK on success. Classic personal access tokens need security_events for private or public repositories, or public_repo for public repositories only. Use the least-privileged supported token for your automation and never place it in the script, command history, or repository. Start with one alert. A batch saves orchestration time, but it makes attribution harder: which edit fixed which alert, which check failed, and which change should be reverted? Batch only alerts that share a component, scanner, owner, and rollback unit. Licence, Policy, and Cost Boundaries The preview requires both sides of the product boundary: - An active GitHub Code Security or GitHub Advanced Security licence. - A GitHub Copilot licence with Copilot cloud agent enabled. - Copilot Autofix available in the repository. Administrators can disable Copilot Autofix at enterprise, organisation, or repository level. Because agentic autofix relies on that setting, disabling classic Autofix also blocks the agentic experience. Administrators can separately opt repositories out of Copilot cloud agent; when the cloud agent is unavailable, assigning an alert falls back to classic Copilot Autofix instead of starting an agentic session. An enterprise policy set to not allowed cannot be overridden lower in the hierarchy; allowing it merely lets organisation and repository administrators choose. The billing model is materially different from classic Autofix: | Route | Copilot licence | AI Credits | Actions minutes | Output | |---|---|---|---|---| | Classic Copilot Autofix | Not required | No | Not for generation | One suggested fix to review and apply | | Agentic autofix | Required | Yes | Yes | Iterative agent session and draft PR | | Manual remediation | No | No | Normal CI usage | Engineer-authored change and PR | During public preview, AI Credits are consumed only when a fix runs on alerts assigned to Copilot. The announcement says this usage is not itemised separately from other Copilot activity during the preview. Agent activity also consumes GitHub Actions minutes. Do not promise a fixed cost per alert: session depth, batch size, tests, runner type, and retries can change consumption. Set a pilot budget before enabling broad campaigns. Track assigned alerts, agent sessions, Actions duration, PR outcome, and human review time together. A cheap generated patch that creates an expensive review queue is not a successful control. A Reproducible Sandbox Evaluation Use an organisation-owned sandbox that has the required licences and policies. Do not test with production secrets, customer data, or a live deployment path. 1. Prepare representative findings Create three small, intentionally vulnerable examples with deterministic tests: - A CodeQL alert from the documented code-scanning suite. - A CodeQL alert produced only by your custom or security-extended configuration. - A third-party alert uploaded as SARIF by the scanner you actually use. Keep each case on the default branch, record the alert number and tool, and ensure the scanner also runs on pull requests. Add functional tests that fail if a simplistic security patch changes intended behaviour. 2. Establish the baseline Before assignment, record: - Commit SHA, scanner version, query suite, rule ID, severity, and alert URL. - Passing and failing test results. - Existing alert count for every enabled scanner. - Repository instructions and agent setup workflow revision. - The expected secure behaviour, including negative test cases. Without that baseline, an alert disappearing could mean a real fix, a changed path, a scanner configuration change, or stale analysis. 3. Run isolated assignments Assign one alert at a time through the UI. Reserve the API helper for a second pass after the manual flow is understood. Capture the agent session log, draft PR, changed files, expla