Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions

GitHub Copilot, released in preview in June 2021, suggests code from the surrounding context. Its model was trained on public code, which includes insecure patterns, and its documentation warned that suggestions can contain them. This paper measures how often that happens in situations where a security mistake is easy to make.

The authors wrote short program prefixes that stop at the point where a security-relevant line comes next, such as building a database query, copying into a buffer or hashing a password. Each prefix targets one weakness from MITRE's 2021 list of the 25 most dangerous software weaknesses (CWEs). For each, they asked Copilot for up to 25 suggestions, kept those that formed a valid program, and checked each one with GitHub's CodeQL scanner or by hand. They counted a suggestion as vulnerable only when the weakness was definitely present, so partial completions were scored as safe.

They tested three axes. Diversity of weakness covered 18 CWEs with 3 scenarios each in C and Python: 477 of 1,084 valid programs were vulnerable (44%), and the top-ranked suggestion was vulnerable in 24 of 54 scenarios. Diversity of prompt took one SQL task and changed the prompt in 16 small ways, such as rewording comments, adding an author name or adding nearby code: 152 of 407 were vulnerable. Diversity of domain asked for hardware descriptions in Verilog for 6 hardware CWEs: 56 of 198 were vulnerable, and Copilot often failed to produce valid code.

Across all 89 scenarios, about 40% of suggestions contained the weakness. Some weaknesses were far more common than others: NULL pointer dereference and OS command injection appeared in more than 80% of suggestions for their scenarios. The clearest prompt effect was the code already present: with a safe database function nearby, none of 18 SQL suggestions was vulnerable; with an unsafe one, 17 of 18 were. The authors also note outdated practices, such as weak password hashing, persisting from older training code.

Generated code needs security review like any other code. About 40% of suggestions in these risky spots had a known weakness, and the top suggestion was vulnerable in 39% of scenarios. When a customer team adopts a coding assistant or an agent that writes code, keep static analysis, dependency checks and review in the path. Do not let the speed of generation remove them.

The surrounding code sets the pattern. The strongest effect in the study was the code already in the file. If a codebase builds queries by string concatenation, the assistant will copy that. Fixing the existing patterns, and putting safe helper functions where the assistant can see them, is a better lever than prompt wording.

Prompt wording is mostly noise at this sample size. Adding "in a secure manner" or a note that SQL injection was fixed did not reliably help. In our recount, only the unsafe-neighbor case clears chance after allowing for 16 comparisons. Do not promise a customer that comments will make generated code safe.

Some weakness classes are much riskier. Memory-safety bugs in C, OS command construction and credential handling were the worst. Point scanners and reviewers at those first in generated code.

This also applies to agents. Code agents that write and run code at scale produce the same patterns faster. Put the security scanner inside the agent's loop or the merge gate, not after deployment.