I played with GitHub Copilot and compared it to simple "Googleing" code snippets to see what is more secure. Please see the video below for a quick recording of the experiment.

The task I selected was pretty trivial: A PHP script/page to collect data from an import form and insert it into a SQL database. There were two specific challenges I looked for:

• Cross-site Scripting: I wanted the data the user entered to be "prefilled" into the form as it is returned to the user
• SQL Injection: The data should be inserted into a SQL database

Both Copilot, as well as the "Google" solution, ignored the XSS issue. As far as SQL injection went, Copilot did use prepared statements, which is nice. The code was not as "clean" as I would have written it, but not that my code is always that great. When specifically asked to, Copilot did escape the data to avoid XSS.

The "Google" solution came from a random PHP tutorial with SQL injection and XSS vulnerabilities. In that sense, Copilot was better.

One important issue I noticed when using Copilot is that the code it comes up with varies in quality. Some of this may also be related to how "busy" Copilot is, as sometimes it responds slowly or not at all.

I also experimented with some other input validation with Copilot, which went okay if the prompt was done correctly.

(This is not a secure coding tutorial but just a quick experiment.)

---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
Twitter|