"Please perform a comprehensive security audit" - and why it doesn't work AiSafe - AI-Powered Security Platform Code Audit is live! Try it now Source Code Audit is now live! Start your first assessment today Blog "Please perform a comprehensive security audit" - and why it doesn't work How Claude Code or Codex won't secure your application, and why AISafe found 7 CVEs in a file hosting app where they couldn't. Šimon Šustek March 16, 2026 Fineas Silaghi March 16, 2026 Introduction Hi there! We are AISafe Labs, a crew of security researchers with only one goal: making security accessible to everyone! We've all witnessed the recent rise of LLMs that has been nothing short of revolutionary. We're reaching a point where the power of a single prompt surpasses any expectation. From generating entire codebases to replacing entire workflows , the bar keeps getting raised. Building this much trust in the power of a prompt naturally raises the question: "Can you secure your application with a single prompt ?" In this article, we put that question to the test. We will assess how far a single prompt can take you when it comes to securing a real application, then compare that to a more targeted prompt armed with domain knowledge and specific instructions, and finally, stack both against AISafe, our specialized source code auditing platform. Same codebase, same goal, will the difference be significant or negligible? Product Findings Vulnerabilities with direct impact Weaknesseses without direct impact Accepted risk False positives Claude Code 24 1 8 11 4 Claude Code w Skills 21 0 5 2 14 Claude Code /security-review 1 0 1 0 0 Codex 4 2 2 0 0 Codex Security 44 6 18 8 11 AISafe 20 9 4 7 0 Let's begin! 🍿 The Target Gokapi is an open source file hosting application with around 2.7k stars on GitHub, 1.5M Docker pulls, and roughly 35k lines of code ( Go & JavaScript ). Preview of Gokapi app It sits in a sweet spot for our case study: large enough to present a complex threat model and with modern web app features such as file handling, API keys, and user permissions with different access levels, yet small enough to audit in a reasonable time. All experiments were run against commit a7c4273b819b8a48f85b866e1803632c089f60a2 , pushed on March 1st, 2026. Anthropic Claude Code Our first candidate is Claude Code , running Opus 4.6 with high effort. We are going to start with the most simple setup possible: a single prompt , just the kind that it's being recommended on social media every other day. To begin, we drop Claude Code into the Gokapi folder, and type "Please perform a security audit of this repository" . It will spin up a few agents and start looking at different parts of the codebase: A few minutes later, we are already looking at 24 findings 👀 4 criticals ? Let's break them down! The first two are "SHA-1 Password Hashing" and "Zero-Filled Nonces in AES-GCM Encryption". While Gokapi does use SHA-1 for password hashing, and sure, that's definitely not a great idea, still, it is a weakness rather than an exploitable vulnerability. A Low severity finding at best. The "zero-filled nonce" finding sounds scary until you take a closer look and realize that the nonce gets handed off to the sio-go library, which appends its own nonce on top, and the server also uses per-file keys. Again, the real-world impact is practically none . The other two critical findings flag "XSS in templates" and an "open redirect". Unfortunately both are false positives : the templates rely on Go's html/template package which handles sanitization out of the box, and on the other hand the redirect URL is only ever set by the instance admin. Four critical vulnerabilities reported, but no real security issues were identified. Moving on to the high impact findings. Three of the seven are about missing cookie attributes and security headers like HttpOnly , Secure , SameSite , Content-Security-Policy , and X-Frame-Options . Not a bad report, but again, not really vulnerabilities either. Another finding flags missing CSRF protection, which has had generally pretty limited impact ever since SameSite=Lax became the default . Finally, there is one finding that points us to something actually vulnerable: a "Header Injection" issue, where file.Name is potentially attacker-controlled: w . Header ( ) . Set ( "Content-Disposition" , "attachment; filename=" " + file . Name + "" " ) However, the impact is overstated. Go validates headers and blocks CRLF injection, so the possible corruption is limited only to the header. Lastly, the final two high impact findings describe header-based auth, which is intentional by design, and a missing TLS configuration that turns out to be another false positive. The mediums are all theoretical with no meaningful real-world impact. Full report CRITICAL 1. SHA-1 Password Hashing internal/configuration/Configuration.go:224-236 Passwords are hashed with SHA-1 + salt, which is cryptographically broken. SHA-1 can be cracked at billions of hashes/second on commodity GPUs. hash := sha1 . New ( ) hash . Write ( pwBytes ) return hex . EncodeToString ( hash . Sum ( nil ) ) Fix: Migrate to bcrypt or Argon2id. 2. Zero-Filled Nonces in AES-GCM Encryption internal/encryption/Encryption.go:156, 168, 195, 202, 209, 216 Nonces are always initialized as zero-filled byte arrays. Reusing a nonce with the same key in GCM mode completely breaks confidentiality and authenticity. nonce := make ( [ ] byte , stream . NonceSize ( ) ) // always zeros Fix: Generate random nonces via crypto/rand.Read() . 3. XSS via Unescaped Filenames in Templates Multiple templates embed user-controlled filenames without proper escaping: File Context web/templates/html_redirect_filename.tmpl:5-16 , tags web/templates/html_redirect_filename.tmpl:24,29 JavaScript string and web/templates/html_download.tmpl:25,126 HTML