I Was Out-Hunted by My Own AI: Where Human Researchers Still Win and How to Make It Count

Every smart contract team I talk to is already running AI against their code. Pasting a repo into a model and asking it to find bugs costs close to nothing, so everyone does it.

Then they reach for one of the cheap scanners, pick one more or less at random, run it, and patch what comes back. The benchmarks meant to show which tool is superior are mostly noise, and teams know it. They try to get as much coverage as possible to feel like they’ve put an effort into using AI to secure their codebase.

Meanwhile, a fresh exploit lands in my feed nearly every morning. More and more of them bear the unmistakable signature of attackers who are also leveraging AI.

So the question I hear, in some form, almost every week is: Will AI replace security researchers?

I have a more nuanced answer than the two extremes you usually hear. One camp claims we’re all doomed. The other dismisses AI as mostly slop.

The truth I’ve lived lies between them. I spent five years hunting bugs manually before co-founding Grego AI. I have watched the tool surpass me in areas where I excelled, and I’ve seen it fall short in others. That gap is precisely where the future of professional security research is headed.

Hunting is not the same as auditing

For five years, hunting was my full-time job. I spent my days trying to break code that other experts had already audited. That experience taught me the sharp distinction the industry often blurs.

Auditors review code to verify it behaves as claimed, within a fixed time and budget, while communicating regularly with the paying client. Bounty hunters face no such constraints, rules, or oversight. Their sole goal is to find any way to break the system using a hacker/red-team mindset rather than a blue-team approach. There are no “out of scope” limitations and if the bug is valid- it’s valid.

When the first wave of LLM-powered audits appeared, I tested them and quickly wrote them off. Most produced confident-sounding but unusable results.

Then my co-founder, Greg Maspero, showed me a system he had built that could uncover bugs in live, heavily audited deployments (bugs I would have been proud to discover myself). That moment completely changed my view of AI.

At the heart of Grego AI is our proprietary Deep Invariant Analysis. It maintains the logic of a large system across many interacting layers simultaneously (including external dependencies) and traces those interactions far deeper than any human can hold in their head.

When Grego AI suspects a bug, it generates a proof-of-concept exploit and attempts to execute it. What you receive is a reproducible exploit you can run and verify yourself.

Where the tool already beats me

Breadth. I get tired like every human. On a large codebase, my attention span becomes the bottleneck. Grego AI examines every path with consistent intensity, no matter how long it runs.

Depth across dependencies. The bugs that survive thorough audits are rarely superficial. They hide five, six, or seven layers deep, often at the boundary between a protocol’s own code and a dependency imported years earlier. Keeping that entire chain in your head exceeds the human cognitive limit, and it’s exactly what Grego AI was built to handle.

Combinations. Vulnerabilities come in many classes, each with distinct patterns. Most human-discovered bugs combine at most two classes. Grego AI, however, cross-references the entire known vulnerability landscape (including obscure edge cases) to identify the most damaging combinations and maximize impact.

One team I worked with had paid a top-tier firm for a manual audit, ran two separate AI auditors, and patched everything. When we scanned it, we surfaced a stack of new medium-severity issues on top of all that.

Most “AI scanners” offer low-cost, high-level cursory checks that look impressive on the surface but lack the depth of a serious human audit. We designed Grego AI specifically to uncover the latent, needle-in-a-haystack bugs that can destroy a protocol in a single transaction.

Give AI the grunt work, keep the judgment

The most important question in security research today is this: How do you extract maximum value from AI while keeping it away from the tasks it still struggles with?

Security researchers pulling ahead have all made the same strategic shift. They direct AI toward the tireless, high-breadth portions of the hunt and reserve their own time for areas that demand human judgment. In the age of AI, the most valuable contribution a human can make is asking the right questions.

Run the tools before you touch anything. There’s no longer a reason to bounty hunt cold on a fresh codebase. Let the system clear the surface issues and map the architecture first. Breadth has become cheap. Focus your attention on depth.

Mine false positives instead of discarding them. The reasoning behind a mistaken finding often points directly to a real issue nearby. Better tools fail in interesting ways: they explore paths cheap scanners never consider and chase shapes that are genuinely suspicious even when the final exploit doesn’t materialize. Learn to read these signals and develop a system for separating gold from noise.

Own the part only a human truly understands: the protocol’s purpose. AI sees code and predicts the next token. It has no concept of what the protocol is meant to do that is not in its context window. It may flag an intentionally unset slippage parameter or mislabel a deliberate design tradeoff as a front-running vulnerability. The mechanic is real. Recognizing it as intentional requires someone who has not only read the documentation but also understood the team’s intentions.

Teach it your protocol as you go. AI should learn from your codebase. Every time you mark a finding as intentional and explain why, a strong system carries that context forward and stops re-flagging your design decisions. This is the direction we’re building Grego AI toward: a true Security Operating System that grows sharper with every run.

Leverage your instincts with AI. Tracking down leads once took hours or days. Now you have an expert auditor on your shoulder that can rapidly validate or invalidate hunches on the fly.

The part that doesn't change

AI tools will continue to improve rapidly. Yet the fundamental approach to the work remains the same, whether you hunt manually or with an AI system: distrust what the code claims to do, start from the assets an attacker would target, and follow the interactions past the point where everyone else stopped looking.

That instinct is the one element that transfers no matter how powerful models become. It’s a skill worth mastering.

We built Grego AI on the belief that a machine could handle the deepest parts of this work better than I ever could. It has proven me right, and that’s the good news. However, until we can guarantee 100% security, there will always be some obscure edge case that either a human or an AI missed, which means there will always be room for bounty hunters.

Check out what Grego AI can find in your codebase: https://scan.grego.ai