Penetration Testing Isn’t Just Scanning—Nor Is It Hacker Mode

Penetration testing services have evolved a lot over the years. Speaking from experience — roughly two decades' worth — I’ve seen the shift firsthand.

I still remember what the CEO, Víctor Chapela, at my first job used to say: companies hired us like we were some kind of luxury item. They had no idea what we were actually for. And back then, there were no regulations forcing organizations to conduct penetration tests at all.

There were no formal definitions, scopes, or scenarios. They’d give us a network cable, a chair, and basically say: “Hack me.”

And the results? Always impressive. If we found juicy stuff (and we always did), we were top-notch hackers who had just breached the defenses of a major national or international corporation. If, by some miracle, we didn’t find anything (which never happened), we’d still come out as the heroes — “Your company must be ultra-secure.”

Fast forward to today, and things are wildly different. I’m not just talking about how some firms have watered down pentesting into glorified vulnerability scans or breach and attack simulations (BAS). These “results” are usually potential issues, with zero evidence and zero use beyond ticking compliance boxes.

What’s changed is that regulations, standards, and a growing anxiety about real-world security have pushed the expectations for penetration tests way higher.

For starters, modern penetration testing isn’t about “hacking” in the romanticized sense. It’s a security assessment that does borrow techniques used by bad actors, but doesn’t cross the same lines.

And here’s the catch: Today’s malicious actors have access to all sorts of tools that ethical testers don’t — 0days, privileged access, toolkits honed over years to bypass EDR/XDR agents like butter. Meanwhile, certified pentesters are still using the same scripts, hash dumpers, and enumeration tools taught in GPEN, OSCP, or CEH programs.

Let’s be real: Today’s pentesters are often a far cry from the raw, hands-on hackers of 15 or 20 years ago.

Meanwhile, clients now come with clear expectations. The golden era of “full scope” tests — plug in and break everything — has been replaced by highly targeted scenarios tailored to specific business risks.

Some examples:

Payment Gateways
Building a payment app from scratch? Not happening. Most companies integrate with third-party gateways via exposed APIs. One of the most common pentest scenarios now is testing these integrations for logic flaws and token handling mishaps on both sides of the exchange.

Sensitive Environments
In Mexico, “SPEI” is a household name. It’s the interbank transfer system — heavily regulated and strictly controlled. These environments require pentests at least once a year, from various perspectives: from outside (isolation checks), from inside with and without privileges, and as a trusted third-party vendor.

Cardholder Data Scenarios (PCI-DSS)
Thanks to the fintech boom, PCI-DSS is trending again. And this is where pentesting has strayed the furthest from its roots. In PCI, testing focuses solely on cardholder data environments and segmentation validation — nothing more, nothing less.

And yet, I just saw a PCI pentest report last week where the only evidence was an Nmap output. No kidding. Even worse — it came from the QSA himself.

So yes, these scopes are wide open to abuse and misrepresentation. Organizations sometimes downplay critical findings just to stay compliant. It’s the auditor’s job to make sure the report actually meets the minimum standards for a true pentest.

Need help spotting a fake? Here are some red flags:

• Talks about ongoing services based on a BAS tool. While BAS tools have a specific purpose and can help maintain constant security checks, they’re nowhere near a real pentest. No matter how hard they try to sell their solutions as “assessments,” the reports usually prove otherwise.

• A ridiculously cheap service in the market. This usually means one of two things: either it’s fake, or the people behind it don’t have the experience to do it right.

• SSL certificates, TLS versions, and misconfigurations flagged as “high.” These findings are often marked as critical by automated scanners, but in a real penetration test, they need to be correlated with other vulnerabilities to truly justify that severity. If the report doesn’t include reproducible evidence and impact, chances are it was just a scan, nothing more.

• Non-reproducible vulnerabilities. At a minimum, a technical report should include the vulnerability description, its impact, and evidence, usually screenshots or clear visuals. If the evidence doesn’t clearly show the vulnerability in action, then it likely wasn’t actually tested.

A classic anecdote: Kevin Mitnick used to leave his pentest report right on the computer of the person who hired him — so they’d feel the impact first-hand.

But it’s not all doom and scans. There’s a rising star in this space: Red Teaming. It’s basically the classic pentest from the old days, now rebranded and reserved for mature clients with solid budgets and security posture—  and for vendors who can actually pull it off.

In Mexico, very few companies actually perform this kind of service. To be honest, our own company hasn’t had the chance to execute one yet. Still, there are a few key considerations any organization should keep in mind when hiring for this type of engagement.

First, the background and qualifications of the people performing the test must be thoroughly verified, not just to ensure high-quality results, but also to minimize the risk of service disruptions due to mistakes. These tests are far more aggressive and often involve real users, so they can’t be handled by inexperienced personnel.

Second, companies offering these services must have the right resources. If real malicious actors have access to 0days and custom-built tools, then professional red teams should have their own equivalent arsenal: persistence tools, command and control platforms, and payloads developed or tailored specifically for the client, among others.

And finally, my favorite sarcastic tip:

If someone offers to “hack you” in a never-before-seen scenario with secret techniques that only they know, they’re probably full of it.

Effective penetration tests, even black box ones, require back-and-forth communication. Anyone who promises to breach your most isolated infrastructure via internet magic is either a scammer or a character from Mr. Robot.

So, what will pentesting look like 10 years from now?

Stay tuned.