ARTIFACT SECURITY
THE WORLDS FIRST
Scam & Phishing Evaluation
The latest AI powered security tools evaluated against Scams & Phishing.
INNOVATIVE
INDEPENDENT
TRANSPARENT
Discover the first independent evaluation on how AI Assistants are reshaping the fight against scams. Can LLMs finally provide a decisive solution in this fast evolving landscape?
With unparalled transparency in our testing, full detailed results are available on our Github for every single test scenario.
Backed by completely independent, unsponsored testing.
The Reports
GenDigital
Norton 360 Advanced &
Genie AI Scam Protection
The GenDigital suite of products was able to offer protection during each of our tested Scenarios while also keeping a 100% legitimate accuracy when handling the same legitimate banking portals we maliciously hosted during the Phishing part of the testing.
Read more in our report!
McAfee Total Protection & Scam Detector
McAfee's offering was proactive in dealing with scams and had good efficacy in dealing with malicious text messages. It's AI assistance was able to complement the email and browser protection offered giving users clear verdicts on on submissions.
Read more in our report!
Bitdefender Premium Security & Scam Copilot
Bitdefender’s suite stands out for its exceptional protection against scams and phishing, particularly through its advanced AI assistant and comprehensive SMS text security. These features highlight Bitdefender’s commitment to innovative, intelligent defenses
Read more in our report!
TRANSPARENCY REPORT
DISPUTE PROCESS OUTCOMES
All participating vendors had the same right to dispute results throughout the test. They were given preliminary results and submitted feedback on any misses.
During the dispute phase, all three participants requested an extension to each test scenario where their respective AI technology can give a response. This was executed as an extension from the original timeline in our test plan and the results are already included in our reports.
A key limitation of this test was the small sample size. Our "honeypot target users" had similar digital footprint to a what a typical user has. To expand the dataset, we generated crafted scenarios based on what was captured already.
Our rating system has only 3 stages of granularity and is quite punishing for FPs. We've received several piece of feedback addressing this and we are evaluating our options for our upcoming methodology changes.
KEY IMPROVEMENTS IN
2026
-
Increased sample size
-
Component specific evasion techniques
-
Domain variety in crafted scenarios
-
Education rating
-
Increased granularity of rating
Media Resources
We provide detailed test results, advance copies and eager to tailor content to help your editorial needs.
Leave your suggestions and ideas on what you want to see next and what resonates with your audience. We can help provide the technical accuracy necessary.
For Interview requests, data access or collaboration inquires don't hesitate to contact us!
The only independent evaluation looking at how AI helps tackle Scams & Phishing.
Access to advanced report copies, customised analysis based on your needs!
Backed by a completely independent, unsponsored testing.
Partner with us to bring trusted, cutting-edge cybersecurity stories to your readers!
Contact us
Follow us and sign up for our newsletter to get cool free stickers!
Comment "Bubbles the Phish", "Betsy the Blue Whale" or "Randall Ransom" alongside your feedback and what you would like to see in the test in the future.
Limited Time!
