AI vs Scams & Phishing in 2026

Scams, Scams, Scams

In September of this year we ran the first Scam & Phishing evaluation. We would like to thank all participants for their feedback during the test and the ongoing work to make testing in this area better. For us it was a great learning experience and a warm up for expanding the test in 2026.

Weaknesses of the previous test

Sample size

One of the core weaknesses of the first run of the test was the limited sample size. Testing in this area is very difficult and expensive to run accurately. We wanted to first see the reception of it before we can expand it further as such one of the biggest improvements we are making in 2026 is severely increasing the sample size to over 100 scenarios, up from 20 tested in 2025.

Domains

Another urgent issue that we had to fix is increase the variety of domains used in our crafted samples. As such going in 2026 each scenario will have a unique domain for use.

Rating

We've taken onboard feedback on how we rate each scenario. During the testing we had to extend the test to allow for each component of the solution to have a chance to give a verdict on the threat, specifically the AI component of solutions. This will now be the default in 2026. Alongside it we have rehauled our Rating system to allow for more granularity and changed the scaling.

New Major Additions

Scam types

This is a given and we always going to adapt to represent the threat landscape however it's worth highlighting some major changes. We've now added Video Scams (both AI Generated and Real Actors), Romance (pig butchering), Investment and more.

As a quick note we are not measuring if something is AI generated or not. We believe this is a bit of wasted metric as there is much AI generated content on social media platforms. What matters is the intent of it.

Platform rating

Most social media platforms have built in security tools that are supposed to help users stay safe online. These are usually not enough to prevent against everything that is going on. It's important to highlight the difference of results between a commercial solution and what you get as a standard user. As such, all our reports in 2026 will contain a platform rating alongside the tested solution as a comparison.

Education Rating & PUA

We've removed PUA as a consideration from the test. We are strictly working in testing against scams & phishing. We will keep an eye on this and how the landscape changes.

Solutions with AI Assistants have a responsibility to also accurately educate the users in recognising what a threat is in this domain. For the first time we will give a separate rating assessing the education and accuracy of an assistant.

There is more to read in our methodology available here.

Look out for the test announcement with dates and participants at the beginning of December!