SEOTesting Review: AI Summaries, Content Decay, and Where It Really Fits

Setting up a test takes a couple of minutes and saves hours of reporting later on. Budget buy-in from management is now easier.

What It Is

SEOTesting is an SEO experimentation and reporting platform that connects to your Google Search Console and GA4 accounts and lets you measure whether the changes you make to your site are actually moving the needle. You log a change, a title tag rewrite, a content update, a schema addition, and the platform compares performance before and after using your real GSC data. No estimates. No third-party clickstream approximations. What Google reported is what you see.

The platform supports five test types. Time-based single-page tests and group tests across multiple URLs have been available for some time. Split tests divide similar pages into a test group and a control group to isolate your change from external noise like algorithm updates or seasonal shifts. A URL switch test handles redirect and migration scenarios. The newest addition, the LLM test type, works like a time-based test but tracks GA4-reported sessions from AI platforms including ChatGPT, Claude, Perplexity, Gemini, and Copilot before and after you make content changes.

Beyond testing, the platform stores your GSC data continuously (Google’s native interface only retains 16 months), surfaces pre-built reports for content decay, CTR gaps, keyword cannibalization, and queries you rank for but never mention on-page, and groups URLs into content clusters you can track over time. The most recent update added AI-powered summaries across individual tests, content groups, and end-of-month reports, plus PDF export and a one-click client-ready copy function. If you are evaluating where SEOTesting fits within a broader ecommerce SEO reporting stack, that context is worth reviewing before you commit to any single tool.

Who It’s Actually For

I haven’t used SEOTesting personally on eCommerce Fastlane. My assessment is based on verified user reviews, direct research into the product, and conversations with ecommerce SEOs working with brands in the $500K to $5M range. I’ll flag where I’m drawing on direct experience and where I’m not.

The merchants I hear from who get the most out of SEOTesting are the ones who were already making regular changes to title tags, meta descriptions, or content clusters and had no systematic way to measure whether those changes worked. That is the exact problem this tool solves. If you are not making regular changes, or if your site does not have the traffic to generate meaningful test data, the value proposition collapses quickly.

One thing worth being direct about for the ecommerce SEO teams reading this: the LLM test type tracks clicks from AI platforms to your site via GA4. It does not track whether your brand is being cited or mentioned in AI responses. Those are different measurements that require different tools. I see this distinction get blurred frequently, and buying SEOTesting expecting the latter will leave you disappointed.

What It Does Well

It produces before-and-after evidence from your own GSC data, and that is the core job this tool was built for. Every test produces a clean comparison using actual clicks, impressions, CTR, and position data from Google, not estimates or third-party approximations. Verified users at Better Collective and Decathlon cite this as the thing that changed how they work: being able to show a client or a leadership team the exact numbers before and after a change, rather than pointing at a graph and asking them to trust the interpretation.

The new AI summaries explain causation rather than just flagging correlation, and that distinction matters more than it might initially sound. When impressions jump but CTR drops, the summary explains that the CTR decline is likely a side effect of impression growth, not a real problem. When average position appears to worsen, it notes that this often happens when a page starts ranking for more queries and the average shifts across a broader, lower-intent query set. Most experienced SEOs know this. Most clients and most founders do not. Having the tool explain it automatically in the summary is the difference between a report you can send and one you have to rewrite first.

Content groups give you cluster-level performance tracking that native GSC does not offer, and the September 2025 update added query-based grouping using exact match, contains, or regex logic. For ecommerce brands with distinct page categories, product pages, collection pages, blog content, landing pages, this replaces a significant amount of manual spreadsheet work. Nilesh, a verified user managing a site with over 30 page types, puts it directly: whenever there’s a Google update, he can immediately check which page types dropped and plan next steps without building a custom report.

Sixteen-plus months of GSC data is retained and queryable. Google Search Console only retains 16 months natively and does not guarantee that data indefinitely. For ecommerce brands with strong seasonal patterns, Black Friday, holiday, back-to-school, the ability to run clean year-over-year comparisons without losing historical context is worth real money at reporting time.

Where It Falls Short

Split tests require meaningful daily traffic, and the tool does not make this constraint prominent enough at signup. The platform recommends at least 25 pages per group with daily organic clicks to generate statistically significant results. Brands under 500 organic clicks per day will run tests that return inconclusive results for weeks. This is a fundamental constraint of statistical testing, not a flaw in the methodology, but it is something early-stage merchants consistently discover after signing up rather than before. If your site is in the $100K to $300K revenue range, time-based tests will still work, but the split testing capability that differentiates this tool from manual GSC analysis will underdeliver.

AI summaries are not saved between sessions and the wording changes every time you run them. Because the summaries are generated fresh each time using a probabilistic LLM, you get a different version each run. Generate a summary on Monday, share it with a client, go back Friday to regenerate it, and you will get a different document. For agency teams that need a record of what was reported and when, this is an operational problem right now. SEOTesting has indicated saved summaries are on the roadmap. Do not build a client reporting workflow around these summaries until that ships.

Several of the most useful reports are buried and easy to miss. Multiple verified users across Capterra and SoftwareAdvice flag the same issue: the Striking Distance Keywords report, the Content Decay report, and the Keywords Not Mentioned on Page report are among the highest-value features in the tool, but new users often do not find them without going through the full onboarding sequence. One user who had been on the platform for four years still noted that features are sometimes hidden in the tool. That is a discoverability problem that directly affects how much value you extract from the subscription in the first 90 days.

There is no direct Shopify admin integration. SEOTesting connects via Google Search Console, not directly to Shopify. For teams that want to correlate organic traffic shifts with conversion or revenue data at the product level, you will need to bridge that gap with GA4 or a separate BI tool. For most SEO use cases this is not a problem. For teams that want product-level attribution, it is a real gap.

Pricing and Value Assessment

Pricing as of April 2026. Plans are month-to-month. Annual plans are available at a discount by contacting the team directly.

Value at early stage ($0 to $500K): The $50/month cost is not the issue. The issue is that split testing, the highest-value capability, requires traffic volume most early-stage sites do not have. You will get value from the content decay reports, striking distance keywords, and the end-of-month AI summaries. But you are paying for a testing platform and may not be able to use the testing core for 12 to 24 months. Manual GSC analysis is a reasonable alternative at this stage. Wait until you are consistently above 500 organic clicks per day.

Value at growth stage ($500K to $5M): This is the clear sweet spot. You have enough organic traffic to run meaningful tests, enough content to benefit from group tracking, and enough reporting obligations to make the AI summaries and PDF export genuinely useful. At $50 to $125/month with no per-user fees, the ROI case is straightforward if you are making any kind of regular content or technical changes. The time saved on monthly reporting alone typically covers the cost for agency teams managing three or more clients.

Value at scale ($5M and above): SEOTesting works at scale, and Better Collective and Decathlon are both documented users at this level. But if you need server-side test implementation, neural network statistical significance modeling, or the ability to run 20 or more simultaneous tests across large programmatic page sets, SearchPilot is the more appropriate tool. SEOTesting is not the wrong choice at this stage, but it has a ceiling that large enterprise programs will eventually reach.

How It Compares

SearchPilot uses a neural network statistical model and integrates at the server or CDN level, which means Googlebot sees exactly what users see during a test. It supports full-funnel testing that measures SEO and CRO impact simultaneously and is genuinely more statistically rigorous for large-scale split testing. If you are running 500K or more monthly organic sessions across large programmatic page groups and need the highest possible confidence in your test results, SearchPilot is the more accurate tool for that specific job. The trade-off is price and engineering overhead. SearchPilot is priced for enterprise budgets and requires meaningful implementation work. SEOTesting wins on breadth of test types, content group tracking, reporting features, transparent public pricing, and no per-user fees. They are not competing for the same customer in most cases.

SplitSignal is Semrush’s split testing module. If you are already paying for Semrush and want split testing without adding another vendor, SplitSignal is worth evaluating. Where SEOTesting is stronger: time-based testing (SplitSignal does not offer this), LLM traffic testing, content group tracking, end-of-month reporting, and historical GSC data retention. SplitSignal also does not have public pricing. SEOTesting’s pricing is transparent from the first click.

Steve’s Take

I do have an affiliate relationship with SEOTesting. If you sign up through a link on this page I may earn a commission at no additional cost to you. That doesn’t change what I’m about to tell you.

I’m actually running SEOTesting on eCommerce Fastlane, so I can speak to this from direct experience. The feature I keep coming back to is the Content Decay report. Every month it flags content that has become dated and is losing ground, which means I’m not guessing what to refresh next. I have a prioritized list waiting for me. For a site like mine where I’m publishing consistently and the archive goes back years, that report alone justifies the subscription cost.

Beyond my own use, the pattern I hear consistently from the ecommerce SEOs I talk to, the ones managing $1M to $5M Shopify brands, is that they are making SEO changes constantly and have almost no systematic way to know which ones worked. They’re looking at GSC before and after, eyeballing it, and calling it a win or a loss based on feel. That is the exact gap SEOTesting fills, and it fills it at a price point that makes the ROI case easy to justify.

The AI summaries are the genuinely new development here. The context-awareness, explaining why a CTR dip is a side effect of impression growth rather than a real problem, is the kind of interpretation that takes an experienced SEO five minutes to write and most clients cannot do themselves. The fact that summaries are not saved between sessions is a real limitation for agency teams right now. I wouldn’t build a client reporting workflow around them until that ships. But the output quality is already strong enough to use with light editing, and the PDF export means you’re not copying from a screen.

The LLM testing feature needs to be understood for what it actually is before you buy on the basis of it. It measures whether content changes increased the number of users clicking through from AI platforms to your site via GA4. That is a useful and increasingly important data point. It is not the same as tracking whether your brand is being cited or mentioned in AI responses, which requires a different kind of tool entirely. If that distinction matters to your use case, get clarity on it before you sign up.

My recommendation is direct. If you are doing $500K or more in revenue with consistent organic traffic, start the 14-day trial, set up one content group for your most important page category, and log one test on a change you have already made. If those two use cases don’t save you meaningful time in the first month, the subscription is not right for your workflow yet. If they do, and for most ecommerce SEO teams at this stage they will, the $50 to $125/month is one of the better-value line items in your stack.

If you are under $500K/year with limited organic traffic, wait. Come back when you have the volume to run meaningful tests. The tool will be here.

Try SEOTesting free for 14 days — no credit card required, full access to all features.

Frequently Asked Questions

What is SEOTesting and how does it work?

SEOTesting is an SEO experimentation platform that connects to your Google Search Console and GA4 accounts and measures whether changes you make to your site actually improved performance. You log a change, a title tag rewrite, a content update, a schema addition, and the platform compares clicks, impressions, CTR, and position before and after using your real GSC data. It supports five test types: time-based single-page tests, group tests across multiple URLs, split tests with test and control groups, URL switch tests for migrations, and LLM tests that track traffic from AI platforms like ChatGPT and Perplexity. Unlike tools that estimate traffic from third-party data, SEOTesting uses only the data Google actually reported to you.

How much traffic do I need for SEOTesting split tests to work?

SEOTesting recommends at least 25 pages per group with each page receiving daily organic clicks for split tests to reach statistical significance in a reasonable timeframe. In practice, sites under 500 organic clicks per day will often see inconclusive results that run for weeks without resolution. This is not a flaw in the tool. It is a fundamental constraint of statistical testing that requires volume to produce reliable results. If your site is below that threshold, time-based tests still work and can produce useful before-and-after comparisons, but the split testing capability that most distinguishes SEOTesting from manual GSC analysis will underdeliver until your traffic grows.

What do the new AI summaries in SEOTesting actually do?

The AI summaries, currently in Beta, generate plain-English analyses of your test results, content group performance, and end-of-month data. For individual tests they produce a verdict (successful, inconclusive, or failed), a metric-by-metric breakdown, seasonality checks, and causation versus correlation analysis. The context-awareness is the most useful part: if CTR drops while impressions grow, the summary explains the drop is likely a side effect of impression growth rather than a real problem. Summaries are generated fresh each time you run them and are not currently saved between sessions, which is a meaningful limitation for agency teams that need a consistent record of what was reported to clients.

Does SEOTesting track AI brand mentions and citations?

No. SEOTesting’s LLM test type tracks GA4-reported sessions from AI platforms, meaning users who clicked a link from a ChatGPT, Claude, or Perplexity response and landed on your site. It measures whether content changes increased that click-through traffic before and after an update. It does not measure how often your brand is mentioned or cited in AI responses without a click, and it does not track your share of voice against competitors in AI-generated answers. Those are different measurements that require a dedicated AI visibility platform. The two data sources are complementary, not interchangeable.

How does SEOTesting compare to SearchPilot for ecommerce SEO testing?

SearchPilot uses a neural network statistical model and integrates at the server or CDN level, which means Googlebot sees exactly what users see during a test. It is genuinely more statistically rigorous for large-scale split testing and supports full-funnel testing that measures SEO and CRO impact simultaneously. For enterprise ecommerce programs running 500K or more monthly organic sessions across large programmatic page sets, SearchPilot is the more accurate tool. SEOTesting wins on breadth of test types, content group tracking, end-of-month reporting, transparent public pricing, and no per-user fees. For most Shopify brands at $500K to $5M, SEOTesting is the right fit. SearchPilot is priced and built for enterprise programs that have outgrown what SEOTesting can do.