BusySeed tracked ChatGPT, Claude and Gemini weekly in 2026 and found the company named first for a buying question changed from one week to the next 56.4% of the time. Store owners should judge AI visibility on a monthly trend across several assistants, not one answer.
When the top answer changes more often than it stays the same, a screenshot of one ChatGPT reply tells you about that afternoon. It tells you very little about your brand.
A store owner types “best weighted blanket for hot sleepers” into ChatGPT on a Monday, sees their brand named first, and screenshots it for the team. The next Monday the same question names a competitor first, and the screenshot goes to the agency with a question mark. Both screenshots are accurate. Neither one says much on its own.
Store owners who check whether ChatGPT recommends their brand may be reading too much into a single answer. New research from BusySeed, a New York marketing agency, found that the company an AI assistant names first for a category changes from one week to the next 56.4% of the time.
That number matters for anyone selling online, whether you’re doing $10K months or $1M months, because a shopper can now ask an assistant for a product recommendation before ever reaching a search box. If the top slot changes more often than it holds, the useful question is no longer “are we first today?” It is “how often do we show up, on which assistants, and is that number moving over a month?”
BusySeed’s study measured which company three AI assistants named first for the same set of buying questions, asked every week for roughly three months, and how often that first name changed.
BusySeed asked ChatGPT, Claude and Gemini the same 160 buying questions every week from April 27 to August 3, 2026. That produced 480 separate question and engine pairs, and 6,385 usable answers naming 1,664 companies.
A “question and engine pair” is one buying question asked on one assistant. The same question asked on ChatGPT and on Gemini counts as two pairs, because each assistant builds its own answer and the study tracks each one separately. That design choice is important: it means a brand can hold steady on one assistant while churning on another, and a blended “AI score” would hide the difference.
According to BusySeed, 82.3% of those pairs saw their top answer change at least once. Only 85 of the 480 kept the same leader for the entire period.
Read those two sentences together and the pattern is clear. Stability is the exception. If you picked any one buying question and any one assistant at random from this study, the odds were heavily against seeing the same company named first every week. The methodology page notes that the study ran on one fixed model version per assistant and that the median pair was observed for 13 weekly reads, so these figures describe a summer 2026 window, not a permanent law of AI behaviour.
The full study is available at https://busyseed.com/research/category-leader-volatility-check.
The top pick most likely changes because AI assistants build each answer fresh, so small differences in what they retrieve and how they generate text can reorder the companies they name, even when the question is identical.
The churn was not equal across assistants. BusySeed’s methodology page reports a week to week leader change rate of 61.2% for Gemini, 58.9% for ChatGPT and 49.1% for Claude. Claude was the steadiest of the three in this window, and even it changed its first name in roughly half of all weekly transitions. That gap is a practical reason to track assistants separately rather than averaging them together.
BusySeed is candid about what its data cannot explain, and lists prompt wording among the possible causes of instability. The study did not log the sources each assistant retrieved, so it cannot separate changes caused by the assistant finding different web pages from changes caused by the randomness built into how these models write. It was also a single pass each week, so it does not measure how much an answer would vary if the same question were asked twice on the same day. Some of the weekly churn is likely that same day randomness showing up on a weekly schedule.
Independent research points the same way. In January 2026, Rand Fishkin of SparkToro and Patrick O’Donnell of Gumshoe published research on how inconsistently AI tools recommend brands, running each of 12 brand recommendation prompts 60 to 100 times per platform. They found less than a 1 in 100 chance that ChatGPT or Google’s AI would return the same list of brands twice, and concluded that visibility across many prompts is a reasonable metric while a single “ranking position in AI” is not.
Phrasing, location and whether the assistant searches the web all add more variation on top. Fastlane covered a controlled test of six phrasings of one running shoe question in why you can’t just ask ChatGPT about your brand, where answers drawn from web search moved far more than answers drawn from the model’s memory.
After one bad week, store owners should log the result and wait for the next two or three weekly reads before changing product pages, content plans or ad budgets, because a single drop is more likely normal churn than a real loss.
“Check the trend, not the day,” said Omar Jenblat, BusySeed’s founder and CEO. “Of the 480 question and engine pairs we tracked, only 85 kept the same leader the whole time. Don’t panic over one bad week.”
The cost of overreacting is real. A team that sees its brand disappear from one ChatGPT answer can burn a week rewriting product descriptions, commissioning new blog posts or signing up for a monitoring tool, all in response to movement that would likely have reversed on its own. That is the same premature complexity that stalls a lot of stores in the $500K to $2M range: adding effort and software before the underlying signal is clear.
Waiting has a cost too. If the first slot drives most of your sales, especially in peak season, a real loss left unchecked for a month is expensive. That is why the trigger below looks for breadth as well as time: a drop across several questions on the same assistant deserves a look within two weeks, while a drop on one question can wait for the monthly read.
A more useful response is a simple rule. Treat a change as real only when it persists for three consecutive weekly reads, or when your four week average moves. If your brand drops out of answers for one question on one assistant for one week, write it down and move on. If it drops out across several questions on the same assistant for most of a month, that is worth investigating, starting with whether something on your site changed: a product went out of stock, a page was blocked from crawling, or a collection was restructured.
If you’re just starting out, the same rule saves you time you don’t have. If you’re scaling, it gives you a defensible answer when someone upstairs forwards a screenshot and asks why the brand vanished from ChatGPT. The answer is the trend line, not the screenshot.
A real AI visibility gain looks like a sustained jump in the share of answers that name your brand, measured across many questions on each assistant, not a single first place mention.
The flip side is that real gains show up in the trend. BusySeed tracks about 100 buyer questions every week for a family-owned cacao manufacturer that sells wholesale online. In August 2026, Gemini named the company in 1.3% of its answers. In September, it named the company in 40.9%. Claude went from naming it in none of 500 answers in August to 219 of 700 in September, or 31.3%.
Notice what is being measured in that example. It is not “were we first?” It is the share of all logged answers that named the company at all. That measure is less exposed to the weekly churn described above, because a brand can stay in the answer without holding the first slot, and SparkToro’s research found presence across many runs a steadier signal than position. The September counts also cover more answers than August’s (700 Claude answers against 500), so the question set grew between months, which is why the comparison is made in shares rather than raw counts. Even so, a move from almost never named to named in about a third of answers is too large to explain as weekly noise.
Two conditions belong next to that result. It is one client’s outcome, reported by the agency that did the work, so it shows what a real gain looks like rather than what a typical store should expect in a month. And the client is a wholesale manufacturer answering business buyer questions, a category where a single well documented supplier can stand out more easily than a consumer brand competing against household names. The case does not say which changes produced the jump, and readers should not assume a particular tactic caused it.
The lesson that does transfer is the measurement. If you want to know whether AI visibility work is paying off, compare your share of answers month over month on each assistant. A gain that shows up on two assistants at once, as it did here on Gemini and Claude, is more convincing than a gain on one.
Store owners should track a fixed list of real customer questions on at least two assistants each week, record whether their brand is named, and judge the share of answers that mention them over a full month.
For store owners, the practical lesson is to track a short list of the questions customers actually ask, on more than one assistant, and judge the results over a month rather than a single week.
Here is how that looks in practice. Illustrative example, assuming 20 questions checked weekly on ChatGPT, Gemini and Claude: that is 60 answers a week and roughly 240 a month. In a spreadsheet, log one row per answer with the date, the assistant, the question, whether your brand was named, and which competitors were named. At the end of each month, calculate the share of answers that named you for each assistant separately. If ChatGPT named you in 12 of 80 answers this month and 22 of 80 next month, that is a movement worth paying attention to. If it went from 12 to 14, it is probably noise. Treat these thresholds as rules of thumb rather than tested cutoffs; with samples this small, only large or repeated moves are worth acting on.
The questions matter more than the tool. Pull them from your support inbox, your on site search terms and the way customers describe their problem in reviews, not from the keywords you wish you ranked for. Include a mix of broad category questions (“best organic cacao supplier”), comparison questions (“brand A vs brand B”) and specific need questions (“fair trade cacao nibs in bulk for a bakery”). Keep the wording identical every week so you are measuring the assistant, not your own phrasing.
When a spreadsheet stops being enough, Fastlane’s guide to AI visibility tools for tracking what ChatGPT says about your brand covers the metrics and tool categories worth comparing. Whatever you use, insist that it reports results per assistant and as a share of answers over time.
Your store needs only as much AI visibility tracking as someone will act on: a monthly spreadsheet check for early stage stores, a weekly log for growing brands, and dedicated tooling only once a team owns the results.
If your store does under $500K a year, 10 questions on two assistants, checked once a week and reviewed monthly, is plenty. Your time is better spent on the product pages and reviews the assistants are reading. Between $500K and $2M a year, expand to 20 to 30 questions on three assistants and assign the weekly log to one person so the data is consistent. This is also the stage where the temptation to overbuild is strongest, so resist buying software until the spreadsheet has shown you something worth acting on.
Between $2M and $10M a year, a paid monitoring platform starts to make sense, but only if someone reviews its output monthly and has authority to change content or product data in response. If nobody owns the follow through, a well kept spreadsheet will tell you more for less. Above $10M, especially with several product lines or international markets, dedicated tooling earns its cost because the question set gets too large to run by hand, and per assistant reporting becomes a budgeting input.
At every stage, reading results monthly is how you tell whether changes to your site and product data have registered with the assistants. Google’s guidance on how AI features in Search treat your website says there are no special optimizations required to appear in AI Overviews or AI Mode; pages need to be indexed and eligible to show in Search. The groundwork in why generative engine optimisation depends on strong SEO foundations still applies, and for Shopify stores, product data quality is a large part of what assistants read, as covered in Fastlane’s agentic commerce guide for Shopify merchants. If you sell through ChatGPT directly, the ChatGPT Shopping playbook for Shopify brands explains why blocking OpenAI’s search crawler would keep your products out of its answers.
In BusySeed’s 2026 study, ChatGPT changed the company it named first for a buying question in 58.9% of week to week transitions. Across ChatGPT, Claude and Gemini combined, the top name changed from one week to the next 56.4% of the time, based on 160 buying questions asked weekly from April 27 to August 3, 2026. Gemini changed most often at 61.2% and Claude least often at 49.1%. Separate SparkToro research found ChatGPT and Google’s AI returned the same list of brands less than once in 100 repeated runs. The takeaway is that the first slot rotates constantly, so a single answer is a poor measure of how often an assistant actually recommends a brand. Tracking the share of answers that mention you over a month gives a far steadier read.
No, one week without a recommendation is usually normal churn rather than a real loss. BusySeed found that only 85 of 480 question and assistant pairs kept the same top company for the whole study period, so week to week movement is the norm. Log the result and check again over the next two or three weeks. Investigate only if your brand drops out across several questions on the same assistant for most of a month. When that happens, start with your own site: check for out of stock products, pages accidentally blocked from crawling, or recent changes to collections and product data, since those are the changes you control and can reverse.
Most stores should track 10 to 30 questions, kept identical every week. A store under $500K a year can learn plenty from 10 questions on two assistants, while a brand between $500K and $2M a year benefits from 20 to 30 questions on three assistants. Choose questions customers actually ask, pulled from support emails, on site search and reviews, and mix broad category questions with comparison and specific need questions. The goal is enough answers each month to see a trend: 20 questions on three assistants weekly produces roughly 240 answers a month, which is enough to tell a real shift in your share of answers from ordinary weekly movement.
A Shopify store should track at least two AI assistants, and ChatGPT, Gemini and Claude are a sensible starting set because they behave differently. In BusySeed’s study, Gemini changed its top recommendation most often and Claude least often, so a brand can look stable on one assistant and volatile on another. Track each assistant separately rather than blending them into one score. If your customer data shows shoppers arriving from a particular assistant, weight that one most heavily. Adding Google’s AI Overviews or AI Mode makes sense if Google search is already a major traffic source for your store.
Judge AI visibility improvements over at least a month, because weekly results move too much to show a trend on their own. In one case reported by BusySeed, a wholesale cacao manufacturer went from being named in 1.3% of Gemini answers in August 2026 to 40.9% in September, and from none of 500 Claude answers to 219 of 700. That is one client’s result, not a typical timeline, and the case does not specify which changes produced it. Compare your share of answers month over month on each assistant before deciding whether work on crawlable pages, product data or third party coverage is paying off.
BusySeed is a New York marketing agency founded in 2013 by Omar Jenblat, its founder and CEO. The company is headquartered at 1178 Broadway in New York and has offices in California, Texas, Florida, Massachusetts and Rhode Island. It has worked with more than 500 businesses and is a Google Partner, Meta Business Partner and HubSpot Solutions Partner. BusySeed also publishes research on how AI assistants such as ChatGPT, Claude and Gemini recommend companies.
