Most Shopify brands under $2M should buy an AI feature rather than build one. The model call is the cheapest part of a custom build. The queue, the two rate limit budgets, the retry policy and the person who owns it at 2am are where the money goes.
The demo works on the first try. That is the problem. Nothing about a feature that works once tells you what it costs to make it work ten thousand times.
Drafting product descriptions for a 5,000 SKU catalog costs about $1.90 in model spend. That is not a rounding error in a budget, it is the entire budget for the part everyone talks about. Building the thing that runs those 5,000 calls reliably, survives a throttled provider, and does not leave a customer staring at a spinner in the middle of a checkout, is where a five figure quote comes from.
The gap between $1.90 and $40,000 is the whole story of custom AI on a Shopify store, and most quotes never explain it. Vendors quote the model because the model is cheap and impressive. The operational layer around it is neither, so it arrives later as change requests, retainers and an on call arrangement nobody priced.
This piece is written for the operator at $500K to $2M who has been handed an AI mandate and a developer’s estimate, and for the founder past $2M whose custom backend already exists and is about to have AI grafted onto it. If you are earlier than that, the honest answer is that the interesting work in your store is not a custom model call. It is your product data.
The model is the cheapest line item in a custom AI feature, usually by three or four orders of magnitude, and pricing it first is how brands end up with a budget that is wrong by a factor of a thousand. OpenAI’s published per million token rates in September 2026 put its fast model, GPT-5.6 Luna, at $0.20 per million input tokens and $1.20 per million output tokens, with its flagship GPT-5.6 Sol at $5.00 and $30.00. Cached input runs at a 90% discount and batch processing at half price.
Assume a product description takes 400 input tokens and returns 250, which is a working estimate rather than a measured figure for your catalog. Five thousand SKUs on Luna is 2 million input tokens at $0.40 and 1.25 million output tokens at $1.50, so $1.90 one time for the whole catalog. On Sol the same job is $47.50. Still not a budget line.
Now price what surrounds it. A job queue and the infrastructure to run it. Error tracking. A status field on every record so your team can see which items succeeded. Retry logic that does not make a bad afternoon worse. A review step, because nobody publishes 5,000 machine written descriptions unread. And a maintenance arrangement for year two, when the model you built against is deprecated and the replacement behaves differently. Fastlane’s stage by stage look at how app budgets and infrastructure choices shift with revenue puts stack remediation projects at $15K to $80K, and a custom AI feature that was built without the operational layer becomes exactly that kind of project within eighteen months.
There is a sequencing point underneath that table. A custom AI feature inherits every inconsistency in your catalog and multiplies it across every record it touches. Clearing out app bloat and operational drift first is the cheapest way to make the eventual build smaller.
Every custom AI feature on a Shopify store spends from two separate rate limit budgets at the same time, and both return the same 429 error when you overdraw them. This is the single most common reason a feature that passed testing falls over in its first busy week, and it is invisible in a demo because a demo makes one call at a time.
The first budget is Shopify’s. Shopify’s rate limits on the GraphQL Admin API are metered in cost points that refill continuously, and the refill rate is set by your plan: 100 points per second on standard Shopify, 200 on Advanced, 1,000 on Plus and 2,000 on Enterprise. A single query may not exceed 1,000 points on any plan, however much you pay. So a bulk job that reads and writes product records is rate limited by your subscription tier, not by your server, and a feature that behaved fine while you were on Plus in staging can throttle on Advanced in production.
The second budget is your model provider’s, and it is metered differently. OpenAI’s rate limit tiers and its recommended backoff cap you on requests per minute, tokens per minute, requests per day and tokens per day at once, and you hit whichever ceiling arrives first. Tier placement is set by cumulative spend: $5 paid gets you Tier 1, $1,000 paid gets you Tier 5. A 429 means your request rate climbed too fast. A 503 means the model itself is overloaded, which is not your fault and not something you can fix by retrying harder. OpenAI’s own guidance is exponential backoff with jitter, respecting the Retry-After header it sends you.
Ask your developer which of those two ceilings your feature hits first, at what volume, and what the shopper sees when it does. If the answer is a shrug, the feature has not been designed for your store. It has been designed for a laptop.
An AI call belongs in a background job, never inside the request that renders a page or completes a checkout. This is the one architectural rule worth a non technical operator understanding in full, because it is the difference between a feature that degrades quietly and a feature that takes revenue with it when the provider has a bad hour.
Model completions routinely run for 30 seconds or longer. A web request that waits on one is a web request that has stopped serving anybody. Put that wait in front of a customer and you have converted a slow API into an abandoned cart. In the Ruby ecosystem that most custom commerce backends are built on, the standard answer is Sidekiq, which processes jobs outside the request cycle and claims scale to thousands of processes and billions of jobs per day. The specific tool matters less than the pattern: the customer’s click creates a job, the job calls the model, the job writes the result, and the interface updates when there is something to show.
The corollary is a list of places an AI call must never appear, and it is short enough to hold in your head. Not in checkout. Not in the add to cart path. Not in anything that has to resolve before a page paints. Not in a webhook handler that Shopify expects to acknowledge quickly. Everything else is negotiable; those are not.
What the customer sees while the job runs is a product decision, not a technical one, and it is yours to make rather than your developer’s. A status that says generating and then resolves is honest. A spinner that never resolves because the job failed silently is the worst outcome available, and it is the default unless someone specifies otherwise. Insist that every record carries a state the interface can read: pending, done, or failed with a reason.
Semantic product search is the custom AI feature that most reliably earns its build cost, and only above roughly $2M with a catalog that native search genuinely cannot handle. Everywhere else it is a solved problem you can install for nothing, and paying to rebuild it is the premature complexity that stalls brands in the $500K to $2M band more than any other single decision.
Start with the free option and prove it inadequate before spending. Shopify’s own Search & Discovery app costs nothing and handles synonym groups, product boosts and filtering, and its recent reviews are genuinely mixed on result quality for large catalogs, which is useful information rather than a reason to dismiss it. Install it, configure the synonyms properly, and read your own search analytics for 90 days. If shoppers are finding what they came for, the build conversation is over and you have saved five figures.
Where it does break down is a catalog with technical attributes, compatibility rules or heavy variant logic, the kind where a shopper types a description of a problem rather than a product name. That is the case semantic search answers, and the mechanism is embeddings: a numeric representation of meaning that lets the system match intent instead of keywords. Pgvector is the reason this is now an ordinary piece of engineering rather than a project. It stores and queries those vectors inside the PostgreSQL database your application already uses, with HNSW and IVFFlat indexes and six distance functions, which means no separate vector store to run, pay for and keep in sync.
Two adjacent moves usually cost less and return more. Replacing overlapping apps with a smaller set of purpose built integrations tends to fix the discovery problem that search was being asked to paper over. And getting your product data ready for AI shopping agents improves how your catalog reads to every system that touches it, your own search included. Clean attributes make a cheap search app perform like an expensive one.
The failures that cost you money are the quiet ones: a 200 response carrying truncated output, a retry storm against a provider that is already throttling you, and a record stuck in a state nobody is watching. None of these announce themselves. All three are cheap to design for in week one and expensive to retrofit in month six.
A successful HTTP status is not a successful result. A model can return 200 with output that is cut off mid structure, and code that assumes a 200 means valid data will write that garbage straight into your product records. The fix is validation before use, and logging the raw response before anything is parsed, so that when a record looks wrong you can see what actually arrived. Sentry and comparable error monitoring tools do this well enough that there is no excuse for a build that does not.
Retries are the failure mode that turns a provider’s bad minute into your bad afternoon. Unbounded retries against a rate limited API flood your own queue, spend real money on calls that were never going to succeed, and delay every legitimate job behind them. Cap retries at three or four attempts with exponential backoff between them, and make the final failure visible rather than silent. The source of the original engineering advice behind this piece put it well: never swallow the error quietly.
Visibility is where a non technical operator has genuine leverage. Ask for a place you can see the state of the feature without asking a developer: how many records are pending, how many failed today, what the last error was. Shopify Flow is free and will notify a Slack channel when a tagged record enters a failure state, which is a crude monitor but an enormous improvement on finding out from a customer. The alternative is discovering in week nine that the feature stopped working in week three.
Build when the workflow is specific to how your business actually runs, no app models it, and you can fund the second year as well as the first. Those three conditions together are rarer than most quotes imply, and the third one is where brands get caught: the build is approved as a project and then discovered to be a commitment. The signals that a standard platform has genuinely stopped keeping up are worth reading against your own operation honestly before you accept that you have hit them.
If the conditions hold, six questions separate a partner who will still be useful in year two from one who will hand you a maintenance problem. First, which model provider are we locked to, and what specifically changes if we switch? A build that isolates model calls behind a single boundary can change providers in one place; a build that scatters them cannot, and vendor lock in is a cost that arrives later at a worse moment. Second, where does customer data go? If any part of your catalog or customer records cannot leave your infrastructure for regulatory reasons, that constrains the architecture from day one and cannot be retrofitted.
Third, how are credentials stored, and can they end up in source control? Fourth, what is the retry ceiling, and what does the customer see when it is reached? Fifth, what does maintenance cost per month once this is live, in writing. Sixth, what version of the underlying framework will this run on, and how long is it supported? That last one has a checkable answer. The Rails team’s own end of support dates put Rails 7.0 and 7.1 past end of life entirely, Rails 7.2 out of security support since 9 August 2026, Rails 8.0 supported only until 7 November 2026, and Rails 8.1 covered through 10 October 2027. A partner proposing new work on an unsupported series is telling you something about the next two years.
Agencies that do this work at volume, including firms offering custom Ruby on Rails development services to commerce teams, should be able to answer all six without preparation. If the answers arrive as reassurance rather than specifics, you are not being quoted for the operational layer, which means you will be billed for it later.
Treat a custom AI feature as infrastructure with a running cost, not as a feature with a price. The model call is trivially cheap and will keep getting cheaper. The queue, the two rate limit ceilings, the validation, the retry policy, the status your team can read and the partner who answers the phone in year two are the actual purchase, and none of them appear in a demo.
For most brands under $2M the right answer is still an app, measured honestly for a quarter, with the effort that would have gone into a build spent on product data instead. That is not a counsel of timidity. It is the same pattern that decides most outcomes in the $500K to $2M band, where brands stall on complexity they added before the fundamentals were solid rather than on any shortage of ambition.
Above $2M, where the workflow is genuinely yours and no app models it, build carefully. Put the feature in a queue, cap the retries, give every record a state your team can read, isolate the model call behind a boundary you can swap, and get the maintenance number in writing before you approve the first invoice. Do that and the feature is still working in eighteen months, which is the only test of a build that matters.
The model spend is usually a few dollars and the build is usually five figures, which is why quotes that lead with model pricing are misleading. Drafting descriptions for a 5,000 SKU catalog costs roughly $1.90 at September 2026 rates on a fast model, assuming about 400 input and 250 output tokens per item. The cost sits in everything around that call: a job queue and somewhere to run it, error tracking, a status field per record, retry logic, a human review step, and a maintenance arrangement for when the model you built against is deprecated. Fastlane’s stage by stage tech stack guide puts comparable stack remediation work at $15K to $80K, which is the range a build without an operational layer eventually becomes.
Buy below roughly $2M in revenue and build above it only where no app models your actual workflow. Apps carry the operational layer for you: the queue, the retries, the error handling and the upgrades are the vendor’s problem, spread across thousands of merchants. A custom build makes all of that yours, which is worth doing when the workflow is genuinely specific to your business and worth avoiding when it is not. The practical test is to install the closest app, configure it properly, and measure it for 90 days. Most brands discover the app was adequate. The ones who discover a real gap now have evidence for a build, which makes the build smaller and the quote easier to judge.
Intermittent failures almost always trace to one of three causes: a rate limit, a model completion running longer than the request that is waiting on it, or a successful response carrying invalid output. Rate limits come from two directions at once. Shopify meters the GraphQL Admin API in cost points that refill at 100 per second on standard plans and 1,000 on Plus, and model providers cap requests and tokens per minute by spending tier. Both return a 429. Separately, a 503 from a model provider means the model is overloaded rather than that you did something wrong. If the failures are random rather than load related, suspect validation: a 200 response can carry truncated output that breaks code assuming the status was enough.
No, and for most Shopify catalogs you do not need semantic search at all yet. Shopify’s free Search and Discovery app handles synonyms, product boosts and filtering, and configuring it properly resolves the majority of search complaints at brands under $2M. Where a catalog has technical attributes, compatibility rules or heavy variant logic, semantic search does earn its place, and even then a separate vector database is optional. Pgvector stores and queries embeddings inside the PostgreSQL database an application already uses, with HNSW and IVFFlat indexing, which removes an entire piece of infrastructure from the build. Clean product attributes improve search results more cheaply than any of this, so do that first.
Ask six questions and judge the specificity of the answers rather than the confidence. Which model provider are we locked to and what changes if we switch. Where does customer data travel, and does any of it need to stay inside our own infrastructure. How are API credentials stored, and can they reach source control. What is the retry ceiling, and what does a customer see when it is hit. What does monthly maintenance cost once this is live, in writing. And what framework version will this run on, and how long is that version supported. The last one is checkable: Rails 7.2 left security support on 9 August 2026 and Rails 8.0 leaves on 7 November 2026, so a proposal built on either is a conversation worth having now.
