Best AI model for support tickets in 2026: what the benchmarks show

The question I get asked, and the one that actually decides it

I build AI agents at eesel, and the single most common question I get from technical support leads evaluating AI customer service software is some version of "which model should we use?" It is a reasonable question. It is also, most of the time, the wrong first question.

I know that because I have watched what happens when teams answer it and stop there. A few of our own customers have churned to go build their own thing directly on the frontier APIs. Looking at that list honestly, it includes an AR and construction-tech firm and a DTC beauty brand, the latter going straight to the Claude API. That is not a knock on them. Those are competent engineering teams and the API is genuinely good. But every one of them was solving for the model, and the model was never the hard part.

The engineering lead at a Bitcoin-ATM company on our Team plan, running a 300-article Confluence base, put the other side of it plainly:

"We could try to write our own LLM application but we didn't want to invest our time into that. We wanted something that we would not have to maintain."

Two teams, same starting question, opposite conclusions. The difference between them is not which model they picked. It is whether they had a way to know if the thing worked.

This post is my attempt to answer the model question properly, with real numbers, and then show you why the answer matters less than the thing sitting underneath it.

There is a benchmark built for exactly this, and almost nobody quotes it

Every model launch cites MMLU, SWE-bench, GPQA, or one of the aggregate intelligence indexes. None of those tell you anything useful about whether a model can close a support ticket, because closing a support ticket is not a knowledge test. It is a sequence: understand a messy request, find the relevant policy, call the right tools in the right order, refuse the things you are not allowed to do, and leave the database in the correct state. That is the difference between a language model and a working AI ticketing system.

Sierra built a benchmark for precisely that, and it is called τ-bench. The original τ-bench paper landed in 2024 with retail and airline domains. τ²-bench added a telecom domain in 2025, and its key contribution was making the user act too, so the agent has to talk a customer through steps only the customer can perform. That is technical support as it actually happens. The paper's own finding is blunt: performance drops significantly the moment the agent has to guide a user rather than just act for them.

Then in 2026 came the version that matters most for support teams. τ³-bench added τ-Knowledge, a banking domain where the agent has to work from a knowledge base rather than a tidy tool list.

Here is what that knowledge base looks like, from Sierra's own writeup: 698 documents across 21 product categories, roughly 195,000 tokens. Each task needs an average of 18.6 documents and 9.5 tool calls, with some requiring up to 33. Tasks are scored on whether the agent produced the correct final database state, not on whether the reply read nicely.

If you have ever wondered why your help center feels like it fights the AI, that is the benchmark version of your help center. It is also why deflection guides spend more time on content than on model settings.

The retrieval cliff

This is the chart I would put on the wall of anyone evaluating AI for ticketing systems.

One model. One evaluation run. Four domains. Telecom 92.3%, airline 84.0%, retail 79.6%, banking 24.7%.

The first three domains hand the agent a clean set of tools and the answer is reachable by calling them. The fourth makes the agent go find the answer in documents first. That single change costs 68 points, which is more than the entire spread between the best and worst frontier models on the same test.

Qwen3.5-397B shows the same shape even more starkly: 97.81% on telecom against 9.79% on banking. An 88-point collapse from the same weights.

I want to be careful about what this does and does not prove. It is one benchmark, and a synthetic one. But the mechanism it exposes is exactly the mechanism I see in production. When a customer tells me their AI knowledge base chatbot "isn't smart enough," roughly nine times out of ten the model is fine and the retrieval is bringing back the wrong three paragraphs.

The 2026 leaderboard for support tickets

Sierra ran a fresh batch on 2026-08-04 covering most of the current frontier. All of these entries used the same retrieval configuration, the same GPT-5.2 user simulator, the same benchmark version, and four trials, so they compare cleanly against each other.

A few things jump out of that table.

The most expensive model is not the best one. Claude Fable 5 bills at $10 per million input tokens and $50 per million output. Qwen 3.8 Max bills at $2 and $6. The cheaper model scores 15.5 points higher on the support task. If you have been assuming the frontier price tag buys frontier support performance, it does not.

The gaps between the top four are small. Qwen 3.8 Max to GPT-5.6 Sol is 8.3 points, and Opus 5 to Grok 4.5 is under a point. These are not different classes of tool. They are neighbours.

Progress is real but slow. Claude Opus 4.5 scored 24.7% on this domain back in February. Opus 5 doubled that to 48.7% by August. Genuine progress across two generations. It still means the model gets it wrong more often than right, which is a very different starting point from what a chatbot vendor comparison usually implies.

Two obvious models are missing. Neither Gemini 3.7 Flash nor DeepSeek V4 Flash has a submission on this domain, so I cannot rank them here honestly. Older Gemini entries used a different retrieval setup, which makes them non-comparable rather than simply lower.

One methodology note worth carrying: the older entries in the leaderboard used different retrieval configurations, so cross-batch comparisons are shakier than they look. Sierra also re-graded the banking domain in July 2026 and says results from before that fix are not comparable. Anyone quoting a banking score from earlier in the year is probably quoting a stale one.

Right once is not right every time

The column most people skip is the one I would look at first.

τ-bench reports pass^k, which asks whether a model got the same task right on every one of k independent attempts. Pass^1 is "it can do this." Pass^4 is "it does this reliably."

The best model in the field is reliably correct on 35.1% of these tasks. Opus 5 drops from 48.7% to 32.0%. GPT-5.6 Sol drops from 46.9% to 27.8%. GLM-5.2 falls off a cliff, from 37.1% to 13.4%, which tells you it is guessing well rather than reasoning consistently.

That spread between pass^1 and pass^4 is the number that decides whether you can leave an AI alone with a queue. A model at 48.7% pass^1 and 32.0% pass^4 is not a model that resolves half your tickets. It is a model that will handle the same ticket type differently depending on the day.

A CX lead at a DTC supplements brand running roughly 7,000 Gorgias tickets a month said the operational version of this to me better than any benchmark could:

"The AI will never be able to answer 100% of the questions, but if it tries and just answers 'sorry I don't know this,' I cannot go and check all my 7,000 tickets to see if the AI actually made a good answer, then the point is a little bit gone. I need an AI who is only handling the tickets that it's confident to handle and all the other ones, leave them alone."

That is pass^k stated as a business requirement. Variance is not a quality problem you fix with a better model, it is a routing problem you fix with confidence-based escalation. Every serious support deployment I have worked on ends up here, and it is the reason a copilot setup is often the safer first phase than full autonomy.

The three ways models actually fail on a real ticket

Sierra published the failure taxonomy alongside the benchmark, and reading it felt uncomfortably familiar. These are not exotic AI failures. They are the exact three things support managers complain about.

It misses the policy that depends on another policy. In the benchmark's worked example, a customer wants to dispute a transaction and request a credit limit increase. The agent searches the dispute policy, finds it, and files the dispute first because that is the order the customer asked in. It never searches the credit limit policy, which says limit increases are automatically rejected when a dispute is pending. The customer gets a denial that a human agent would have avoided by doing the two steps in the other order.

It reports success for something that failed. Many of the benchmark's tools have to be unlocked before use. When the agent skips that step and gets an error back, Sierra's writeup notes that some agents then hallucinate a success response to the user. That is the failure mode that costs you a customer rather than a ticket. A Danish B2B vehicle-telematics team on Zendesk, doing about 200 tickets a month and scaling toward 2,000, hit the consumer version of it when their bot confirmed support for car brands that were not in the database, because the knowledge base said "we support all models."

It believes the customer. When the simulated user claims their dispute was approved, agents frequently apply the credit without checking the actual status. Every support team has a version of this. One eCommerce support manager was blunt in our own dashboard chat:

"stop promising customer things we cant do. we cannot guarantee this customer's order to them by friday"

None of these three are solved by a bigger model. They are solved by scoping what the agent is allowed to touch. There is a comment on Hacker News that frames this better than most vendor documentation:

"Give your LLM the same level of permissions that you would give a human you just hired in the same role. The examples given, tricking the customer support LLM into sending text messages to all users, or into transferring money, are not things that you would ever give a human customer support agent the tools to do."

soerxpso, Hacker News

That is the whole security and reliability posture in two sentences, and it costs nothing to adopt. It also happens to be how most durable customer service automation gets built, one narrow permission at a time.

What each model actually costs per ticket

Let me put real numbers on this, because "it depends on your usage" is a non-answer.

A support ticket answered with retrieval is a reasonably predictable shape. Call it 15,000 input tokens (system prompt, policy, the retrieved knowledge base chunks, conversation history) and 1,500 output tokens (the reply plus reasoning tokens, which bill at the output rate on every major provider). Your numbers will differ, but the ratios between models will not.

Rates come from each provider's own pricing page. The Anthropic and Qwen figures are linked in the section above; the rest sit on OpenAI's rate card and Google's, with DeepSeek's linked further down.

Four traps hide in that table.

Gemini's price is promotional. Google's $0.75 and $3.75 run through December 31, 2026, then double to $1.50 and $7.50 on January 1. If you are building a cost model on Gemini 3.7 Flash today, build it on next year's rate.

DeepSeek now charges by time of day. DeepSeek's rate card lists peak hours of 01:00 to 04:00 and 06:00 to 10:00 UTC at exactly double the off-peak rate, so the $0.22 and $0.66 above become $0.44 and $1.32 during those windows. Those hours track Chinese business hours, which for a US or European support queue mostly land outside your busiest period. That is a genuine quirk in your favour, and it is also fragile.

Sonnet 5's $2 and $10 is introductory. Anthropic's published rate reverts to $3 and $15 on August 31, 2026. That is two weeks from this writing.

Data residency carries an uplift. Anthropic applies a 1.1x multiplier when you pin inference to US-only, and OpenAI charges a 10% uplift on regional processing endpoints for models released after March 5, 2026. If you are handling customer PII under a residency requirement, add it in.

Try the numbers on your own volume

The model is around 7% of your bill

Run that calculator with any realistic platform meter attached and the shape of the answer stops changing.

Zendesk publishes $1.50 per automated resolution on a commitment and $2.00 pay as you go, sitting inside the plan comparison table rather than on the plan cards. Gorgias meters $1.50 per automated interaction. Freshdesk charges $0.49 per session past the included 500. Help Scout is $0.75 per resolution, with the fairest rule in the category, one resolution per conversation no matter how many questions the AI answers inside it.

Put Claude Opus 5 behind Zendesk's committed rate and your per-ticket cost is $1.61, of which the actual intelligence is $0.11. Downgrading to the cheapest model on the market saves you six cents and costs you nine benchmark points. That is the worst trade in this entire post.

The billable unit matters more than the rate, too. A session meter charges again when a solved ticket reopens the next day. A per-interaction meter charges even when the AI hands off to a human. If your reopen rate is high, those two details cost you more than any model decision. Our breakdown of AI resolution rate goes through how the definitions differ between vendors. For the comparison against a headcount baseline, see AI versus human agent cost.

So which model should you actually pick

Here is my honest read, with the reasoning attached rather than a shrug about your specific needs.

If you want the best support scores and residency is not a blocker: Qwen 3.8 Max. It tops the knowledge domain at 55.2% and costs $2 and $6, which is less than half of Opus 5's output rate. The catch is real and you should weigh it properly: it is served from Alibaba Cloud, which for most Western support teams handling customer PII is a compliance conversation before it is a technical one. Check your DPA obligations before you check the benchmark.

If you need a Western-hosted default: Claude Opus 5 or GPT-5.6 Sol. They are within two points of each other on the benchmark and within eight cents per thousand tickets. Opus 5 has slightly better four-of-four reliability, 32.0% against 27.8%, which matters if you are automating unsupervised. Both are well documented and have real enterprise agreements behind them.

If you are running high volume with a clean, small knowledge base: Gemini 3.7 Flash or Claude Haiku 4.5. No τ-bench banking submission exists for either, so I cannot rank them against the field honestly. What I can say is that the retrieval cliff cuts both ways: when your knowledge base is 40 well-written articles rather than 698 messy documents, the task looks a lot more like the telecom domain where nearly everything scores above 80%. Price the promotional rate carefully on the Gemini side.

If you are cost-constrained and can accept variance: DeepSeek V4 Flash. It is by far the cheapest credible option, and the off-peak windows genuinely favour Western business hours. Two caveats. Its paid-API terms are silent on training use rather than explicitly protective, and there is no published zero-retention option, so it is not a drop-in for customer data. And Artificial Analysis measured a high hallucination rate on it, which is exactly the wrong failure mode for a customer-facing reply.

Skip Claude Fable 5 for this job. It is a superb model and it is the most expensive one here, at $10 and $50. On the support benchmark it ranks sixth, below Opus 5 and GPT-5.6 Sol, both of which cost half as much or less. There is no reading of the data where it is the right pick for a ticket queue. The same goes for Kimi K3: at $3 and $15 it costs more than Qwen 3.8 Max and scores 18 points lower.

One more thing worth naming plainly: hallucination rates went the wrong way across this generation. Artificial Analysis measured Claude Opus 5's rate climbing 14 points to 50%, and Gemini 3.7 Flash regressing to 64.5% from Gemini 3.6 Flash's 55.6%, even as accuracy improved. Models got smarter and more confidently wrong at the same time. For AI in customer service, that trade is bad in a way it is not for a coding assistant, because a wrong answer goes straight to a customer.

What to fix before you touch the model dropdown

If the retrieval cliff is real, and I think the evidence is strong, then here is where the leverage actually lives.

Fix the knowledge base first. This is the single highest-return change available. The τ-bench discussion on Hacker News has a demonstration of the principle that stuck with me: someone rewrote the benchmark's policy documents into clear decision trees and numbered steps, without touching the model or the tasks, and lifted GPT-5-mini by 22%. The author's own explanation of why is the sharpest line in the thread:

"If a model only performs well once the rules are clarified, that's still revealing something important about its agency: it's brittle when policies are ambiguous, but much stronger when they're structured."

blndrt, Hacker News

Twenty-two points from rewriting documentation. No model upgrade comes close to that. Our guides on AI for knowledge management and the best knowledge base tools cover the practical side.

Scope the tools like you would scope a new hire. Back to the permissions point. If the AI cannot issue refunds above a threshold, it cannot be talked into issuing one. This is cheaper and more reliable than prompting it not to, and it is what separates a real AI teammate from a chatbot with API keys. Working agent examples tend to have narrow tool lists, not clever prompts.

Route on confidence, not on ticket type. Pass^k says the same model handles the same ticket differently across runs. The answer is not a better model, it is letting the AI decline. Set up handoff rules that treat "I am not sure" as a valid, common outcome rather than a failure. This is also where agent coaching pays for itself, because the declined tickets tell you what to write next.

Measure the right things in production. One Hacker News commenter described watching a vendor track τ-bench airline on a 32-day rolling average, alongside tool call error rate and structured output error rate. That is a better dashboard than resolution rate alone, because it separates "the model got it wrong" from "the model called the wrong tool." Our writeup on customer service metrics covers what to pair those with.

Test on your own history before go-live. Every benchmark in this post is somebody else's queue. Yours has different products, different edge cases, and a different tone. The only test that tells you whether a model will work for you is a replay of your own resolved tickets.

That last one is not optional, and it is the step teams building on the raw API most often skip. It is also the reason a benchmark ranking should inform your shortlist and never decide it.

Try eesel for your ticket queue

Everything above points to the same conclusion: the model is a component, and the part that decides whether your support automation works is the scaffolding around it. That scaffolding is what eesel is.

The piece I would point at specifically, given this post's argument, is simulation. You can ask eesel to run a simulation over your recent tickets, and it replays hundreds of your past tickets and scores the AI's answers against what your team actually sent, then hands back the gaps and suggested instruction changes. It is the pass^k problem addressed with your own data instead of a benchmark's. It does not forecast a resolution rate for you, and I would be sceptical of any vendor that claims to.

It connects to Zendesk, Gorgias, Freshdesk, Front, Help Scout, HubSpot, Salesforce and Jira Service Management. Pricing is $0.40 per ticket or chat handled, with no per-seat fee. There is a $50 free trial with no card required, which for most teams is a few hundred real tickets of testing.

If you have been comparing model names in a spreadsheet for the last week, this is the shortcut: pick any of the top four, connect your helpdesk, replay your own tickets, and let the results tell you whether the model was ever the problem.