Laya AI review: the open 33ms decision model, tested honestly

My verdict at a glance

If you only read one section, read this table. It is the review compressed into the six dimensions I actually care about when a model like this lands on my desk.

The shape of that scorecard is the whole review: brilliant at being fast, honest, and cheap; middling until you invest in fine-tuning; weak on wide classification. If you want the full architecture walkthrough rather than the verdict, I wrote that up separately in the Laya AI deep dive.

What Laya AI is, in one paragraph

Most AI you have used is autoregressive: it predicts the next token, then the next, building a sentence. That is great for writing a reply and wasteful for a yes-or-no call, the kind of gap that separates a real conversational AI platform from a raw model. Laya is the other kind, a bidirectional encoder that reads the whole input at once and answers a fixed question by scoring your options instead of writing them out. Convai calls it a System 1 model, the instant reflexive judgment, against the slow System 2 reasoning of a generative LLM. The builder, Nandakishor Mukkunnoth, sums up the motivation in one line:

"Not every AI problem requires an autoregressive chatbot."

He is right, and if you have ever bolted an 8B model onto a prompt that ends in "respond only with JSON" just to route a ticket, you have felt the exact pain he is solving. This review assumes you already buy that premise, and focuses on whether the execution holds up.

Should you use Laya AI? A quick fit check

The buying decision here is unusually clean, so rather than bury it in prose, here is the decision I would walk a colleague through. Pick the branch that matches you.

What the benchmarks actually say

Here is the full comparison the project published against Jev. It is dense, but the headline is that Laya (routed) edges Jev on the shared public datasets while being far faster and better calibrated.

The three numbers worth carrying, all measured on a single Tesla T4 with Jev figures third-party published (Convai had no API access):

• Latency: 32.8ms p50 for one routed question, against Jev's 236 to 276ms. That is the 7.8x.
• Calibration: ECE of 0.081 after temperature refit, against Jev's 0.246, a 3x better calibration score.
• Accuracy: 0.766 on the 2,000-decision typed set, against Jev's 0.727, and it clears the 0.735 teacher ceiling.

And then the asterisk that dominated every serious discussion of the launch. That 0.766 is the fine-tuned checkpoint. The base models, untouched, score around 0.362 on the same benchmark, a hair above the 0.318 random baseline. Convai's own limitations section says so plainly, which I respect. On Hacker News, where the launch cleared 1,300 points, the top critical comment was exactly this:

"This should be way up in the article. Fine tuning is a pain, requiring it for good results put Laya in a whole different category vs Jev"

cjalmeida, Hacker News

That is a fair hit, and it is the single most important line in this whole review. Jev is sold as usable zero-shot; Laya, read honestly, is a base you specialize.

Where Laya genuinely shines

Strip away the Jev drama and Laya is a fast classifier with honest confidence, which is worth a lot. The calibration is the piece I would actually reach for it to get, and it lines up with something I learned the hard way running support automation: the dangerous failure is not a bot that says "I don't know," it is one that is confidently wrong. A model that reports honest probabilities lets you gate on confidence and escalate the shaky calls, and that is worth more to me than a point of raw accuracy. Convai trains it with a scheme they call RLCD, reinforcement learning against strictly proper scoring rules, and Nandakishor's line on why naive training fails is the sharpest in the write-up:

"Naive RL maximizes accuracy by destroying calibration. It turns your model into a confidently wrong machine."

The per-workflow results back up where it is strong. Laya reports:

• Email spam filtering: 0.993
• Phishing detection: 0.980
• LLM guardrails and jailbreak detection: 0.755 to 0.762 (0.931 at 50% selective coverage)
• RAG passage relevance: 0.657
• Support ticket routing, 10-way: 0.522

The binary safety calls are excellent, and the routing score is a solid base for the kind of AI agent examples teams actually ship. The operational story is just as good: it runs 45 of 51 languages above 3x random when routed, against zero published multilingual numbers from Jev.

On cost it is Apache 2.0 and self-hosted, so it is $0.00 per million tokens against Jev's $0.042, a gap that adds up fast once you compare it to the cost of a human agent, and it runs air-gapped on your own hardware, which matters for anyone with HIPAA or GDPR constraints. Installation is a one-liner, pip install laya, and Router(preload=True) keeps every checkpoint resident so you dodge a 7-to-10-second cold reload on language switches.

Where it falls short

I want to be fair here, because the gaps are real but narrower than the internet made them sound. Three things stood out in testing.

First, the fine-tuning requirement is not optional for accuracy work, which rules Laya out as a no-code chatbot for non-engineers. The zero-shot 0.362 is the honest number, and a practitioner on HN put the cost of that better than I could:

"Idk, your limitations section sure makes it seem less drop in and less general than Jev... I don't want to manage GPUs / CPUs now. I don't want to maintain my corpus and retrain as my product's data distribution shifts... I do just want a reasonably good general classifier served to me with a great devex and calibrated confidence scores to help me figure out when to fallback."

ianbutler, Hacker News

Second, high-cardinality classification struggles, which matters if you were hoping to replace a fleet of custom AI models with one router. On Banking77 (77 labels) Laya scored 0.425 against Jev's 0.870, because each option only gets 3 to 4 tokens to be distinct at the default budget. The fix is to raise the token budget or split into a coarse-to-fine hierarchy, but out of the box a 10-way router is comfortable and a 77-way one is not.

Third, the multilingual failure mode is nasty if you ignore the router. The English root model collapses outside Latin scripts: on the 51-language sweep it scored 0.000 accuracy on Khmer while reporting 0.952 confidence, the exact "confidently wrong" trap the model is meant to avoid. Laya ships a built-in Router that detects script in under half a millisecond and picks the multilingual checkpoint, so this is solvable, but you have to use it.

None of these are dealbreakers. They are the boundaries of a component, and they are all documented, which is more than most launches manage.

What the community actually thinks

The reaction was loud and, once you filter the priority-claim drama, useful. The single most-repeated criticism was the near-random zero-shot caveat, pasted verbatim by several people who thought it belonged at the top of the announcement. The counter-take from the enthusiast camp was just as consistent:

"In the limitations you say that 'Out-of-the-box base models score ~0.35 on the typed-decisions benchmark (near random)...' - but that's the whole point, you can obviously fine-tune specialized models but having a model follow you instructions and be promotable and fast makes it massively easier to use."

lopuhin, Hacker News

And the practical value, when someone cut through the noise, was clear:

"The ability to knock out any arbitrary classification problem in minutes instead of in a week is a big deal... I don't want to spend a week to create a dataset, finetune a model, and set up infrastructure to run the model, every time I have a small routing or classification problem."

soerxpso, Hacker News

There was also broad relief that the open version exists at all, even from skeptics who read the sibling Jev launch's "can't hallucinate" language as VC-bait. That is roughly where I land too, and it echoes how the community reacted to Anthropic's Claude updates for support: earned trust beats loud claims.

The controversy, briefly

You cannot review Laya without the origin story, because it is half the reason it went viral. Nandakishor's framing is a priority claim: he says he published this non-autoregressive, RL-trained approach in a March 2025 arXiv paper a full year before TypeSafe launched Jev.

The community split on it fairly, some noting that clean branding and a usable API are themselves the product. My review take is that the credit fight is a distraction from a strong release. Whether or not Laya was first, it is open, fast, honestly documented, and it works well for the jobs it is honest about. I unpack the full timeline in the Laya AI overview if you want it.

Where a decision model fits versus an AI teammate

Here is the part I care about most, because I build this for a living. A decision model and an AI teammate are not competitors, they live on different layers.

Laya is infrastructure. It answers "which queue does this ticket belong in" or "is this urgent" in 33ms, and it does that one thing beautifully. But it does not read your help center, it does not draft the reply, it does not check the order status, and it does not close the conversation. Those are the actual jobs a support team needs done, and they need a model that acts, not one that only classifies. That is the difference between a component and an employee.

eesel is an AI teammate platform, and the AI helpdesk teammate is the one hired for support. It joins your existing queue in Zendesk, Freshdesk, or wherever you already work, reads your past tickets and docs, and resolves the whole ticket end to end. Under the hood it is making the same fast, calibrated routing and confidence calls that Laya is good at, but it wraps them in the reading, drafting, and action that turn a classification into a resolved conversation. If you are a developer who would rather script that teammate than click it, the eesel CLI and MCP server let a person, a script, or a coding agent like Claude Code drive the same workspace the dashboard uses, which is the natural home for exactly the kind of programmatic classification Laya fans want.

Try eesel

If you are looking at Laya because you want smarter, faster support decisions, it is worth being clear about what you are buying. A decision model gives you a component to build with. An AI support teammate gives you the finished worker.

The reason I trust the calibration argument Laya makes is that I have lived the alternative. I have watched confident-sounding bots quietly give wrong answers on live queues, which is why every eesel rollout runs a simulation over your own historical tickets first, so you see the exact resolution rate and the exact answers before a single customer is touched. Pricing is usage-based at roughly 40 cents per resolved ticket with no per-seat fees, and it is free to start. If you want the decision acted on and not just labelled, that is the layer to look at.

What I would tell a friend

Laya is a real contribution, and I am glad it is open. If you are a developer with classification or routing problems and the willingness to fine-tune, it is one of the most interesting new tools of the year: fast, calibrated, multilingual, and free. Go read the model card first, it is refreshingly honest about its own limits, and that honesty is a big part of why this review is positive.

But read that limits section before you build on it. The zero-shot numbers are near random, the confidence needs refitting on your data, and wide classification wants a fine-tune. Treat it as the excellent foundation it says it is. And if what you actually need is the whole support job done rather than a single decision returned, reach for a teammate, not a component.