Laya AI: the open 33ms decision model that can't hallucinate

What Laya AI actually is

The tagline on the Laya homepage is "decisions, not text," and that is the whole product in three words. Most AI you have used is autoregressive: it predicts the next token, then the next, building up a sentence. That is great for writing a reply and terrible for a yes-or-no call, because you are running a language model to get a single bit of information and then parsing free text to recover it.

Laya is the other kind of model. It is a bidirectional encoder that reads the entire input at once and answers a fixed question by scoring your options, not by writing them out. Convai calls it a System 1 model: the instant, reflexive judgment, as opposed to the slow, deliberative System 2 reasoning you get from a generative LLM. The builder, Nandakishor Mukkunnoth of Convai Innovations, puts the motivation bluntly:

"Not every AI problem requires an autoregressive chatbot."

He is right, and if you have ever wired an 8B model up to a prompt that ends in "respond only with JSON," you have felt the pain he is describing.

The idea: System 1, not System 2

The pitch rests on a real inefficiency. When you call a generative model to route a ticket or flag a phishing email, you pay for it three times over: 500 to 2,000ms of latency, real inference cost, and brittle string parsing on the way out. Worse, the confidence number an LLM gives you is theatre. As Nandakishor puts it:

"When an LLM outputs 'confidence: 0.95', it is just predicting tokens that sound confident. There is zero mathematical calibration behind it."

Laya's answer is to make the reflex decision its own model, one that returns an actual probability distribution instead of a confident-sounding sentence. The contrast is the entire reason the model exists.

This is the part I find most interesting, because it lines up with something I have learned the hard way. The dangerous failure in support automation is not a bot that says "I don't know." It is a bot that is confidently wrong. A model that reports honest probabilities is worth more than one that is a point higher on raw accuracy, because you can actually gate on the confidence and escalate the shaky ones. Laya is built around that idea from the ground up.

How Laya works under the hood

Mechanically, Laya takes a state plus one or more typed questions and answers them all in one parallel pass. There are three question primitives, and once you see them you realize they cover most of the "decisions" a support pipeline makes:

• choice picks one option from a set you define (routing, intent, topic).
• score places the state on an ordinal rubric like 0 to 3 (urgency, frustration, severity).
• noul is a boolean that returns a calibrated P(true) from 0.0 to 1.0 (phishing, spam, jailbreak, churn risk).

The clever bit is how it scores options. Every option is packed with its own [MASK] token, and the model reads the hidden state at that marker to produce a logit, then softmaxes over the question's options. Because the answer space is defined at request time, new schemas need no retraining, you just pass different options. The English checkpoint is ModernBERT-large at 421M parameters, with a small decision head trained on top.

Then there is the training, which is where the calibration comes from. Laya is trained with what Convai calls RLCD, reinforcement learning against strictly proper scoring rules. The short version: naive training teaches a model to be overconfident, because pushing the winning probability toward 1.0 maximizes a simple accuracy reward. Nandakishor's line on this is the sharpest in the whole write-up:

"Naive RL maximizes accuracy by destroying calibration. It turns your model into a confidently wrong machine."

RLCD instead rewards the model only when its reported probability matches reality, so honest confidence is the way to win. That is a real, verifiable design choice, not a tagline, and it is the thing I would actually reach for Laya to get.

The three checkpoints

"Laya" is really a family of three checkpoints bundled under one Hugging Face repo, and picking the right one matters more than the marketing suggests.

The English root model collapses outside Latin scripts, and not gracefully. On the 51-language sweep it scored 0.000 accuracy on Khmer while reporting 0.952 confidence, which is the exact "confidently wrong" trap the whole model is supposed to avoid. Confidence gating cannot save you there, so Laya ships a built-in Router that detects the script in under half a millisecond and picks the multilingual checkpoint before the forward pass ever runs.

The benchmarks, and the caveat everyone missed

Here is the full comparison the project published against Jev. It is a lot, but the headline is that Laya (routed) edges Jev on the shared public datasets while being far faster and better calibrated.

The numbers worth carrying, all measured on a single Tesla T4 (with Jev figures third-party published, since Convai had no API access):

• Latency: 32.8ms p50 for one routed question, versus Jev's 236 to 276ms. That is the 7.8x.
• Calibration: ECE of 0.081 after temperature refit, versus Jev's 0.246, a 3x better calibration score.
• Accuracy: 0.766 on the 2,000-decision typed set, versus Jev's 0.727, and it beats the 0.735 teacher ceiling.

And then the asterisk that dominated the discussion. That 0.766 is the fine-tuned checkpoint. The base models, out of the box, score around 0.362 on the same benchmark, barely above the 0.318 random baseline. The project's own limitations section says so plainly: treat Laya as a fast foundation to specialize, not a zero-shot oracle. On Hacker News, where the launch hit over 1,300 points, the top critical comment was exactly this:

"This should be way up in the article. Fine tuning is a pain, requiring it for good results put Laya in a whole different category vs Jev"

cjalmeida, Hacker News

That is a fair hit. Jev is sold as usable zero-shot; Laya, honestly read, is a base you specialize. The defenders had a fair rebuttal too, which is that a fast, promptable, calibrated model you can fine-tune in an afternoon is still a big step up from managing a fleet of bespoke classifiers.

What you would actually use it for

Strip away the Jev drama and Laya is a fast classifier with honest confidence, so it fits anywhere you are currently over-paying a generative model to make a small structured call. The project reports real per-workflow accuracy, and the spread tells you where it is strong:

• Email spam filtering: 0.993
• Phishing detection: 0.980
• LLM guardrails and jailbreak detection: 0.755 to 0.762 (0.931 at 50% selective coverage)
• RAG passage relevance: 0.657
• Support ticket routing, 10-way: 0.522

The binary safety calls are excellent; the wide, fuzzy classification is where it needs the fine-tune. The practitioner case for a model like this was put well on HN, and it matches what I hear from teams drowning in one-off routing rules:

"The ability to knock out any arbitrary classification problem in minutes instead of in a week is a big deal... I don't want to spend a week to create a dataset, finetune a model, and set up infrastructure to run the model, every time I have a small routing or classification problem."

soerxpso, Hacker News

There is a real tension in that thread worth naming, because it is the actual buying decision. Some developers want a drop-in they never have to train:

"I don't want to manage GPUs / CPUs now. I don't want to maintain my corpus and retrain as my product's data distribution shifts... I just want a reasonably good general classifier served to me with a great devex and calibrated confidence scores to help me figure out when to fallback."

ianbutler, Hacker News

That is the honest limit of Laya for a lot of teams: it is a superb building block if you are willing to run and specialize it, and less of a fit if you wanted something that just works out of the box.

Speed, languages, and cost

Where Laya is unambiguously strong is the boring operational stuff. It runs 45 of 51 languages above 3x random when routed, against zero published multilingual numbers from Jev.

On cost, it is Apache 2.0 and self-hosted, so it is $0.00 per million tokens against Jev's $0.042, and it runs air-gapped on your own hardware, which matters for anyone with HIPAA or GDPR constraints. Installation is a one-liner, pip install laya, and Router(preload=True) keeps every checkpoint resident so you avoid a 7-to-10-second cold reload on language switches. The GitHub repo is past 9,000 stars, and there is a live demo Space if you want to poke at it before writing any code.

The one caveat for high-cardinality problems: choice questions degrade past about 20 options at the default token budget. On Banking77 (77 labels) Laya scored 0.425 against Jev's 0.870, because each option only gets 3 to 4 tokens to be distinct. The fix is to raise the budget or split into a coarse-to-fine hierarchy, but out of the box, a 10-way router is comfortable and a 77-way one is not.

The controversy: who built it first

You cannot write about Laya without the origin story, because it is half the reason the post went viral. Nandakishor's framing is a priority claim: he says he published this non-autoregressive, RL-trained decision approach in a March 2025 arXiv paper and a September 2025 follow-up, a full year before TypeSafe launched Jev to fanfare.

"It is incredibly frustrating when something you poured your heart into for months as an open-source researcher gets overlooked because it was built for a vertical use case, while a funded lab packages the same core idea horizontally and gets all the glory."

Nandakishor M, Convai Innovations

The community split on it, fairly. Some pointed out that clean branding and a usable API are themselves the product, and that his earlier work was marketed as a niche "sales conversion" tool that nobody read. Others were sympathetic, and several were glad the open version exists at all even while rolling their eyes at the launch hype around both models:

"I had used versions of bert to achieve the same functionality years ago. But to me it seems like they were able to trick the VCs with 'can't hallucinate' etc. To the above author, kudos for sharing your work and making it open."

prometheus1992, Hacker News

My take: the priority argument is a distraction from a strong release. Whether or not Laya was first, it is open, fast, honestly documented, and it works well for the jobs it is honest about. That is worth more than the credit fight.

Where a decision model fits versus an AI teammate

Here is the part I care about most, because I build this stuff for a living. A decision model and an AI teammate are not competitors. They live on different layers.

Laya is infrastructure. It answers "which queue does this ticket belong in" or "is this urgent" in 33ms, and it does that one thing beautifully. But it does not read your help center, it does not draft the reply, it does not check the order status, and it does not close the conversation. Those are the actual jobs a support team needs done, and they need a model that acts, not just classifies.

That is the difference between a component and an employee. eesel is an AI teammate platform, and the AI helpdesk teammate is the one hired for support: it joins your existing queue in Zendesk, Freshdesk, or wherever you already work, reads your past tickets and docs, and resolves the whole ticket end to end. Under the hood it is making the same kind of fast, calibrated routing and confidence calls that Laya is good at, but it wraps them in the reading, the drafting, and the action that turn a classification into a resolved conversation. If you are a developer who wants to script or automate that teammate directly, the eesel CLI and MCP server let a person, a script, or a coding agent like Claude Code drive the same workspace the dashboard uses.

Try eesel

If you are looking at Laya because you want smarter, faster support decisions, it is worth being clear about what you are buying. A decision model gives you a component to build with. An AI support teammate gives you the finished worker.

The reason I trust the calibration argument Laya makes is that I have lived the alternative. We have watched confident-sounding bots quietly give wrong answers on live queues, which is why every eesel rollout runs a simulation over your own historical tickets first, so you see the exact resolution rate and the exact answers before a single customer is touched. Pricing is usage-based at roughly 40 cents per resolved ticket with no per-seat fees, and it is free to start. If you want the decision acted on and not just labelled, that is the layer to look at.

What I would tell a friend

Laya is a real contribution, and I am glad it is open. If you are a developer with classification or routing problems and the appetite to fine-tune, it is one of the most interesting new tools of the year: fast, calibrated, multilingual, and free. Go read the model card, it is refreshingly honest about its own limits.

But read that limits section before you build on it. The zero-shot numbers are near random, the confidence needs refitting on your data, and wide classification wants a fine-tune. Treat it as the excellent foundation it says it is, and if what you actually need is the whole support job done rather than a single decision returned, reach for a teammate, not a component.