Inkling-Small review: a quarter the size, and mostly as smart

Inkling-Small in 30 seconds

What Inkling-Small actually is

The architecture is where the interesting part lives, and it explains both halves of the results.

Inkling-Small is a 42-layer decoder-only transformer with a sparse mixture-of-experts feed-forward stack. Each token gets routed to 6 of 256 experts, plus 2 shared experts which fire on every single token. Total parameters come to 276B; only 12B of those are active per token. The model card confirms hybrid local and global attention layers, BF16 and NVFP4 numerics, then training on NVIDIA GB300 NVL72 systems.

Compare that against the parent. Inkling is 975B total and 41B active across 66 layers, so the routing shape is the same, but with roughly a quarter of the total mass and under a third of the active mass. Thinking Machines describes the result as "comparable performance to Inkling at a quarter of its size."

That ratio is the whole story here. A big total-to-active ratio is what makes a model cheap to serve, and a small total parameter count is what makes it forgetful. Inkling-Small pushed hard on the first one and paid for it on the second. For the general version of that mechanism, custom AI models walks through the same trade without all the benchmark tables.

It takes text, then images (40 to 4096 pixels works best), and audio as WAV at 16 kHz, ideally under two minutes. Output is text only. Reasoning effort is a dial from 0 to 0.99, where low is 0.2, medium sits at 0.7 and max is 0.99, so per call you can trade thinking tokens against latency.

Native audio input is the rarest thing on that list, which is why this model keeps coming up in voice conversations. The limit is worth to flag though: audio goes in, text comes out, so there is no speech synthesis here at all. Vendors that do close the loop are covered in AI voice companies.

Which job is Inkling-Small right for?

The answer changes a lot depending on workload, and for each one the vendor's own table has the evidence. Pick the job you actually have:

The benchmarks: it really does beat its parent

A distilled or shrunken model normally lands a bit behind the flagship it came from. Inkling-Small does not, at least not on the categories most people buy a model for.

On coding it takes SWEBench Verified 80.2% against 77.6%, then SWEBench Pro 55.9% against 54.3%, with Terminal Bench 2.1 at 64.7% against 63.8%. On the agentic side it takes Toolathlon Verified 54.4% against 45.5% and MCP Atlas 79.6% against 76.0%, plus a GDPval-AA v2 Elo of 1269 against 1238. On general reasoning it takes GPQA Diamond 89.5% against 87.2%, and Humanity's Last Exam 31.6% against 29.7% text-only.

It also posts numbers which the parent's card does not report at all. SciCode 48.7%, CritPt 8.3%, then ARC-AGI-1 84.0% and ARC-AGI-2 40.1%. On instruction-following, IFBench comes in at 82.2% against 79.8%, and that matters more than it sounds for anything which has to obey a system prompt reliably. If that is the axis you care about, AI agent vs rule-based chatbot explains why in production obedience beats raw intelligence.

The losses on the reasoning side are real, but narrow. AIME 2026 drops to 95.5% from 97.1%. Tau 3 Banking drops to 15.5% from 23.7%, which is the sharpest single regression anywhere in the agentic block, and worth noting if your workload looks like structured financial workflows.

One more result deserves a flag, because it is easy to miss. Global-MMLU-Lite, the multilingual test, slips to 86.7% from 88.7%. A two-point drop, so nothing dramatic, but it points the same direction as the factuality numbers do: the cut came out of breadth. If you serve several languages then test in each of them rather than trusting the aggregate, and multilingual support agents covers what that testing looks like in practice.

Independently, Artificial Analysis scores it 40 on their Intelligence Index at rank #15 of 101, against 41 and #13 for the full Inkling. One index point apart, at a quarter of the size. Third-party scoring and vendor scoring landing on the same conclusion is unusual, and it is the strongest thing in this review.

The one number that should stop you

Here is the part which did not make the launch post.

SimpleQA Verified, which tests whether a model knows short factual answers, comes in at 20.6% for Inkling-Small against 43.9% for Inkling. Less than half. And AA Omniscience, which nets correct answers off against confidently wrong ones, reads -9.0 where the parent gets +2.1.

A negative Omniscience score means the model states more wrong things confidently than right things. This is not a model which says "I'm not sure." It is a model that fills the gap. Thinking Machines lists this themselves under known limitations, naming "hallucination (generating plausible but factually incorrect or unsupported content)" and "degraded performance in long multi-turn conversations," and they recommend against medical, legal or safety-critical deployment without further fine-tuning. Credit where it is due, that is a more candid limitations section than most labs bother to publish.

For a coding agent this barely matters, since the compiler is the fact-checker. For anything customer-facing it matters a great deal, and I have watched the failure mode in production. One support team I worked with had a knowledge base saying they support all vehicle models, and so their bot cheerfully confirmed coverage for car brands which were not in their database at all. Nothing hallucinated in the dramatic sense. The model just filled a gap with something plausible. That team described their early setup as "trial and error in the beginning."

A model at -9.0 on Omniscience is that same failure mode with the volume turned up. The mitigations are known and boring, which is retrieval over sources you control, citations a person can check, plus a refusal path. AI hallucinations in support covers the full set, and the shorter practical version is AI hallucination prevention.

Price and speed: the real reason to care

This is where the small model earns its place in the world.

Output tokens cost 3.4x less and arrive 1.5x faster, for one index point of intelligence. For agent loops, where output tokens dominate the bill and latency compounds over dozens of turns, that is not a marginal difference. It also shows up in the numbers a support team actually reports on, because faster first tokens move first contact resolution along with the rest of the customer service metrics which follow from it.

Two caveats before you go and budget on it. Input pricing is not settled yet: Artificial Analysis lists $0.30 per 1M, while the OpenRouter listing shows $0.45 and its API returns $0.50, so check your own route. Also there are only two providers so far, against four for the parent, and that is thin if you need failover.

The second caveat is the more surprising one. The model card says 1M context. OpenRouter currently serves it at 524,288 tokens, which is exactly half. The weights support the full window, the routed API just does not expose it yet. If a million tokens is the reason you picked this model, then self-host or verify with your provider first. Context window size covers why the advertised number and the usable number diverge so often.

The actual business at Thinking Machines is Tinker, their hosted LoRA fine-tuning platform, and Inkling-Small is supported on it. Checkpoint storage runs $0.10 per GB-month; per-token training rates are not published on the overview page.

Running Inkling-Small yourself

Self-hosting is the clearest upgrade over the parent model, and here the numbers are not close.

Unsloth's numbers put Inkling-Small at 543 GB in BF16, then 132 to 170 GB at 4-bit, 128 GB at 3-bit, and 89 GB at 2-bit. The full Inkling needs 1,900 GB at BF16 and it still wants 270 to 285 GB even at a 1-bit quant. The model card frames the same thing in hardware terms: 600 GB of aggregated VRAM for BF16 (4x B300 or 8x H200), dropping to 180 GB for the NVFP4 checkpoint, so one B300 or a pair of H200s.

The 89 GB figure is the one that changes who can run this. That fits inside a 128 GB unified-memory machine, which is a workstation purchase and not a datacentre one. And because only 12B parameters are active per token, a heavily quantized build stays usable instead of crawling.

Recommended sampling settings are temperature 1.0, top_p 1.0 and min_p 0.0, on a 1,048,576-token context. Day-one support covers transformers, vLLM, SGLang, TokenSpeed, Unsloth and Docker Model Runner, with quantized builds through llama.cpp, Ollama, LM Studio and Jan. One thing to watch, llama.cpp needs PR #25731 merged in. Broader self-hosting options sit in best open source AI agents.

Pros and cons

What's good

• Beats a model 3.5x its size on coding and tool use, plus instruction-following
• $1.20 per 1M output tokens, so 3.4x cheaper than the parent
• 131 tokens per second, which is meaningfully faster
• Apache 2.0, so commercial self-hosting is not restricted
• Runs at 89 GB quantized, inside workstation range
• Native audio and image input, still uncommon at this price
• A candid limitations section from the maker itself

What isn't

• SimpleQA Verified at 20.6%, which is less than half of the parent's
• AA Omniscience at -9.0, so more confident wrong answers than right ones
• 1M context on paper, but 524K through OpenRouter today
• Only two API providers, which makes failover thin
• Tau 3 Banking drops to 15.5% from 23.7%
• Audio scores trail the parent slightly across all three of the tests
• Text output only, so no image or audio generation

What the community says

The reaction was a lot quieter than the parent launch, which drew over 1,200 points on Hacker News. Inkling-Small's own thread took 33 of them. Sizing it up against the field was the first instinct:

"better than haiku 4.5 smaller than nemotron 3 ultra"

tosh, Hacker News

The other comment placed it against the obvious open-weights rival, noting it is "About the same size as DeepSeek Flash 4, but also supports audio and image input." A fair framing. DeepSeek V4 Flash is cheaper per token and text-only, so the audio and image support is the real separator here, and the head-to-head detail sits in the Flash review.

The sharpest comment came before the quants had even shipped, over on the Hugging Face submission:

"I didn't see a gguf yet, going to be most interesting if there's a quant that fits nicely into about 90 [GB] so it can run in 128GB unified memory. It's 12B active so should hopefully be pretty fast at 2 bit quant if it fits"

andy99, Hacker News

Unsloth's 2-bit build landed at 89 GB. The local-inference crowd called the target number before the release existed, and it hit it. In its first month the Hugging Face repo has taken 15,500 downloads.

One thing I noticed: nobody in those threads mentioned the factuality regression at all. The launch framing was speed and size, so that is what got discussed.

What this means if you want it answering tickets

Here is where I will be direct, because this part is my actual job.

A cheap fast model looks like an obvious win for support. Ticket volume is high, replies are short, and $1.20 per 1M output tokens against $4.05 is real money at scale. That's the arithmetic behind AI customer service cost, and it's the same maths that drives the human agent comparison. Teams tracking it as a unit rate usually land on cost per resolution.

But support is the one workload where -9.0 on Omniscience becomes the disqualifying number rather than a footnote. A coding agent which invents an API gets a stack trace back. A support agent that invents a return window sends it to a customer instead, in writing, with your logo sitting on it. Buyers I talk to already know this, and it is the objection which comes up first. One CX lead at a DTC supplements brand put it about as plainly as it gets:

"The AI will never be able to answer 100% of the questions... I need an AI who is only handling the tickets that it's confident to handle and all the other ones, leave them alone."

a DTC supplements CX lead, from an eesel sales call

That is a request for a layer, and not for a model. Four things have to sit in between any model and your queue:

1. Retrieval over sources you own, so that answers come from your help centre and past tickets instead of from the weights. AI knowledge base chatbot covers the pattern; for the tooling side there is best AI knowledge base tools.
2. Confidence routing, so the model answers what it's sure of and escalates the rest. The routing side is AI escalation management; the clean transfer to a human is agent handoff.
3. A dry run before go-live, simulated against your own historical tickets rather than against somebody's benchmark suite. This is the step teams skip, and it is also the one which catches the confident-wrong answers.
4. Scoped autonomy, starting as a draft-writing copilot before it replies on its own. AI copilot is the on-ramp; tier 1 deflection is where it ends up, measured on deflection rate.

Do those four and the factuality gap in Inkling-Small stops being a liability, because the facts stop coming out of the model. Skip them and what you bought is a fast, cheap, confident source of plausible answers. That is the difference between an AI helpdesk agent and a raw API key, and it is the same conclusion which AI agents vs AI chatbots reaches from the other direction.

It is also the reason a good support layer stays model-agnostic. Inkling-Small is not the first model this year to become the obvious cheap default, and it will not be the last one either. Anything hard-wired to a single model has to get rebuilt every time the leaderboard moves, which is a bad way to run customer service automation.

Verdict

For most jobs Inkling-Small is the better buy over its own parent, and on price it is not close. Same intelligence tier by independent scoring. Better at coding and at tool use by the vendor's own numbers, 3.4x cheaper output, 1.5x faster, Apache 2.0, and small enough that you can self-host it on a workstation. Fifteen days after the flagship, the small model made the flagship hard to justify.

The asterisk is specific and easy enough to state: it knows measurably less, and it does not act like it. Give it a compiler, a tool loop, or a retrieval layer and it's excellent. Ask it to be the source of truth and it will confidently be wrong.

If you want it for code or agents, take it today. If you want it near customers, budget for the layer above it, not just the tokens.

For the wider field of what else fits this slot, start with Inkling alternatives. The closest rival on price and size gets its own breakdown in Kimi K3 review.

And if the actual job is sorting an inbox rather than writing code, support ticket triage is the more useful read, with ticket deflection covering what happens after triage lands.

Try eesel

Picked Inkling-Small and now you need it to safely answer real tickets? That gap is exactly the thing eesel closes. It plugs into Zendesk, Freshdesk, Gorgias or whichever helpdesk you run in a few minutes, learns from your resolved tickets and your help centre instead of from the model's weights, then answers only what it is confident about and leaves the rest for your team. Before it touches a live queue you can simulate it over your own ticket history and see the actual replies it would have sent, which is the check that a -9.0 factuality score makes non-negotiable. It stays model-agnostic as well, so the next cheap model is a settings change and not a rebuild.

Free to try, and usage-based, so you can check the pricing before you commit. Try eesel or see how it fits as AI customer service software.

Sources

• Inkling-Small model card
• Introducing Inkling-Small
• Inkling model card
• Inkling-Small on Hugging Face
• Artificial Analysis: Inkling-Small
• Artificial Analysis: Inkling
• Unsloth: running Inkling locally
• OpenRouter: Inkling-Small
• Tinker fine-tuning platform
• Hacker News: Inkling-Small
• Hacker News: Hugging Face submission