Strands Decider 2B: AWS's free decision model, tested against the hype

What is Strands Decider?

Strands Decider 2B is a small decision model from Strands Labs, the experimental arm of AWS's open-source Strands Agents project. The launch post lists Marc Brooker, Mike Chambers and Fabio Nonato de Paula as authors, and everything ships under Apache-2.0.

A decision model doesn't write text. Think of it as a small language model with the talking removed. It answers questions you define, picked from options you define, and gives you a probability for each. The category took off after TypeSafe Jev launched in September, and the Strands team says so plainly: it is "a type of model that has been gaining a lot of attention since TypeSafe AI's launch of Jev."

I build eesel's integrations and helpdesk API surface, and the triage call (which team, how urgent, is this spam) is the step I've wired into more helpdesks than any other. So I read this release the way I'd read a part I might drop into a pipeline: what does it return, how fast, where does it break, and what's left for me to build.

Here is the short spec sheet:

Every row comes from the project's GitHub README and its Hugging Face model card.

How does Strands Decider work?

The trick is subtraction. Strands takes a normal language model, keeps its "torso" (the layers that understand text), and throws away the head that would normally generate words. In its place sits a tiny pointer head, just over a million parameters, that scores each answer option. The README sums it up as "One forward pass, no generation, no decoding loop."

Because it never generates, it can't wander off format. If you ask for one of billing, sales or retail, you get one of those three, every time. That's the same promise as function calling on a big LLM (or JSON mode, its looser cousin), but cheaper, because there's no token-by-token loop and no JSON to parse.

It also means more questions cost very little. The model reads your text once, and "each question adds only its own tokens." So asking "which team?", "is it urgent?" and "how frustrated is the writer?" in one call is close to the price of asking one.

The Hugging Face card's own demo is a support ticket, which tells you exactly who AWS thinks the buyers are:

To run it as a service, you start the bundled server and post to it:

pip install strands-decider
strands-decider serve StrandsAgents/strands-decider-2B-hobson-v21 --port 8000

The weights download from Hugging Face on first use. Then POST /v1/systemone with a state and a questions object, and you get back per-question answers, token usage (output_tokens: 1) and latency_ms. One thing to know before you get excited about swapping it in for Jev: the docs say JevBench's adapter runs against this endpoint unmodified, then add, "Compatibility with the Jev API itself is not verified" (docs/inference.md).

The vendor is also upfront about limits. The launch post says the one-pass design "makes it significantly worse at solving complex problems than reasoning models", and that it is "unsuited for coding, chatbots, document summarization." The model card adds that long, multi-step documents are its weak spot.

How accurate is Strands Decider?

On the JevBench public set (231 tasks), the numbers from the repo are:

The sentence I respect most in the whole release is the one where the team talks down its own upgrade. Across six training runs each, the v19 and v21 recipes "average 172.8 and 172.3 JevBench tasks: the same accuracy." v21's higher headline number is mostly seed luck, and they say so. That honesty is worth more than the extra four points.

Where does that leave it on the board? On the JevBench v1.4.2 leaderboard (89 systems, September 25), v19 sat 50th overall and 3rd of 33 in the 2B class, behind FlyMy.AI's decision-2b (0.753) and Gemma-based system-one-open (0.732), per the evaluation notes. The top of the board belongs to reasoning models, with GPT-6 Luna at 0.996, and then Jev rebuilds on 12B-plus torsos at 0.83 to 0.93. If you want the whole field side by side, the Jev alternatives roundup ranks them.

So the honest framing: Strands Decider is a strong small model, not a strong model. It trades about 23 points of accuracy against a frontier LLM for speed, price and the ability to run on your own box.

A note if you look at the training-history chart in the launch post: it labels two comparison points "strands-decider-2b v10/v11". Marc Brooker corrected that on Hacker News: those are a different model called "decider-2B", "which isn't ours."

Do the confidence scores mean anything?

Calibration is the whole selling point. A confidence score is only useful if it's honest: a label is only useful for automation if you can say "act above 0.9, send to a human below." The repo claims that on unseen short classification tasks, "answers at a confidence of 0.9 or more are right about 95% of the time."

That's encouraging, with two caveats the model card itself raises: the confidence bands were set on short classification only, so "measure on your own traffic before you trust a threshold," and the model reads questions less closely than documents, so a reworded question often gets the same answer. Early users have found a third:

"From what i can tell from my (limited) experiments, there is a lot of variation caused by simply reordering options, plus a heavy bias toward the first option listed"

fxwin, Hacker News

The fix is boring and it works: shuffle option order in your tests, and score the model on a few hundred of your own labelled tickets before you trust any number. Brooker himself suggested on HN that at 2B you can fine-tune on your questions, or simply recalibrate without fine-tuning.

How fast is Strands Decider on real hardware?

The launch post's headline figure is a 115 ms median on an RTX 3090. That's real, and it's a GPU number. Here is the wider picture, with every figure taken from the repo, its pull requests or Hacker News:

The pattern matters more than any single number. Latency grows roughly in line with input length, so a long email thread costs far more than a one-line chat message. And on the kind of CPU box most teams spin up by default, a decision takes seconds. One HN user saw about half a second on six cores; another measured 1,732 ms in a CPU-only Docker container. If the decision sits in front of a customer waiting on a chat reply, or inside an intelligent routing step, you want a GPU or Apple silicon.

How much does Strands Decider cost?

The model is free. Running it is not, and there's no one to pay for it but you.

• No hosted endpoint. The Hugging Face page says "This model isn't deployed by any Inference Provider," and there is no Bedrock listing. Bedrock only appears in the launch post as the LLM in the agent example.
• No authentication. The local server binds to 127.0.0.1 and, in the card's words, "has no authentication: use it for local experiments." Putting it behind a load balancer, auth and monitoring is your job, the same chores covered in the Strands harness alternatives for agent hosting.
• No official llama.cpp build yet. Brooker explained on HN that you can't convert just the LoRA, because "there's a whole pointer head and some custom layers." A community GGUF port exists, as does an ONNX one, but both are unofficial.

Here's how it stacks up against the two hosted options most teams compare it to:

Jev's rate comes from its Workers AI listing, and Clef's from Cloudflare's model page. The full math lives in the Clef pricing breakdown.

At Jev's rate, a million 500-token decisions cost about $21 in input tokens. That's the bar a self-hosted GPU has to beat once you add the engineer-hours to keep it running. For most teams it won't (the same math as any self-hosted model, see Qwen pricing), until volume is very high or the data can't leave your network. That last case, regulated data, is where Strands Decider is easiest to justify.

What did AWS actually release?

This is where Strands Decider stands apart from Jev, and it's the part I'd point a builder at. The launch post says the release includes "all the training data and scripts we used to build the model."

• Training data. Hugging Face datasets, ContractNLI and MuSiQue, synthetic rows checked by open-weight models, and the output distributions of a frozen Qwen3.5-4B teacher. Each source and its license is listed in data/sources.md.
• The recipe. One script, training/recipe.sh all. A retrain takes about 11 hours on one RTX 3090 or about 1 h 10 min on eight H100s.
• A preregistered research log. Every training run states its predictions and failure conditions before it runs. I've rarely seen a model release that does this, and it's the reason I trust the "v21 is the same as v19" admission.

That makes Strands Decider less a Jev competitor and more a kit for building your own triage classifier. If your categories are specific (your product lines, your refund rules, your escalation tiers), retraining on a few thousand of your own labelled ticket classification rows is the path the authors themselves point to. If fine-tuning is new to you, the recipe is one of the cleaner places to learn it.

The repo is moving fast, too. Pull requests merged on October 7 and 8 add support for Gemma 4 torsos (E2B, E4B, 12B and 26B-A4B) and a soup command that averages several training runs into one checkpoint. The PRs name five upcoming checkpoints, and Brooker wrote on HN that "we've got some new ones coming this week." None have published benchmark numbers yet, so I'd wait for them before planning around them.

What are developers saying about Strands Decider?

The Hacker News launch thread passed 270 points within a day. The tone is warmer than the Jev threads were, mostly because the weights are open. The clearest positive report came from someone running it on-device:

"This is a great model. I've been running it on device in chrome extension to filter things like email. It is just the right mix of size, capability and speed to make it generally useful for adhoc bulk classification tasks."

miguelspizza, Hacker News

Co-author Marc Brooker framed where he sees it going, which is a useful mental model for anyone building agents:

"There are a ton a ways to use models like this. Model routing is a popular emerging one. The one that really interests me is hybrid agentic workflows - filling the gap between deterministic workflows (e.g. AWS StepFunctions) and fully agentic workflows that use frontier intelligence."

mjb, Hacker News

The pushback is fair and specific. One thread argued that strong benchmark calibration "does not establish reliability on unfamiliar production inputs" (soltanov). Another commenter noted that Cloudflare's Clef runs on llama.cpp, while Strands needs its own CLI and server for now:

"clef from cloudflare runs on llama.cpp - being locked-in to strands cli would be a bummer and will slow down adoption."

hrpnk, Hacker News

That matches my read. The model is good for its size, the tooling is young, and the people getting value right now are developers comfortable hosting their own inference.

Where does Strands Decider fit in a support queue?

The obvious use is ticket triage: which team, how urgent, is it spam. It's classic intent classification, and decision models are a good fit for that call. This is where I'd set expectations carefully.

A label is a fraction of a triage system. Someone still has to host the model, pick a threshold, test it on real tickets, then actually tag, assign or escalate the ticket in Zendesk or Freshdesk. And after all that, the customer still needs an answer.

Here's what that looks like when the confidence threshold is doing its job:

That last box is the one teams underestimate. On a cross-checked eesel trial on a real e-commerce Zendesk inbox, AI triage hit 93% accuracy and caught 100% of spam (about 22% of the inbox), but only 12% of drafted replies were sent as-is. Sorting is the easy half; the reply is where the work is.

What buyers ask for is exactly what a calibrated decision model promises. A CX lead handling about 7,000 tickets a month put it this way on an eesel sales call:

"I cannot go and check all my 7,000 tickets to see if the AI actually made a good answer... I need an AI who is only handling the tickets that it's confident to handle and all the other ones, leave them alone."

So here's my recommendation by situation:

• You're a developer building an agent or a custom pipeline, your data can't leave your network, and you have a GPU: Strands Decider is a great base. Retrain it on your own labels.
• You want a decision call with zero hosting: start with Jev instead. Clef and OpenAI's Decisions API are the other hosted options.
• You run a support team and want tickets sorted and answered: you don't need a decision model at all. You need something that makes the call and acts on it inside your helpdesk. The AI triage tools roundup covers that route, and the auto-triage glossary entry explains the terms.

eesel for triage, routing and the reply

If the job you're picturing for Strands Decider is "sort my support queue," eesel's AI helpdesk teammate does that job end to end. It plugs into Zendesk, Freshdesk or Gorgias in a few minutes (the Zendesk integration is the most common setup), learns from your past tickets and help center, then tags, routes, escalates and writes the reply.

The same split Strands Decider is built on, a light model for the quick call and a heavier one for the hard part, is already a setting. Each eesel automation has a "Model strength" option: the usual model for drafting and multi-step work, or a lighter one for jobs like tagging or ticket routing. Events that don't match your instructions are skipped and logged with a reason, which is the "leave them alone" behaviour that CX lead asked for. Every action can be set to Auto, Needs approval or Disabled in actions and approvals, so refunds can stay behind a human click while tagging runs on its own.

If you'd have reached for Strands Decider because you like working from a terminal, the eesel CLI gives you the same teammate from the command line. npx @eesel/cli works with no account (a temporary workspace with $5 of usage), every command prints JSON, and errors come back as one JSON line with a retryable flag, so Claude Code, Cursor or Codex can drive the whole setup. A developer or a coding agent can wire a triage job with eesel automations enable <platform> <key> --instructions "...", check a change first with --dry-run, and clear held actions with eesel approvals approve <id>. It's the same agent as the dashboard, and changes sync both ways. More on that in my look at the agent CLI space.

Before anything goes live, the simulation skill runs the agent over your real past tickets and scores each answer, so you see accuracy on your own traffic instead of a benchmark. The free plan includes 100 credits with no card, and paid plans start at $299 a month for 500 tickets or chats on eesel pricing, with no per-seat fee.

Try eesel on a slice of your queue and see how many tickets it would sort, and answer, this week.