Laya vs Jev: Self-Hosting the Decision Model in 2026

TL:DR: A self-hosted decision model is Laya, a 421M-parameter classifier that answers typed questions like "which department" or "should this escalate" in a single 33 ms forward pass, no text generation. Renting the hosted equivalent, Jev, costs $0.042 per million input tokens. Laya is Apache 2.0, so on a single T4 it costs nothing per call, and the numbers that matter are 0.766 against 0.727 accuracy with 7.8x lower latency.

Every app you have ever shipped that needed a decision made about a piece of data has a language model doing it. Is this a billing question or a technical one. Is this review positive. Is this message a prompt injection attempt. Is this customer about to churn. An LLM answers all of these by generating text, which you then parse back into a value your code can branch on. That is a lot of machinery to get a label.

This post looks at the alternative: a model whose only job is to answer typed questions, and what it costs to run it yourself instead of paying per token. That is Laya, and it is the same category of product as TypeSafe's Jev, the hosted decision API it is built to be compatible with.

What a "decision model" actually means in 2026

TypeSafe calls the category System One: you send a document and a set of typed questions, the model returns structured answers your code already understands. Three question types come back with a confidence attached:

• choice — pick one label from a list, with a probability per option. "Which department should handle this."
• score — a value on an ordinal rubric, 1 to 5, with a distribution. "How urgent is this ticket."
• noul — a calibrated probability that a statement is true. "Does this message threaten to cancel."

Nothing is generated, so there is no output to parse, no JSON to validate, and no hallucination to defend against. The contract is the interesting part, not the weights: you ask a question that has a fixed answer space, and you get an answer in that space with a number saying how sure it is.

Laya ships three checkpoints. laya runs a ModernBERT-large encoder at 421M parameters for English, laya-multilingual runs mmBERT-base at 322M parameters for 100+ languages, and laya-typed-decisions is the fine-tuned variant for production workflows. A Router picks between them per request by detecting the script in pure Python, in under a millisecond, before any forward pass runs. The Router exists because of a documented failure mode: on Khmer text the English checkpoint scores 0.000 accuracy at 0.952 confidence. A model that is confidently wrong cannot be confused out of it by thresholding, so the routing decision has to happen upstream of the question.

That is the whole product. Everything below is arithmetic.

The two seats that matter for routing decisions are the top two rows and the last three. The Banking77 row is the honest exception and worth understanding, because it is an architectural limit rather than a tuning gap: options share a fixed token budget per question, so 77 labels get about 3 to 4 tokens each and the text stops being distinguishable. If your label set is that wide, either raise head_max_len and max_len at runtime or shortlist with embeddings and run one pass over the top 20. Above roughly 20 options, Jev wins out of the box.

The cost of self-hosting, computed properly

The free tier of this comparison is where the marketing usually stops and the invoice starts, so run the numbers yourself.

Laya's own published costs per million decisions, on one T4, at the measured batch rate:

At a single T4's measured 72.3 ms per 10 questions, a million decisions is two hours of active GPU. Ten million decisions is twenty hours. On a rented T4 at roughly $0.35/hour, ten million decisions costs about $7. Jev's own rate, at their published $0.042 per million input tokens, is per token, and a support ticket in a routed decision state is easily 300 tokens, which puts a million decisions at roughly $12.60 in input tokens alone. The two are close enough that the honest framing is not "self-hosting is free," it is "self-hosting is a flat cost you control instead of a rate that scales with your prompt length."

The real trade is different. Self-hosting buys three things money cannot: your data never leaves the machine, which matters the moment the documents are customer messages or medical notes; the latency is yours, so a p50 of 32 ms instead of a network round trip; and the weights are yours, which means you can fine-tune on your own labels and the model gets better at your problem rather than the average problem. Renting buys you no GPU, no ops, and a model you cannot change.

For most small teams, the crossover is around a few million decisions a month. Below that, a hosted API is the boring correct choice. Above it, or with any data-residency constraint, the arithmetic flips.

Fine-tuning is where the value actually sits

The base checkpoints are near chance on the typed-decisions benchmark out of the box: 0.362 and 0.342 against a 0.318 random baseline and a 0.461 majority-class baseline. All of the headline 0.766 comes from a checkpoint fine-tuned on that benchmark's own training split. Reading that headline as "this model classifies your data" is the mistake, and the repo says so plainly: treat Laya as a fast base to specialise, not a zero-shot decision engine.

The worked specialisation is the best argument in the project's favour. Using Laya as the operation and target decider for a browser agent, element top-1 accuracy among about 45 candidates went from 0.10 zero-shot to 0.66, and real task success went from 0% to 62%, at 17 to 23 ms per step, on a single 16 GB GPU. That run took roughly four to five hours for four epochs over 30k questions on two free Kaggle T4s. The weights and pipeline are published, and the self-hosting case is the same one we make elsewhere.

That is the shape of the opportunity: a task where a language model currently generates a choice, a few thousand labelled examples, and a free GPU budget. The fine-tuned model then answers the same question 7.8x faster than the hosted option, at a fixed cost, on your hardware. The repo ships a fine-tuning notebook that does the whole loop including calibration fitting, and a browser-agent write-up covering what worked and what did not.

How to choose, the decision tree

• You need under 20 labels, English only, no data-residency constraint. Use the hosted API. Pay the token bill. The wider stack around it stays simple, which is the same argument behind why time tracking apps are overengineered.
• You need under 20 labels and your data cannot leave your machine. Self-host laya. This is the clear win on both control and cost.
• You need over 20 labels. Either raise head_max_len and max_len, or shortlist with embeddings and run the top 20. Do not ship 77 options at default settings.
• Your traffic is not English. Use laya-multilingual or the Router, and fit calibration temperatures before you trust the probabilities. The English checkpoint is useless on non-Latin scripts and will not tell you.
• You are classifying something no one has ever labelled. Fine-tune. The base model is not a product, it is a starting point, and the gain from one domain adaptation run is larger than any accuracy claim on the readme.
• You only need a score or a yes/no, not a label. Use noul as a two-option choice with neutral keys instead. On the English checkpoint, noul has been documented following its false: and true: option labels rather than the input, which is a bug you do not want to debug in production.

That last set of sharp edges is not a reason to avoid the project, it is a reason to read the honest-limits section before you deploy. Confidence gating on act_probability does not work, it reads 1.0 for almost everything; gate on confidence, which reaches 0.77 AUROC on the same labelled decisions. Calibration is shipped over-confident, and refitting one temperature per question type on held-out data moves mean ECE from 0.466 to 0.081. Every one of those is documented with an issue number, which is more than most hosted APIs give you.

Integrating with the rest of your stack

Because Laya's answer payload is schema-identical to Jev's, the server is a baseUrl change away for anything already written against the hosted API. pip install "laya[serve]" adds FastAPI and uvicorn, LAYA_PRELOAD=1 laya-serve binds 0.0.0.0:8000 with all three checkpoints resident, and LAYA_API_KEY turns on bearer auth. There is a NixOS module that runs it as a hardened systemd service reading the token through LoadCredential, and a Docker quickstart under docs/docker.md.

Local runtimes already cover the hardware spread: laya-mlx is a native MLX runtime at 7 to 14 ms per short decision on Apple silicon with no PyTorch and no cloud, and laya-coreml puts it on the Neural Engine at about 5 ms. If you run the Apple half of your stack, that is a decision endpoint inside a laptop with no network dependency at all.

The integration story for the rest of your business is simpler than the model story. You do not need a decision-model platform. You need whichever classifier you are currently paying per token for, moved to a process you own, pointed at the questions you already ask. If you are already running your own CRM and support pipeline, the documents are already there and the questions are the ones your team answers by hand today, and the same ownership argument applies to what runs them.

Bottom line

For a small team making under a few million decisions a month with no residency constraint, rent the hosted model and spend the saved effort on the product. For everything else, and specifically for anything touching customer data, self-hosted Laya at 32.8 ms with Apache 2.0 weights is the boring correct answer, provided you fine-tune and calibrate before you trust a single confidence number.

FAQ

Is Laya better than Jev?

It depends entirely on the axis. Laya is faster, 32.8 ms against 236 to 276 ms p50, better calibrated after temperature fitting, 0.081 ECE against 0.246, free to self-host, and marginally more accurate on the typed-decisions benchmark, 0.766 against 0.727. Jev is substantially better at very wide label sets, 0.870 against 0.425 on Banking77's 77 options, and is better out of the box because it requires no fine-tuning to be useful. Most teams should compare on their own label set, not a public benchmark.

What does a self-hosted decision model actually cost per month?

Server cost only. On a rented T4 at about $0.35 an hour, ten million decisions is roughly 20 hours of active inference, so about $7 a month at that volume. There is no per-call charge and no per-token charge. If you already own the GPU, the marginal cost of a decision is electricity. The comparison that matters is against Jev's $0.042 per million input tokens, where a 300-token document and a million decisions is about $12.60.

Can a decision model replace an LLM for classification?

For fixed label sets, yes, and it is the better tool. An LLM generating text has to be prompted to emit one of a fixed set of options, parse the output, and handle the case where it emits something else. A decision model returns a distribution over the options you gave it, with a confidence, in one forward pass. It cannot replace an LLM where the answer space is open-ended, where the output needs to be prose, or where the task requires reasoning over several steps.

Do I need a GPU to run Laya?

No, though it helps. The docs give 193 to 464 ms per question on CPU, which is around 7 to 14x slower than a T4 but still fine for background workloads like nightly triage or batch classification. If you have an Apple machine, the MLX and Core ML ports run at 5 to 14 ms with no GPU at all. A single T4 is enough for production at realistic volumes.

What is the catch with self-hosting a decision model?

Three, and they are all in the fine print. The base checkpoints are near chance on production workloads until you fine-tune, so there is a data-labelling step you cannot skip. The shipped calibration is over-confident and must be refit on held-out data, or your confidence thresholds are lies. And the upkeep is yours: you own the version pinning, the security patches, and the monitoring, which is the same tradeoff that applies to self-hosting anything else. For the common failure modes of a stack that looks free until you count the hours, see why most expense tracking apps suck. If none of that sounds like a good time, rent the hosted model, that is what it is for, and the cost math stays visible instead.