Laya vs Jev: Is the Open-Source Alternative Actually as Good?

TypeSafe shipped Jev on September 15. Days later, Convai Innovations released Laya under Apache 2.0: a 421M-parameter model that answers the same typed questions, runs on hardware you own, and costs nothing per call.

At least seven open-source Jev clones have appeared since. Most of the coverage is a list of which ones exist. Almost none of it answers the question you actually have, which is whether any of them is good enough to use.

For Laya the short answer is: yes, for a narrower job than the headlines suggest. It is genuinely faster, genuinely cheaper, and beats Jev on several classification tasks. It also scores below the majority-class baseline out of the box, and falls apart on questions with many options. The reason this comparison is possible at all is that Convai published both halves on its own model card, including the parts that hurt.

What Laya is#

Laya does the same job as Jev. You hand it state, you hand it typed questions, and it answers them in a single forward pass with calibrated probabilities. It supports the same three primitives: choice to pick an option, score to rate against ordered levels, noul to ask a yes/no question and get a probability. Nothing is generated, so there is nothing to parse.

It even shares the training method. Convai trained Laya with RLCD against strictly proper scoring rules, the same approach TypeSafe describes for Jev.

Three checkpoints ship:

It installs with pip install laya and its published benchmarks run on a Tesla T4.

Where Laya wins#

Convai's published head-to-head, all figures from its model card:

"Routed" matters. Those accuracy figures come from a router picking between the English and multilingual checkpoints per input, not one model. The English checkpoint alone scores 0.947 and 0.573, so the routing is doing real work, and you have to build it.

Latency is the real result. Jev's 70ms floor is measured at the model; what you experience is a hosted API call over a network, measured by third parties at 236 to 276 milliseconds. Laya answers in 32.8 milliseconds locally, and batching improves it: ten questions take 72.3 milliseconds total, about 7.2 each.

Emotion is the more meaningful accuracy win, because it was held out of Laya's training mix. AG News was not, so treat that one lightly.

Where Laya loses#

Two failures, and both are large.

High cardinality. On Banking77, which asks the model to pick a single banking intent from a long list, Jev scores 0.870 on a 72-label version and Laya scores 0.425 on all 77. The label counts differ, which cuts against the point being made here, so treat the gap as large rather than exact. Convai explains the mechanism rather than hiding it: at default settings, a 77-option question allocates roughly three to four tokens per label, which is not enough to represent them. There are workarounds in the docs, involving longer heads, shortlisting with embeddings, or splitting one question into a coarse and a fine pass. The card's own summary is blunt: "Jev is currently better suited for 50+ options without tuning."

Zero-shot accuracy. This is the one that matters most and gets reported least. The base checkpoints score 0.362 and 0.352 on typed decisions. Random guessing scores 0.318. Picking the most common answer every time scores 0.461. Out of the box, Laya is closer to random than to a trivial baseline, and Convai states the conclusion plainly: "Laya is a fast base to specialise, not a zero-shot decision engine."

Three smaller constraints round it out. Ordinal scoring is the weakest primitive, at 0.372 on SST-5. Both checkpoints ship over-confident and need temperature recalibration before you trust a probability. And the English checkpoint collapses outside English while the multilingual one is weaker inside it, so you route between them or pick one deliberately.

The number worth checking#

Laya's headline is 0.766 on typed decisions against Jev's 0.727. That looks like a clean win until you read two more rows of the same table.

That 0.766 belongs to laya-typed-decisions, a checkpoint fine-tuned on that benchmark's own 1,200-case training split. And the table lists a teacher self-agreement ceiling of 0.735, which the fine-tuned model exceeds. Scoring above the ceiling of the process that generated your labels is the signature of a model that has learned the benchmark rather than the task. Convai's own description of the checkpoint agrees: "Expect it to behave like the base laya checkpoint, or worse, on anything else."

On that same table, Jev wins the two columns nobody quotes. Soft accuracy: 0.580 against 0.471. Calibration error: 0.144 against 0.213. On the benchmark Laya was specifically tuned for, Jev is better calibrated and closer when it is wrong.

What neither set of numbers can tell you#

Three asymmetries, so you read both vendors the same way.

Jev's row on Convai's tables is labelled "published." It was not run in the same harness, on the same day, against the same inputs. It is a number lifted from TypeSafe's materials and placed next to a number Convai measured.

Laya's 7.8x latency win compares a local GPU against a hosted API over the public internet. That is the honest comparison for a developer choosing between them, but it is not a like-for-like model speed measurement.

And Jev's own headline is softer than it looks. As we covered in what the 193x benchmark doesn't measure, TypeSafe's workflow evals define the correct answer as the average of GPT-6 Astra and Fable 5.1, so they score agreement with two frontier models rather than accuracy. Neither side of this comparison is standing on ground truth.

Which one to use#

The decision is not about which model is better. It is about what you have.

Use Laya if you have labelled data, or can produce it, and somewhere to run a GPU. You get 33-millisecond answers, no per-call cost, Apache 2.0, 100+ languages, and full control. You will spend real time fine-tuning and recalibrating first, and you should budget that work rather than discover it.

Use Jev if you need it working zero-shot this week, your questions have many options, or you do not want to operate inference. It handles high cardinality without tuning, and the hosted path is the whole point.

Laya is not a drop-in replacement, and Convai never claimed it was. It is a fast, free, honest base that becomes competitive once you specialise it. The open-source alternative is real. It just asks you for labelling days instead of API credits.

Shipping either one#

Whichever side of the table you land on, the work between the model and production is the same shape: the fine-tuning and recalibration budget if you go Laya, the schema and threshold design either way, and the self-hosted inference or agent and automation engineering around it.

Sources

• Laya model card and laya-typed-decisions, Convai Innovations: checkpoints, benchmark tables, limitations.
• Laya repository: install, API example, speed tables, cardinality guidance.
• Introducing System One Models & Jev, TypeSafe AI: Jev pricing, latency and eval methodology.