Skip to content
heyaican

Released Sep 2026

Record sheet · No. 145

Laya

Small encoder models that read a document once and return a probability for every option of every typed question, with nothing generated and a router that picks the checkpoint by script.

Open it Source Submit a correction
Screenshot of Laya
Editor screenshot, 15 Oct 2026Laya ↗

What it is

Software that classifies, routes or guards usually pays a large model to write a sentence and then parses the decision back out of it.

Laya is built the other way round. A choice question takes named options and returns a distribution over them;

noul returns the probability that a statement holds; score returns a distribution over ordered levels plus an expected level.

Nothing is decoded, so there is nothing to parse. Three checkpoints ship under Apache-2.0: a 421M English model on a ModernBERT-large base, a 322M multilingual one on mmBERT-base, and a 421M fine-tune for the project's own workflow benchmark.

A Router reads the script and language of the input and chooses between them before any weights are loaded, because on the English checkpoint a Khmer document scores 0.000 accuracy at 0.952 confidence. The package also carries an HTTP server that speaks TypeSafe Jev's /v1/systemone contract, integrations for LangChain, LlamaIndex, CrewAI and MCP, and SDKs in TypeScript, Go, Java and .NET.

Who built it

A GitHub account created on 2019-03-16 with 87 public repositories and 1,948 followers, whose profile reads CEO Convai Innovations and names a previous post as principal investigator at the IIT Palakkad Technology IHUB Foundation. Convai Innovations owns the convaiinnovations organisation on Hugging Face where the three checkpoints live. He owns 815 of the repository's 2,213 commits; the contributors endpoint returns 149 accounts, and 162 with anonymous ones.

How it is put together

The parts · 7

The organising idea is that a decision is a scoring problem, so the model should be an encoder with a head on it and the question should be part of the input rather than part of a conversation. A request is rendered into one flat sequence in which the question's options appear as [MASK] markers ahead of the state. The encoder is bidirectional, so the markers read the document and the document reads the markers in the same pass. A two-layer transformer head then reads each marker, a scorer turns each into one logit, and a softmax across the markers is the answer. Everything else in the package follows from that shape. No vocabulary is generated, so there is nothing to parse. The primitives are three rather than open-ended, so the head can be trained against a proper scoring rule instead of a likelihood. And because the encoder is the whole cost, questions batched into one request share one pass. The router sits in front of that rather than inside it: the English checkpoint cannot read non-Latin script and does not know it, so the checkpoint is chosen from the script and language of the input before any weights are touched.

laya/common.py
The sequence builder and the model. build_sequence renders [CLS] <type> instructions [SEP] [MASK] opt0 … [SEP] state [SEP], build_head returns the marker position of every option, and DecisionModel pairs a Hugging Face encoder with a two-layer transformer head, a scorer, an act_head and three temperature buffers.
laya/agent.py
The runtime: checkpoint loading with revision and digest pinning, autocast and dtype chosen per batch size, collection of a request into rows, the OOM fallback that demotes one request to CPU and puts the runtime back afterwards, and the decode of logits into typed answers.
laya/router.py and laya/lang.py
Three named checkpoints, lazy loading with two resident models by default and LRU eviction, the script and language detection that decides which one to load, and predict, predict_batch and predict_long.
laya/calibrate.py and laya/confidence.py
Temperature fitting per question type and option count under a [0.5, 5] clamp, abstention thresholds, and the three separate confidence numbers: entropy-based confidence, answer_confidence for the reported answer, and jev_confidence in Jev's own form.
laya/train.py
The fine-tuning loop: rlcd_loss over proper-scoring rewards, optional option-order shuffling with the target permuted to match, the same temperature fitter the runtime uses, and warnings for a run whose temperatures never moved or whose head collapsed onto the class prior.
laya/serve.py, laya/mcp/, laya/integrations/
The ways in: an HTTP server on the Jev /v1/systemone route with a strict mode for clients that validate the response shape, an MCP stdio server, and adapters for LangChain, LangGraph, LlamaIndex and CrewAI.
laya-ts, laya-go, laya-java, laya-dotnet and sdk/
Four re-implementations of the same contract in other languages, each with its own fixtures and tests, regenerated from Python goldens in continuous integration where the runtime can be shared.

Choices, and the alternative

  • Score the options instead of generating them over a decoder that writes the answer and parses it back

    There is no call to generate and no sampling parameter anywhere in the package. The answer is a softmax over per-option [MASK] positions, which makes the output parseable by construction and lets the training reward be a proper scoring rule rather than a likelihood.

  • Train against a strictly proper scoring rule over cross-entropy on the argmax label alone

    proper_reward is log score plus 0.75 of the spherical score plus a ranked probability score term on ordinal questions, and the README reports the trade-off in its own numbers: the fine-tuned checkpoint beats Jev on argmax accuracy at 0.766 against 0.727 and loses on soft accuracy at 0.471 against 0.580.

  • Choose the checkpoint before the forward pass over asking the English model and falling back when confidence is low

    The README supplies the measurement that forces it: Khmer scores 0.000 accuracy at 0.952 confidence on the English checkpoint and its mean confidence never drops below 0.885 at any accuracy level, so its own confidence cannot gate anything. The router reads script and language first.

  • Keep the head small and expect a fine-tune over one large model asked to cover every domain zero-shot

    head_layers is 2, and the shipped config records 7,313 updates in 1.96 hours for the English base. The documentation states the position directly: Laya is a fast base to specialise, not a zero-shot decision engine.

  • Speak Jev's wire protocol over publishing a bespoke API

    laya-serve answers on /v1/systemone, LAYA_JEV_STRICT strips the extra fields for clients that validate against the Jev contract, and x_jev_confidence reports Jev's own confidence formula so a threshold can be moved across without being recalibrated.

Read fromREADME.md, BENCHMARKS.md, LOCAL_SETUP.md, AGENTS.md, CONTRIBUTING.md, pyproject.toml, docs/ (35 files including benchmarks.md, evals.md, finetune.md, http-api.md, routing.md and guardrail-limitations.md), .github/workflows/, laya/common.py, laya/agent.py, laya/router.py, laya/train.py, laya/fast.py, laya/revisions.py, laya/confidence.py, laya/lang.py, research/README.md, the three rl_agent_config.json and encoder/config.json files on Hugging Face, and the repository tree of NandhaKishorM/laya, read 2026-10-15 and 2026-10-16.

Build log

6 stages
  1. 01

    One wire contract, three different machines

    TypeSafe opened Jev on 2026-09-15: a hosted model that answers typed questions with probabilities and returns no text. The category went public within hours, and people started rebuilding the interface. Laya's repository was created on 2026-09-18, three days later, and it is the largest of those rebuilds. What Laya shares with Kev (jaredpalmer/kev, created 2026-09-17) and with Jeff (firelex/jeff, 2026-09-28) is the contract, not the code. All three take one state and a set of typed questions and all three can be dropped behind a Jev client. Kev matches /v1/systemone closely enough that TypeSafe's own Python SDK drives a local Kev server unchanged; Laya ships laya-serve on the same route and adds x_jev_confidence, Jev's own confidence formula, to every choice and score answer so a threshold carried over from Jev has something comparable to read. Underneath, the three are different machines. Kev is a decoder family on Qwen bases and reads its answer out of a <decide> token. Jeff is a 0.8B Qwen model with per-task LoRA adapters and a cascade. Laya is encoder-only: a bidirectional ModernBERT or mmBERT backbone with a decision head bolted on, and the question rendered into the input rather than asked in conversation. The name is not a spelling of Jev; it is the author's own. The author also claims to have published the idea first. Nandakishor told Tom's Hardware that a March 2025 paper of his, SalesRLAgent, already proposed a probability-estimation model with weights and datasets released, and that the earlier work went unnoticed because it was aimed at sales rather than at a general engine. That is his account of the order of events and this record carries it as one.

  2. 02

    What "non-autoregressive" means in this code

    The claim is checkable and it holds. The laya/ package contains no call to generate, no max_new_tokens, no beam search and no sampling schedule; searching the whole package for those returns nothing. A request is rendered into one flat sequence: [CLS], the question type and its instructions, [SEP], then one [MASK] token per option, then [SEP], then the state, then [SEP]. The encoder runs once over that sequence. A two-layer transformer head reads the hidden state at each [MASK] position, a scorer (LayerNorm, a linear layer, GELU, a linear down to one value) turns each into a single logit, and a softmax across the markers is the answer distribution. The answer is read out of the model at fixed positions. Nothing is emitted. "Single forward pass" means one pass per request, not one per question. Every question in a request becomes its own row in one batched call, so eight questions are still one encoder pass, which Router.predict states in its docstring: route, then answer every question in one forward pass on the chosen checkpoint. Long documents are the exception and the code names it: predict_long cuts a state into overlapping windows and scores each one, which is a forward pass per window. Two more pieces sit on top. Each checkpoint carries fitted temperatures per question type and option count, clamped to [0.5, 5] and applied to the logits before the softmax. And an act_head predicts act or escalate from the pooled representation plus four features. The README reports that the second one currently reads 1.0 for almost every input and that its raw logits run against correctness, AUROC 0.30 across 396 labelled decisions.

  3. 03

    "System 1" here is a training objective, and the training took hours

    Both base checkpoints are fine-tunes of public encoders: answerdotai/ModernBERT-large for the English one and jhu-clsp/mmBERT-base for the multilingual one. The method the project labels RLCD is in the package as rlcd_loss. Gaussian noise is added to detached logits; each noisy sample is scored against the target distribution by proper_reward; the advantages are normalised across the group; and that advantage weights the Gaussian log-density of the sample. Soft cross-entropy against the target is added to it. proper_reward is a strictly proper scoring rule: log score, plus 0.75 of the spherical score, plus a ranked probability score term on score questions. Training a distribution against a proper rule is what the project means by reporting honest probabilities being the only way to raise the reward. That is reinforcement learning in the technical sense, and the shipped configuration files state its size rather than leaving it to be guessed. The English checkpoint records 7,313 updates, one epoch, 1.96 hours on a single device. The multilingual checkpoint records 15,987 updates, four epochs, 4.97 hours on a single device. The workflow fine-tune inherits the English figures. Weights are public in three Apache-2.0 repositories under convaiinnovations: 842,609,210 bytes for the English model, 643,835,514 for the multilingual one and 842,609,220 for the fine-tune, with the last two also nested inside the first repository as subfolders. The training data is the part that is not public. The benchmark tables mark each evaluation set as in training mix or held out, the demo Space describes the checkpoint as trained with pure RL on public datasets, and no page in the repository names those datasets or releases them. The weights can be downloaded and re-run; the corpus behind them cannot be read.

  4. 04

    Where the shipped checkpoints do not hold up

    The number the README leads with is 0.766 on the typed-decisions benchmark, and the same table prints what it belongs to. The two base checkpoints score 0.362 and 0.352 on the same 2,000 decisions, against a random baseline of 0.318 and a majority-class baseline of 0.461. The README states the reading itself: the base checkpoints sit below majority class, and all of the capability on that benchmark comes from a checkpoint fine-tuned on that benchmark's own training split. The gold labels there are the mean of three samples from a teacher endpoint, and the teacher agrees with itself 0.735 of the time, so the fine-tuned score measures agreement with a teacher. The published limits list is long and specific. Moderation scores 0.530 with macro-F1 0.400 on held-out toxic-chat, which the project describes as barely above chance on a balanced split. A choice question's options share one fixed token budget, so 77 labels get roughly three to four tokens each and Banking77 falls to 0.425 where Jev scores 0.870. Ordinal score is the weakest primitive, 0.372 on SST-5. Labels such as true/false can dominate an answer instead of the state. The English checkpoint collapses outside English while staying confident, Khmer at 0.000 accuracy and 0.952 confidence before the temperature clamp. The multilingual checkpoint ships with no fitted temperatures at all, temperature as [1, 1, 1] and an empty per-option table, and it has a position bias on score questions that makes it rarely pick the first level. A negation probe recorded in the README selected cancel_account for all four negated cancellation requests on the English checkpoint, one multilingual answer at probability 0.9998. An independent reproduction later found the direction of the calibration problem to be the opposite of the card's framing; that result is in the next section rather than here.

  5. 05

    33,025 stars in twenty-seven days

    The counts were read at 2026-10-15T17:48Z: 33,025 stars, 2,968 forks, 117 watchers and 111 open items, against a repository created at 2026-09-18T04:46:33Z and last pushed at 2026-10-15T10:46:25Z. That is twenty-seven days. The stars arrived with a category rather than with the project alone. Jev opened on 2026-09-15 to a Hacker News thread that a third-party write-up puts past 1,900 points and 520 comments, and Laya shipped three days later as the open alternative, the largest of the rebuilds within a week. One account published on 2026-10-05 recorded it past 19,000 stars with 5,000 added in a single day. None of that is a measurement of the model. Behind the number there is more work than a star spike usually carries. main holds 2,213 commits, the first at 2026-09-18T04:46:27Z and the last at 2026-10-15T10:45:46Z, from 159 distinct author names; the contributors endpoint returns 149 accounts and 162 with anonymous ones. The largest are NandhaKishorM with 815 commits, aashish254 with 425, keemsisi with 207, Bruce-Yii with 84 and PerryLink with 48, and the rest of the list is long rather than top-heavy. 993 pull requests have been opened and 812 merged, by 188 accounts; 194 issues have been filed by 109 accounts. Commits by month read 1,032 in September and 1,181 in October, and the two heaviest days are 204 on 2026-09-24 and 180 on 2026-10-01. Subjects run 534 fix, 309 docs, 243 feat, 198 test, 59 ci and 27 release. 236 commits carry a co-author trailer and 175 of those name a model or an agent, with 172 trailer lines naming a Claude model; a Copilot code-review workflow is registered alongside them. Distribution tracks the same shape and is Python-only. PyPI holds 46 releases of laya, from 0.1.0 at 2026-09-18T04:38Z to 0.4.8 at 2026-10-15T10:52Z, which is a release every fourteen hours across the twenty-seven days. pypistats.org reports 272,233 downloads in the last month, 67,422 in the last week and 10,603 in the last day. GitHub carries 41 releases, each with the wheel and the sdist attached and 1,619 downloads between them. On Hugging Face the English checkpoint has been pulled 48,894 times with 5,624 likes, multilingual 11,529 with 390 and the fine-tune 2,406 with 164; the demo Space has 276 likes and was running on a zero-a10g at the time of writing. Continuous integration is not decorative: fourteen workflows are registered, twelve in the repository. ci.yml alone has completed 2,082 runs. In the 500 most recent runs across all workflows, covering 2026-10-12 to 2026-10-15, 416 succeeded, 42 failed, 30 were cancelled and 12 were waiting for a maintainer to approve a fork's workflows. Two lanes have never run at all. release-ts.yml and release-java.yml have each completed 0 runs, the 41 tags are every one of them v0.x, and npm install laya-ts, which the README prints as the TypeScript install, returns 404 from the npm registry.

  6. 06

    Independent checks, and what it costs to run

    Third-party work on this project exists, which is not true of most records filed here. An independent reproduction and preregistered study was posted as arXiv 2609.33843 in September 2026, with a stated absence of any affiliation to Convai Innovations, the benchmark's publisher or TypeSafe. It re-runs the released workflow checkpoint on all 400 test cases and reproduces accuracy at 0.767 against the card's 0.766, ECE at 0.2142 against 0.213 and score MAE at 0.2424 against 0.242. Soft accuracy is the one headline figure that does not reproduce: 0.5086 measured against 0.471 reported, with the confidence interval excluding the reported value. The paper also measures the direction of the miscalibration and finds it opposite to the card's framing, uniformly under-confident at a signed gap of −0.214, where a single temperature fitted on a held-out half, T = 0.469, takes held-out ECE from 0.204 to 0.037 while the shipped per-option table softens where it should sharpen. Its frozen escalation gate misses its own 10% error target on both tracks. A second paper, arXiv 2610.03387 (Jiawen Lu and Tongtong Wu, 2026-10-02), pairs Laya and Jev on candidate coverage with 23,932 predictions per model. At five TREC candidates with natural names, Laya detects 97.2% of missing-answer cases and falsely rejects 69.7% of present controls; Jev's rates on the same construction are 24.8% and 0.0%. Calibration-only none-score thresholds move those to 33.9%/3.7% and 45.0%/1.8%. A preregistered test registered on the Open Science Framework before it ran, published as Zenodo record 10.5281/zenodo.23293732 on 2026-10-11, gave seven decision models the same 799 unpublished yes/no decisions about coding-agent work and destructive shell commands, three times each. Against a pre-set expected calibration error of 0.05, Laya recorded 0.051 and its verdict in the abstract is inconclusive; Perplexity's open pplx-decider was the only model consistent with a calibration claim, at 0.026 and 94.7% accuracy. Three of the data sets inside the repository came from outside accounts: a CUDA capacity study on an RTX PRO 5000, an RTX PRO 6000 and an H100 NVL with a 24-hour replay, kept in a separate repository; a zh-CN and zh-TW evaluation of 219 items and 284 questions; and 64 synthetic Chinese workplace scenarios with paired Laya and Jev responses. LOCAL_SETUP.md is a fourth. It is a full set-up and verification record written on a 2019 Mac Pro that reproduces the project's own published tables on CPU and on Metal, and that found two portability bugs the project then fixed: a RoPE base read wrong on transformers 4 for the mmBERT checkpoint, and an MPS autocast call that raised on every request. Running it is one line. pip install laya on Python 3.10 or newer pulls torch, transformers, safetensors, huggingface_hub and numpy and nothing else; serve, mcp, langchain, llamaindex, crewai, onnx, fast and structured are extras. Checkpoints download from the Hub on first use and load from the cache afterwards. CPU, CUDA, MPS and Intel XPU are supported, and the repository publishes measured latency for a T4, an RTX 4070 Ti SUPER, an Intel Arc B390, an AMD Radeon Pro Vega II over Metal, a Ryzen 9 laptop CPU, a four-core EPYC and an NVIDIA GB10. On a T4 one question takes 39.5 ms on the English checkpoint and 32.8 ms on the multilingual one.

What they would tell you

  • The star count and the training budget are separate measurements, and here they are three orders of magnitude apart. 33,025 stars arrived in twenty-seven days; the English checkpoint that earned them was trained for 1.96 hours on one device for one epoch, and the multilingual one for 4.97 hours over four. Nothing in the repository hides this: it is a field in rl_agent_config.json, next to the temperatures.
  • The headline accuracy belongs to a fine-tune on the benchmark's own training split, and the README says so in the sentence after the table. Zero-shot, the two base checkpoints score below the majority-class baseline on the same decisions. What ships is a base to fine-tune, and the fine-tuning documentation is the part of this project that a reader with a real decision to make should read first.
  • The multilingual checkpoint ships with no fitted temperatures, temperature as [1, 1, 1] and an empty per-option table, so its probabilities are not usable as probabilities until somebody fits them. The English checkpoint ships with a table, and an independent reproduction found that table softens where the measurement says it should sharpen.
  • Confidence is not a warning signal on the English checkpoint. Khmer scores 0.000 accuracy at 0.952 confidence and never drops below 0.885 mean confidence at any accuracy level, which is why routing happens before the forward pass. The same failure appears in the opposite direction on the workflow benchmark, where an independent preregistered study measured the checkpoint as uniformly under-confident.
  • Third-party work exists and does not all point the same way. A preregistered study reproduced the headline accuracy to within a point and found one metric, soft accuracy, that does not reproduce at all; a second paper found the model answering a well-formed candidate list by rejecting valid options 69.7% of the time; a third, preregistered on the OSF before it ran, recorded a calibration verdict of inconclusive against a pre-set line.
  • The TypeScript lane the README advertises does not exist. release-ts.yml and release-java.yml have completed zero runs, all 41 tags are v0.x Python releases, and npm install laya-ts returns 404 from the registry. Third-party npm packages named after this project do exist and are not it.
  • The volume is real even where the maturity is not. 2,213 commits in twenty-seven days from 159 author names, 812 pull requests merged from 188 accounts, 46 PyPI releases, 2,082 runs of one workflow, and 272,233 PyPI downloads in the last month are not the profile of a repository that was starred and abandoned. They are the profile of a category that went public all at once and a project that kept up with it.

Adjacent records

All records →

No. 067

book-to-skill

A converter that reads a document (one file, a folder, a glob or a list of paths, in PDF, EPUB, DOCX, HTML, RTF, MOBI or plain text) and writes an agent skill: a core file of mental models with a chapter index, one file per chapter that loads only when a question touches it, and a glossary, a patterns file and a cheatsheet beside them.

No. 058

LeanCTX

A local layer that sits beside a coding agent and decides what reaches the model: file reads are compressed and cached, command output is compressed by per-command rules, session findings persist across chats, and a local proxy rewrites each request without breaking the provider’s prompt cache — with a savings ledger, a budget and a dashboard for what it measured.

No. 146

Jeff

A 0.8B open model that answers first and hands the query to a large model only when its own confidence falls below a threshold.