Skip to content

Released Sep 2026

Record sheet · No. 148

Jeff

A 0.8B open model that answers first and hands the query to a large model only when its own confidence falls below a threshold.

GitHub avatar of firelex, not the project’s own logo — taken from github.com on 2026-10-02.

Screenshot of Jeff
Editor screenshot, 2 Oct 2026Jeff ↗

What it is

Jeff is a 0.8B open model that sits in front of a larger one and decides the easy cases itself, swapping LoRA adapters per task for guard, triage and tool choice.

It answers first; when its confidence falls below a per-task threshold, the query continues to a model such as Qwen3.8-27B and that model’s answer is used.

One base model, Jeff-Qwen3.5-0.8B v1.2, is paired with LoRA adapters that swap per task, so a single setup covers guard, triage, support intent, tool choice, grounding, navigation, emotion, spam and contract clauses. The README’s own comparison, on a fixed sample of held-out rows, puts the cascade at 95.3% against 86.6% for the 27B deciding everything, at 0.25 s per decision instead of 8.1 s.

Those two numbers describe that one setup rather than small models against large ones in general.

Who built it

Thirteen of the repository’s fourteen commits are his, the fourteenth from a second contributor. Nine of the fourteen carry a co-author trailer and all nine name Claude Opus 5.5 at 1M context.

How it is put together

The parts · 5

One base model plus swappable adapters, with the routing decision made by the small model’s own confidence, not by a separate classifier. Each adapter has its own threshold, fixed before the test rows were scored: it is the fastest setting that still beats the 27B by at least one point on the task’s separate calibration rows. Below that threshold the 27B answers too, and its answer is the one used. The usual cascade consults the expensive model by default and filters what comes back. This one consults it by exception, which is how a 6.9% memory increase buys a 38× drop in latency on the tasks the adapters cover.

The v1.2 base
Jeff-Qwen3.5-0.8B, 1,706,027,688 bytes on disk, released as a community preview on 2026-10-01. v1.3 is announced as the long-term-support base, with the official adapters retrained on it.
Nine adapters
LoRA add-ons over that one base, about 41 MB each: guard for prompt injection and jailbreaks, triage for urgency and sentiment, support-intents for what the customer wants, tools for which tool an agent should call, ground for whether an answer is supported by its sources, nav for voice commands to on-screen items, emotion for the strongest of 27 emotions or neutral, spam for spam and phishing, and legal-clauses for contract clause types.
The routing rule
Jeff answers first and passes the query on when its confidence is under the adapter’s threshold. The README reports the pass-through rate per task: 0.0% on most of them, 1.7% on grounding.
The comparison harness
Both models run on MLX on one Apple M4 Max with 128 GB, one at a time, the 27B in 8-bit. Each task is scored on a fixed random sample of its held-out test set, and the threshold is fixed before the test rows are scored.
docs/data-sources.md
Where the training data comes from, at 7 KB, next to the published data guidelines. Those guidelines are the part that lets somebody else’s data set outlive a base-version change.

Choices, and the alternative

  • Put the small model in front and pass on by exception over asking the large model every time

    The README’s position is that the strong local model does the writing and planning while Jeff makes the quick decisions in front of it, and the two tables are the argument: same rows, 95.3% against 86.6%, 0.25 s against 8.1 s, with the 27B consulted for 0.0% of most task types.

  • Keep adapters swappable over one base over one fine-tuned model per task

    Adapters are LoRA add-ons of about 41 MB each, and the README gives the cost of all nine loaded at once as 1.96 GB on top of a 28.6 GB setup.

  • Release as a community preview and name the next base rather than waiting for it over holding the adapters until v1.3 is stable

    The README announces v1.2 with nine adapters, asks for issues, and says v1.3 is about thirty-six hours away, together with the warning that adapters do not survive a base change while data sets do.

Read fromREADME, the results tables and the repository tree of firelex/jeff, read 2026-10-03.

Build log

4 stages
  1. 01

    Four days from an empty repository to a preview with nine adapters

    The repository was created on 2026-09-28 and holds 14 commits, twelve in September and two in October. It ships as a community preview, not a release: the README announces Jeff v1.2 with nine adapters, asks readers to file what works as issues, and says a stable long-term-support base, v1.3, is about thirty-six hours away with the official adapters retrained shortly after. It also states the constraint that follows from that schedule instead of burying it. Adapters do not carry between base versions and have to be retrained; data sets do carry over, if they follow the published guidelines.

  2. 02

    Who wrote it: nine commits out of fourteen carry a model’s name

    Thirteen of the fourteen commits are the author’s and one is a second contributor’s. Nine carry a co-author trailer, and all nine name Claude Opus 5.5 at 1M context: two thirds of the history. That is the highest proportion in this batch. It is evidence that the model wrote a substantial part of the code in those nine commits. It is not evidence about the training data, and it is not evidence about how the benchmark numbers below were produced.

  3. 03

    The claim is a comparison, and it is printed with its conditions

    The README does not argue that a small model is good. It puts two configurations side by side on the same rows. Across eight adapters, the 27B deciding everything scores 86.6%, and Jeff with adapters deciding while the 27B handles the rest scores 95.3%. Time per decision falls from 8.1 s to 0.25 s and wrong answers from 13.4% to 4.7%. The measurement is described rather than asserted: an Apple M4 Max with 128 GB, both models on MLX one at a time, the 27B in 8-bit and prompted to answer directly, and each task scored on a fixed random sample of its held-out test set of 300 rows, or 500 for emotion and contract clauses. A second table breaks the same comparison down per task: guard 84.0% to 98.0%, triage 81.3% to 91.0%, support intents 86.0% to 95.3%, tool choice 90.3% to 98.0%, spam 88.0% to 98.7%. Grounding is the one row where the small model does not win, 96.7% to 96.3%, and the column next to it says why: 1.7% of those queries were passed on.

  4. 04

    What you get, and what it is for

    Each adapter is a LoRA add-on of about 41 MB, trained in one epoch on one GPU in half an hour to four hours, on top of a base that is 1,706,027,688 bytes. The repository also carries the material for building your own: docs/data-sources.md on where training data comes from, and the data guidelines that let somebody else’s dataset survive a base-version change. Every number has a source on jeffhub.ai, and the comparison can be rebuilt from the reference application.

What they would tell you

  • Emotion is left out of the averages on purpose. Picking the strongest of 27 emotions in a short comment is hard for people too, and the human labels often disagree. Including it makes the gain smaller rather than larger: across all nine adapters the average is 91.4% for Jeff with adapters against 80.9% for the 27B.
  • Grounding is the one task where the 27B is strong. Jeff passes on its least sure 1.7%, ends up level with it at 96.3% against 96.7%, and is still 20× faster. One question in 300 is the whole of that difference.
  • Adapters do not carry between base versions. Data sets do, if they are built to the published guidelines, which is a more useful promise than it sounds: it means a labelled set is not thrown away when the base moves.
  • The two headline numbers were measured on different hardware. Accuracy comes from an M4 Max with both models on MLX; memory comes from an RTX PRO 6000 with all nine adapters loaded and the adapter switched on every request, at a median of 30 ms per decision.

Revision log

1

This record is kept current rather than frozen: when the project moved and a paragraph stopped being true, the paragraph was corrected in place. Every corrected paragraph carries a marker, and each one appears here with what it said before.

  1. 2026-10-03RecordRewritten for a reader who has not heard of the project. The ninth adapter is named (emotion, which the README leaves out of its averages and this record had left out entirely), the benchmark now carries the conditions the README states, the phrase “when it is unsure” has been replaced with the threshold mechanism it actually describes, and the repository inventory has been cut back to what a reader can use.

Adjacent records

All records →