FreshContextA context lab · Fife
Drawing
02 · Benchmarks
Sheet
2 of 3
Rev
A · 2026-10-01

LFM2.5-8B-A1B · file← LFM2.5-8B-A1B

LFM2.5-8B-A1B Q4_K_M

The only file: Liquid AI's own Q4_K_M. Tested twice, never given a job, deleted in October. Liquid AI's GGUF.

Size
5.2 GB
Status
Deleted 11 Oct 2026
Scored runs
8
Smallest Mac
16 GB estimate, 8K context
Results8 scored · 95% intervals
SuiteScore95% interval PassedRuntime · profileRecorded
groundinglfm2.5-8b-a1b47.2%38.7–55.9Passed 59/125llama.cppextraction2026-08-31
nullschemalfm2.5-8b-a1b27.8%18.8–39.0Passed 20/72llama.cppextraction2026-08-31
nullschemalfm25-8b-a1b-midrange27.8%18.8–39.0Passed 20/72llama.cppextraction2026-09-10
screeninglfm2.5-8b-a1b73.3%61.0–82.9Passed 44/60llama.cppextraction2026-08-31
synthesislfm25-8b-a1b-midrange53.3%40.9–65.4Passed 32/60llama.cppextraction2026-09-10
toolcalllfm25-8b-a1b-midrange1.7%0.3–8.9Passed 1/60llama.cppextraction2026-09-10
toolscalelfm25-8b-a1b-midrange82.5%68.0–91.3Passed 33/40llama.cppextraction2026-09-10
worldviewlfm25-8b-a1b-midrange30.0%18.1–45.4Passed 12/40llama.cppextraction2026-09-10

Runs marked unenforced were measured on a suite that requires constrained decoding, on a server shown not to apply the schema: mlx_lm accepts it and ignores it, and a few other servers and models break it. Those numbers do not measure what the suite is designed to measure. They are kept and labelled rather than deleted.

The recordHow it was made · how it was served · what went wrong · what happened to it

Rebuilt from the lab's records by Claude, an AI model. The results above come straight from the automated runs. How these pages are made.

FileLFM2.5-8B-A1B-Q4_K_M.gguf
FormatQ4_K_M GGUF, 5,155,564,768 B (4.80 GiB)
Where fromdownloaded 2026-08-31 01:33–01:43Z from LiquidAI/LFM2.5-8B-A1B-GGUF; bytes verified against the HF manifest
Statusdeleted 2026-10-11 (Tier B sweep)
Hugging Facehttps://huggingface.co/LiquidAI/LFM2.5-8B-A1B-GGUF

Where it came from

  • Repo / file: LiquidAI/LFM2.5-8B-A1B-GGUF / LFM2.5-8B-A1B-Q4_K_M.gguf. HF revision: not on record (resolve/main).
  • Command:
curl -sL --retry 20 --retry-delay 4 --retry-all-errors -C - --no-progress-meter \
-o models/lfm2.5-8b-a1b/LFM2.5-8B-A1B-Q4_K_M.gguf \
"https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-GGUF/resolve/main/LFM2.5-8B-A1B-Q4_K_M.gguf"

The parallel curls stopped at 3.86 GB (01:39Z) and were resumed sequentially with --retry 30... -C -.

  • Verification: local 5,155,564,768 B against remote 5,155,564,768 B, "OK" (01:43Z); GGUF magic 47475546. The same check caught the Guardian file 701 MB too large after a -C - resume. This file was clean.
  • Date: 2026-08-31, 01:33–01:43Z. sha256 not on record.
  • Who asked: Alan, "Forgot about granite and lfm models too" and "Try the 2 main candidate models if they make sense".

How it was served

Both labels ran on upstream llama.cpp/build/bin/llama-server, b10618 (eb25b7263). That build came from the 2026-08-25 rebuild and was replaced at 2026-09-10 12:17Z, after both runs.

labeldates (UTC)ctx / -np / KV / -fa / --jinja / --reasoning-formatprofilewhy this label exists
lfm2.5-8b-a1b2026-08-31 01:49Z–01:51Z--ctx-size 16384 --n-gpu-layers 99 --jinja; no -np (server default, not on record), f16 KV, no -fa, no --reasoning-formatextraction (thinking off, max_tokens 2048)second-seat test: screening, grounding, nullschema, co-resident beside qwen3-next-80b-coder at 41.5 GB
lfm25-8b-a1b-midrange2026-09-10 10:47Z–10:51Z--jinja --reasoning-format deepseek -c 32768 -np 1 -fa on -ctk q8_0 -ctv q8_0 -ngl 999, alias lfm25-8b-a1bextraction (temp 0, seed 42)mid-range cohort with qwen3-8b and gemma4-12b: toolcall, toolscale, worldview, nullschema, synthesis. The label spelling lfm25 comes from the script's run name, so the two labels differ only by the dot

Never seated. Footprint: 47.02 GB total llama RSS with the 41.5 GB coder resident on 2026-08-31; 5.1 GB RSS alone at -c 32768, q8_0 KV, on 2026-09-10.

Decisions

  • Downloaded for the second-seat test.
  • Not taken for the judge seat: "not accurate enough".
  • Re-benched 2026-09-10 for the mid-range question; "the weakest model tested".
  • Kept 2026-09-12 as the only build of the model and again 2026-09-16.
  • Deleted 2026-10-11, Tier B, on Alan's "nuke tiers a & b" (RESTORE-deleted-20261011-tiersAB.sh; a lab note). The exact time is not on record; the sweep was done by one of our agents before the working session began at 08:45Z.
  • Not published (the vendor's own file).

What went wrong

  • First run tested a loading server. curl -s.../health && break exits 0 on a 503, so the suites ran against a model still loading. All 60 screening cases were excluded with HTTP 503, coverage 0%. The results files were deleted and the run repeated with a gate that required HTTP 200. The void attempt left no run-log row.
  • Empty think tags in content. <think></think> appears in 41/60 toolcall, 34/125 grounding and 2/60 screening responses (results files), including under --reasoning-format deepseek. The graders still scored those rows; none is unparseable in nullschema.
  • Run-log build stamps: the five lfm25-8b-a1b-midrange entries say llama_build: 0.4.0-dev (build 10893, commit c32d1dabe). They ran 10:47–10:51Z on 2026-09-10, before the 12:17Z upgrade to b10893. runtime_version b10618 is right. The 2026-08-31 stamps (b10618) are right.
  • Configuration differs between labels (ctx 16384/f16 vs 32768/q8_0 KV/-np 1/-fa on). Only nullschema overlaps; it scored identical totals, with 2 case flips.

What happened to it

Downloaded 2026-08-31 → benched 2026-08-31 (3 suites) and 2026-09-10 (5 suites) → never seated → kept 2026-09-12 and 2026-09-16 → deleted 2026-10-11 (Tier B; RESTORE-deleted-20261011-tiersAB.sh). Re-downloadable: the RESTORE line is hf download LiquidAI/LFM2.5-8B-A1B-GGUF, which pulls the whole repo. The exact file is hf download LiquidAI/LFM2.5-8B-A1B-GGUF LFM2.5-8B-A1B-Q4_K_M.gguf, which should match 5,155,564,768 B. The run-log rows survive.