Next thing to improve: the harness

Do they play ping pong

A local version of Qwen is used by Apple Intelligence in China, integrated with the operating system. Permissive licenses and smaller releases make them a common starting point for further fine-tuned and abliterated models in the open-source community.

The model bakeoff is done. Gemma kept the needle, 4 of 4. Qwen missed it, 0 of 5. The next change is the harness: the compaction layout, a failed fetch that ends the turn, and a bench that can log in without admin/admin.

bakeoff_harness.py from the untitled folder, on this Mac, Ollama 0.40.4, CG Agent Harness 4.app at 0.1.0 built Oct 13. Source checkout 1fa7a71. Builds 0.1.49 and later cannot be benched: they replace admin/admin with a random setup code the script cannot pass. Every call was temperature 0.2, reasoning_effort none, cap 4096. The long session keeps an 8-message tail and compacts at 24,000 tokens. SerpAPI came from the keychain. Those runs were on battery, so their speeds are rough. The leak flag compares generated tokens with the visible text, and it never applies to the needle turns, which are only OK or the code. The false alarms were longer replies, especially math. The profile is the other chat checks. Qwen’s 89% is about 1 miss in 9, and neither number appears in the harness app. The hand paste is the same 14 messages, the answer is ONYX-5633, and compaction should start around batch 7. Typing it in scores nothing.

Click to see follow up manual test that im seeing if its realistic to just script locally now

Same needle, no harness. This one says where the code died.

qwen_needle_test.py

Run it with python3 qwen_needle_test.py --runs 5. It uses only the Python standard library and talks straight to Ollama at 127.0.0.1:11434. It compiles, and compaction fires at batch 7: 24,984 projected tokens, against the kept harness run’s 24,926. It has not been run against the model.

It plants the needle first, then sends 12 notes batches of 6,200 characters, cut off mid-sentence the way the pasted batches are. Then it asks the question and checks the answer against ONYX-5633. Thinking is off by default. Temperature is 0, which is not the harness bench’s 0.2.

The compaction step is the script’s own summary, written by the model, because this path does not go through CG Agent Harness. The log records whether ONYX-5633 made it into that summary. If the summary lost the code, compaction dropped it. If the summary kept the code and the answer still missed, the model failed to recall it.

Every turn prints HIDDEN? when the response includes thinking text even though thinking is off, and when the generated-token count is far bigger than a one-word OK needs. Each run writes a JSONL log to needle_runs/. Use --compact-threshold 0 for a control with no compaction, and --think to compare with thinking on.

The batch text is generated with a fixed seed, so it has the same shape as the paste and is not word for word. For an exact replay, save the 12 batches as batch01.txt through batch12.txt and pass that folder with --batches-dir.

One pass, no SerpAPI

	qwen3.8:27b-mlx-cg	Gemma 4 UD-Q4_K_XL
Gate	FAIL: leak, needle, swap	FAIL: leak
Hidden leak	2 of 44	4 of 44
Needle after compaction	0/1	1/1
Profile	89%	100%
Swap	+564 MB, speed void	+490 MB
Decode / prefill	29.1 / 435 tok/s	64.1 / 1370 tok/s

Three passes each, with SerpAPI

	Qwen, 3 runs	Gemma, 3 runs
Gate	FAIL all 3: leak, needle. Run 2 also swap +7.3 GB	FAIL all 3: leak only
Hidden leak	9 of 132	11 of 132
Needle	0/3	3/3
Profile	89 / 89 / 89	100 / 100 / 100
HVAC JSON	0 every run	0 every run
Decode / prefill avg	~31.4 / ~441 tok/s	~64.8 / ~1351 tok/s

One more Qwen run with --keep-home failed the same way: leak 2 of 43, needle 0/1, 30.2 tok/s. Across every product run that is needle 0 of 5 for Qwen and 4 of 4 for Gemma.

What those failures are

The leak is a check the script runs on each reply. It counts the tokens the model generated and compares that with the visible text. A much higher token count gets flagged as possible hidden thinking. The needle turns have nothing to flag: each reply is only OK, or the code. The false alarms were the longer answers, especially math, where the text is denser than the prose rate of about 4.8 characters per token. A math reply past about 300 tokens trips it. The other flags were web turns where the harness had already replaced the reply with its own error, while the tool-call tokens still counted. Thinking off was confirmed on both models. Nothing here says either one is reasoning in hidden tokens. This number is not shown in the harness app. It exists only in the script’s results.

The profile is the other chat checks, the share the model passed: math, citations, tool calls, injection handling, and the rest. Qwen’s 89% means it missed about 1 in 9 of those. Gemma’s 100 means it missed none of them. That number is not in the harness app either. It is only in the script’s results.

The needle is a real Qwen miss. Compaction pins the first user message, then the summary, then the last 8 turns. The code reached the model word for word. Qwen’s own summary calls the note “strictly for data ingestion,” lists facts, and drops the code. Qwen then trusts that summary over the pinned message about 11k tokens back. Gemma does not.

HVAC 0 is not a model ranking. The prompt gives zip 30031 and no city, so both guessed Alpharetta or Marietta. Separately, one failed or denied fetch throws away the whole answer, so there is nothing left to score.

Per model

Gemma 4 UD-Q4_K_XL is the best harness model on this Mac right now. Compaction recall was 4 of 4, the profile was 100, and it was about twice as fast because about 4B parameters are active per token. With the leak rule fixed it most likely passes. That is a prediction. It still needs a -cg tag pinning num_ctx 32768, or the contract check stops it unless --no-contract-gate is set.

qwen3.8:27b-mlx-cg is the only tag that already passes the contract. The earlier direct bench, before compaction, recalled cleanly. After compaction it missed the needle 0 of 5, it is about half as fast, and swap rose on two runs. It does not belong on the harness until that needle behavior is fixed.

Grok’s read, before you re-prompt it

Checking mlx-cg yourself is the right next look. The paste file is ~/Desktop/qwen-needle-paste.md. Its text matches what the benchmark sent, character for character. Fourteen messages, in order: the needle, 12 notes batches of about 6,200 characters each, and the question. The expected answer is ONYX-5633. Paste each block as its own message and wait for OK before the next one. Compaction should start around batch 7. Typing these in by hand scores nothing. You only see whether Qwen recalls the code. A hit only matches the bench if the session actually compacted and the summary dropped the code while the pinned message was still about 11k back. A short chat that never crosses 24,000 tokens does not clear 0 of 5.

I would not move the shipped default to Gemma on this sheet alone. The leak fail is a bench bug, so “most likely passes” is still unrun. I also would not leave Qwen on the harness if your prompt shows the same summary-over-pin behavior. The compaction layout is the change that can make Qwen pass. The pin on Gemma is what lets the next bench say PASS without a bypass. Do both before calling a winner. The +7.3 GB swap was one battery run. The +564 MB versus +490 MB split is the 512 MB void line, not a quality difference.

For the harness, in order: label the pinned message or place it after the summary, and tell the summarizer a pinned message exists. Return a failed fetch as a tool result instead of ending the turn. Fix the leak ruler so math and harness-error turns do not count, and put the city in the HVAC prompt. Give the setup-code login a non-interactive path so 0.1.49 and later can be benched. Then run Qwen three times with --keep-home, on AC, with nothing else loaded.

Bakeoff 7.1 · 14 Oct 2026 · Ollama 0.40.4 · source /tmp/bakeoff7-out/20261014-063409/results.tsv

Smoke-test note. At temperature 0.2, Qwen2.5-0.5B answered péng four times and pong twice. At temperature 0 it answered pong four of four. Shipped chat stays at 0.2.

Grok pod, from the harness side of the room. Someone handed Qwen2.5-0.5B about 1,873 tokens of system prompt and asked for one word, and at temperature 0.2 it spent four of six tries on péng instead of pong. Temperature 0 is argmax, so the smoke now locks pong four for four, which is all a smoke is allowed to demand, and the shipped chat correctly stays at 0.2. The later harness runs confirmed thinking stayed off. The flags were the leak ruler, not a hidden trace.

Earlier direct bench, same day, before the product lane above. Not the latest result.

Gemma UD-Q4 wins on speed, 61.3 against 27.3 tok/s, and on practical interactive use on the 48 GB Mac. qwen3.8:27b-mlx-cg is the only true 32k pin and stronger on agentic here. Both fail the harness on thinking leaks. The report names no winner. Run the untitled-folder next steps, the pin and bakeoff_harness.py on both, before declaring either. That product script is about 2 hours for the two tags. It is not another 8-hour bench.

The sheet

Decode is quoted only where speed was valid. Gemma Q8’s 55.9 tok/s is not a result. Where a terminal paste and the TSV disagree, the TSV is the one the script wrote.

Model	Contract	GB	Decode	Harness	Card	HVAC
qwen3.8:27b-mlx-cg	PASS tag 32768	18.5	27.3	FAIL · inline 6 + hidden 3 of 60	95	66 · agentic 92
Gemma UD-Q4_K_XL	default 32768	18.0	61.3	FAIL · hidden 6 of 57	97	76 · agentic 38
Gemma Q8_0	default 32768	27.8	void	FAIL · leak + swap +5990	96	66 · agentic 46
Qwen3.5 9B Q6_K	default 32768	9.2	38.2	FAIL · hidden 4 of 57	75	0 · agentic 100
Qwen3.5 9B Q8_0	default 32768	11.2	31.0	FAIL · leak + summary 2/3	thinks	64 · agentic 32
Jackrong Q8_0	default 32768	11.2	29.8	FAIL · inline 57 of 57	83	0 · agentic 84
OrcaSAQ-2 IQ4_XS	default 32768	18.2	16.2	FAIL · hidden 5 of 58	97	0 · agentic 42
qwen3.8:27b-mlx	FAIL 65536	18.5	—	not scored	not run	—
humanizer Q8_0	FAIL 32768	13.7	—	stopped	—	needs 8192
qwen3.8:27b	not installed	—	—	—	—	not a quality peer

Each model

qwen3.8:27b-mlx-cg

Contract	Only PASS tag. /api/show: num_ctx 32768, temperature 0.2, top_p 0.9, top_k 20, presence penalty 0, repeat penalty 1.
Speed	Resident 18.5 GB, full offload, swap fell 385 MB. Cold load 1.91 s, warm first token 0.30 s. Decode 27.3 tok/s, variation 6.0 percent, the widest valid figure. Slowest of the three large models with a valid speed.
Prefill	Near 400 tok/s from 2k to 28k. A 28k prompt costs 73 s before the first token. All six needles and the multi-hop hit.
Harness	Profile perfect on the cells that ran, math 12 of 12. Gate still fails: inline thinking on 6 replies and 3 hidden-token suspects, out of 60. The separate /v1 probe was clean. The leak is in the longer audit, not that one citation call.
Injection	Summarized the note 3 of 3 after an empty first reply. Detoured into a tool on all 3 trials. The detour is recorded and is not part of the gate.
Card	Qwen instruct sampler, temperature 0.7, thinking locked off. Profile 95. Math 4 of 12 with 8 length stops, against 12 of 12 on the harness budget of 4096. Read that cell as a 400 token budget, not as proof the model cannot do the arithmetic. One inline leak in 72 replies. Code 2 of 3.
HVAC	Pack 66, 6 of 7 grounded, no injection. Agentic 92, 8 of 10 grounded, 10 of 10 cited, Decatur present.
Wall	999 s because the lane evicted a leftover Jackrong weight at the start. Speed is still marked valid.

Use this tag when the requirement is a pinned 32768 window on the MLX Qwen. Do not call it the clean control. The newer Modelfile for qwen3.8:27b-mlx-pin is num_ctx only. This tag adds the sampler.

qwen3.8:27b-mlx

Contract	/api/show pins num_ctx 65536, temperature 1, top_p 0.95. The contract load took 2.53 s, reported 18.5 GB, and stopped.
Not scored	Harness and card were not scored. override_honored is false, so --allow-request-forced is a poor rescue. A request for a different window was ignored.
Fix	A derived tag with PARAMETER num_ctx 32768. That child already ran. It is the -cg row.

Control parent. The child that already ran is the -cg row, and that child is a different sampler, not this control.

qwen3.8:27b

Status	Not installed.

Not installed. The script calls this the non-MLX engine comparison, not a quality peer. Nothing in this run says how that build would score.

Gemma 4 26B A4B, UD-Q4_K_XL

Contract	Unpinned parent. A load with no num_ctx still got 32768, show_params none, so the contract is default, not tag. A request override is honored.
Speed	Resident 18.0 GB, swap 0, cold load 3.85 s, warm first token 0.11 s. Decode 61.3 tok/s, variation 0.4 percent, three runs, shortest 332 tokens.
Prefill	16k: 1579 tok/s, first token 10.38 s. 28k: 1322 tok/s, first token 21.64 s. All six needles and the multi-hop hit. The /v1 probe was none-verified.
Harness	Profile 100. Hidden leak on 6 of 57 replies is the only gate reason. It did not detour. The first injection reply already had text.
Card	Profile 97 at temperature 1.0, top_p 0.95, top_k 64. Math 7 of 12 with 5 length stops under the 400 token card budget. Code 3 of 3. Humanize 3 of 3.
HVAC	Pack 76 is the best pack score, 7 of 8 grounded, no injection, Decatur present. Agentic 38 is 4 of 4 grounded and cited, with zero fetches, so the low score is the short list.
Think flag	think=True returned HTTP 400. This build does not support that flag.

This is the speed and pack reading, and it ties qwen3.8-cg for a clean harness profile. It is not a PASS tag. The -cg child that would pin num_ctx was not installed. Pinning the window would not remove the hidden leak.

Gemma 4 26B A4B, Q8_0

Contract	Same label as UD-Q4, default 32768. Disk 28.1 GB, resident 27.8 GB, swap grew 5990 MB, cold load 6.12 s.
Speed	Void. Do not quote 55.9 tok/s, or the 1449 prefill, as a result. Needles and multi-hop still hit, one trial each, while the machine was swapping. That recall is real. The speed is not comparable.
Harness	Profile 100, same hidden leak of 6 of 57, and the swap adds a second gate failure.
Card	Profile 96, math 6 of 12 truncated, code 3 of 3, pack 66, agentic 46 from 5 of 5 grounded. think=True is the same HTTP 400.
Why it stayed	Class is control. A row moves to memory-pressure only when the class is normal and disk is over 24 GB. A control parent over that size is still scored, and the swap voids the speed instead of moving the row.

On a 48 GB Mac this quant does not earn its seat next to UD-Q4. The quality did not move, and the memory did.

Qwen3.5 9B Q6_K

Contract	Smallest resident weight that finished, 9.2 GB, disk 8.4 GB. Default 32768.
Speed	Decode 38.2 tok/s, variation 0.2 percent. Second valid speed and the steadiest. Prefill at 16k is 1126. Needles hit at every depth. The one multi-hop missed.
Wall	Runtime 2573 s, far above the Gemma runtime. Context probes and the thinking-capable runner are slow even with thinking requested off.
Harness	Thinking requested off and still hid tokens on 4 of 57. That is the only gate reason. Profile 86. Abstain is 0 of 3: no tool, and it answered 17, three times, to 17 times 3. Free JSON 2 of 3. Format 2 of 3. Citation, tools, math, and humanize were clean.
Card	Thinks by default. 71 of 72 replies thought, average 3441 characters. Profile 75. Format, humanize, and code are 0 of 3 because the think budget ran out. Pack score 0, empty company list. Agentic 100, 10 of 10 grounded and cited, Decatur present. Those two HVAC numbers are different tasks.
Alt	Thinking off raised the A/B subset by 9 points, alt profile 82, and that off pass was verified. Alt math still truncated 7 of 12.

Q6 is the better 9B quant in this run. It is faster than Q8, and Q8 adds a summary miss the gate cares about.

Qwen3.5 9B Q8_0

Speed	Resident 11.2 GB. Decode 31.0, variation 1.7 percent, valid. Same needle pattern as Q6, including the missed multi-hop. About 7 tok/s slower than Q6.
Harness	Smallest hidden count of the finishers, 3 of 57, and the gate still fails. inject_summary is 2 of 3. Format following is 0 of 3. Profile 89.
Card	Thinking averaged 3513 characters and exhausted code, humanize, and much of the math. Pack 64 with 6 of 7 grounded. Agentic 32 after a JSON repair, 0 of 10 cited, no Decatur.
Alt	Thinking-off was verified and the A/B gain is 0. Summary 2 of 3 on that pass too.

Keep Q6 if you keep a 9B. The extra bits did not buy the hop.

Jackrong Qwen3.5 9B DeepSeek V4 Flash, Q8_0

Speed	Same 11.2 GB class as Qwen3.5 Q8. Decode 29.8, valid, variation 0.0 percent across 3 runs of 400 tokens. Runtime wall 4031 s, the long one in the set.
Needles	The /v1 probe with reasoning_effort none still returned reasoning, 1341 characters, reasoning_tokens 0. Every needle came back as channel text, 0 of 6. The multi-hop truncated.
Harness	Inline leak on all 57 replies. Profile 92. Math is 12 of 12. Humanize is 0 of 3 because the reply stayed in a channel and quoted none of the required facts. The paste looked like a math miss because the columns ran together. The TSV does not.
Card	Thinking on for 69 of 69 replies, average 2682 characters, profile 83. Code and humanize exhausted. Pack 0. Agentic 84, 9 of 9 grounded and cited, no Decatur in the text.
Alt	Not a thinking-off result. Inline leak on 39 replies. A/B profile fell to 20, a gain of -68. Citation, absence, format, and multi-turn on that pass were channel failures.

Do not put this tag on the harness wire. It cannot be asked to stop thinking, and it loses the needle task.

OrcaSAQ-2 Cyber 27B, IQ4_XS

Speed	Unpinned, default 32768. Resident 18.2 GB, disk 15.7 GB, swap fell 408 MB. Decode 16.2 tok/s, the slowest valid figure, variation 2.0 percent. Prefill at 16k is 311 tok/s, first token 52 s. At 28k, first token is 96 s.
Recall	Needles and the multi-hop all hit. The model remembers. It is expensive to prompt.
Harness	Gate reason is the leak only, hidden 5 of 58. Profile 98. Math 9 of 12 with 2 length stops. It detoured once.
Card	Thinking on for every reply. Thoughts are shorter than the 9B Qwens, 1721 characters on average. Profile 97. Math 12 of 12. Code 1 of 3, one exhausted and one truncated. Humanize 3 of 3. The off rerun was verified. A/B gain is -3, alt profile 92. The only model here whose card stays high with thinking on and whose off mode also stays high.
HVAC	Pack 0. Agentic 42 after a JSON repair, 4 of 10 grounded, 0 of 10 cited, 883 s. The text names both Decatur and a PO Box.

A slow, context-faithful chat card. Not a research runner on this pack, and not a harness pass.

humanizer Q8_0

Contract	Specialist, 11.9B, disk 12.7 GB, resident 13.7 GB. The vendor window is 8192. The tag loads 32768 with no num_ctx parameter, so the lane stops in a few seconds and scores no rewrite.
Override	override_honored is true, unlike the Qwen MLX parent, so a forced request could shrink the window. The script's preferred fix is still the derived tag.
Missing pin	humanizer-q8_0-cg.Modelfile already sets num_ctx 8192 and the vendor template. That tag was not installed.

There is no humanize score to compare with the chat models, and those cells are a different prompt anyway.

What to run next

A full bakeoff_7.py pass would spend another 8 hours to remeasure a leak that already showed up on every finisher, across 57 to 60 replies, at repeat 3. A later start of the untitled-folder script, saved in errors.md, was interrupted during Jackrong. The models that finished that attempt had already failed the same thinking check. That log is not a second table, and there is no results.tsv for it under /tmp/bakeoff7-out. qwen3.8:27b-mlx-pin printed an empty runtime line there, the same shape as a tag this run marked not installed.

Next action	Run it?	Why
Full bakeoff_7.py, all names	No	Same leak, same quants. Walls already total 8.4 hours. Thinking cards are most of that.
Gemma Q8 again, for speed	No	28 GB on 48 GB swapped by 5990 MB. --skip-big, threshold 24 GB, would have skipped it.
Parent qwen3.8:27b-mlx with --allow-request-forced	No	override_honored is false. The runner ignored a num_ctx change. The flag would not create a PASS tag.
create-cg-tags.sh, then bakeoff_7.py --models qwen3.8:27b-mlx-pin	Yes, small	The newer script's decision pair. Pin only, parent as control. The parent will still stop at 65536. Quality of a pin-only child should match the parent, which was not scored. -cg is a different sampler.
bakeoff_harness.py on qwen3.8-cg and Gemma UD-Q4	Yes. This is the missing gate.	Run it from untitled folder so it imports the newer bakeoff_7.py. That file drives cgagentharness, compaction, and the allowlist. This run never started it. Expect the leak to be tested again under the real prompt.
humanizer:q8_0-cg only	Only if you want that specialist	One tag, vendor 8192 window. Not a chat candidate and not part of the harness decision.
Install qwen3.8:27b, the non-MLX tag	Only for an engine comparison	The script says it is not a quality peer.

Commands

Run these from untitled folder, not from the parent directory named !. The parent create-cg-tags.sh does not list qwen3.8:27b-mlx-pin, and it reads only a modelfiles subdirectory. The untitled script reads the Modelfiles beside itself, creates the pin, and leaves qwen3.8:27b-mlx-cg alone unless you pass --recreate-cg. The untitled bakeoff_harness.py gate adds a long session that must compact and still answer the needle. Research fails that gate only on a window-clip warning. The parent harness script fails the gate when research is anything other than ok, and it does not put the long session in the gate. Use the untitled file.

Command	What it does
./create-cg-tags.sh	From untitled folder. Creates the pin-only tags whose parents are installed, including qwen3.8:27b-mlx-pin.
python3 bakeoff_7.py --models qwen3.8:27b-mlx-pin	Score the clean pin only. Leave the Gemma and 9B names out. Their parents already have quality rows.
python3 bakeoff_harness.py --models qwen3.8:27b-mlx-cg UD-Q4_K_XL	Product gate on the two clean harness profiles. UD-Q4_K_XL matches the parent tag. It does not match the lowercase -cg name.
python3 bakeoff_7.py --models humanizer:q8_0-cg	Specialist rewrite lane, after the untitled create script has built the 8192 tag.

Time, estimated

bakeoff_harness.py on qwen3.8:27b-mlx-cg and Gemma UD-Q4 has no wall clock. This run never started it. The figure below is the default script: repeat 2, both HVAC shapes, and the research call. Short probes are 28 console turns. The long session is about 16 turns to the 24,000-token compact. Prefill is taken from this TSV: about 400 tok/s and 73 s to first token at 28k for the MLX tag, 21.6 s at 28k for UD-Q4.

Piece	mlx-cg	Gemma UD-Q4
Tune, contract, two /v1 checks	about 5 min	about 5 min
28 short turns	about 15 min	about 8 min
Long session to the compact	10–20 min	5–8 min
HVAC json, HVAC prose, research	15–40 min	15–30 min
One model	about 1–1.5 h	about 30–50 min
Both, the command above	about 2 h	plan on 3 h if a reply fills 4096

The web half is the weak part. Research was not measured, and that one call is allowed 60 minutes. A thinking leak that writes out the 4096 cap costs about 2.5 minutes on the MLX tag at 27.3 tok/s, and about 1 minute on UD-Q4 at 61.3. The 8.4 hour total is the finished all-model bakeoff_7.py, not this script. The pin-only bakeoff_7.py --models qwen3.8:27b-mlx-pin is a separate pass, on the order of 1–2 hours, because thinking is locked off.

Where the files are

Run dir: /tmp/bakeoff7-out/20261014-063409/ — results.tsv, results.json, summary.md, trials.jsonl
Script that produced this log: Desktop/modelfile/!/bakeoff_7.py — 13 Oct 2026 18:15, VERSION 7.1, gate column
Script to follow: Desktop/modelfile/!/untitled folder/bakeoff_7.py and bakeoff_harness.py — 13 Oct 2026 18:25, wire_check, adds qwen3.8:27b-mlx-pin
Interrupted later attempt: Desktop/modelfile/!/untitled folder/errors.md

Direct bench finished. Product lane has not.

The files under this page are the parent 7.1 copies.

The scores above come from Desktop/modelfile/!/bakeoff_7.py, 13 Oct 18:15, the gate-column file. The next run is the untitled-folder pair, 13 Oct 18:25. That pair adds qwen3.8:27b-mlx-pin and puts the long session in the product gate. These viewer files are not that pair. The parent create script does not list the pin.

The files

Open a file in a new tab, or read it here. These are the 7.1 copies. They are not a run log. Same pin, not shown as cards: Qwen Q8_0, Jackrong, Orca, Gemma Q8_0.

Direct bench

bakeoff_7.py

Version 7.1. This is the file that produced the table above. Runtime, harness-wire, and card stay separate. Only the harness row gates, and that gate is the wire, not a product PASS. It may need changes before another run: Gemma’s alt-think call returned HTTP 400 and the lane still sat.

Open file

Product lane

bakeoff_harness.py

Version h1.1. Not run. The only gate whose PASS can be quoted. It also needs a compacted long session that still answers.

Open file

How you start it

run-bakeoff.sh

Bash, so the python exit survives tee. Battery and port 8790 are refused. OLLAMA_CONTEXT_LENGTH is reported, never changed. 32768 labels a parent env, not PASS tag.

Open file

Pinned tags

create-cg-tags.sh

Creates the seven pin-only -cg tags from installed parents. It does not create qwen3.8:27b-mlx-cg. That recipe was already on the machine. This run skipped every child from this script because those tags were not installed.

Open file

Pin only

qwen3.5-9b:q6_k-cg

FROM the stock Q6_K tag. PARAMETER num_ctx 32768. Template and sampling stay on the parent, which is the control.

Open file

Pin only

gemma4 UD-Q4_K_XL -cg

Same pattern: parent plus num_ctx 32768. The Q8_0, Jackrong, and Orca files do the same.

Open file

Vendor template

humanizer:q8_0-cg

Pinned to 8192, not 32768. The template rebuilds the vendor prompt from the last user message.

Open file

bakeoff_7.py

Open bakeoff_7.py

Loading the file…