robotnurseLarge language models on hardware you own and can touch.
Field notes from building, breaking, and fixing local LLM setups: quantization, inference tuning, hardware that actually matters, and the honest benchmarks behind each claim. Written slow, tested twice, updated when something stops working. If you would rather have it set up than read about it, book a free 30-minute call.
- lived
- 11 months
- hardware
- gpu ×8 · apple silicon ×2
- entries
- 79 field notes
- hype
- off
The notebook is what we learned. This is what we do with it.
Private, on-site AI on machines you own. Your hardware. Your weights. No invoices from a cloud provider.
On-site in NYC. We install, quantize, and tune models on your own machines, then leave you running.
A short, paid look at your current stack: what is slow, what is wasted, what to fix first. Numbers, not opinions.
Fixed-scope bundles for individuals, small teams, and clinics that need private inference with no cloud dependency.
Latest entries
All 79 posts →The spec sheet is not the machine: two checks before you spend on local-AI hardware
Most local-AI hardware advice treats the spec sheet as the machine. Two checks from our own fleet: do the fit arithmetic before the money moves, and measure the machine you received, not the one you ordered — two identical units differed by 4.7× until a full shutdown brought the slow one back in line. For readers at the buying decision now, the distilled toolkit and config pack are linked at the end.
A Review Found Our Approval Gate Could Be Forged by the Agent It Gated
Four agents flagged a synthetic cath-lab case packet in one Switch room, and a human chair had to click Allow before a fifth agent could write any PDF. A pre-submission review found the agent itself could have faked the Allow record. What we fixed, what we could not, and the numbers.
Turning On the Drafter Moved the Whole Model to the Slow Path
Three times, enabling a speculative-decoding module added a few hundred experts to a mixture-of-experts model, overflowed a count derived without them, and silently dropped the entire model onto the paged fallback: 485 ms per pass instead of 28, and different tokens. The different tokens were a second bug.
Our Best Validation Loss Was Our Worst Model
A fine-tune beat production's validation loss by a wide margin (1.086 vs 1.427) and leaked Thai, Khmer and Chinese into English replies. Our harness scored it clean, our checkpoint sweep scored it 0 defects twice over, and four of mlx_lm's defaults meant something other than what our config said.
The Model Booted, Returned 200, and Spoke Fluent Nonsense
A filtered download skipped one shard of 141, the one holding the final norm; the server booted with it uninitialized and raised nothing. A shard audit then crashed and reported green on a node holding 17 of 141 shards. Liveness checks cannot see any of this.
Two Crash Fixes That Cost Throughput and Fixed Nothing
A daily inference-server crash got two mitigations, both matched to the symptom by name, both 'validated' by a load test that passed 12 of 12. One mechanism was impossible, the other flag routed traffic into the failing path, and one of them cost 24% of decode.
Tools
Free calculators we use before buying anything.
Buyers’ notes
- 01Best Mac Studio config for local LLMs — 2026
- 02Mac mini vs Mac Pro for local AI — which is worth it
- 03How much RAM do you need for 7B, 13B, and 30B models?
- 04amdgpu vs Mac for local inference — a fair fight
- 05Mac Studio vs GPU PC for local AI — an honest comparison
- 06Best local AI setup under $3,000 — three builds that work
- 07RAM vs VRAM: how much memory do local LLMs actually need?
- 08Turn an old gaming PC into a local AI box
More in all buyers’ notes.