Field notes on AI, automation, and building software that lasts.
Twelve posts, in the order the levers actually work.12 posts · 219 min
Start hereLLM cost optimization: the 5 levers that actually cut an AI bill, in the order that works. Most teams start with caching and save less than they hoped.
DiagnoseRead the shape of the bill first: spike, step, compounding, or always high.
MeasureThe five fields to log, and why the 95th percentile is the number to read.
Lever 1 · The loopInput tokens grow with the square of the step count. Cap the loop before anything else.
Lever 1 · RetrievalTop-k times chunk size decides the bill, and most systems retrieve far more than the answer needs.
Lever 1 · The payloadOutput bills 4x to 5x input, so most prompt-trimming advice optimises the cheap side.
Lever 2 · Right-sizeA cheap model breaks even at roughly 1 over how many times cheaper it is.
Lever 3 · Per requestUp to 90% off repeated input, and the traffic level below which it costs you money.
Lever 3 · On BedrockSame models, stricter rules: minimums up to 4x the direct API, and it fails silently.
Lever 4 · ProvidersAn optimised Claude workload beats an unoptimised GPT one at 2.4x the per-token price.
Lever 5 · Where it runsThe break-even volume, and why published figures disagree by a factor of ten.
ProofThe same method run on one of our own systems, line by line.
Thirteen posts on the discipline that replaced prompt tweaking.12 posts · 100 min
Start hereContext engineering explained: the attention budget, why long context fails, the levers that manage it, and what each one costs to run. With real numbers.
First principlesThe prompt is one input among many. Prompt engineering is a subset, and treating it as the whole job is how context problems stay invisible.
The budget · WindowsWhat a window physically is, what the million-token numbers actually buy, and what it costs to fill one.
The failure · RotRecall is U-shaped: strong at the edges, weak in the middle. More tokens make it worse, not better.
The failure · PoisoningOne bad tool result contaminates every turn after it. Hygiene beats cleverness.
Lever · MemoryWorking memory IS the context window. Everything else is a read that costs tokens on every step.
Retrieval · The decisionCorpus size, change rate and query overlap decide it. Price each branch before you pick one.
Retrieval · AgenticWhen the agent drives retrieval it searches better and spends more, and both effects compound.
Plumbing · The standardWhat MCP actually adds over a plain API, and the honest cases where you do not need it.
Plumbing · Build itHello world is in the official docs. Day two, with logging, testing and versioning, is not.
Plumbing · Secure itOAuth 2.1 with PKCE has been mandated by the spec since March 2025. Most tutorials still skip it.
Packaging · SkillsA folder and a SKILL.md that load only when needed: context engineering made reusable.
Haystack is three years older than LangChain and has a fifth of the stars. The real difference is abstraction level, and the most-upvoted practitioners on this question answer neither.
SGLang leads by 29% on shared-context multi-turn and loses on single-turn unique prompts. Same hardware, opposite results. Which means "which is faster" is not answerable without naming the workload.
Two published benchmarks agree vLLM wins and disagree by 2.5x on the numbers. And there is one metric where Ollama beats vLLM outright, which turns out to be the clearest proof of the problem.
Most comparisons rank these two by GitHub stars. The repos show something more useful: OpenClaw ships native Swift and Kotlin clients, Hermes ships five sandbox backends. Reach versus containment.
Every comparison frames this as control versus convenience, with self-hosting as the safe choice. Self-hosting a gateway means owning a database full of every provider key you have, and owning the clock when it needs patching.
Nearly every comparison says these two perform identically because both run llama.cpp. That stopped being true, twice. Both ship MLX now, but Ollama's needs more than 32GB of unified memory, which most Macs do not have.
Function calling and MCP are not rivals. The real difference is who writes the tool schema and when it arrives: yours at build time, or somebody else's at runtime. That single change is the value and the risk.
ACP connects your editor to a coding agent. MCP connects that agent to tools. They are not alternatives: the editor hands the agent its MCP servers through the ACP session, so one protocol literally carries the other's configuration.
MCP connects an agent to tools, A2A connects agents to each other, and ACP connects a coding agent to your editor. Three layers, plus the ACP that most comparisons still describe as a live option a year after it merged into A2A.
MCP connects an agent to tools. A2A connects agents to each other. They are not competing standards, they solve different problems, and most real systems need both, not one instead of the other.