Patrick McCanna writes up moving 35 kB agent prompts from Anthropic and OpenAI to a self-hosted Ollama box

Patrick McCanna, writing on his personal blog, has published notes on porting long agent preprompts from Anthropic and OpenAI to a self-hosted Ollama setup running an abliterated 27b open-weights model. The Hacker News thread on it has 105 points at the time of writing. The system he tested on is a 128 GB AMD Ryzen AI Max+ 395 with 32 GB reserved for the host OS and everything else given to inference, running through opencode.

The number that decides the migration

The finding a working developer will care about first is the context math. McCanna reports that his self-hosted setup ran at a 65k-token context window, and that a 35 kB prompt "immediately consumes 14% of total context window". With session history added, the window fills within a few turns. "Large prompts founder and thrash," he writes. He describes the effect as an agent "briefing a man who is reincarnated every ninety seconds," carrying out the most recent instruction with no memory of the preceding ones.

Frontier APIs hide this by running much larger context windows. McCanna argues that some of the perceived quality of a frontier model is actually the chain-of-thought room a large window buys, which lets a model recover from a poorly written prompt.

Failure signals worth logging

He lists five log patterns that indicate a local agent is running out of context, worth watching for in any self-hosted setup: identical tool calls back to back, multiple reads of the same file, an agent restating its objectives, tool-call parse failures, and a high turn count relative to file changes. He calls tool-call parse failures a particular hazard, because a bad parse can "shove so much raw data into context that it destroys sessions like a burst pipe at your dinner party."

What to change in the prompt

McCanna's fixes are prompt-shape changes, not model choices. Split preprompts into single-objective units, one problem and one resolution each. Move to opencode's declarative agent definitions in ~/.config/opencode/agents rather than reading a Markdown file at run time. Tune Ollama's context length explicitly, since the defaults are small. Have agents log session state to disk and re-read only the slice they need. Cut the number of tool calls per step. Replace negative directives ("don't do X") with positive ones ("only do Y").

The motivations he names

McCanna frames the migration in privacy terms, citing the recent public argument over whether frontier providers may have trained on user session data during the Navier-Stokes discussion in August, and states his position that customers cannot verify a provider's retention or training pipeline. He also complains that frontier safety filters refuse security-testing questions in ways that block defenders from finding real vulnerabilities in their own code. Those are his views; the migration notes stand on their own and are the part a team can act on today.

For a team running coding agents against a private codebase, McCanna's write-up is the shape of a real trade-off: a self-hosted stack removes the provider from the loop and forces the prompt discipline that a large frontier window otherwise lets you skip.

Source

Primary source: Notes on migrating 35kb prompts away from Anthropic/OpenAI to self-hosted Ollama+opencode, by Patrick McCanna. Hacker News discussion: thread 49697014.