·
The engineering behind typed: a free tier that runs on your own machine with no account and no key, Anthropic Messages API fidelity, the prompt-cache discount that paid for the hosted side, and what still does not work.
Update, 23 September 2026: the economy tier is now spelled typed (it was typed--), the free tier that runs on your machine is typed-local (it was typed), and with a paid account the CLI defaults to typed.
I built typed because I was spending ~$200/month on subscription AI CLIs and wanted to keep the workflow at a lower price. One of its tiers is free: typed-local runs on your own machine, needs no account and no key, and never reaches typed's servers -- that is why it costs nothing. The paid hosted tiers are a proxy that speaks the Anthropic Messages API and serves open-weight models from typed's side. After six weeks of building and two months of running my own coding traffic through it, my upstream costs are around $15/month for the same volume.
This post is about the engineering side -- the API surface that turned out to be subtler than I expected, the cache-discount discovery that paid for the whole project, the cost-of-quality knob I ended up adding on the top tier, and the things that still don't work. It assumes you've used Claude Code or one of the other Anthropic-API clients. If you just want pricing and a signup link, the product post covers that.
Subscription AI CLIs are excellent products. The thing that bothered me was the cost curve when sessions got long. The 5-hour usage windows would reset mid-thought during a heads-down day, and the per-token Extra Usage bills added up faster than felt right for what I was actually doing -- reading a lot of code and emitting smaller diffs.
I started watching the per-token economics on the open-weight side. Several reasoning models had landed in the last 12 months that were close-enough on coding tasks for my workflow, at a tenth of the per-token price. The gap was real -- the open models trail frontier Claude by maybe 10-15% on hard agentic tasks -- but most of my work isn't hard agentic tasks. Most of it is refactors, debugging, generating boilerplate, writing tests. The kind of work where 90% of frontier quality at 10% of the cost is a good trade.
What I didn't have was a way to use those models from the same CLI I was already running. The subscription tool talks to api.anthropic.com over the Messages API. The open models live on different inference providers with different APIs. Bridging the gap was the project.
The naive version of this is a CLI that wraps each open-model provider's SDK. That throws away the existing CLI -- and Cursor and Cline and aider and every other tool that already speaks the Anthropic API. I wanted the existing tooling to keep working.
So the paid side of typed is a wire-compatible server. A key is only needed here -- the free local tier needs neither key nor account. Set three environment variables in your shell:
export ANTHROPIC_BASE_URL=https://api.typed.cloud
export ANTHROPIC_AUTH_TOKEN=<typed key>
export ANTHROPIC_MODEL=typed++and the client hits typed instead of Anthropic. Streaming SSE works the same, tool_use blocks work the same, system blocks work the same, image input works the same, the cache_control hints work the same. The proxy translates Anthropic shapes to the upstream's shape on the way out, the upstream's shape back to Anthropic shapes on the way in.
The trick is that "the same" has more surface than I initially understood.
Three things turned out to be load-bearing in ways I didn't anticipate.
The system field shape. The Messages API accepts system as either a string OR an array of content blocks. Existing CLIs send an array -- about 92K tokens of tool definitions, environment context, and cached instructions, spread across multiple cache_control'd blocks. Most open-model APIs only accept a single string for the system prompt. Flattening the array preserves the bytes but loses the cache markers, which matters a lot (more on caches below). I ended up writing a translator that flattens the array into one string for upstream while remembering the cache boundaries for the cache_control round-trip.
Tool definitions vs tool calls. Anthropic's tool definition shape has input_schema (JSON Schema, top-level). OpenAI-flavored APIs (which most open-model providers speak) have parameters nested under a function object. The mapping isn't 1:1 -- tool_choice semantics differ, parallel tool calls are encoded differently, and the streaming format for tool_use blocks (incremental input_json deltas) doesn't have a clean equivalent in the OpenAI streaming shape. Most of my early bugs were in this surface.
Cwd injection. Existing CLIs prepend the user's working directory as a separate system block. With a 92K-token system already in flight, that block kept getting attention-drowned by the upstream model -- the model would acknowledge the system instructions but forget which directory it was working in. The fix was injecting cwd both as a terminal system block AND prepending it to the last user message. Fixed about 15% of my early failed-edit reports.
(After two more model upgrades the prepend became unnecessary -- the newer flagship pays enough attention to the system block alone. I kept the dual-injection code path with a feature flag in case I need to re-enable it for a future regression.)
This is the single biggest engineering surprise of the project.
Several open-model providers advertise a cache discount on repeated-prefix workloads -- which is exactly the shape of CLI coding traffic. Every turn re-sends the system prompt, the tool definitions, and recent file context. If even 70% of those input tokens are billed at the cached rate (typically ~10% of the uncached rate), the COGS math changes by an order of magnitude.
The advertised cache discount and the realized discount are not the same thing. I lost about two weeks figuring this out.
Some specifics:
The pin discovery is what made the unit economics work. With the cache pin in place, my blended input cost dropped from around $0.30/M to around $0.04/M effective. That's the difference between "this idea kind of works" and "this idea is profitable at 50% gross margin on the cheapest tier."
The general lesson: a model card's advertised cache discount is a hypothesis. The realized discount depends on which inference platform you're on, whether their routing preserves cache locality, and which exact provider knobs you set on each request. Test before you commit.
The customer-visible model surface is two families. Paid and hosted: typed++ and typed -- typed serves these, and each maps internally to (a) which upstream model serves the request and (b) what reasoning-effort budget gets passed to it. Free and local: typed-local, the default without a paid plan, which runs on your machine with typed not in the request path at all.
I picked this shape for two reasons.
First, an id says where its tier sits and nothing more. It names no model and carries no version number, so nothing in it goes stale when the thing behind it changes.
Second, it abstracts away the upstream model. If I want to swap one upstream for another -- because a new model lands that beats the current pick, or because a provider's cache discount breaks -- I can do that without breaking customer integrations. The customer-visible ID is stable; the routing decision underneath isn't.
The way to pick a tier is --model, with the tier name as the value: typed --model typed++. With no flag you get typed with a key on a paid plan and typed-local without one. The launcher also still accepts --effort for anyone who has the words in shell aliases. That map is a reference, not a ladder, because its two halves have opposite pricing:
--effort max and --effort xhigh -> typed-local, the free tier on your machine.--effort high -> typed, paid and hosted.--effort medium and --effort low -> typed++, paid and hosted.Read top to bottom the map looks upside down: the two highest words land on the free tier and low is a word that bills, which is the reason --model is the way to choose. The free tier is server-shaped rather than model-shaped: anything that answers POST /v1/messages plus GET /props works -- llama-server. The CLI probes /props for the context window, probes tool support, and adapts to whatever answers.
Both hosted tiers carry 1M tokens of context -- no beta header, no separate long-context id, no tier change when a request gets big. typed is a lighter model with the lowest draw per token; each hosted tier is its own serving configuration. A request that carries images runs on the same tier and the same window as text. The free local tier is bounded by whatever window your own server reports on /props; the CLI reads it and sizes the session to fit.
The single biggest quality lever on agentic coding is variance reduction. A flagship model running once will sometimes wander -- enter an unproductive chain of thought, get stuck on a wrong premise, emit a broken diff. The same model running three times in parallel and stopping at the first one that passes a quality gate will wander much less often, because the wander-trajectories get aborted while a good trajectory wins.
So I built a best-of-N orchestrator: N parallel rollouts racing on one request, the first one to emit a complete and grading-pass response wins, the losers get aborted with their wasted tokens accounted in a separate column on the usage record. At N=3 the customer is billed 1x and my COGS is up to 3x on those requests.
This is a textbook cost-of-quality trade, and I have not taken it. Every tier is set to N=1, so nothing fans out in production and the whole path sits behind a kill switch. The architecture is there and the tests pass. Raising N for a tier is a one-number change, and I would rather make it against a real customer baseline than against my own guess at what a wander costs.
The fun engineering bit: aborting an in-flight upstream rollout has to happen before the model emits a meaningful number of tokens, or you've paid for them. I added a "length-band" gate -- if any live rollout exceeds 2.5x the median output of its siblings, kill it as wander-class. Empirically this catches the wander trajectories before they get expensive.
Only the paid hosted tiers have a quota at all -- the free local tier never reaches typed, so there is nothing to meter and nothing to bill. On the paid side: I don't use rolling 5-hour windows. typed bills monthly. When a customer hits their monthly quota, the default response is a 429 with a one-click top-up prompt -- pick $5, $10, $20, or $50 and continue. Customers who don't want the friction can enable auto-top-up in the dashboard; the next request past quota auto-charges their configured denomination, emails a receipt, and the session continues uninterrupted.
The engineering on the quota side ended up more interesting than I expected. The naive version is "subtract input + output tokens from the customer's remaining budget after each request." That's correct but leaves a concurrent-write window: a customer right at 99% quota can fire two streaming requests in parallel and have both admitted, each thinking the other one didn't exist. The actual debit can exceed the available balance.
The fix was making the debit-then-stamp path explicit -- one transaction that decrements the top-up balance OR the monthly quota (whichever the customer is on) and writes the usage record. There's still a small accepted-shortfall window where a request admitted under the prior gate-check ends up costing more than the balance covered; I stamp it anyway and emit a structured warn so I can quantify how often it happens at scale.
Auto-top-up failure is its own surface. Card declined, a payment processor timeout or 5xx -- the customer needs to know without losing their session. I return HTTP 402 with an auto_topup_failed error body and a 60-second cool-down so the client doesn't retry-spam against a dead card.
A few things I didn't expect.
The Anthropic API is the moat. I spent more time on Anthropic API fidelity than on routing. Every release of the reference CLI ships with assumptions about the API surface -- new tool shapes, new cache_control patterns, new streaming events -- and matching those exactly is the bar for "works seamlessly." There is no half-credit; if cache_control doesn't round-trip cleanly, the existing CLI's cache stops engaging and the customer's bill triples. Tests are the only defense.
Cache locality dominates per-token rates. A model at $0.30/M uncached input is more expensive than a model at $0.50/M with a working cache discount. The order-of-magnitude rate sheet matters less than the realized cost on YOUR workload's actual cache shape. If you're evaluating inference providers, measure the realized cost on a real CLI session before you commit.
Streaming is the hard part of API fidelity. The non-streaming path is straightforward translation. The streaming path requires emitting an event sequence that mirrors Anthropic's exact event types (message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop) in the right order, with token counts and tool_use deltas that match what the client expects. Get the order wrong and clients silently corrupt their internal state.
Customer-facing model identifiers are forever. Once I named the tiers, customers wrote scripts against those names, and any rename after that is a breaking change I have to absorb rather than hand to them. So a long tail of ids keeps resolving: typed-pro, typed-medium, typed-long-context and typed-plus-plus all land on typed++; typed--, typed-high and typed-minus-minus on typed; typed-xhigh and typed-max on typed-local. Not everything survived as an alias: typed-low and typed-fast were removed outright, and a request for either now returns a 400 listing the valid ids rather than quietly picking one for you. Which is the right call -- an id that silently means something else is worse than an id that errors.
I have to be honest about one exception to "forever". On 23 September 2026 a bare typed stopped meaning the free local tier and started meaning the hosted economy one. Every spelling kept resolving (typed-- still lands on the same tier it always did, and typed-local had been an alias of the free tier for a month before it became its name), no request that had ever been served changed carrier or price -- a bare typed sent to the hosted endpoint had only ever been refused -- and the change is dated in the API's own error text, on the docs pages and in the legal pages. But a meaning change is a meaning change, and "forever" has to be read as: every id keeps resolving, and any change in what one means is announced, never silent.
Honest list.
/props and works within it. If you want a big window and someone else's GPUs, that's what the paid hosted tiers are.Customer pricing starts at zero: the local tier is free, with no account and no card, because the model runs on your machine and typed is never in the loop. Above that the paid monthly tiers are deliberately boring -- $10 Starter, and $20 Pro / $100 Max 5x matching the reference subscription plans, anchored there because I think anchoring is honest. Ultra sits above that anchor at $200, where no subscription CLI publishes a plan to equate it to. Team plans are per-seat and start at $25/seat. typed matches those plans on coding quality too -- I ran a head-to-head against the reference flagship on test-graded tasks and it tied 14/14.
What actually differs is the billing shape: monthly with optional top-ups, instead of 5-hour rolling windows and weekly caps. The per-tier breakdown and top-up mechanics live on the pricing pages; the interesting engineering is the COGS side above, not the sticker price.
The fastest path is the launcher:
curl -fsSL https://app.typed.cloud/install.sh | bash # no key, no account
typed local # sizes a known-good model and prints the command to start it
typed # free, runs on your machine, never reaches typed
typed disable # bounce back to your direct providerFor the paid hosted tiers, pass a key at install time and name a hosted tier:
curl -fsSL https://app.typed.cloud/install.sh | bash -s <your typed key>
typed --model typed++ # typed++, served by typed.cloudOr wire the three environment variables yourself. Either way the client doesn't know it's hitting a different backend.
If you sign up and run a real session, the honest comparison page at typed.cloud/migrate calls out where typed and the subscription CLI diverge. Read it before committing.
I built this to keep using my existing coding CLI at a lower price. If you have the same problem, give typed an afternoon and decide.
Try typed -- free on your own machine, no account, one curl to install. Hosted tiers from $10/month, teams from $25/seat. Read the migration guide -- where typed and Claude differ. Source-of-truth pricing -- tiers, top-ups, quota policies.
Questions: support@typed.cloud.
Published by Yaw Labs.