17 messages · 8 compactions · 5 judged · context 389 / 500
A public chat room with a small model in it, haiku, which replies when someone writes @haiku, or keeps talking to it without the mention. Beside the room is haiku’s context window, exactly as the model will see it, live for everyone: the system prompt, pinned messages, summaries of earlier conversation, and the latest messages, each as tall as its share of the token budget.
What fold does here
The context is a fold view. Messages are an append-only log, and a summary is a row that stands for some of them. Fold explodes each summary into the messages it covers, and the context is the system prompt, the pins, the summaries, and every message no summary covers: an anti-join, kept current as rows come and go. The prompt sent to the model is that view, read in order.
The token budget is a fold rule: the context’s summed tokens, joined with the budget, and any transaction that would leave it over is refused. So a message that wouldn’t fit is committed together with a summary of the messages that matter least, in one transaction: inserting the summary takes them out of the context, and the new message goes in. Raising the budget retires summaries, newest first (a set of retired summaries, so the summary itself is kept), and their messages come back exactly as they were. Nothing is ever deleted from the log.
What matters least is Jev’s call too: a Score question with five levels, from “nothing” (greetings, thanks, filler) to “essential” (a fact, name, decision or request the conversation depends on), asked of every message, in the same request as its judgment when there is one. Each rating is a row. A view ranks the latest messages by it, never the newest four or the pinned, and shows what compaction would take next, so you can see it coming and pin what you want to keep.
Every reply records what it saw: a row per segment of its prompt, by reference (a message, a summary, a pin, the system prompt by its hash), in the same transaction as the reply. Since summaries are never deleted and messages never leave the log, any reply’s prompt can be read back exactly, however the context has moved since. Ask for a reply “without this” and haiku is asked again with that one segment left out; the alternative is a row too, at most three to a reply and each segment once, both fold rules, spending from your mentions like any other call.
The rest is fold too. Each day’s spending is a view under the daily cap: a mention sets the call’s worst case aside first, in the same transaction as the message, and what it really cost replaces it when the reply commits. Each visitor’s mentions are a refilling tank, like Canvas’s ink. Pins are counted per visitor, three each, and may take at most half the budget.
Without a mention, haiku has to tell whether a message was meant for it. Jev, TypeSafe’s decision model, judges each message: is it addressed to haiku, and would a reply be welcome? The code replies when the first is at least 0.70 and the second at least 0.50. Each judgment is a row (the two probabilities, the model, what it cost), and the replies it leads to draw from a tank the whole room shares, twenty an hour, with a cooldown of 20 seconds after haiku last spoke. Both are fold rules, so a slow judgment that lands just after haiku spoke is refused by fold, not by a check that could race.
Why it shows off fold
Context management is usually a loop of app code: count tokens, decide what to drop, rebuild the prompt, hope it matches what the page shows. Here it’s incremental view maintenance. Compaction and expansion are ordinary transactions that insert or retract one summary, fold works out what leaves and what returns, and the same subscription that pushes the change to every open page is the prompt the model gets next.
Worth trying
- Ask
@haiku something, and watch its reply stream into the room as it’s written.
- Then keep talking to it without the mention. Each message gets a quiet “→ haiku 0.82”: how likely Jev thought it was meant for haiku. Hover it for both probabilities and the decision; the Attention strip in the panel keeps the latest few.
- Drag the budget down: the stack rescales as you drag, and when you let go, the least important messages fold into a summary, even if they weren’t said in a row. Drag it back up and they come out again.
- Watch the quiet dots on each message: its importance, 0 to 4. When the context nears its budget, the next ones to go are marked “next to go”. Pin one to save it.
- Click a reply of haiku’s (or “what it saw”) to see its prompt as it was, marked with what has since been compacted or expanded. The link is shareable.
- There, press “Without this” on a segment: haiku answers again without it, and the alternative appears under the original for everyone.
- Click a summary to see the messages it stands for. In the room, those messages are quieter, marked “in a summary”.
- Pin a message, then talk until it would have been summarised: it stays, above the summaries, counted once.
- Show the exact prompt: that’s what the next reply will be sent, segment by segment.
- Open the inspector and send a message that doesn’t fit: one line, the message and the compaction, with the context’s tokens falling.
Still app code
Choosing what to fold from the ratings (least important first, oldest first among ties, leaving the newest few alone), deciding on Jev’s probabilities, calling the model for the summary and the reply (through curl, outside the lock), estimating tokens as characters ÷ 4, and the page. With no TypeSafe key, only mentions get a reply. With no OpenRouter key, haiku is offline: mentions get a note saying so, and summaries are placeholders that say what they cover.
Press \ in the app to watch fold's commits as they happen.