The word “chat” in ChatGPT always amuses me. It’s brilliant branding: one syllable that makes machine output feel like human conversation. But chatting with LLMs is rarely participatory – at least not in the way we’re used to.
The best thing about human conversation is locality. You can interrupt – “wait, what do you mean by that?” – request a rephrase, and get the updated version right where the confusion happened. The whole experience is nonlinear: it emerges, loops back, and self-corrects.
Texting narrowed that. It forced conversation into sequential, computer-like steps – the interface is basically a log file. But texting preserves some locality because turns are short, which creates natural openings to cut in and repair the thread before it runs away.
ChatGPT inherits the messaging-app template. The irony is that LLM doesn’t speak like a person, yet we force it into that UI. Locality disappears, and that makes real knowledge hard to build.
Who chats this long?
The first problem is non-interruptible, verbose generation. The knowledge is happening on stage; you’re watching in the dark. That builds a small fourth wall. If a sentence trips you up, the only move is endurance: let the model finish its performance. Even when you spot an error, your objection can’t land where the error is. You can only respond after.
The interface quietly assumes a linear user: read the whole output, hold it in mind, then reply cleanly.
Most of us can’t do that. The moment we feel lost, our eyes drop to the composer. We fire a follow-up – and the page scrolls, appending the answer at the bottom. The clarification you needed is now physically detached from the sentence that triggered it.
Once repairs are forced to the bottom of the page, every clarification carries a tax: scrolling, reloading context, re-reading. Each new turn pushes the earlier structure further out of reach. After two or three of those, the thread is a nest of follow-ups and the structure is gone.
The cheapest move stops being “understand” – it becomes “try again.” There’s nothing left to do with the answer except re-roll it. The interface starts to feel like a slot machine. Learning research has blunt phrases for what this behaviour is missing: desirable difficulty. When an answer arrives smooth and complete – without requiring any meaningful participation – you confuse ease of reading with depth of grasp.
Why GPT doesn’t read like real knowledge
Another weird thing about LLM is its content. They are structured, coherent, easy to read. But for some reasons, you can only rely on them to solve problems, not to learn.
This disposability starts with our intention of using AI, which is to get things done (履约), and that is why email composing and coding agents remain to be LLM’s killer applications. The mindset affects us while we are learning from LLMs. We are inclined to ask a locally optimal question, that is, the very question that clears the comprehending roadblock and pushes you forward. Most of these questions are about sensemaking, to quickly grasp a meaning, and get a response that feels coherent and complete for the question in front of it. It fits the moment so well that it reads like closure.
What we really need is composable knowledge: reusable pieces – definitions with boundaries, assumptions made explicit, a map of what depends on what. No matter you are studying something from academia, or fixing your toilet, that should be the goal when crafting prompt or asking a question.
But it’s not enough to just have a proper mindset. AI-generated texts don’t read like human, which makes it terrible format to host such knowledge. Grant Sanderson, the creator behind 3Blue1Brown, made this comment on Wikipedia, which I think bears the same issue with LLM outputs:
There’s a difference in feel for trying to learn something from a Wikipedia page versus, if it’s a philosophy topic, going to the Stanford Encyclopedia of Philosophy…. The difference there is the articles are deliberately written by one individual who tries to actually craft a motivation around it. Whereas on Wikipedia, it’s this local minimum that’s reached where every sentence has to be correct.
Dwarkesh Patel replies:
The way I was learning it, I’d have his (Steven Strogatz) university lecture on one-third of the screen, that part of the textbook on another third, and an LLM on the last third… I’m pausing, reading the textbook, talking to LLMs, and then restarting again. But with him curating the right order to understand concepts and the right problems to motivate understanding them…
This is what separates human-generated knowledge from machine-generated: a curated structure. It’s like a story arc, with ups and downs, and behind is the motivation to really make you understand — even at the expense of making mistakes.
For example, when explaining concepts, good teachers would have to say something that’s not quite right, and correct it later. These semi-wrong words scaffold our understanding, and removing them after the truth is reached would give us strong sense of closure. The scaffolding works because of something more general: knowledge lives in structure, and dies outside it. The Chinese writer Mu Xin saw this in Dream of the Red Chamber: “The poems in Dream of the Red Chamber are like water plants — take them out of the water, and they lose their beauty. Left in the water, they are lovely to behold.” Out of the water, they are the same poems — what they lose is the story that made them mean something. Knowledge behaves the same way: even a perfectly correct explanation is close to unlearnable when it arrives with nothing around it. That is what LLM output feels like — a piece of jello: polished, every sentence fine on its own, but nothing submerged in anything.
It’s tempting to conclude that we should wait for models to write like Strogatz. But look again at what Dwarkesh was actually doing on that screen: pausing, cross-reading, asking, restarting. Strogatz supplied an arc, but the arc that mattered was the one forming in Dwarkesh’s head — assembled from his own stops and starts. The lecture was raw material. The learning was the editing.
That’s the reframe I want to propose: the arc doesn’t have to arrive pre-built. It can be built by the reader, if the interface lets them.
How we did with books
Books already taught us half of this. You can’t change the ink—but you can change what the text does for you, and your interaction stays local: next to the sentence you’re wrestling with. You underline, write your own thoughts in the margin, and return later to the same spot with the history of your confusion still visible. Unfortunately, there is not much blank space on the margin, so we cannot document everything in our brian.
With printed books, that limitation was physics. With generated text, it’s superstition — an inherited norm that the model’s words have “integrity,” and changing them would be tampering. That norm makes sense for human messages: rewriting what a person said really is a kind of forgery. But generated text has no author to betray. It’s material, not testimony. Refusing to rewrite it doesn’t protect anyone; it only keeps you in the spectator seat.
And rewriting is not just permitted — it’s where the learning happens. Rewriting a passage forces the exact moves that reading lets you skip: deciding which claims survive, which were scaffolding, and how the pieces actually depend on each other. You can’t rewrite a paragraph you haven’t understood; the attempt is the test. (Yes, you might rewrite an error into your notes.)
Based on this proposal, I built coo. To see what it’s for, look at the workflow it replaces. Every day we generate a small library’s worth of knowledge with LLMs — explanations, comparisons, walkthroughs. And our instinct for anything worth keeping is the one we trained on the human web: clip it, send it to the notes app. That instinct made sense for human writing, because when you clip a paragraph from a good essay, you’re saving the author’s arc along with the words — the water comes bundled in. Clip a paragraph of jello and you’ve saved nothing but surface. It reads the same in your vault as it did in the chat: smooth, finished, inert. Clipping is the most efficient way to store generated text, and close to the least efficient way to understand it.
coo is a different way to save this kind of knowledge: interact first, keep what survives. Ask a question, drag-select any words in the reply and an in-place editor opens right inside the conversation: you can ask a follow-up that stays anchored to that word, rewrite it once the confusion resolves, or delete it outright. Last month I was reading a generated explanation of formal verification and one sentence stopped me cold. In the old workflow, that sentence and my question would have ended up five scrolls apart, and the whole answer would have been clipped — unread, really — into notes. In coo, the question, the answer, and my eventual rewrite all live at that sentence, and what finally lands in my notes is a passage I could teach from, because I built it. That’s the real product: notes that started as model output but end up ultimately mine.
One honest caveat. coo does not fix the locally optimal question. My own default prompt inside coo is still “what is it.” No interface can force you to ask for boundaries, assumptions, and dependencies when all you want is to get past the roadblock in front of you. What coo changes is what happens after the cheap question. In a chat thread, a “what is it” answer scrolls away and dies; the question was cheap and the answer stays cheap. In coo it stays pinned to the sentence that provoked it — and pinned questions accumulate. Three lazy “what is it"s hanging off one paragraph are a diagnosis: this paragraph needs rewriting, and by then you’re holding the pieces to do it. The tool doesn’t upgrade your questions. It composts them.
coo also has a plugin for Obsidian, which is more suitable to the reader scenario. When a line from a piece of well-structure content trips you up, coo gets you answers in foldable callouts. If coo thinks the question is a bit trivial to the goal of understanding the document, it will add tag [minor] so you can skip it first. The plugin will record chain of conversation, so you can even select words in the answer for follow up questions.
Computers are fun; interaction is human
Everyone is talking about building agents. But most of what people actually do in ChatGPT is still simple: they ask questions. That means epistemology and interface design aren’t side quests – they’re under-provided public goods, because better ways of asking, checking, and revising help everyone.
You can already hear the default reply: a better model will solve all of this. But this isn’t mainly a model problem. It’s human science – how people notice confusion, test claims, form concepts, and revise beliefs. The dream is frictionless knowledge, answers delivered like electricity. Learning doesn’t work that way.
Computers aren’t just appliances. They’re tools people like using because they invite agency: you poke, you try, you iterate, you make something, you change your mind. That’s the pleasure of software at its best. It’s also the route to understanding. Learning is not merely receiving; it’s participating.
So the goal isn’t to delete friction. It’s to keep the meaningful friction – the moments where you have to ask, rewrite, test, and connect – while removing the dumb friction of scrolling, losing context, and being forced into a spectator seat.