I Fed a Model Facts Through Its J-Space Instead of Its Prompt

The invisible KV cache, the workspace band, and the answer: the fact enters through the drawer, never through the prompt.

I typed "The capital of Kevin's home country is the city of" into Qwen3.5-4B and it answered " Beijing". Nothing in the prompt mentions China. No system message, no retrieval snippet, no hidden instruction. I had written a China vector into layers 12 through 17 of the model's residual stream, at one specific token, at a specific per-layer volume, and the model treated it as a fact it had read.

This took an afternoon on a single RTX 4090, and the whole thing runs on code Anthropic published alongside the paper.

The paper that made this possible

Anthropic's global-workspace paper landed with a claim that sounds philosophical but is actually mechanical: a small set of activation patterns inside Claude behaves like a workspace. They call it the J-space, and the model can report what's in it, deliberately hold things in it, and route multi-step reasoning through it. Their measurement tool is the Jacobian lens, a per-layer matrix that translates mid-network activity into vocabulary space, and their headline intervention is the swap: replace the silent "spider" pattern while the model works on "the number of legs on the animal that spins webs" and the answer flips from 8 to 6.

Two things made me drop what I was doing. They released the code under Apache 2.0, with the exact prompt sets from the paper. And Neuronpedia published pre-fitted lenses for Qwen3.5-4B, a model that fits on my GPU with 16GB to spare.

The paper's swap replicates on this little model, by the way: 34 of 36 trials on the clean templates. A 4B model on consumer hardware carries the same causally load-bearing workspace the paper found in Claude. That result alone would have justified the afternoon.

The question the paper doesn't ask

Every swap in the paper edits a concept that is already in the J-space because it appeared in the text. What happens when the slot is empty?

That question is worth asking because of what RAG does all day. Retrieval fills the context window with sentences whose only job is to resolve references: who is this user, which account, which country. Each of those sentences costs prompt tokens and attention. If the workspace accepts writes, maybe retrieval could skip the prose and deposit the entity directly.

The workspace holds pointers to concepts the model already knows, one vocabulary token per pointer, a few dozen slots. You cannot write "Q3 revenue was $4.2M" into it, because that is a novel proposition, not a pointer. Documents still need the context window. The testable slice is reference resolution, so I built a battery for exactly that: 16 countries, four questions each (capital, official language, continent, currency), asked about "Kevin's home country" with Kevin's country never stated. With no information the model scores 6%. With the fact written in the prompt, 92%. Everything interesting happens between those two lines.

Writing to a brain is not writing to a file

My first attempt was the obvious one. Take the lens direction for " France", add it at every token position across the workspace band, generous strength. Accuracy: zero out of sixteen, and not a quiet zero. The model's top predictions became " France", "France", "法国". I hadn't given it a thought; I had given it a tic.

Same France vector, same strength; only the write positions change.

The failure has clean mechanics. A lens direction is, by construction, the pattern that makes the model more likely to say that word. Added at the final position, the position that predicts the next token, it is a raw output bias. The paper's swaps never hit this because coefficient transfer scales the edit by how strongly the old concept is expressed at each position, which is roughly zero everywhere except where the entity actually lives. Injection into an empty prompt has no such gate. You have to build it yourself, and it took me three fixes.

The first was the address. Instead of writing everywhere, I injected only at the token where a resolved entity ought to be bound, and I guessed the noun "country". That one change took accuracy from zero to 41%.

The second was volume. When France genuinely appears in text, it expresses along its own direction at a coefficient of about 2.4 at layer 12, rising to 7.1 by layer 24; the concept fades in with depth. Reproducing that measured profile instead of a flat magnitude pushed accuracy to 53%. Doubling it made things worse, so the reading circuits apparently expect signals at natural amplitude and treat anything louder as distortion.

The third fix was to stop guessing and ask the model where. When a transformer processes a word, it chooses which earlier words to pull information from, and an attention map is the record of that choice: for each position, a list of where it looked and how hard. Qwen3.5-4B complicates this, because only 8 of its 32 layers have the full kind of attention that can look back at any word; the rest keep a compressed running summary. So I pulled the maps from those layers, at the position about to produce the answer, and checked which words of the question it reads before answering. Not "country", where I had been injecting. It reads " Kevin", the name whose country it doesn't know, and the question's own subject word, "continent" in the continent question, "currency" in the currency one. Moving the injection to the spots the model actually reads brought the total to 64%, and the capital and continent questions stopped missing entirely, 16 out of 16.

Every method, one chart: residual injection climbs with better calling conventions; the cache route jumps past all of it.

The two-token cheat that nearly closes the gap

Residual injection fights the model's expectations at every step. There is a lazier move: give the entity a position of its own. Forward the bare tokens " France." through the model once, keep the resulting KV-cache state, and run the question on top of that cache. The prompt the user sees contains nothing. The cache contains a two-token phantom.

When a transformer reads text, every token leaves behind a key-value pair at each attention layer. The key is how that token advertises itself to future lookups; the value is what gets handed over when a lookup matches. That per-token ledger is the KV cache, and it is the only trace a past token leaves. The model never rereads your text, it rereads the cache, which makes the cache the real interface and the prompt just the usual way of filling it. A phantom slot is a cache entry with no counterpart in the visible text. I let the network write the entry itself by forwarding " France." once and continuing from there, rather than splicing tensors by hand: Qwen3.5-4B is a hybrid, so "cache" also means recurrent state in its 24 linear-attention layers, and hand-splicing across two kinds of state is how you get subtle garbage. To the attention heads downstream, France is just another token they once read. It gets attended to like any real word and never shows up on screen.

Some trials straight from the run log. With " Japan." in the cache, the capital question comes back " Tokyo", and " Kyoto" sits in the top four. With " Peru.", the language question returns " Spanish", and the runner-up token is " Que", the start of Quechua, an alternative nobody asked about and one that only makes sense if the model is actually thinking about Peru. With " Kenya.": " Nairobi" for the capital, and the language answer starts spelling out Swahili. The misses stay in-country too. With " Egypt." cached, the currency question answers " Egyptian" with " pound" right behind it, which reads as a tokenization stumble, not a retrieval failure.

84%. Against a 92% text ceiling. And the model does the reference binding entirely on its own; there is no sentence anywhere saying Kevin has anything to do with France. A dangling entity in cache plus an unresolved referent in the question is apparently enough. Even the currency question, which every injection variant failed at 0/16, comes back at 50%, which is exactly as well as that clunky template performs with the entity written in plain text.

One detail I didn't expect: the trailing period is worth 17 points. " France" alone scores 67%; " France." scores 84%. Punctuation seems to close the entity into a finished statement that downstream attention retrieves far more reliably. I would not have bet on a period being a load-bearing component of retrieval.

Same retrieval, three delivery routes: what the user sees, and what the model scores.

Where injection breaks, the cache slot doesn't.

Then I tried to break it

The obvious objection to a two-token phantom: real retrieval is never one fact. So I stress-tested it with same-shaped noise. For each trial, k distractor facts about other people ("Fact: Maria's home country is Japan.") joined Kevin's fact at a random position, for k of 4, 16, and 48. Text-RAG got them as visible sentences; the phantom lane got them as cache entities.

The bare-entity phantom didn't degrade; it died. 84% clean, 6% with four distractors, which is the no-information floor. And it should die, which is the useful part. Cache entries written for " Japan. Brazil. France." are context-free: nothing in them records whose country is whose. With one dangling entity, the model binds it to the one dangling referent and looks clever. With five candidates, the cache genuinely contains no answer to "which one is Kevin's," and the collapse is the model being honest about that.

The fix costs one token per fact. Forward " Maria: Japan. Omar: Brazil. Kevin: France." instead, and the France entries get written in the context of Kevin: the association is baked into the keys and values themselves, because a cache entry records a token-in-context, not a token. Models have read a lifetime of name-colon-value structures, so the retrieval heads know what to do with them. Bound this way, the phantom tracks text-RAG at every noise level and ties it at 48 distractors, 84% to 83%, at roughly half the cache tokens per fact.

The clean condition forced me to revise this post's own framing. Bound " Kevin: France." scores 95%, a point above the full fact sentence. The 84%-versus-92% gap I reported above was never the cache's limit. It was a binding problem, and three tokens solve it.

One token of binding survives 48 distractors; none survives four.

Does it hold when the model grows?

I reran the noise battery on the 2B and 9B siblings. The unbound collapse replicates everywhere, as it must; no amount of scale can find binding information that isn't there. The bound slot is the interesting curve. At 2B it trails prose badly under noise, 45% against 67% at sixteen distractors. At 4B they tie. At 9B the cache lane reads at 91% against 92%. Reading name-colon-entity shorthand as reliably as prose is something models grow into. One more scale effect surprised me: the bigger the model, the less willing it is to bind a bare, unattributed entity in the clean condition (80%, then 84%, then 61% at 9B). The 9B isn't worse at using the cache; it's more suspicious of an unlabeled fact, which is probably the behavior you'd want.

By 9B, cache shorthand reads like prose.

Where the phantom lives: the J-space

Does a cache trick have anything to do with the global workspace this post started from? I measured it directly. With " Kevin: France." sitting in cache and nothing in the prompt, the Jacobian lens finds France in the workspace during the question at median rank 1 for all sixteen countries, indistinguishable from when the fact is written in text. The model lifts the cached entity into the same broadcast space Anthropic described and answers from there.

Stored once in the cache, broadcast in the workspace when the question needs it.

Then I tried to cut the wire. Erasing France's workspace direction during the question barely mattered: 95% dropped to 88%. My first reading was deflating: the workspace displays the fact but doesn't carry it. The real story turned out better. That erasure left the cache entry itself untouched, and attention can re-read the source at every layer, rebuilding whatever I scrubbed. So I corrupted the entry instead, applying the same erasure while the prefix was being processed, so the stored keys and values never contained a clean France: 64%. Still alive, because now the leftover residual copy compensates. Both interventions at once: 17%, essentially the floor. The same erasures with a control country's direction changed nothing anywhere.

The fact lives twice: either copy alone carries retrieval, and only losing both kills it.

So the fact lives twice. The cache stores it, the workspace broadcasts it, either copy can carry retrieval alone, and only losing both kills it. That redundancy also explains, in hindsight, why the phantom beat injection all along. Injection was me writing a single copy into the workspace by hand, once, with hand-tuned position and amplitude, and nothing left behind for attention to re-fetch. The phantom hands the model a stored source and lets it do its own broadcasting, with native timing and native volume. I spent three script versions learning the workspace's calling convention; the cache route is the model already knowing it.

One edge stays open. This model is a hybrid: 24 of its 32 layers keep recurrent state rather than key-value pairs, and my prefix erasure corrupted both kinds of memory at once. How the entity divides between them is the next question down.

Could you skip the text entirely?

The phantom slot is still context. I built the cache by forwarding two real tokens; information did not materialize from nowhere. The claim is narrower and, I think, more useful: retrieval does not need the fact sentence. It collapses to a constant-size, precomputable KV entry per entity, invisible to the prompt, and composable, since two entities can occupy two phantom slots without writing a paragraph to link them.

If the phantom is just text delivered in a hidden turn, the purer method would write vectors into the cache and skip text entirely. I tried that too, four ways: the France embedding clamped into a cache slot at every layer, the same thing rescaled to natural residual magnitude, the lens direction at its measured amplitude written into a dedicated slot attention could re-read, and a two-slot Kevin-plus-France version. Every one of them scored at the no-information floor, in the same run where the text-built references hit 84 and 95 percent. The outputs weren't even wrong about France; they were identical across countries, generic guesses. Measuring why: the embedding slots were never even attended, their lookup keys unreadable to every query, and the lens-direction slot was attended and even displayed France in the workspace readout while the model remained unable to use any of it. A cache entry the model can use has to come from real reading: the notes the model takes, layer by layer, while the token sits in actual text. Everything cheaper than that got ignored or went unread in the ways that matter. The methods that do bridge this gap, soft prefixes, gist tokens, learned context compressors, all train their way across it. Without training, the only way to write a memory is to read.

One variant did work, and it deserves spelling out because no text is involved at question time. Offline, I ran " Kevin: France." through the model as ordinary text, once, and recorded its internal notes at every layer for those four tokens. At question time I fed the model four meaningless placeholder tokens and overwrote its notes with the recording, layer by layer, so the memory it filed was the one real reading would have produced. The battery came back at 95%, identical to real text. The catch is what that amounts to: recorded at positions one through four and played back at positions one through four, it is prompt caching with extra steps. When I played the recording back at different positions than it was made, accuracy fell to 63%, the language question going to zero while continent didn't move at all, even though the moved recording was still fully attended and its country still showed in the workspace at rank one. The recording works where it was made. Move it and it breaks, unevenly, in the reading circuits rather than at the point of retrieval.

What I'd trust and what I wouldn't

This is one instruct model with completion-style prompts, so I reran the headline cells at 256 trials each, with four different names, before trusting them. They held. Phantom and text-RAG are statistically indistinguishable (McNemar exact p=0.61), and under any grading laxer than strict first-token both are perfect, which also quantifies how much the strict grader undercounts: about nine points, all sentence-style artifacts. The precision worries turned out unfounded. With two people's facts in cache, the model fetches the right person's entity at 91% with a 1.6% intrusion rate, and an irrelevant phantom in cache doesn't corrupt unrelated questions at all. Two caveats earned their place instead: a second cached fact costs about five points on recall, so multi-fact capacity needs measuring before anyone builds a hundred-entity cache, and the 8% floor is nonzero because continents and languages are guessable. One more caveat the last experiment forced: the lens readout can display a concept the model cannot use, so workspace presence alone is weaker evidence than it looks. Nothing here says anything about Claude, about other scales, or about the consciousness questions the source paper carefully brackets. Idea-wise, this sits near soft prompts, function vectors, and REMEDI-style fact injection; the new parts are the free vocabulary-indexed direction dictionary the J-lens provides, and the workspace framing that told me where to aim.

The phantom stays in its lane: 1.6% worst-case intrusion at n=256.

What stays with me is the shape of the lesson. The workspace has a write API, and its calling convention is strict: right address, natural amplitude, correct depth profile, and it still varies by which downstream circuit does the reading. Text in context needs none of that care, which is a good explanation for why context windows won, and the phantom cache slot inherits that robustness at a fraction of the size. The open question I actually care about is scale: does injection consumability emerge somewhere between 0.8B and 27B, or is the calling convention this fussy all the way up? The lenses for the sweep are a weekend of backward passes away.

Everything here is reproducible on one GPU. The scripts, the raw per-trial data behind every number, and the figures are at github.com/phenx-inc/J_space_prompting.

---

An independent experiment on a single RTX 4090 (plus an RTX 3090 for the scale sweep) using Anthropic's open-source Jacobian lens and Neuronpedia's pre-fitted lenses on Qwen3.5 dense models. All accuracy numbers are strict greedy next-token grading unless stated. The scripts, raw per-trial data, and figures are public at github.com/phenx-inc/J_space_prompting.