Deleting a Model's Thoughts Changed Nothing. Then I Changed the Task.

Conceptual illustration of vector feedback and discrete token generation.

I took a model trained to reason in vectors, removed its six extra reasoning steps, and watched its accuracy go from 98.4% to 98.6%.

That was one more correct answer out of 500 held-out questions. The accuracy barely changed, but the model needed about 35% less time per question.

I was reproducing Meta's Coconut method on a 4090 and a 5090, using GPT-2, a 124-million-parameter model. When I repeated the test on arithmetic, removing the same six steps dropped accuracy from 33.3% to 5.1%.

Other researchers had already reported this difference between the two tasks. My experiment is an independent replication, with additional measurements of speed and what happens as you vary the number of reasoning steps.

Feeding a model's thoughts back in

Written chain-of-thought gives a language model a place to put intermediate calculations. Each step becomes input for the next. Producing that text takes time: the model scores its vocabulary, selects a token, looks up that token's input vector, and runs again. A token can be a word, part of a word, or punctuation.

Coconut skips those text-generation steps during reasoning. It feeds the model's final internal vector, or hidden state, directly back in as the next input. The paper calls each such step a continuous thought. After a fixed number of these steps, the model generates its answer as text.

One motivation is to avoid spending generation steps on wording. Another is to let a vector represent several possible next steps without immediately selecting a token. Whether a trained model uses that flexibility is something to test.

Meta introduced Coconut in December 2024. Its training schedule builds on Deng et al.'s work on gradually removing written reasoning. The model starts by learning to write the steps. During later stages, more of those steps are replaced with vector inputs. Both of my Coconut runs finished with six vector slots and no written reasoning between the question and answer.

The schedule matters. In the paper's direct-training comparison, Coconut scored 14.4% on grade-school arithmetic, below the baseline that answered without a reasoning chain.

The paper also compares Coconut with pause tokens, an idea explored by Goyal et al. in 2023. These are learned placeholder inputs. They give the network extra positions to compute over without feeding the preceding hidden state back as the next input. Keeping the same training schedule makes them a useful control for the vector feedback.

Graph search and arithmetic

ProsQA is a synthetic graph-search benchmark. A prompt lists roughly fifty invented facts and asks which of two conclusions follows. A shortened example is:

Sally is a scrompus. Every scrompus is a rempus. Every rempus is a sterpus. Is Sally a hilpus or sterpus?

The answer is sterpus, reached in three hops. The full prompt includes other facts, but exactly one of the two answers is reachable. Random guessing would score 50%.

GSM8K consists of grade-school arithmetic word problems. The training data includes worked calculations, such as:

Out of 600 employees in a company, 30% got promoted while 10% received bonus. How many employees did not get either a promotion or a bonus?

600×30/100 = 180 · 600×10/100 = 60 · 180+60 = 240 · 600−240 = 360

That supplied solution assumes the promoted employees and bonus recipients don't overlap. Following it requires calculating intermediate values and reusing them. In ProsQA, the relevant relationships are already written in the prompt. I suspected that difference might affect how much the model needed the extra vector steps.

Coconut's paper reports a large advantage over written reasoning on ProsQA: 97.0% against 77.5%. On arithmetic, it trails written reasoning, 34.1% against 42.9%.

Reproducing the results

I trained the models with one random seed, using about 85 GPU-hours. I started from Meta's implementation, added the missing control configurations, and later patched an inefficient tensor operation for the arithmetic run. The patch matched the original loss and gradients in the verification batches.

The eight comparisons below broadly reproduce the paper's results. My results contain one run per method; the paper reports means and standard deviations across runs.

• Coconut, ProsQA: My accuracy (%): 98.4; Paper accuracy (%): 97.0 ± 0.3

• Pause tokens, ProsQA: My accuracy (%): 96.4; Paper accuracy (%): 96.6 ± 0.8

• Removal curriculum without vector slots, ProsQA: My accuracy (%): 95.2; Paper accuracy (%): 95.5 ± 1.1

• Written reasoning, ProsQA: My accuracy (%): 75.8; Paper accuracy (%): 77.5 ± 1.9

• Direct answer without a reasoning chain, ProsQA: My accuracy (%): 76.8; Paper accuracy (%): 76.7 ± 1.0

• Written reasoning, arithmetic: My accuracy (%): 42.8; Paper accuracy (%): 42.9 ± 0.2

• Coconut, arithmetic: My accuracy (%): 33.3; Paper accuracy (%): 34.1 ± 1.5

• Pause tokens, arithmetic: My accuracy (%): 24.6; Paper accuracy (%): 24.1 ± 0.7

The two controls without reasoning steps use different training procedures. One gradually removes the written steps; the other trains on direct answers. They are separate training controls, not two names for the same baseline.

On the RTX 4090, with one question at a time, written reasoning took 0.210 seconds per ProsQA question and Coconut took 0.071 seconds. Coconut was about three times as fast and 22.6 percentage points more accurate in these runs.

Removing the reasoning steps

Those scores left me with a question: did the trained Coconut model still need its vectors at inference? I added a hook that let me delete or replace them. With the hook inactive, the model reproduced the original accuracy and generated-token count.

Deleting all six vectors gave 98.6% accuracy. Replacing them with random vectors of the same magnitude also gave 98.6%. So did adding noise twice as large as the vectors themselves.

The most striking result came from taking another question's six vectors and substituting them for the model's own. All 500 answers stayed the same. Across the other interventions, at most three answers changed.

I checked that the replacements reached the model. Random replacement moved the input vector by 139% of its original length and changed the model's vocabulary scores, the values used to choose an output token, by up to 2.9. The intervention affected the computation even though accuracy barely moved.

I also passed each reasoning vector through the model's output layer to see which token it would predict. Every vector, at every position across all 500 questions, strongly favored ###, the marker that introduces the answer. Successive vectors also became nearly parallel. Those readings suggest the states were changing little in direction, but confidence in an answer marker doesn't establish that the model had already computed the correct answer.

The training controls help explain the accuracy. Keeping the removal schedule and replacing feedback with pause tokens scored 96.4%. Training with the removal schedule but no vector slots scored 95.2%. Both were far above the 75.8% written-reasoning baseline. Most of Coconut's advantage on this task survived without vector feedback, although the separately trained controls don't isolate every effect of the curriculum.

There was a useful inference saving on this checkpoint. In the ablation run on the RTX 5090, removing the six steps reduced time per question from 0.061 to 0.040 seconds, about 35% less time, while accuracy stayed essentially unchanged.

On arithmetic, deletion mattered

ProsQA proofs average fewer than four hops, and GPT-2 has twelve layers. One possible explanation is that the model could solve these questions without the extra passes. The ablations don't reveal its exact algorithm.

I trained the arithmetic version for about seventeen hours on the 5090, with a pause-token control on the other card, then ran the same interventions. These are separately trained models with the same architecture and test code.

• Keep the six vectors: ProsQA accuracy (%): 98.4; Arithmetic accuracy (%): 33.3

• Delete them: ProsQA accuracy (%): 98.6; Arithmetic accuracy (%): 5.1

• Replace them with random vectors: ProsQA accuracy (%): 98.6; Arithmetic accuracy (%): 6.1

• Use another question's vectors: ProsQA accuracy (%): 98.4; Arithmetic accuracy (%): 2.9

On arithmetic, deleting the vectors cost 28.2 percentage points. Another question's vectors did even worse than random replacements: 2.9% against 6.1%. That is evidence that the vectors contain information specific to the question. It doesn't tell us which calculation each vector represents.

I then varied the number of vector steps at inference, from zero to eight.

Arithmetic accuracy rose from 5.1% with zero vectors to 33.3% with six. The increase averaged about five percentage points per added vector, though the individual gains were uneven. ProsQA accuracy barely changed.

The model had only been trained with up to six vectors. Seven scored 34.0%, and eight scored 34.4%. Those small gains are worth checking across more training seeds before treating extra steps as a reliable improvement.

The arithmetic model depended on the vector loop, but it still scored below written reasoning: 33.3% against 42.8%. On the RTX 4090, it took 0.035 seconds per question against 0.131 seconds for written reasoning, roughly 3.7 times as fast at a cost of 9.5 percentage points of accuracy.

These two tasks gave different answers to the deletion test. Removing the extra steps left the ProsQA checkpoint's accuracy essentially unchanged and badly damaged the arithmetic checkpoint's accuracy. A high benchmark score alone wouldn't have told me that.

Earlier work found the same split

After the ProsQA result, I looked for earlier experiments that had tested whether the vectors affected the answer. Three groups had studied this question.

Do Latent Tokens Think?, from December 2025, reported that perturbing Coconut's latent tokens changed answers much less often than perturbing written reasoning tokens. An April 2026 study, The Illusion of Superposition?, found no accuracy drop when deleting the latents from a fine-tuned Coconut on ProsQA, but a large drop for small models trained from scratch.

In June 2026, Aswal, Ferraz, Zhou and Peyrard published Observable Patterns Are Not Explanations. They tested both benchmarks, including deletion and a pause-token control. With Coconut's latents removed, their ProsQA accuracy went from 98.0% to 97.8%, and arithmetic went from 35.7% to 7.7%. My corresponding results were 98.4% to 98.6% and 33.3% to 5.1%. They also tested swapping vectors between examples.

Their study adds a qualification to the ProsQA result. Some patterns interpreted as search also appeared in a control without vector feedback. Yet targeted changes to particular directions in the latent vectors could still flip answers. Little change in aggregate accuracy after deletion doesn't mean the vectors have no influence under every intervention.

My runs reproduce the broad difference between the tasks. The additional measurements cover intermediate vector counts, seven and eight steps beyond the trained count of six, several noise magnitudes, internal-state readings, and timing on consumer GPUs.

A related approach: Soft Thinking

Coconut feeds back a hidden state. Soft Thinking, introduced in May 2025, instead feeds back a weighted average of token embeddings. Each embedding is weighted by the model's predicted probability for that token. The method can be applied to an existing reasoning model without Coconut's training schedule.

The original Soft Thinking study reported QwQ-32B math accuracy rising from 83.84% to 86.32%, with reasoning chains 12–22% shorter. A later analysis, LLMs are Single-threaded Reasoners, found that the most likely token often dominated the average, making the method behave much like greedy decoding, which always selects the highest-scoring token.

In that analysis's QwQ comparison, soft inference scored 80.06%, sampled reasoning scored 82.35%, and a version that added noise to the blend scored 83.04%. Meta's Soft Tokens, Hard Truths also distinguished benefits during training from benefits during inference, and did not confirm the earlier claims for soft inference.

These are separate methods and experiments. The connection to my Coconut runs is the choice of control: for Soft Thinking, compare against greedy and sampled decoding; for Coconut, compare against the same curriculum with the feedback removed.

What I would test next

This experiment covers GPT-2, one seed, and two tasks. I would next repeat the deletion and count tests on a modern half-billion-parameter model, then check whether the result holds across training seeds.

Later work gives reasons to test scale and supervision separately. CODI distills written reasoning into latent vectors. SIM-CoT adds supervision through a decoder. LOTUS uses repeated transformer blocks and supervises intermediate states against written reasoning tokens. Their results concern different training and inference procedures; this Coconut replication doesn't settle how those methods behave.

For anyone reproducing the small-model experiment, the largest practical saving came from a code change. Meta's implementation rebuilt the input tensor in Python at every vector step. Replacing that operation with a clone and an indexed assignment preserved loss and gradients in our checks and made the six-vector training step about six times as fast. That brought the arithmetic run within reach of an overnight session on the 5090.

For the next model, I would measure the effect of removing vector steps before choosing a fixed reasoning budget. That would tell me whether the extra passes improve accuracy enough to justify their inference cost.

References

• Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, arXiv:2201.11903

• Goyal et al., Think before you speak: Training Language Models With Pause Tokens, ICLR 2024, arXiv:2310.02226

• Deng et al., Implicit Chain of Thought Reasoning via Knowledge Distillation, arXiv:2311.01460. Source of the augmented GSM8K training set

• Deng et al., From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step, arXiv:2405.14838. The removal schedule Coconut borrows

• Hao, Sukhbaatar, Su, Li, Hu, Weston, Tian, Training Large Language Models to Reason in a Continuous Latent Space, arXiv Dec 2024, COLM 2025, arXiv:2412.06769 · code

• Shen et al., CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation, EMNLP 2025, arXiv:2502.21074

• Zhu et al., Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought, arXiv:2505.12514

• Zhang et al., Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space, arXiv:2505.15778

• A Survey on Latent Reasoning, arXiv:2507.06203

• LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking, arXiv:2508.03440

• Butt et al., Soft Tokens, Hard Truths, arXiv:2509.19170

• SIM-CoT: Supervised Implicit Chain-of-Thought, arXiv:2509.20317

• Do Latent Tokens Think? Probing the Causal Role of Latent Reasoning Tokens, Dec 2025, arXiv:2512.21711

• Latent Reasoning with Supervised Thinking States, Feb 2026, arXiv:2602.08332. Qwen2.5-0.5B and 1.5B

• Zou, Xiong, Liu, Capabilities and Fundamental Limits of Latent Chain-of-Thought, Feb 2026, arXiv:2602.01148

• How Do Latent Reasoning Methods Perform Under Weak and Strong Supervision?, Feb 2026, arXiv:2602.22441

• The Illusion of Superposition? A Principled Analysis of Latent Thinking in Language Models, Apr 2026, arXiv:2604.06374

• Aswal, Ferraz, Zhou, Peyrard, Observable Patterns Are Not Explanations: A Causal-Geometric Analysis of Latent Reasoning Models, Jun 2026, arXiv:2606.12689. Ran the same two-task ablation with a pause control

• LOTUS: Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers, Jun 2026, arXiv:2606.31779

• Training Continuous Chain of Thought Models: A Tale of Two Regimes, Jul 2026, arXiv:2607.16972

• Recirculation, Aug 2026, arXiv:2608.17981. Inference-time recurrence with no weight changes