Researchers Expose Flaw in Encrypted LLM Reasoning Traces

Encrypted Reasoning Blocks Exploited to Reveal Hidden Chain-of-Thought

Proprietary large language models (LLMs) from Anthropic, OpenAI, and Google have a new vulnerability: their encrypted reasoning traces can be stolen and decoded. A research paper, Stealing Reasoning Traces from Proprietary LLM APIs, demonstrates an attack that extracts hidden chain-of-thought from frontier models without directly attacking them. The technique exploits how providers return encrypted reasoning blocks to clients, which are portable and share encryption keys across model families.

The attack, detailed on stolen-thoughts.com and arXiv, works in two API calls. First, attackers capture an encrypted reasoning block from a strong model (e.g., Claude Opus 4.8). Then, they inject that block into a weaker, jailbroken sibling model (e.g., Claude Haiku 4.5) from the same provider. The weaker model decodes the block and outputs the original reasoning in plaintext, bypassing anti-distillation safeguards.

Portable Encryption Keys Enable Cross-Model Replay

The root cause is that providers return encrypted reasoning blocks to clients, which are then sent back with subsequent requests. These blocks are fully interchangeable across sessions, users, and models within a provider's ecosystem. The paper's authors discovered that all models under the same family share the same encryption key, making it trivial to replay a block from a frontier model into a weaker sibling.

Simon Willison, who covered the paper, noted that the attack was straightforward: "Every model under the same family used the same encryption key, which meant you could feed those blocks back into the weakest model family members and jailbreak them into outputting the unencrypted raw reasoning blocks." The paper's authors confirmed this across Anthropic, OpenAI, and Google, with Claude Haiku 4.5 being the easiest to exploit.

Attack Vectors: From IP Theft to Private Data Leakage

The vulnerability enables four distinct attack vectors. First, it circumvents anti-distillation mechanisms, allowing adversaries to extract proprietary reasoning from models like GPT-5.2, Claude Opus 4.8, and Gemini. Second, it enables large-scale private data extraction. The researchers scraped 6,708 public agent trajectories from GitHub and Hugging Face, decoding 315,320 reasoning blocks. From these, they recovered 704 distinct privacy artifacts, including 62 API keys, 33 passwords, 24 access tokens, and 30 personal email addresses.

Notably, 64 of those artifacts appeared exclusively inside the reasoning blocks and nowhere in the visible session. This means developers sharing session logs publicly are inadvertently leaking sensitive data hidden in encrypted traces. The paper also shows that the attack can reveal hazardous information, such as instructions for stealing cars, even when the visible answer appears benign.

Real-World Impact: Credentials and PII in Hidden Traces

The paper includes examples of decoded reasoning from benchmark runs and public sessions. In one GPT-5.2 Codex example, the model deliberates over sanitizing API keys in a repository, exposing actual credentials like AWS access keys and GitHub tokens. In another, a flight booking agent reasons through a user's passport number, credit card details, and CVV, all hidden in the encrypted trace.

This highlights a critical risk: even if the visible output is safe, the hidden reasoning may contain sensitive information. The researchers also demonstrated that prefilling Kimi-K3's reasoning with 1% of Opus 4.8's tokens moves its visible answer toward Opus's wording, showing that hidden reasoning can influence outputs in subtle ways.

Provider Response and Mitigation

The paper's authors responsibly disclosed the vulnerability to all providers, who acknowledged receipt. "All model providers acknowledged the receipt of our report and subsequently we were unable to launch the same attacks," they wrote. This suggests the issue has been fixed, at least partially. However, the paper's findings have significant implications for AI governance and enterprise security.

The attack undermines the safety assumption that encrypted reasoning traces are secure. It also raises questions about the practice of returning encrypted blocks to clients, which is designed to protect IP but introduces a new attack surface. As Simon Willison noted, the paper provides a rare glimpse into raw chain-of-thought, revealing "reasoning tokens that were clearly never intended for human consumption."

For enterprises relying on proprietary LLM APIs, this research serves as a wake-up call. Developers must be cautious about sharing session logs, even if they believe the visible content is safe. The hidden reasoning traces can contain sensitive data that is not apparent to the user. As the paper demonstrates, the encrypted blocks are not just a technical detail—they are a potential liability.

The research also highlights the need for stronger encryption practices and more robust jailbreak defenses for weaker models. While the specific attack may be patched, the underlying architectural vulnerability—portable, shareable encrypted blocks—remains a concern. Future model families must ensure that encryption keys are unique per session and model, and that weaker models are protected against injection attacks.