ICML 2026 · Agent Dynamics

Escaping Mode Collapse in LLM Generation via Geometric Regulation

Xin Du · Kumiko Tanaka-Ishii

Mode collapse is reframed from token-level repetition as a geometric collapse of internal trajectories, enabling online low-rank regulation of the Transformer value cache.

Mode Collapse Is a Long-Horizon Dynamical Problem

Autoregressive instability is often treated as a decoding problem. Repetition penalties, top-$k$, top-$p$, and higher temperatures all redistribute probabilities at the current step. Yet explicit loops are only the most visible failure. Generation may instead become semantically stationary, lose lexical range, or converge prematurely to a few safe patterns. These cases need not have abnormal one-step entropy and cannot be defined by one repeated span.

The paper shifts attention from output symbols to the internal trajectory. As decoding proceeds, the KV cache continually absorbs history and constrains which states remain reachable. If local bias accumulates along slowly decaying directions, the trajectory can become confined to a low-dimensional metastable region. The model still writes grammatical sentences while losing access to new internal states. Mode collapse is therefore formulated as geometric collapse: different surface symptoms share a decline in state-space accessibility.

Correlation dimension measures the effective degrees of freedom still active in the trajectory. Token entropy and Distinct-$n$ describe the output distribution or lexical diversity; correlation dimension instead measures the multiscale recurrence of internal states. Both abrupt loops and gradual degeneration coincide with sustained dimensional loss.

An Analysable Minimal Model

To isolate mechanism rather than reproduce every detail of a Transformer, the paper constructs a two-dimensional, state-dependent iterated function system. Map selection depends on both instantaneous randomness and the historical mean state $m_t$. This feedback resembles mean-field Ising dynamics: at high temperature the trajectory moves among regions, whereas below a threshold the historical bias dominates, splits the dynamics into stable branches, and localizes the trajectory.

The mechanism is explicit: history creates a slow variable; the slow variable changes transition probabilities; the altered transitions reinforce that variable. Below the critical point, accessibility and correlation dimension fall together. Weak damping,

breaks the feedback and restores exploration. This is not a proof of a thermodynamic transition in a language model, but it identifies an actionable control target: persistent historical directions rather than undifferentiated randomness.

From Slow-Mode Identification to Geometric Regulation

The intervention modifies only the Transformer value cache. Changing keys would alter attention weights and their global dependencies; a low-rank transform of values gives a more localized way to regulate aggregated state content. Let $\Sigma$ be the value-state covariance and $\Sigma_{\Delta}$ a symmetrized lag-one cross-covariance. Slow modes satisfy

Under stationarity, $|\lambda|\leq1$; directions with $\lambda$ close to $1$ decay very slowly and can accumulate into trajectory-dominating modes.

Reinforced Mode Regulation (RMR) updates these statistics online, selects the few directions $U$ above a persistence threshold, and damps them in the value cache:

The threshold keeps the intervention sparse. Before unregulated collapse, the leading generalized eigenvalue approaches $1$ while the second falls, creating a spectral gap and indicating takeover by one slow mode. RMR keeps the leading value below the threshold and preserves multiple accessible directions. A random subspace of the same rank does not reproduce the gain.

Moving the Stability Boundary

Experiments compare fixed-temperature sampling with fixed target entropy in long continuations. Standard decoding frequently collapses at an entropy rate near $2.0$ nats/step; RMR remains stable near $0.8$ nats/step. At temperature $0.7$, the non-collapse rate rises from 8% to 56%. At target entropy $1.0$, it rises from 5% to 33%. Low entropy is therefore not sufficient for collapse; the decisive factor is whether persistent internal modes lock the trajectory.

At 4,000 tokens, the non-collapse rate of standard temperature-$1.0$ decoding is 6%, and even temperature $1.2$ reaches only 55%. RMR at temperature $1.0$ reaches 99%. On continuations that do not collapse under the baseline, coherence, syntax, and information progression show no significant degradation. The result is not explained by making all outputs more random.

Scope of the Conclusion

The paper supports a specific conclusion: long-horizon degeneration can be diagnosed as contraction of internal geometry, and targeted inference-time regulation of slow modes can alter the stability boundary. Critical behaviour in the minimal model supplies a mechanism; correlation dimension, spectral gaps, and interventions provide empirical counterparts in real models. They should not be conflated with proof of a strict thermodynamic phase transition.

RMR requires access to the KV cache and adds online covariance estimation and low-rank eigensolving. Threshold and damping strength are currently fixed, and experiments focus on long-form Transformer generation. Extending the method to tool-using agents requires external memory, planning state, and tool feedback to be embedded in a non-stationary state space, then testing whether spectral persistence gives reliable warning before ineffective loops begin.

← Agent Dynamics