arXiv 2026

Cliff Tokens: Analyzing Failure Trigger Tokens in LLM Mathematical Reasoning

Jaeyong Ko1, Jinu Lee2, Pilsung Kang1, Yukyung Lee3,†

1Seoul National University    2University of Illinois Urbana-Champaign    3Boston University

†Corresponding author

Overview

Large language models reach high accuracy in mathematical reasoning, but individual traces on the same problem diverge; some arrive at the correct answer while others fail. We introduce the cliff token, a token at which the estimated probability of reaching the correct answer drops beyond an adaptive threshold.

Cliff token identification overview
Figure 1: Cliff token identification. The token “7” shifts an example trace toward an incorrect answer. The adaptive threshold distinguishes a cliff token from a smaller drop in success probability.
  1. We formalize the cliff token with a z-test-based adaptive threshold that separates statistically significant drops in success probability from sampling noise, showing that these tokens act as failure triggers.
  2. We introduce a cliff taxonomy—deterministic, uncertain, and sampled-off cliffs—showing that the proportion of each type varies across model families and sizes.
  3. We show that uncertain and sampled-off cliffs are more effective as training signals than deterministic ones under Cliff-DPO.

What Is a Cliff Token?

Success probability is the probability of reaching the correct answer conditioned on the reasoning trace generated up to a specific token position. We estimate it by running continuations from each token prefix and measuring the fraction that reach the correct answer.

A cliff token is a token where success probability drops significantly when it is generated. The trace may still be recoverable before that token; after it is fixed, continuations are more likely to fail.

We identify drops using an adaptive threshold based on a one-sided two-proportion z-test at a 95% confidence level. The base threshold is Ď„ = 0.1, and success probability is estimated with 64 rollouts at each evaluated token position.

Do Cliff Tokens Trigger Reasoning Failures?

Across seven models and three mathematical reasoning benchmarks (GSM1K, MATH500, and AIME 2025), cliff tokens occur more frequently in incorrect traces than in correct traces for most models, with Llama-3.2-1B as the exception.

Average cliff tokens per trace for correct and incorrect traces across seven models
Figure 2: Average cliff tokens per trace. Correct and incorrect traces are compared across GSM1K, MATH500, and AIME 2025.

Resampling before the first cliff token (Cliff-del) generally yields higher pass@k than resampling after it (Cliff-keep) at the same sample count, suggesting that cliff tokens trigger reasoning failures.

Cliff-del and Cliff-keep pass at k on incorrect traces across three benchmarks and seven models
Figure 3: Cliff-del and Cliff-keep pass@k on incorrect traces. Cliff-del generally has higher pass@k at the same sample count. It reaches pass@64 = 1.0 in all evaluated panels; Cliff-keep ranges from 0.67 to 1.00 where cliff tokens occur.

Recovery and Self-Correction

A cliff token does not necessarily mark an irreversible failure or coincide with the actual error token. In Figure 4, success probability recovers after the model writes “But wait” and corrects its arithmetic. Neither of the two cliff tokens coincides with the miscomputed value 443, where the arithmetic error occurs.

A Qwen3-4B reasoning trace with two cliff tokens that later self-corrects
Figure 4: Recovery after cliff tokens. Qwen3-4B on AIME 2025 Problem 1.

Cliff Taxonomy

We characterize cliff tokens by greedy choice and token entropy. Their distribution varies by model family and scale: as model scale increases within the Qwen3 and Llama-3 families, sampled-off cliffs become less common and deterministic cliffs more common. Qwen3-8B also produces more deterministic cliffs than the similarly sized Llama-3.1-8B.

Deterministic cliff

A greedy token with entropy below 0.0561; the model samples it with near certainty.

Uncertain cliff

A greedy token with entropy at or above 0.0561; the top choice is sampled under uncertainty.

Sampled-off cliff

A non-greedy token with entropy at or above 0.0561; another choice is sampled under uncertainty.

Paper Table 1: model-wise deterministic, uncertain, and sampled-off cliff proportions alongside all-token proportions, with the original grayscale shading
Table 1: Distribution of cliff types: deterministic (D), uncertain (U), and sampled-off (S). Cliff and all-token proportions give each type’s percentage among identified cliff tokens and all reasoning-trace tokens, respectively. Darker shading indicates larger values.

Do Cliff Types Differ as Training Signals?

Cliff-DPO applies single-token preference optimization at identified cliff positions, pairing a non-cliff candidate token with the original cliff token. The uncertain and sampled-off variants achieve higher mean accuracy than the base model and deterministic variant across the three evaluated models on GSM8K, GSM1K, and MATH500.

Cliff-DPO accuracy and cliff-token probability by cliff type across three models
Figure 5: Cliff-DPO results. The sampled-off variant has the highest average accuracy for Qwen3-0.6B and Llama-3.2-1B; the uncertain variant ranks highest for Llama-3.1-8B. Deterministic cliff-token probabilities begin above 99% and change little after training.

Citation

@article{ko2026clifftoken,
  title={Cliff Tokens: Analyzing Failure Trigger Tokens in LLM Mathematical Reasoning},
  author={Ko, Jaeyong and Lee, Jinu and Kang, Pilsung and Lee, Yukyung},
  journal={arXiv preprint arXiv:2606.25524},
  year={2026},
  eprint={2606.25524},
  archivePrefix={arXiv}
}