NeurIPS 2026

From Noise to Diversity

Random Embedding Injection in LLM Reasoning

Heejun Kim^1,* Seungpil Lee^1,* Jewon Yeom² Jaewon Sok²

Seonghyeon Park² Jeongjae Park² Taesup Kim^2,† Sundong Kim^1,†

¹ Gwangju Institute of Science and Technology (GIST) · ² Seoul National University

* Equal contribution · † Corresponding authors

(a) Attention from the token being generated. Attention on the RSP tokens is large at first and fades as generation proceeds (measured on Qwen2.5-Math-7B). (b) The same prompt sampled 8 times: RSP paths branch early and converge. Temperature-only paths stay close. Drag the timeline to scrub.

Append a few random vectors to the prompt. No training, no learned content. The model explores more reasoning paths early and still commits to one answer.

Random Soft Prompts

Soft-prompt methods improve reasoning by inserting trained vectors into the input, but whether the gain comes from the learned content or from the act of injection has not been separated. We study the simplest possible soft prompt: a Random Soft Prompt (RSP) drops the training step and appends a freshly drawn sequence of random embedding vectors to the input.

Each vector is sampled from an isotropic Gaussian fitted to the entrywise mean and variance of the pretrained embedding table, so it has the scale of a real token but no preferred direction. The sequence carries no learned content, yet it reaches accuracy comparable to optimized soft prompts (TTSV[1], SoftCoT[2], LTPO[3]) on MATH-500, GSM8K, and AIME24.

h_j ~ N(μ_E·1, σ_E²·I), j = 1 … L, input = [X ; H]

Table 1. Comparison with optimized soft prompt methods. Accuracy (%) under a unified evaluation, mean ± std over five initialization seeds for RSP, TTSV[1], SoftCoT[2], and LTPO[3]. The baseline uses greedy decoding and RSP uses suffix injection. Bold and underline mark the highest and second-highest mean in each row.

Model	Benchmark	Baseline	RSP (suffix)	TTSV	SoftCoT	LTPO
Qwen2.5-Math-7B-Instruct	MATH-500	83.20	83.52±0.56	83.44±0.55	82.88±0.32	82.64±0.43
	GSM8K	95.45	95.60±0.08	95.12±0.11	94.74±0.28	94.97±0.11
	AIME24	13.33	16.67±0.00	15.32±4.11	18.00±2.67	12.67±1.33
Qwen2.5-Math-1.5B-Instruct	MATH-500	73.00	74.12±1.36	72.32±1.75	72.88±2.27	73.68±0.97
	GSM8K	85.29	84.99±0.22	84.04±1.07	82.97±2.97	84.76±0.52
	AIME24	10.00	13.33±5.27	6.02±1.22	9.33±2.42	9.33±1.33

The same holds across injection positions and model families. Where the baseline is already strong, suffix RSP matches it. On the harder AIME24 it adds a few points. When a base model is evaluated with a chat template it was never trained on, a random prompt recovers much of the lost accuracy.

Table 2. Accuracy across model configurations and benchmarks. RSP accuracy (%) is mean ± std over five initialization seeds, and the baseline uses greedy decoding. On AIME24, suffix RSP adds 3.3 pp at both Qwen2.5-Math-Instruct scales. When a ChatML template is applied to a base model not trained for it, RSP recovers up to +27 pp (prefix, 1.5B) and +17 pp (suffix, 7B) on MATH-500.

Model	MATH-500			GSM8K			AIME24
Model	Baseline	RSP (prefix)	RSP (suffix)	Baseline	RSP (prefix)	RSP (suffix)	Baseline	RSP (prefix)	RSP (suffix)
Instruct models
LLaMA-3.1-8B-Instruct	52.20	41.24±2.16	49.52±1.48	85.75	76.30±0.75	86.70±0.39	6.67	3.33±2.36	6.67±3.33
Qwen2.5-Math-7B-Instruct	83.20	81.72±0.46	83.52±0.56	95.45	94.92±0.28	95.60±0.08	13.33	13.33±2.36	16.67±0.00
Qwen2.5-Math-1.5B-Instruct	73.00	75.60±0.51	74.12±1.36	85.29	85.26±0.39	84.99±0.22	10.00	12.00±1.83	13.33±5.27
Base models + ChatML (format mismatch)
Qwen2.5-Math-7B	52.20	57.40±1.71	69.40±1.03	58.30	56.25±1.14	75.88±0.36	23.33	9.33±4.35	20.00±2.36
Qwen2.5-Math-1.5B	34.40	61.56±1.42	48.80±2.09	37.45	66.07±1.03	50.24±2.09	13.33	14.67±2.98	8.00±5.06

Content or injection? Replace the trained vectors and see.

We keep each method's injection length and position fixed and only change what is injected. Replacing all trained vectors with random draws does not consistently hurt. Shuffling token order or permuting coordinates inside each vector, which destroys the learned structure but keeps the norms, does not hurt either. The channel is not inert: overtraining the same vectors with a 1000× learning rate collapses accuracy.

Table 4. Intervening on learned prompt content. Qwen2.5-Math-1.5B-Instruct, accuracy (%) as mean ± std over three seeds. Injection length and position stay fixed within each method (20 prefix tokens for TTSV, 8 infix for LTPO, 4 infix for SoftCoT). 50% front / 50% random / 100% RSP replace trained vectors with RSP draws, Token-shuffle reorders the vectors, and Dim-permute permutes coordinates inside each vector. Bold marks the best mean in each column.

Intervention	TTSV[1]			LTPO[3]			SoftCoT[2]
Intervention	MATH-500	GSM8K	AIME24	MATH-500	GSM8K	AIME24	MATH-500	GSM8K	AIME24
Trained	72.33±1.46	84.08±1.08	6.67±0.00	74.07±0.82	84.69±0.21	7.78±1.57	72.87±1.18	84.84±1.24	8.89±1.57
50% front	75.40±0.28	84.99±0.70	10.00±0.00	74.27±0.47	84.63±0.34	10.00±2.72	72.47±0.98	84.15±1.29	10.00±2.72
50% random	72.93±1.95	81.07±3.54	12.22±1.57	73.87±0.90	84.91±0.39	8.89±1.57	73.33±0.25	84.53±0.32	12.22±1.57
100% RSP	75.13±0.25	85.09±0.44	11.11±1.57	73.80±0.91	85.06±0.57	11.11±1.57	73.67±0.52	85.01±0.19	8.89±1.57
Token-shuffle	72.40±0.99	83.83±0.38	6.67±2.72	73.60±0.49	84.61±0.06	7.78±1.57	72.80±1.56	84.96±0.29	11.11±1.57
Dim-permute	71.93±0.57	83.62±0.55	5.56±1.57	74.00±0.43	84.00±0.77	12.22±1.57	74.80±0.16	84.99±0.16	13.33±0.00

Branch early, anneal later.

RSP flattens the distribution over the first few generated tokens: entropy and varentropy go up, top-1 probability goes down, and the curves return to the baseline within a few percent of the generation. Optimized soft prompts show the same early pattern despite different training objectives.

The mechanism is attention mass. With a fixed number of RSP tokens and a growing KV cache, the share of attention on the random positions shrinks as generation proceeds, so the perturbation that opened a new branch fades and the response commits to a single completion.

Entropy, top-1 probability, and varentropy over the first 5% of generation for Baseline, RSP, LTPO, SoftCoT, and TTSV — First 5% of generation, Qwen2.5-Math-1.5B-Instruct on MATH-500. RSP (orange) lifts early entropy and varentropy, then converges to the baseline.

Diversity that turns into Pass@N.

Temperature rescales the same distribution and never changes the ranking of tokens. RSP reranks: the Spearman correlation with the baseline first-token distribution is 0.71 instead of 1.0. Matching RSP's distributional shift with temperature alone (τ = 3.0) drops accuracy to 1.4%, while RSP keeps 73.3%.

Drawing a fresh RSP for every sample and combining it with temperature sampling gives the highest Pass@N on both MATH-500 and AIME24. Sharing one RSP across samples gives no gain, so the benefit comes from varying the perturbation, not from any single one.

Pass@n curves on MATH-500 and AIME24 for Baseline, RSP single seed, RSP independent seed greedy, and RSP independent seed with temperature — Pass@N with 16 samples per problem, Qwen2.5-Math-1.5B-Instruct. Independent RSPs with temperature (red) reach 92.8% on MATH-500 and 36.7% on AIME24 at N = 16.

+27.2pp

MATH-500 accuracy recovered on Qwen2.5-Math-1.5B when a mismatched chat template is used, with a 20-token prefix RSP.

36.7%

Pass@16 on AIME24 with an independent RSP per sample, versus roughly 20% for temperature sampling alone.

0.71

Spearman correlation between RSP and baseline first-token rankings. Temperature keeps it at exactly 1.0.

Beyond inference: better training data.

If RSP finds correct solutions that plain sampling misses, it should help rejection-sampling fine-tuning. On 32,737 hard problems that Qwen2.5-Math-7B-Instruct solves at most once in four clean rollouts, we draw eight rollouts per problem with and without RSP, keep the correct ones, fine-tune the same model on each set, and evaluate without RSP.

Table 5. Rejection-sampling SFT with RSP on Qwen2.5-Math-7B-Instruct. Coverage is the share of hard problems with at least one correct rollout out of eight. The SFT average is over Minerva, GaoKao2023-EN, OlympiadBench, AIME24 and AMC23 (Avg@32 for the last two), evaluated without RSP. Mean ± std over three collection seeds.

Collection	Problem coverage (%)	Rollout accuracy (%)	Accuracy after SFT, 5-benchmark avg. (%)
No SFT	–	–	40.26
Baseline sampling	23.75±0.10	5.54±0.03	42.11±0.54
RSP	49.68±0.13	17.45±0.05	43.77±0.54

Citation

@inproceedings{kim2026rsp,
  title     = {From Noise to Diversity: Random Embedding Injection in LLM Reasoning},
  author    = {Kim, Heejun and Lee, Seungpil and Yeom, Jewon and Sok, Jaewon and
               Park, Seonghyeon and Park, Jeongjae and Kim, Taesup and Kim, Sundong},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2026},
  url       = {https://arxiv.org/abs/2605.11936}
}

References

[1] Xinyue Kang, Diwei Shi, and Li Chen. Model Whisper: Steering vectors unlock large language models' potential in test-time. In AAAI, 2026. arXiv:2512.04748
[2] Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. SoftCoT: Soft chain-of-thought for efficient reasoning with LLMs. In ACL, 2025. arXiv:2502.12134
[3] Wengao Ye, Yan Liang, and Lianlei Shan. Thinking on the fly: Test-time reasoning enhancement via latent thought policy optimization. In ICLR, 2026. arXiv:2510.04182