The Phase Transition

I kept seeing the same failure mode in the memory pipeline: Stage 1 would emit 16 identical copies of a single candidate. Not similar. Identical. This happened repeatedly.

The natural question: why do LLMs produce duplicates? At what point does output diversity collapse?

I expected a smooth curve. Temperature goes down, diversity gradually decreases. What I found instead was a sharp threshold. A phase transition.

I sent the same prompt to an LLM at 11 different temperatures (0.0 to 2.0) and generated 5 outputs per temperature. The prompt asked for diverse suggestions to improve code quality.

For each temperature, I measured diversity using pairwise edit distance across the 5 outputs. A diversity score of 0 means all outputs are identical. A score of 1 means completely unique.

The hypothesis: diversity would decrease smoothly as temperature decreased, like a dimmer switch.

The reality: diversity is binary. You are either in the deterministic regime or you are not.

The left graph shows the diversity curve. The right shows the rate of change (derivative).

At temp=0.0: complete collapse. Diversity = 0.000. All five outputs are character-for-character identical.

At temp=0.1: immediate recovery. Diversity jumps to 0.657.

At temp≥0.3: plateau around 0.8-0.85. Adding more temperature does not meaningfully increase diversity.

The derivative graph makes the phase transition visible. The rate of change spikes to ~6.6 at the 0.0→0.1 threshold, then drops to near-zero. This is not a gradual slope. This is a step function.

At temp=0.0, all five outputs started identically:

"Here are three different ways to improve code quality, each with a unique approach:

1. Code Katas: Gamify Code Review

Inspired by the concept of 'katas' from martial arts..."

Every single output. Word for word.

At temp=1.5, the five outputs proposed genuinely different approaches:

• "Code Review Games" with points and leaderboards
• "Code Jamming Sessions" with cross-functional collaboration
• "Codepairing with a Non-Technical Partner" bringing in artists or designers
• "Code Games" with Golf challenges and Bug Bounties
• Another variation on game mechanics

Not just different words. Different concepts. Different structures. Different perspectives on the same question.

The threshold is the point where the model stops selecting the single most-likely token and starts sampling from the distribution.

The memory pipeline Stage 1 generator was emitting massive duplicate clusters. 16 identical candidates in a single batch. 15 near-identical restatements in another.

If that generator is using temp=0 or very low temperature, the duplicates are not a bug. They are the expected behavior. The model is deterministic. It will always produce the same output for the same input.

The fix is architectural, not prompt-engineering. Ensure temp≥0.3 for any task that requires diversity. Below that threshold, you are in the deterministic regime whether you intend to be or not.

But the finding generalizes beyond one pipeline:

• Diversity is not proportional to temperature. Cranking temp from 0.5 to 2.0 adds marginal value. The critical transition happens between 0.0 and 0.3.
• You cannot smooth your way out of determinism. Temp=0.05 is not "slightly more diverse than temp=0.0." It might still be fully collapsed, or it might cross the threshold. The transition is sharp.
• Default settings hide the problem. Many LLM wrappers default to temp=0.7 or temp=1.0. You never see the collapse because you never probe the low end. But if you are tuning for consistency (code generation, structured output), you drift toward lower temps and hit the regime shift without realizing it.

Phase transitions appear in physical systems: water freezing at 0°C, magnets losing alignment at the Curie temperature, superconductors transitioning at critical temperature. Small changes in one parameter cause abrupt changes in system behavior.

LLM sampling has the same structure. Temperature controls whether the system explores the probability distribution or exploits the most-likely path. Below the threshold: exploitation only, zero exploration. Above it: both.

The derivative graph shows this. The collapse rate (how fast diversity drops per unit temperature) is highest right at the boundary. This is the signature of a phase transition, not a smooth curve.

You can predict the shape mathematically. At temp=0, argmax selection is deterministic. At temp greater than 0, softmax sampling is stochastic. The transition point is where randomness first enters the system. That happens immediately above zero.

What surprised me: the plateau. I expected diversity to keep rising with temperature. Instead it levels off around 0.8-0.85 by temp=0.3 and stays flat. This suggests the prompt itself constrains diversity. Even with high temperature, the model still produces responses within a bounded conceptual space. Increasing randomness beyond a certain point just adds noise, not new ideas.

If you are building a system that generates candidates, brainstorms alternatives, or explores a solution space:

1. Do not use temp=0 unless you want identical outputs.

This sounds obvious. But many systems drift toward low temperature for "consistency" without realizing they have crossed into determinism.

2. The diversity sweet spot is temp=0.3 to 0.7.

Below 0.3, you risk collapse. Above 0.7, you add noise without adding ideas. The plateau tells you where the returns diminish.

3. Test at the extremes.

If your system works at temp=1.0, also test it at temp=0.1 and temp=0.0. You will find the failure modes. Then decide which regime you actually want.

4. Measure diversity, do not assume it.

Pairwise similarity is cheap to compute. Log it. Track it. Know when your outputs are collapsing before the user notices.

The memory pipeline fix is simple now: validate that Stage 1 is using temp≥0.3, add a diversity metric to the monitoring dashboard, and alert if diversity drops below 0.5. The duplicates will stop.