CSE 151B

Dive into Thinking Modes

19

CSE 151B · Interactive Survey

Dive into Thinking Modes: An Interactive Survey of Reasoning in Language Models

Direct answers Token reasoning Trained reasoning Latent reasoning

Why Direct Answers Fail

A raw model can recognize a problem’s shape and still skip the computation needed to solve it.

Reasoning creates room to compute, check, and revise before the final answer.

How Transformers Generate Answers

Click a module to see the mechanism, equation, and role in the encoder-decoder path.

Hover or focus a module to isolate it. Click to keep it selected.

Interactive Transformer encoder-decoder architecture Token and positional embeddings enter repeated encoder and decoder stacks. Attention, feed-forward, residual normalization, and the output projection transform them into next-token probabilities. Repeated encoder and decoder stacks N× N× Token embeddings and positional encoding Input embedding Output embedding Masked multi-head self-attention Masked multi-head attention Multi-head attention Multi-head attention Multi-head attention Residual addition and layer normalization Add & Norm Add & Norm Add & Norm Add & Norm Add & Norm Position-wise feed-forward network Feed forward Feed forward Linear projection and softmax output head Softmax Linear
Adapted from Vaswani et al. (2017), Figure 1 Same architecture, reorganized horizontally for projection and interaction.

Figure overview

A fixed sequence of transformations

Embeddings move through repeated attention and feed-forward blocks before the output head predicts the next token.

The encoder builds contextual representations; the decoder uses causal self-attention and cross-attention to generate one token at a time.

Representative equation
h⁽ˡ⁺¹⁾ = Blockₗ(h⁽ˡ⁾)

The number of layers is fixed for every generated token.

Direct answering still computes, but it adds no separate pre-answer scratchpad.

Chain-of-Thought Prompting

Same model. More decoding steps before the answer.

Question Reason 1 Reason 2 Answer

Few-shot CoT: show worked reasoning examples in the prompt.

Zero-shot CoT: add a trigger such as “Let’s think step by step.”

Both methods generate intermediate tokens that act as a reusable workspace.
Standard prompting gives a wrong direct answer while few-shot chain-of-thought prompting uses a worked reasoning example and gives the correct answer.
Wei et al. (2022), Figure 1 Worked reasoning demonstrations establish the pattern before the new question.

Prompted vs. Trained Reasoning

Prompted CoT

Instruction or demonstrations at inference time

Parameters stay fixed

Trained reasoning

SFT, distillation, or reinforcement learning

Parameters change

Prompting elicits a behavior without updating the model.

Training changes the behavior the model produces by default.

Prompting examples PaLM and GPT-3 with reasoning demonstrations
Training examples Quiet-STaR, DeepSeek-R1, OpenAI o-series

Click a component to inspect it.

Interactive Quiet-STaR think, talk, learn algorithm The original text appears across the top. Two sampled thoughts branch below it. Helpful thoughts receive a reward and train the language model, while harmful thoughts are discarded. Original text and tokenwise prediction positions Original Text 4 2 + 1 2 = 5 4 \n Sample candidate thoughts 1) Think Sampled Thought Sampled Thought START 2 + 2 END START 4 + 1 END Mix predictions with and without a thought 2) Talk Thought₆ Measure whether a thought helps prediction Helps Prediction Harms Prediction ✓ ✕ Reward useful thoughts and update the language model 3) Learn Reward Discard LM update θ discard

Figure overview

Think, talk, then learn from prediction

Quiet-STaR samples hidden rationales, tests whether they improve future-token prediction, and reinforces useful thoughts.

thought → prediction gain → learning signal

Ordinary text supplies the supervision signal.

Zelikman et al. (2024), Figure 1. Think, talk, learn. Layout preserved; components redrawn for interaction.

Pause to Think

Pause Tokens are an intermediate step between chain-of-thought and Coconut.

An intermediate step
CoT <pause> tokens Coconut

Chain-of-thought adds readable reasoning tokens. Pause Tokens keep the extra sequence positions but remove the requirement that those positions contain meaningful reasoning text. Coconut goes further by reasoning in continuous latent states.

More positions, more computation
x extra positions y

Pause tokens add sequence positions before the answer. The tokens carry no meaningful reasoning text, but their hidden states can help generate later answer tokens within a standard token-based Transformer.

Pause Tokens give the model more computation through extra token positions, without requiring readable intermediate steps.

Hover or focus a component to isolate it. Click to keep it selected.

Standard pretraining compared with pause-pretraining The original two-panel layout is preserved. Interactive components show standard pretraining, inserted pause tokens, added computational paths, ignored pause outputs, and supervised target tokens. Standard pretraining path kid is playing soccer The kid is playing (a) Standard pretraining Inserted pause tokens <pause> <pause> Additional causal computation paths Outputs that receive no pause-token loss Ignore output Ignore output Ordinary next-token targets kid is playing soccer

Figure overview

Pause positions add computation before commitment

Pause-pretraining inserts learned pause tokens, skips their prediction loss, and keeps the original next-token targets.

Principle or formula
K tokens + M pauses → K + M positions per layer

The delay changes computation while revealing no intermediate rationale.

Goyal et al. (2024), Figure 2 Layout preserved; components redrawn for interaction. Research analogue only.

Token vs. Latent Reasoning

Trace how Coconut replaces sampled reasoning words with continuous hidden-state feedback.

Hover or focus a mechanism to isolate it. Click to keep it selected.

Token chain-of-thought compared with Coconut continuous thought The left panel samples hidden states into tokens and embeds each token for another model pass. The right panel feeds hidden states directly into later input positions before returning to token output. The sampled token is embedded again xᵢ xᵢ₊₁ xᵢ₊₂ xᵢ₊ⱼ … Hidden states become the next embeddings Last hidden states are used as input embeddings The language model performs each update Large Language Model Large Language Model Question enters as token embeddings [Question] Question enters as token embeddings [Question] Token chain-of-thought samples a reasoning token xᵢ xᵢ₊₁ xᵢ₊₂ xᵢ₊ⱼ [Answer] … … A boundary token starts continuous thought <bot> The model returns to language for the answer [Answer] <eot> …
Hao et al. (2024), Figure 1 Lavender marks hidden states. Gold marks embeddings. Gradients mark direct latent-state reuse.

Figure overview

Two ways to allocate pre-answer computation

Token CoT repeatedly converts hidden states into words. Coconut keeps part of that computation in continuous hidden states.

Hover or focus a component to inspect it. Click a component to keep it selected, or start the walkthrough to follow the comparison in order.

Representation
Tokens compared with hidden states
User visibility
Depends on the reasoning path
Computational role
Separates visibility from the representation used to reason.
Hidden describes visibility. Latent describes representation.

Summary: Types of Test-Time Reasoning

Direct answer Language workspace Learned reasoning Flexible inference compute Continuous latent compute
Path Pre-answer steps Representation Visible to user Special training
Direct answer No separate workspace Standard hidden activations Answer only Not required
Prompted CoT Yes Discrete language tokens Usually visible No parameter update
Trained token reasoning Yes Discrete reasoning tokens Visible, hidden, or summarized Yes
Latent reasoning Yes Continuous hidden states Usually hidden Yes

LLM reasoning developed by giving models more flexible computation between the question and the final answer, not simply by making thoughts visible or hidden.

References

Paper links and technical references mentioned throughout the presentation.

  1. Vaswani et al. (2017). Attention Is All You Need.
  2. Brown et al. (2020). Language Models are Few-Shot Learners.
  3. Wei et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.
  4. Chowdhery et al. (2022). PaLM: Scaling Language Modeling with Pathways.
  5. Kojima et al. (2022). Large Language Models are Zero-Shot Reasoners.
  1. Goyal et al. (2024). Think Before You Speak: Training Language Models With Pause Tokens.
  2. Zelikman et al. (2024). Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking.
  3. Hao et al. (2024). Training Large Language Models to Reason in a Continuous Latent Space.
  4. DeepSeek-AI et al. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.
  5. OpenAI (2024). OpenAI o1 System Card.

Figure-specific links on earlier slides open the cited page of each paper.