A
More room to think
A vector can carry several possible next steps at once. A sentence must commit to one sequence of words.
What happens before the answer
Machines do not
think in prose.
Neuralese is a loose name for the numbers AI systems pass around before they produce words. Researchers use it for messages between agents and, more recently, for reasoning inside one model.
What the term meanshuman language
0.18−1.420.732.06−0.340.91−0.081.270.64−0.511.83−0.92
observable output
representation spacen × d
01
What it means
It is not a secret language.
It may not be a language at all.
A language model turns text into numerical states, transforms those states, then turns the result back into text. We can read the input and output. The work in between is much harder to inspect.
The term covers two related ideas. In the 2017 paper, it means messages exchanged by artificial agents. Newer work uses it for internal reasoning that never becomes a chain of written steps. That shift is useful, but it also makes the word imprecise.
02
Why it matters
B
Fewer round trips through text
A model that keeps intermediate work in its hidden state does not need to write and reread every step. Whether that saves time depends on the method.
C
A harder audit
A written chain of thought is already an imperfect record. If the useful work stays hidden, researchers need other ways to test what the model did and why.
03
A short lineage
2017
Agents translate their private messages
Translating Neuralese
Jacob Andreas, Anca Dragan, and Dan Klein train a system to translate messages passed between artificial agents into natural language.
ACL
2024
A model reasons without spelling it out
Chain of Continuous Thought
Coconut feeds a model's hidden state back into itself instead of turning each reasoning step into words.
arXiv
2025
A forecast gives the term a new use
AI 2027
AI 2027 uses "neuralese" for internal recurrence and memory that do not pass through written language.
Forecast
04
Open questions
The interesting part
is also the problem. We cannot
read it directly.
- 01
Can we translate a hidden state without changing what it means?
- 02
Does latent reasoning solve problems that written reasoning cannot, or does it only use fewer tokens?
- 03
How can we tell whether a model's explanation matches the computation that produced its answer?
- 04
What should an audit require when the reasoning itself cannot be read?