Autoregressive generative models, including large language models (LLMs), generate sequences by using the history of previous outputs. Their outputs can depend on long histories, leading to complicated non-Markovian dynamics in general.
This study develops a unified framework of stochastic thermodynamics for these models. The study shows that entropy production, which compares the probabilities of forward and backward processes under the same model, can be estimated efficiently from the model's own output probabilities without enumerating all possible histories.
The analysis also reveals the information-theoretic structure underlying entropy production. In proof-of-concept experiments with LLMs (GPT-2 and Qwen3-4B-Base), the order of sentences is reversed while the order of tokens within each sentence is preserved. The results show that texts describing ordered chains of events tend to exhibit greater entropy production per token than texts listing independent facts. This work lays a foundation for understanding autoregressive generative models through thermodynamics and may contribute to the quantitative study of LLMs' internal representations of the world, or "world models."
The diagram illustrates how an autoregressive generative model uses past outputs (tokens) to compute a latent state and generate the next token. The glass illustrations represent events described by sentences, whose order is reversed between the forward and backward processes.
Papers
Journal: PRX Intelligence
Title: Stochastic Thermodynamics for Autoregressive Generative Models: A Non-Markovian Perspective
Authors: Takahiro Sagawa
DOI: 10.1103/tv38-b23y
Giant spin hydrodynamic generation in laminar flow
An optimal method in diffusion models from a nonequilibrium thermodynamic perspective: Revealing the relationship between thermodynamic dissipation and accuracy of data generation
