By the close of Chapter 5, the theory of generative flows had ramified into an archipelago of related but formally distinct methods. Euclidean flow matching works through vector fields and the continuity equation. Diffusion models work through score functions and the Fokker-Planck equation. Discrete flow matching works through rate matrices and the Kolmogorov forward equation. Riemannian flow matching works through geodesic interpolants on curved manifolds. Each has its own training objective, its own proof that the conditional version equals the marginal version, its own collection of design choices. The methods look and feel like the same idea. But the same idea as what?
The answer came in 2025, in a paper titled “Generator Matching: Generative Modeling with Arbitrary Markov Processes” by Holderrieth, Yim, and Jaakkola. It does not introduce a new generative model. It identifies the right level of abstraction at which all existing models become special cases of a single construction: matching the generator of a Markov process using a Bregman divergence. When you see the framework, the proliferation of objectives does not look like four different ideas. It looks like one idea written in four fonts.
6.1 A Zoo of Objectives
Before presenting the unification, it is worth confronting the fragmentation it resolves. Across the preceding chapters, four training objectives have accumulated — each derived from first principles, each enjoying a conditional-equals-marginal equivalence, each requiring nothing more than sampling from conditional distributions at training time.
The conditional flow matching objective for Euclidean data (, linear interpolant):
The denoising score matching objective for Gaussian-noised diffusion:
The discrete conditional flow matching objective for token sequences:
The Riemannian flow matching objective for manifold-valued data:
The structural repetition is unmistakable. In every case: sample time , draw a data point , sample a noisy state from the conditional path, minimize a distance between a model output and a known conditional target. The differences lie in the object being regressed (, , , ) and the geometry of the distance (, weighted , discrete , Riemannian ). These differences are, as the generator matching framework shows, consequences of a single underlying choice — made once, at the level of the Markov process.
6.2 The Generator of a Markov Process
The central concept is the infinitesimal generator of a Markov process — a linear operator that encodes, at each instant, how the process is about to evolve. It generalizes the velocity field of an ODE to the richer settings of stochastic and discrete dynamics.
[!definition] 6.1 — Infinitesimal Generator Let be a time-inhomogeneous Markov process on a state space . The infinitesimal generator at time is the linear operator on functions defined by:
It encodes the instantaneous rate of change of the expected value of any observable under the process, starting from state at time .
The generator takes a different concrete form depending on the type of process. For a deterministic flow on :
The generator is the directional derivative along — it tells you how fast changes if you move with the flow.
For a stochastic diffusion on :
The Laplacian term reflects the diffusive spread of probability mass; the drift contributes a transport term identical to the ODE case.
For a continuous-time Markov chain with rate matrix on finite :
The generator evaluates at all reachable states , weighted by the jump rate , and computes the expected instantaneous change.
For a Riemannian flow with velocity on a manifold :
where is the Riemannian gradient and is the metric inner product at .
[!remark] 6.1 Every Markov process — deterministic or stochastic, continuous or discrete, flat or curved — is fully characterized by its generator family . This is not a simplification: by the Hille-Yosida theorem, a sufficiently regular generator uniquely determines the associated process and its semigroup. The generator is not a summary of the dynamics; it is the dynamics, stated at the infinitesimal level.
6.3 The Generalized Continuity Equation
The generator acts on functions; its -adjoint acts on probability distributions. The evolution of any marginal density under the corresponding Markov process is governed by a single equation:
This is the Kolmogorov forward equation in its most general form, and (6.2) is the common root of every evolution equation encountered in the preceding chapters.
For the ODE generator (6.1), the adjoint acts by , and (6.2) becomes:
For the diffusion generator, , and (6.2) becomes:
For the CTMC generator, , and (6.2) becomes:
The message is plain: the continuity equation, the Fokker-Planck equation, and the Kolmogorov forward equation are all the same equation, stated in different coordinate systems for different types of generators. This is not a formal coincidence — it reflects the fact that every Markov process conserves total probability, and equation (6.2) is the expression of that conservation law at the infinitesimal level.
6.4 Bregman Divergences
The second ingredient in the framework is the Bregman divergence — the family of loss functions that measure how well a model’s generator matches the true generator of the interpolant.
[!definition] 6.2 — Bregman Divergence Let be a strictly convex, differentiable function on a convex set . The Bregman divergence generated by is:
It measures the gap between the value of at and the tangent-plane approximation to at , evaluated at .
Bregman divergences are non-negative (, equality iff ) but generally asymmetric. They satisfy a generalized bias-variance decomposition: for any random variable and any fixed ,
The second term on the right depends on but not on . This identity is the Bregman generalization of the classical decomposition , and it is the engine of the conditional generator matching theorem.
Two canonical choices cover the main cases encountered in generative modeling:
Squared norm ( on ): . Symmetric, geometrically natural, the basis of least-squares regression. The right choice when the generator output is a vector in a real space — a velocity field, a score, a tangent vector.
KL divergence ( on non-negative sequences): , which reduces to the standard KL divergence when and are probability distributions. The right choice when the generator output is a distribution over next states — as it is for CTMC rate matrices, where each row specifies a signed measure over .
6.5 The Generator Matching Objective
Given an interpolant process with known generator and a parametric family of generators of the same structural type, the generator matching objective is:
where is the local generator statistic at state — the concrete object that the generator produces at each point:
- For flows: (the velocity vector).
- For CTMCs: (the row of outgoing rates).
- For Riemannian flows: (the tangent-space velocity).
The minimizer of (6.4) is the parameter such that matches the true generator on the support of , at every time .
[!remark] 6.2 The objective (6.4) is intractable as stated, because computing requires access to the marginal generator — which in turn requires integrating over all data that could produce the observed . This is exactly the same intractability that made the original flow matching objective (4.2) intractable: the marginal velocity field is the average over all data points weighted by . The resolution is the conditional generator matching theorem.
6.6 The Conditional Generator Matching Theorem
The key observation is that the marginal generator statistic is a conditional expectation of the conditional generator statistics. If is a mixture of conditional paths, then by the linearity of generators and the tower property:
where is the conditional generator statistic of the path . For the linear interpolant, this is ; for the masking interpolant, it is the rate row from (5.3). In every case, the conditional target is computable from first principles, without marginalizing over data.
Define the conditional generator matching objective:
[!theorem] 6.1 — Conditional Generator Matching Let be any Bregman divergence, any state space, any parametric family of Markov generators, and any family of conditional paths whose mixture recovers the marginal . Then:
The marginal and conditional objectives have identical gradients with respect to at every parameter value.
The proof is a direct application of the generalized bias-variance decomposition (6.3). Fix , , and . By (6.3) applied to the random variable with expectation :
Integrating over and and differentiating in annihilates the second term, leaving .
This proof is four lines. It holds for any Bregman divergence, any state space, any generator class, any interpolant family. Every conditional-equals-marginal equivalence from every preceding chapter is an instance of it. The CFM = FM proof of Chapter 4 was the special case of squared-norm Bregman divergence applied to ODE generators on . The discrete case from Section 5.3 was the same proof applied to CTMC generators on a finite alphabet. What changes is the state space and the divergence. What does not change is the theorem.
6.7 Flow Matching as Generator Matching
Recovering conditional flow matching is the most immediate instantiation. Choose:
- State space: .
- Generator class: deterministic flows, .
- Bregman divergence: squared norm, .
- Interpolant: linear, with conditional velocity .
The local generator statistic is , the velocity vector. Substituting into the conditional objective (6.5):
The conditional flow matching objective of Chapter 4 is exactly generator matching with an ODE generator and squared-norm Bregman divergence. Similarly, Riemannian flow matching is generator matching on with a Riemannian-flow generator and the Riemannian squared norm as the Bregman divergence — the geodesic is the conditional generator statistic of the geodesic interpolant (5.4).
6.8 Diffusion as Generator Matching
The diffusion case is instructive because it highlights the role of the Bregman divergence. A diffusion model parametrizes the score function — the gradient of the log-density — rather than a drift directly. The connection arises from decomposing the SDE generator:
Restricting to models that share the fixed diffusion coefficient and vary only in their drift, and identifying the drift with the score via , the generator matching objective reduces to:
This is likelihood-weighted denoising score matching (Kingma et al., 2021). With Gaussian conditional paths and Tweedie’s formula, the score is the Gaussian score , and the objective becomes the DDPM noise-prediction loss of Chapter 3. Generator matching with a diffusive generator and squared-norm Bregman divergence recovers the full family of score-based diffusion objectives.
[!remark] 6.3 The diffusion case shows that the choice of Bregman divergence is not merely aesthetic. The score function is a logarithmic derivative — it measures relative changes in density, not absolute ones. Using the squared Euclidean norm treats it as a plain vector, ignoring its probabilistic origin. A Bregman divergence derived from an entropic potential would be sensitive to ratios rather than differences, yielding a different loss. The freedom to choose within the generator matching framework makes this choice explicit and principled, rather than buried in a reparametrization.
6.9 Discrete Flow Matching as Generator Matching
The discrete case makes the role of the Bregman divergence most transparent. Choose:
- State space: finite (a vocabulary, alphabet, or product thereof).
- Generator class: CTMCs, .
- Local generator statistic: , the row of outgoing rates.
- Bregman divergence: KL divergence, .
With the masking interpolant, the conditional generator statistic at state given target is the point-mass row from equation (5.3). The KL divergence between this point mass and the model’s predicted rate row is:
which is the cross-entropy of the model’s next-token distribution against the true token . Training with the KL Bregman divergence turns generator matching into maximum-likelihood estimation over categorical predictions — masked language model training, with time-dependent weighting by .
The KL divergence is the natural choice for discrete generator outputs for a deeper reason: the rate matrix row is a measure over next states, and the appropriate divergence between measures is information-theoretic. Using the squared norm on rate rows — as the discrete CFM objective of Section 5.3 does — is technically valid (it corresponds to a different Bregman divergence) but ignores the probabilistic structure of the output. The KL version is the one that connects naturally to the standard cross-entropy training of language models.
6.10 The Design Space: A Unified Recipe
The generator matching framework reduces the problem of designing a new generative model to four choices. Every model encountered in this monograph is a point in this space; every future model is a new combination of the same four ingredients.
Choice 1: State space . Euclidean , a Riemannian manifold , a finite alphabet , a Lie group, a product space — the state space determines what kind of data the model generates and constrains the generator class.
Choice 2: Interpolant . The conditional path from reference to data. Linear for Euclidean, geodesic for Riemannian, masking or uniform noise for discrete, Gaussian forward process for diffusion. The interpolant determines the conditional target and controls the geometry of the training signal.
Choice 3: Generator class . The parametric family of Markov processes the model can express. Deterministic flows, stochastic diffusions, CTMCs, or Riemannian flows — and mixtures thereof. This is the parametrization of the neural network: what the model outputs at each state.
Choice 4: Bregman divergence . Squared norm for real-vector outputs (velocities, scores, tangent vectors); KL divergence for distributional outputs (rate rows, token predictors); other Bregman divergences for specialized output geometries.
Given these four choices, training proceeds identically across all settings: sample , draw , sample , evaluate the conditional generator target , compute , backpropagate. No simulation. One forward pass per gradient step. Theorem 6.1 guarantees that this procedure is equivalent — in gradient — to optimizing the intractable marginal objective.
[!remark] 6.4 The four choices are not independent. The state space constrains the generator class (CTMCs are not defined on ; ODE generators are not defined on finite alphabets). The interpolant should respect the geometry of the state space (masking for discrete, geodesic for Riemannian). The Bregman divergence should match the type of generator output (squared norm for vectors, KL for distributions). Within these structural constraints, there is genuine freedom — and exploring the design space has been the source of most practical advances since 2023. The generator matching framework makes these choices explicit rather than implicit, which is precisely its value as a research tool.
6.11 Historical Notes and the Road Ahead
The generator matching framework as presented here is due to Holderrieth, Yim, and Jaakkola, whose 2024/2025 paper unified the continuous, discrete, and Riemannian streams of the literature under a single theorem. The key technical insight — that the Bregman bias-variance decomposition (6.3) is the underlying algebraic fact, and that all the conditional-equals-marginal equivalences are instances of it — was anticipated in scattered form. The connection between flow matching and CTMC-based discrete generation had been noted independently by Campbell, Yim, Barzilay, Jaakkola, and Gat (NeurIPS 2024) and Gat et al. (NeurIPS 2024). The tower-property proof in the continuous case goes back to Lipman et al. (ICLR 2023) and Liu et al. (2022). The Riemannian extension had been established by Chen and Lipman (ICLR 2024). What the generator matching paper contributes is the identification of the common algebraic structure — Bregman decomposition applied to Markov generators — and the proof of Theorem 6.1 in full generality.
The framework has been extended in several directions since. The stochastic interpolant perspective of Albergo, Boffi, and Vanden-Eijnden (2023) fits seamlessly: adding noise to the interpolant corresponds to switching from a deterministic ODE generator to a diffusive one, with the Bregman objective unchanged. Riemannian diffusions — SDEs on manifolds — arise when the CTMC case is replaced by a manifold-adapted diffusion generator; the score function becomes the Riemannian gradient of the log-density, and Riemannian denoising score matching follows automatically from Theorem 6.1. The unification is not just formal: knowing that all these methods are generator matching has guided the design of new ones, by making clear which ingredients can be varied independently.
The most consequential open direction, at the time of writing, is joint generation on product spaces — the subject of Chapter 7. Most real scientific data is heterogeneous: a protein is simultaneously a sequence over a twenty-letter alphabet and a structure in ; a small molecule is simultaneously a graph of atom types and a set of three-dimensional coordinates; a genomic locus is simultaneously a DNA sequence and an epigenetic state vector. The relevant state space is a Cartesian product — combining a discrete factor and a Riemannian factor. The generator decomposes over the product; the Bregman divergence factorizes; Theorem 6.1 continues to hold. But the design choices for the two factors interact in non-trivial ways, and the coordination between discrete and continuous generation during inference is a fresh problem. Chapter 7 develops the product-space theory and its application to the joint generation of protein sequence and structure — arguably the central problem in computational structural biology.
6.12 Historical Notes on the Bregman Connection
A remark on the mathematics is warranted, because the appearance of Bregman divergences in this context is not accidental. Amari (1985, 2016) established that the natural divergences on statistical manifolds are the -divergences and Bregman divergences arising from the geometry of exponential families — and that denoising, regression, and maximum-likelihood estimation are all instances of Bregman projection. Banerjee, Merugu, Dhillon, and Ghosh (JMLR 2005) showed that every Bregman divergence corresponds to an exponential family and vice versa, and that the “generalized mean” defined by minimizing the expected Bregman divergence is simply the expectation. This is exactly the fact exploited in the proof of Theorem 6.1: for any Bregman divergence and any random variable .
The generator matching framework, seen through this lens, is the answer to a natural question: what is the correct divergence between Markov generators? The answer is: any Bregman divergence, applied to the local generator statistics. The choice of divergence determines the implicit statistical model — squared norm corresponds to Gaussian regression, KL corresponds to categorical maximum-likelihood — and the choice of generator class determines the process being modeled. The two choices together determine the method. The theorem holds for all of them.