Artificial Neural Networks and Deep Learning · Insper

Flow Matching

Draw a line, learn the average

Diffusion's derivation, and a shortcut

Diffusion: an SDE (stochastic differential equation), a variational bound, a named noise schedule, a
chain of 1000 steps, three parameterizations (, , ). The method is simple; the derivation is not.

Flow matching reaches the same place in three lines:

  1. Draw a noise sample and a data sample .
  2. Draw the straight line between them: .
  3. Train a network to predict the direction of travel, , at a random point on it.

Is this a different family of model from diffusion?

In this class is the noise and the data: is noise, is data. It is the opposite of the Stable Diffusion class, where was the image.

The whole training loop

x1 = data_batch();  x0 = torch.randn_like(x1)
t  = torch.rand(len(x1), 1)                   # one t per example, in [0, 1)
xt = (1 - t) * x0 + t * x1                    # a point on the line
loss = ((model(xt, t) - (x1 - x0)) ** 2).mean()

No schedule, no bound, no chain. Sampling: draw noise and integrate

CFM: conditional flow matching, because the target is that of one drawn pair. vθ: the network, with weights θ.

The problem: which line?

Many lines pass near the same point at the same time , heading different ways. What can one
network output there?

  • Only their average, over the pairs whose line passes through at : the marginal field

  • You cannot compute it for real data — it averages over every data point that could have produced .
  • Theorem (Lipman et al., 2023): regressing the conditional target has the same gradient as regressing . Train on what you can write; converge to what you wanted.
  • The same move as DDPM regressing instead of the score () — stated once, in general.

One point, two lines, one answer

Two training lines pass through the same point x t at t = 0.25: one from the noise x0 = (1, 1) to B = (−2, 0), one from x0′ = (−0.33, 1) to A = (2, 0). Their targets are (−3, −1) and (2.33, −1). Weighted by how likely each noise sample is, 0.39 and 0.61, they average to the marginal velocity (0.25, −1.00); the spread around it, 6.77, is the floor of the loss at this point. noise 𝒩(0, I) ut​ A = (2, 0)B = (−2, 0) x0​ x0​′ xt​ LINE TO B x0​ = (1, 1) target (−3, −1) weight 0.39 error² 10.56 LINE TO A x0​′ = (−0.33, 1) target (2.33, −1) weight 0.61 error² 4.34 t = 0.25xt​ = (0.25, 0.75) ut​(xt​) = 0.61·(2.33, −1) + 0.39·(−3, −1) = (0.25, −1) floor = 0.61 · 4.34 + 0.39 · 10.56 = 6.77 arrows: 0.5 × velocity

Simulator · training pairs and the learned flow

t = 0.00

Two consequences that surprise people

Trained long enough, does the flow-matching loss go to zero?

  • No. The spread of the targets around their average, , is a floor. Lab, 8 Gaussians on a circle: floor 3.73 (exact; predicting zero gives 6.04), trained network 3.76. Converged, not failed.
  • The learned paths never meet: an ODE has one velocity per point and time.
  • So the flow re-pairs noise and data: each particle ends on the data, not on its training partner.
  • And at , : knowing is knowing the noise and nothing about , so . Everything heads for the mean of the data.

Simulator · one training step and Euler, line by line

x0 · noise
x1 · data
t

● data A, B · ● x0 · ● xt · → target · → v · — Euler (arrows: 0.25 × velocity)

Diffusion is a special case

Method Path
DDPM / diffusion curved: (b, a) on a quarter circle
Flow matching straight, per pair
Rectified flow straight, pairs remade by the model

One framework, one loss, one code path. With Gaussian noise the views are interchangeable: on the linear path, DDIM is the Euler step of the flow (). What differs: the weighting across and the shape of the paths.

Here grows from 0 (noise) to 1 (data): the Stable Diffusion class's schedule read backwards. Linear: , , the target of the loop. Diffusion: , a circle.

One pair, two interpolants

t =
The same noise–data pair under two interpolants. Left, the weights (b, a): flow matching moves along the straight line a + b = 1, diffusion along the quarter circle a² + b² = 1. Right, the pair in the plane: flow matching goes from x0 to x1 in a straight line at constant velocity x1 − x0; diffusion follows a curve and its velocity a′x1 + b′x0 changes along the way. Buttons choose t. THE WEIGHTS: (b, a) 011 b: weight of the noise a: weight of the data a + b = 1 a² + b² = 1 ONE PAIR: xt​ = a·x1​ + b·x0​ 0 x0​ (noise) x1​ (data) arrows: 0.25 × velocity flow matching: a = t, b = 1 − t diffusion: a = sin(πt/2), b = cos(πt/2)

"Straight paths, fewer steps"?

The line between one noise sample and one data point is straight. Is the path the model learns
straight?

  • The conditional path is straight by construction.
  • The marginal path is not: it averages over crossing pairs.
  • On a ring of 8 Gaussians (the next simulator, exact fields), the linear-interpolant paths are 111% longer than the straight line between their ends; the diffusion ones 58%. The lab's trained networks (another ring) measure 88% and 44%.
  • Neither wins at every step count; past ~32 steps both are invisible. The argument is about the few-step regime.

Simulator · linear vs. diffusion interpolant, exact fields

On images: mixed, and the weighting matters

Measurement Linear path Diffusion path
Lipman et al. 2023: solver evaluations (CIFAR-10, ImageNet 32/64) 122–142 183–193
SD3, 61 formulations, mean rank (lower is better), uniform 5.67 /linear: 2.88
SD3, rectified flow with logit-normal 1.54, 1st; best at 5 steps
This lab, MNIST, uniform , FD at 2 / 4 / 8 steps 751 / 226 / 133 510 / 180 / 94.0

At best 22–37% fewer evaluations (CIFAR-10: 1 − 142/183 ≈ 22%; ImageNet 32: 1 − 122/193 ≈ 37%), not 2–10× — and only with the right weighting of .
The interpolant is not where the big step-count savings come from.

ε/linear: ε-prediction with DDPM's linear β schedule. FD: the FID recipe (Fréchet distance between features of generated and real images), with a digit classifier in place of Inception; lower is better, and real digits score 4.6.

The real advantage is the objective

  • One unweighted regression; no schedule to tune; no -vs- choice to get wrong at high noise.
  • Movie Gen (Meta, 2024) trained both: flow matching more robust to the noise schedule and better than the diffusion losses they tried.
  • That, more than step count, is why SD3, FLUX and the video models adopted it.
  • Going from fifty steps to four or one takes something else: reflow or distillation.

Reflow: straighten on purpose

  1. Draw noise , integrate the trained ODE accurately: .
  2. Retrain on the pairs the model produced itself, on the straight lines between them.

Does reflow change what the model generates?

  • No: same distribution of , new coupling.
  • The flow's paths do not meet, so the lines between their endpoints rarely cross — and a straight line is integrated exactly by one Euler step.

Reflow, step by step

Six noise samples and two data points, A and B. Step 1: training joins each noise sample to a data point drawn at random by a straight line, and the lines cross. Step 2: the learned flow bends the paths so that they never meet; four of the six land on the other data point. Step 3: one Euler step on this flow sends every sample to the mean of the data, (0, 0). Step 4: reflow pairs each sample with where its own path ends, by a straight line. Step 5: after reflow, one Euler step lands on the data. noise 𝒩(0, I) mean of the data A = (2, 0)B = (−2, 0) hover a noise sample noise sample zdata point

Reflow, measured in the lab

Before After one reflow
2-D: loss 3.76 0.0010
2-D: curvature of the paths 88% 0.1%
2-D: sliced Wasserstein, 1 Euler step 1.35 0.061
MNIST: FD at 1 / 2 / 4 Euler steps 2255 / 751 / 226 201 / 146 / 124

The cost: a large set of pairs generated with the full ODE, a second training run, and the first
model's errors baked into the pairs. One round on images leaves a gap (CIFAR-10 1-step FID 12.2 →
3.1 only after reworking the recipe, Lee et al. 2024).

Curvature: path length / distance between the ends − 1. SW (sliced Wasserstein) to fresh data: data against data ≈ 0.05. The loss drops below the 3.73 floor because the floor comes from crossings, and the new pairs barely cross.

"Rectified flow" means less than it seems

SD3 and FLUX are "rectified flow" models. Did they reflow?

  • Neither paper describes a reflow stage. In SD3, the name refers to the straight interpolant of Liu et al. (data at ) — the flow-matching path.
  • The few-step models that shipped were distilled:
    • FLUX.1 [schnell]: latent adversarial diffusion distillation, 1–4 steps.
    • SD3.5 Large Turbo: adversarial diffusion distillation, 4 steps.

Fewer steps: learn jumps, not velocities

Method What the network learns Result
Consistency (2023), sCM (2025) any point → the trajectory's end 2-step FID 1.88, ImageNet 512
Shortcut models (2025) velocity and step size : one jump of = two of one model, 1–128 steps
MeanFlow (2025) average velocity over an interval 1-step FID 3.43 → 1.72, ImageNet 256
Adversarial distillation few-step student + a discriminator what shipped: FLUX.1 [schnell], SD3.5 Turbo

All are ways of learning the flow map : where whoever was at at time is at
time . The velocity is the jump of length zero.

Training choices that changed

  • Where in to train. Uniform wastes effort at the easy ends. SD3, HunyuanVideo: logit-normal (, ), concentrated on the middle. π0, GR00T: a Beta favouring the noisy end.
  • Shift with resolution. More tokens make a noise level easier to see through: large images and videos shift toward noise (SD3, LTX-Video) — the SNR panel of the Stable Diffusion class.
  • What the network outputs. Velocity by default; clean data or noise are interconvertible. Li & He (2025): predicting the clean data lets a plain Transformer generate in pixel space.

Where it is used today (1/2)

Domain Models (first release)
Image SD3 (Mar 2024), FLUX.1 (Aug 2024), FLUX.1 Kontext (May 2025), Qwen-Image (Aug 2025), Z-Image and FLUX.2 (Nov 2025)
Video Movie Gen (Oct 2024), HunyuanVideo (Dec 2024), LTX-Video, Wan 2.1 (Feb 2025), Open-Sora 2.0
Speech, audio, music Voicebox (2023), Matcha-TTS, CosyVoice, F5-TTS (2024), Movie Gen Audio, ACE-Step (2025)

Image and video: a VAE latent through a DiT (next class). FLUX.2: 32B Transformer, a 24B
vision-language model as the text encoder.

Where it is used today (2/2)

Domain Models What flows
Robot policies π0 (Oct 2024), π0.5, GR00T N1, SmolVLA a chunk of future actions
Proteins FrameFlow, FoldFlow, Proteina backbone frames or coordinates
Inside multimodal LLMs JanusFlow (2024), BAGEL (2025) an image head next to next-token loss

Not flow matching: AlphaFold 3 (diffusion), CogVideoX and Stable Audio Open (-prediction).
Sora, Veo 3: "diffusion", objective not disclosed. With the equivalence, the label matters less than
interpolant, target and weighting.

Why robots use it

A robot can grasp a mug from the left or from the right. What does a regression head that predicts the
action directly output?

  • The average of the two: a grasp in the middle, which matches neither.
  • A flow represents a distribution with several modes, and samples one of them.
  • π0: an action expert generates chunks of 50 actions with 10 Euler steps, fast enough for control at up to 50 Hz. GR00T N1: chunks of 16, 4 steps.
  • Same loss as an image model; a different (the data distribution, here of actions).

Sampling is a separate choice

x, dt = torch.randn(shape), 1 / n_steps        # t = 0: pure noise
for i in range(n_steps):                       # Euler: 1 evaluation per step
    x = x + dt * model(x, i * dt)

for i in range(n_steps):                       # Heun: 2 evaluations per step
    v1 = model(x, i * dt)
    v2 = model(x + dt * v1, (i + 1) * dt)
    x = x + dt * (v1 + v2) / 2

Count model evaluations, not steps. MNIST, lab, FD (lower is better):

model evaluations Euler Heun
4 226 (4 steps) 550 (2 steps × 2)
16 99.3 (16 steps) 89.7 (8 steps × 2)

One step on MNIST

On MNIST the learned paths are only 2.2% longer than straight lines. Is one Euler step good?

  • No: FD 2255 at 1 step, against 99.3 at 16 — every sample is the same grey smudge, 2.0 from the mean digit (real digits sit 14.0 from it).
  • At the field is : one step lands on the mean digit.
  • "Almost straight in length" is not straight enough: the turn happens early, and it is everything.

Reading a flow-matching codebase

Three conventions differ between papers. Check them before debugging anything else:

  1. Is noise or data? (This class: noise. The SD3 paper and many codebases: data.)
  2. Does the model predict velocity, or ? All interconvertible, given and the interpolant.
  3. Is a number in or an integer timestep index?

Reference: Meta's Flow Matching Guide and Code (2024), with the flow_matching PyTorch package.

Lab · Flow matching from scratch

Colab notebook, ~2 h. Sections 1–5 on CPU (2-D, exact field known); 6–8 on a T4 GPU (MNIST).

  1. the exact field, and the loss floor
  2. train a velocity network
  3. steps, solvers, curvature
  4. the diffusion interpolant, same code
  1. reflow in 2-D
  2. MNIST: the same loss on images
  3. one step, Heun, the interpolant on MNIST
  4. reflow on MNIST; logit-normal : the table is yours

open in Colab →

Takeaways

  1. Flow matching regresses a velocity; the conditional target has the same gradient as the marginal. The loss has a floor.
  2. Diffusion is flow matching with a curved interpolant; on the linear path, DDIM is the flow's Euler step.
  3. "Straight" is the conditional path. The learned one bends and never meets; on images the linear path saves at most about a third of the steps.
  4. The real win is the objective and its recipe: logit-normal , resolution shift.
  5. Reflow straightens at the cost of generated pairs; shipped few-step models are distilled.
  6. Image, video, speech, robot actions: the same line of code, a different .

Outline: (1) the loss in three lines; (2) the marginal field and the theorem; (3) diffusion as a special case; (4) what "straight paths" means, measured; (5) reflow, and few-step generation; (6) training choices; (7) where it is used in 2026. Closes with the Colab lab.

Answer later: no. Diffusion is the same thing with a curved interpolant.

Step through the five stages. Data A = (2, 0), B = (-2, 0), the same as the training-step simulator. Two lines pass through x_t = (0.25, 0.75) at t = 0.25: from (1, 1) to B and from (-0.33, 1) to A. Weights 0.61 for A, 0.39 for B (the noise of the line to A is closer to the origin). The mean, (0.25, -1), is what the network outputs; the floor at this point is 6.77; any v pays |v - u|^2 + 6.77. Hover a line to isolate it.

Start on "training pairs", animate: lines cross, 1569 crossings, 15 meetings. Switch to "the learned flow": paths bend, 0 meetings, mean distance 0.51 against 1.23. The flow re-pairs noise and data.

Default case, "the training loop": data A = (2, 0), B = (-2, 0); pair 1 joins the noise (1, 1) to B. Step line by line; the plot shows x_t (orange), the target (yellow) and v (purple). Line 6: v of pair 1 is (0.25, -1.00) against the target (-3, -1): it alone gives 10.56, loss 3.52 with the network already ideal. Pair 3 crosses too, at t = 0.75, and costs nothing. Line 9 with 4 steps: the noise of pair 1 ends on A, not on B (re-pairing). Switch to 1 step: (1, 1) + (-1, -1) = (0, 0), the mean. Cases: t = 0 gives loss 4 = mean of |x1|^2; t = 0.95 gives 0; v = 0 gives 6.17.

Cosine schedule: abar = sin^2(pi t / 2), so a = sin(pi t / 2), b = cos(pi t / 2). Click t = 0, 0.25, ..., 1. Left: the weights; flow matching on the line a + b = 1, diffusion on the quarter circle, at even steps of angle. Right: one pair. The linear point moves at the constant velocity x1 - x0; the diffusion point leaves the line and its velocity a' x1 + b' x0 turns, from a' x1 at t = 0 (adding data) to b' x0 at t = 1 (removing noise). At t = 0.5: a^2 + b^2 = 0.5 for the line, 1 for diffusion (variance preserving).

Exact marginal fields, nothing trained. Show lin at 1, 2, 8 steps, then vp. At 2 steps vp's error is lower (0.23 vs 0.43), at 8 lin's (0.05 vs 0.08). Then tick the ideal reflow: the limit, not what a trained reflow gets.

Same two data points. (1) Training pairs, random partners, lines cross. (2) The exact flow: paths bend, never meet; 4 of 6 change partners, mean distance 2.25 -> 1.32. (3) One Euler step on the flow: v = -z at t = 0, everyone lands on (0, 0), the mean. (4) Reflow pairs each z with its own endpoint. (5) One step after reflow lands on the data. Hover a noise sample to follow it through all five.