DistillAlign: Coordinating Mode Covering and
Mode Seeking in Autoregressive Video Distillation

Jiaxing Li^1,2*, Kai Zou^1*, Cindy Zhou^1,3, Kaichen Huang^1,2, Junyao Gao¹,
Zile Wang¹, Yang Liu¹, Bin Liu¹, Bo An², Yangguang Li^1†

¹Riemann Dynamics · ²Nanyang Technological University · ³Wellington College, UK

^*Equal contribution. ^†Corresponding author.

Paper Code 🤗Model 🤗Dataset

Overview

Abstract

Existing autoregressive video distillation methods commonly adopt a Distribution Matching Distillation (DMD)-based multi-stage pipeline. However, they typically decouple the initialization and DMD stages—which then pursue different target distributions—and judge the intermediate student mainly by visual scores such as VBench. In this paper, we revisit this design from a distributional perspective. Given the mode-seeking nature of the distribution matching loss, a good initialization should match the mode coverage of the target DMD teacher, rather than merely pursuing high quality.

To analyze this, we introduce a distributional evaluation protocol that measures precision and coverage between student and teacher distributions in a shared latent space. It exposes differences hidden by visual scores: some initializations reach high precision but low coverage, leading to suboptimal refinement, while mode-covering ones preserve broader support. Furthermore, even when the target distributions are aligned, DMD's reverse-KL objective can still drive the student toward high-probability teacher regions in late training, reducing coverage and diversity. To address this, we propose joint distillation, which combines DMD's mode-seeking objective with a Consistency Distillation-based mode-covering constraint. Experiments show that our method improves generation quality, coverage, and diversity; notably, even with a Wan-1.3B DMD teacher, it outperforms baselines refined with Wan-14B, underscoring the importance of distributional alignment in autoregressive video distillation.

Method & Findings

A Distributional View of Video Distillation

We read multi-stage distillation through the divergence each stage implicitly optimizes — initialization covers the teacher's modes, DMD seeks them. With a metric that actually measures coverage, two failure modes turn into testable hypotheses, each backed by its own controlled experiments.

Initialization Mode Covering

ODE / consistency-distillation initialization is mean-seeking: it spreads student mass to cover the teacher's support, reaching more modes but leaving samples blurry or under-refined.

DMD Mode Seeking

Distribution Matching Distillation is a reverse-KL objective: it sharpens high-probability modes around the current student, improving fidelity but able to collapse onto a few modes.

Good distillation is a balance: initialization must cover the teacher's modes, and refinement must seek them without collapsing that coverage.

01Distribution Evaluation

We embed student features U = {u_i} and teacher features V = {v_j} with a frozen V-JEPA2 encoder, and estimate teacher support with an adaptive k-NN radius r_k(v_j) — the distance from each teacher sample to its k-th nearest teacher neighbor. Against this support we report two complementary quantities:

\operatorname{Precision}(U,V)=\frac{1}{N}\sum_{i=1}^{N}\mathbf{1}\!\left[\,\exists\, j:\ \lVert u_i-v_j\rVert_2 \le r_k(v_j)\,\right]

\operatorname{Coverage}(U,V)=\frac{1}{M}\sum_{j=1}^{M}\mathbf{1}\!\left[\,\min_i \lVert v_j-u_i\rVert_2 \le r_k(v_j)\,\right]

Precision asks whether student samples fall inside teacher support (fidelity); Coverage asks how many teacher neighborhoods are reached by some student sample (mode coverage). To compare initializers fairly, we first apply a shared teacher-normalized re-noising that cancels low-level sharpness gaps while preserving the semantic region each initializer reaches — giving us the tool that makes the next two hypotheses testable.

02Matched Coverage Matters

When the initialization and DMD target distributions are misaligned, DMD seeks modes outside the student's support and provides unsupported gradients, causing coverage collapse.

Hypothesis I illustration: mode collapse under mismatched teachers

Hypothesis & validation · Controlled target-distribution swap

Init. Target	DMD Teacher	Matched	VBench ↑	Coverage ↑
Wan-14B	Wan-14B	✓	84.74	0.648
Wan-1.3B	Wan-1.3B	✓	84.50	0.516
Wan-14B	Wan-1.3B	✗	84.16	0.503
Wan-1.3B	Wan-14B	✗	83.89	0.453

Even a weaker teacher/data wins when the two stages are matched: final quality tracks alignment, not teacher scale.

Rethinking distillation pipelines

Fixing the target distribution (Wan-14B) and its distillation data, we vary only the initializer across five pipelines — (D) ODE and (E) AR-CD correspond to Self-Forcing and Causal-Forcing.

Five distillation pipelines (A-E) sharing one aligned target

VBench and coverage across the five pipelines, before and after DMD — The highest-VBench initializer (C, AR-DMD) is **not** the best final model; the mode-covering **AR-CD** (E) keeps the broadest coverage before and after DMD — **coverage, not init visual score, predicts final quality**.

03Balancing Coverage and Seeking

We found even when targets are aligned, long-term reverse-KL refinement keeps pushing the student toward the teacher's high-probability regions, steadily shrinking quality and diversity.

Hypothesis II illustration: late-stage drift toward high-probability modes

Hypothesis & toy experiment · Late-stage drift

Step	0	500	1000	1500	2500
VBench ↑	82.60	84.44	84.74	84.21	83.90
Diversity ↑	1.319	1.297	1.292	1.285	1.272

VBench first rises then falls, while diversity drops monotonically — drifting to high-density modes does not keep improving quality.

Visualizing the drift · distribution trajectory

Tracking checkpoints in a shared V-JEPA2 PCA space makes the drift visible: Causal CD initialization first expands to cover the teacher, but pure DMD then contracts toward its high-density regions, losing coverage and diversity.

Distribution evolution across Causal CD, pure DMD, and joint distillation — **Causal CD init** expands coverage → **pure DMD** contracts and drifts → **joint distillation** refines while holding coverage (precision/coverage tracked over training below each panel).

Our fix · Joint distillation

To counter the drift, we couple the mode-seeking objective with a mode-covering constraint under the same target distribution:

ℒ_joint = ℒ_DMD + λ ℒ_CD

The CD term anchors coverage while DMD sharpens modes, so joint distillation matches teacher high-density regions while preserving broad coverage — the rightmost panel above.

Results

Alignment Beats Scale

On VBench text-to-video, our matched joint distillation achieves the best overall score and the strongest coverage among distilled autoregressive models — even the weak variant, refined with only Wan-1.3B, surpasses baselines refined with Wan-14B.

Method	DMD Teacher	VBench ↑	Precision ↑	Coverage ↑	Diversity ↑
LTX Video	–	81.37	–	–	1.281
Wan2.1-T2V-1.3B	–	83.56	–	–	1.334
CausVid	Wan2.1-14B	83.03	0.51	0.21	1.223
Self-Forcing	Wan2.1-14B	84.21	0.78	0.53	1.269
Causal-Forcing	Wan2.1-14B	84.38	0.61	0.29	1.250
Ours (weak)	Wan2.1-1.3B	84.54	0.77	0.59	1.305
Ours (full)	Wan2.1-14B	84.83	0.79	0.69	1.304

VBench is the weighted overall score; precision and coverage are teacher-normalized against the listed DMD teacher; diversity is the raw Vendi score. Ours (weak) with a Wan-1.3B teacher already beats Wan-14B-refined baselines on overall score and coverage.

Qualitative comparison — visual quality & motion

Sharper subjects, stronger motion, and more temporally coherent details than bidirectional and distilled autoregressive baselines.

Wan2.1-1.3BSelf-ForcingCausal-ForcingOurs (1.3B teacher)Ours (14B teacher)

Qualitative comparison — sample diversity

Under fixed prompts and different seeds, our method preserves richer appearance and motion variation while keeping high fidelity.

Wan2.1-14BSelf-ForcingCausal-ForcingOurs (1.3B teacher)Ours (14B teacher)

Citation

@misc{li2026distillalign,
  title         = {DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation},
  author        = {Li, Jiaxing and Zou, Kai and Zhou, Cindy and Huang, Kaichen and Gao, Junyao and Wang, Zile and Liu, Yang and Liu, Bin and An, Bo and Li, Yangguang},
  year          = {2026},
  eprint        = {2607.26811},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2607.26811}
}