Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Models

Authors

Affiliations

¹ University of Maryland, College Park

² Hugging Face

* Equal advising

Resources

Paper

Code Repository

Abstract

Inference-time scaling for text-to-image generation has progressed from simple Best-of-N (BoN) sampling to guided search methods that verify and steer candidate trajectories at intermediate denoising steps. These approaches focus on when and how often to verify during denoising but largely treat the cost of generation itself as fixed. Moreover, the standard practice of comparing methods by number of function evaluations (NFEs) counts only denoising forward passes and ignores verifier overhead, which can distort efficiency rankings.

We show that under wall-clock evaluation, simple BoN already matches or outperforms several guided search techniques, suggesting that compute is better spent on broader exploration than on repeated intermediate verification. This motivates Flash-BoN, which generates a large pool of inexpensive draft candidates by combining three complementary acceleration knobs: timestep truncation, layer skipping, and activation proxies into a single configuration optimized once per model. An efficient multi-stage verification procedure then identifies the most promising draft, which is refined at full quality.

Across three benchmarks and three model scales, Flash-BoN consistently outperforms all baselines under fixed wall-clock budgets, with gains that grow at larger model scales (+8% AUC). We further show that our strategy combines well and improves existing orthogonal techniques such as reflection-based prompt optimization (+16% AUC). The gains correlate with increased candidate diversity, which also enables draft-guided selection to accelerate RL post-training convergence.

NFE vs. Wall-Clock: Rethinking Compute Accounting

Prior work typically compares inference-time scaling methods using the Number of Function Evaluations (NFEs) as a hardware-agnostic proxy. However, NFEs only count denoising forward passes and often ignore verifier costs entirely. This matters because methods that verify more frequently appear efficient under NFE accounting even when they are not under wall-clock time.

NFE vs wall-clock comparison on GenAI-Bench with Wan2.1 1.3B — **Figure 1:** [**T2I Model:** Wan2.1 1.3B] Comparison under matched NFE and matched wall-clock budgets. **Left:** At a fixed NFE budget, Breadth-First Search (BFS) is marginally best at high NFEs. **Right:** Under equal runtime, the trend reverses. Dashed vertical lines mark the wall-clock time each method needs to reach 4k NFEs (the maximum NFE shown on the left). Methods with frequent verifier calls take longer to reach the same NFE budget, reducing exploration within a fixed runtime. Consequently, BoN-style approaches outperform under realistic latency constraints.

Once compute is adjusted to account for all costs (generation + verification), simple BoN already matches or exceeds guided search methods like BFS, DFS, and ZOS. This motivates our approach: rather than spending compute on repeated intermediate verification, we focus on generating more and cheaper candidates for broader exploration.

Flash-BoN Method

Flash-BoN builds on the observation that promising candidates can often be identified without full-cost generation. We define a draft configuration $\phi$ that combines three complementary acceleration knobs, then apply a multi-stage verification procedure to select and refine the best draft.

Three Acceleration Knobs

Early Stopping. Truncate the denoising trajectory to $S'$ steps instead of the full $S$, producing rough but structurally informative previews.
Activation Proxies. Exploit redundancy across timesteps by forecasting layer activations via a second-order Taylor expansion from cached features at previously visited timesteps.
Layer Skipping. Selectively skip a subset of transformer layers $\mathcal{L}_{\text{skip}}$ during denoising, propagating the previous activation instead of computing the full block output.

These three knobs define a joint configuration $\phi = (S', n_{\text{skip}}, \ell_{\text{start}}, \ell_{\text{end}}, f_{\text{full}})$. We formalize the search for the optimal draft configuration $\phi^*$ as a discrete optimization problem, solved once per model.

Multi-Stage Verification

Given $M$ draft candidates, pointwise scoring alone is unreliable: VLM-based verifiers produce integer scores that compress resolution at the top ranks, with over 80% of prompts producing tied highest scores. We introduce a progressive filter that combines both pointwise and pairwise comparisons with Elo ranking:

Pointwise pruning + Elo seeding. Score all $M$ candidates, retain the top fraction, and initialize Elo ratings from pointwise scores.
Sparse pairwise comparisons. Sort survivors by Elo, run adjacent head-to-head comparisons, and update ratings.
Final selection. Return the candidate with the highest Elo rating for full-quality refinement.

Three-Stage Pipeline

Stage 1 — Draft Generation: Generate $M$ cheap candidates $\{\tilde{y}_1, \ldots, \tilde{y}_M\}$ using $\phi^*$.
Stage 2 — Verification: Score all drafts and select the most promising one via multi-stage Elo ranking.
Stage 3 — Refinement: Regenerate the selected draft at full quality to produce the final output $y^*$.

Main Results

We evaluate Flash-BoN against four baselines — Best-of-N (BoN), Breadth-First Search (BFS), Depth-First Search (DFS), and Zeroth-Order Search (ZOS) — across three benchmarks (GenAI-Bench, GenEval, UniGenBench) and three model scales (Wan 2.1 1.3B, Wan 2.1 14B, FLUX.1-dev).

Method	GenAI-Bench			GenEval			UniGenBench
Method	Wan 1.3B	Wan 14B	FLUX	Wan 1.3B	Wan 14B	FLUX	Wan 1.3B	Wan 14B	FLUX
ZOS	0.52	0.57	0.57	0.29	0.36	0.37	0.45	0.52	0.49
DFS	0.55	0.50	0.51	0.36	0.37	0.35	0.47	0.46	0.45
BFS	0.50	0.43	0.44	0.31	0.28	0.29	0.43	0.39	0.38
BoN	0.65	0.60	0.60	0.43	0.42	0.42	0.54	0.53	0.54
Flash-BoN (Ours)	0.66	0.68	0.68	0.45	0.49	0.47	0.57	0.60	0.61

Table 1: Normalized AUC (↑) comparison of inference-time search techniques across benchmarks and models. Flash-BoN leads across all model–benchmark combinations, with margins that grow at larger model scales.

GenAI-Bench: Wall-Clock Comparison Across Scales

GenAI-Bench results across model scales — **Figure 3: GenAI-Bench under fixed wall-clock budgets.** Performance versus total runtime (seconds). All methods are evaluated under identical hardware and end-to-end time accounting. Best-of-N (BoN) consistently outperforms guided search baselines across models. Flash-BoN achieves the strongest performance at all time budgets, with gains that persist and widen at larger scales.

Per-Category Breakdown

**Figure 4:** Per-category GenAI-Bench scores on Wan2.1 1.3B under wall-clock time budgets. Flash-BoN leads across all ten categories.

Qualitative Comparison

Diversity Analysis

We measure candidate pool diversity using the Vendi Score and find that performance and diversity are positively correlated (Pearson $r=0.75$). Guided search methods cluster in the low-diversity, lower-performance region, while Flash-BoN consistently sits at the high-diversity end — inexpensive drafts enable not only more candidates but also meaningfully different ones.

Diversity vs performance correlation — **Figure 6: Candidate pool diversity vs. final GenAI-Bench performance across three models.** Diversity is measured using the Vendi Score. Each point is one method averaged over all prompts; color indicates technique, shape indicates model. Dashed line shows the linear correlation fit. Methods that explore more diverse candidates consistently achieve higher quality (Pearson $r=0.75$).

Combining with Orthogonal Techniques

The Flash draft strategy functions as a drop-in module that can be layered onto existing inference-time scaling methods. Applying Flash drafts to Reflection-Tuning yields a large improvement (+16% AUC), while gains on BFS are more modest (+6% AUC).

Combining Flash-BoN with existing methods — **Figure 7: Combining the Flash draft strategy with existing inference-time scaling methods on Wan2.1 1.3B (GenAI-Bench).** We report AUC/Time, the area under the score-vs-wall-clock curve normalized by total time, as a summary efficiency metric. Applying Flash drafts to Reflection-Tuning yields a large improvement (0.46 to 0.62), while the gain on BFS is more modest (0.49 to 0.55).

RL Post-Training

The draft-guided selection principle transfers to reinforcement learning: Flash-Flow-GRPO matches baseline Flow-GRPO convergence in 10× fewer steps. Flash-Flow-GRPO opens a clear gap within the first 60 steps and maintains it throughout training, reaching 0.699 vs. 0.692 for the baseline.

Conclusion

We introduced Flash-BoN, a draft-and-refine inference-time scaling method that generates a large pool of inexpensive draft candidates by unifying three complementary acceleration strategies — timestep truncation, layer skipping, and activation proxies — into a single optimized configuration. Combined with a novel multi-stage verification procedure using Elo ranking, Flash-BoN consistently outperforms all baselines across three benchmarks and three model scales under fixed wall-clock budgets.

Key contributions include: (1) demonstrating that NFE-based comparisons distort method rankings by ignoring verifier costs; (2) a unified draft generation framework with a discrete optimization procedure; (3) consistent gains that grow at larger model scales (+8% AUC); (4) modular combination with orthogonal techniques like Reflection-Tuning (+16% AUC); and (5) accelerating RL post-training convergence by 10× via draft-guided selection.