FUSE: Flow-aligned Universal Sketch Editor for
DiT-based Generative Models

Abstract

Hand-drawn sketches provide an intuitive interface for specifying the structure of missing or edited regions in image inpainting. Recent Diffusion Transformer (DiT)-based generative models offer strong visual priors, but their direct adaptation to sketch-guided inpainting still faces two practical challenges: trajectory-level background drift can cause boundary discontinuities after compositing, while global sketch guidance entangles edited and background regions.

To address these issues, we introduce FUSE (Flow-aligned Universal Sketch Editor), a flow-aligned sketch-guided inpainting framework with two inference-time strategies. At its core, Flow-based Background Trajectory Alignment (FBTA) aligns background latents with their forward-diffused counterparts during sampling, mitigating trajectory-induced boundary artifacts. We further incorporate Region-aware Sketch Control (RASC), a lightweight strategy that decouples sketch guidance strength between edited and background regions. To support training and evaluation, we construct UniSketch-60k, a dataset of 59,276 image-text-sketch-mask quadruplets across diverse categories and sketch styles. Experiments on Flux and Z-Image backbones, together with real hand-drawn editing cases, show that FUSE improves background consistency, sketch faithfulness, and practical controllability.