Seamless Sketch-Guided Image Inpainting via
Flow-based Background Trajectory Alignment
Abstract
Sketch-guided image inpainting enables explicit structural control over content synthesis. By leveraging the strong prior of recent large-scale diffusion models, direct fine-tuning within controllable frameworks significantly improves synthesis quality. However, such straightforward adaptation often causes background regions to drift during sampling, leading to severe boundary discontinuities when the original background is restored.
To address this, we propose Flow-based Background Trajectory Alignment (FBTA), a training-free, plug-and-play inference strategy designed to ensure visual consistency. FBTA anchors the denoising trajectory by replacing predicted background latents with their forward-diffused counterparts at each timestep. We further introduce FBTA-fast, which achieves comparable results with negligible computational overhead, and UniSketch-60k, a dataset of 59,276 high-quality image-text-sketch-mask quadruplets. Extensive experiments demonstrate state-of-the-art performance in universal sketch-guided image inpainting.
Qualitative Comparison
FBTA follows the input sketch while maintaining sharper details and more coherent boundaries than existing sketch-guided and general image-editing methods.
Method Overview
At every denoising step, FBTA aligns the predicted background with its forward-diffused trajectory, preventing background drift and visible seams around the edited region.
More Results
Our method generalizes across indoor scenes, landscapes, portraits, crowded images, typography, artistic styles, and other diverse sketch-guided editing cases.
Controllability
The sketch guidance weight provides smooth control over how strongly the generated content follows the supplied structure.
