Multi-Conditioned Diffusion Synthesis of Sand Boils for Low-Resource Earthen-Levee Inspection

A new diffusion-based workflow targets a very specific but important data problem: sand-boil defects on earthen levees are safety-critical, yet pixel-level labels are hard to collect. The pipeline fine-tunes Stable Diffusion XL with DreamBooth and uses a multi-branch ControlNet setup to generate inspection images from a small curated reference set.

One of the practical changes here is the soft-mask inpainting path, which keeps the real defect pixels intact while re-rendering the surrounding scene. That avoids the seams and color shifts that can show up with older compositing approaches. A separate mask-conditioned mode can even create a new boil inside a chosen mask, effectively making the mask the segmentation label by construction, though the default release favors the label-reliable soft-mask preset.

The system also includes a taxonomy-driven Prompt Atlas that expands one domain description into a CLIP-validated prompt bank, with transfer to new defect classes without code changes. From the real training set, it produced 1,020 synthetic candidates and 815 passed the CLIP admissibility filter. The team evaluated quality against the real reference set and a Poisson baseline, while also checking for drift and memorization.

For game developers, the broader takeaway is less about levees and more about data generation under tight annotation budgets. The work shows how multi-condition diffusion, mask-aware workflows, and prompt taxonomies can be combined to create more controllable synthetic datasets, while still wrestling with the tradeoff between fidelity, diversity, and...