SD-𝜋XL: Generating Low-Resolution Quantized Imagery via Score Distillation
DescriptionLow-resolution quantized imagery, such as pixel art, is seeing a revival in modern applications ranging from video game graphics to digital design and fabrication, where creativity is often bound by a limited palette of elemental units. Despite their growing popularity, the automated generation of quantized images from raw inputs remains a significant challenge, often necessitating intensive manual input. We introduce SD-𝜋XL, an approach for producing quantized images that employs score distillation sampling in conjunction with a differentiable image generator. Our method enables users to input a prompt and optionally an image for spatial conditioning, set any desired output size 𝐻×𝑊 , and choose a palette of 𝑛 colors or elements. Each color corresponds to a distinct class for our generator, which operates on an 𝐻×𝑊×𝑛 tensor. We adopt a softmax approach, computing a convex sum of elements, thus rendering the process differentiable and amenable to backpropagation. We show that employing Gumbel-softmax reparameterization allows for crisp pixel art effects. Unique to our method is the ability to transform input images into low-resolution, quantized versions while retaining their key semantic features. Our experiments validate SD-𝜋XL’s performance in creating visually pleasing and faithful representations, consistently outperforming the current state-of-the-art. Furthermore, we showcase SD-𝜋XL’s practical utility in fabrication through its applications in interlocking brick mosaic, beading and embroidery design.
Event Type
Technical Papers
TimeTuesday, 3 December 20242:56pm - 3:08pm JST
LocationHall B7 (1), B Block, Level 7
