Iterative Multi-granular Image Editing using Diffusion Models

Recent advances in text-guided image synthesis has dramatically changed how creative professionals generate artistic and aesthetically pleasing visual assets. To fully support such creative endeavors, the process should possess the ability to: 1) iteratively edit the generations and 2) control the s...

Full description

Saved in:
Bibliographic Details
Main Authors: Joseph, K J, Udhayanan, Prateksha, Shukla, Tripti, Agarwal, Aishwarya, Karanam, Srikrishna, Goswami, Koustava, Srinivasan, Balaji Vasan
Format: Conference Proceeding
Language:English
Subjects:
Adaptation models
Applications
Arts / games / social media
Benchmark testing
Commercial / retail
Image synthesis
Modulation
Process control
Training
Virtual / augmented reality
Visualization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recent advances in text-guided image synthesis has dramatically changed how creative professionals generate artistic and aesthetically pleasing visual assets. To fully support such creative endeavors, the process should possess the ability to: 1) iteratively edit the generations and 2) control the spatial reach of desired changes (global, local or anything in between). We formalize this pragmatic problem setting as Iterative Multi-granular Editing. While there has been substantial progress with diffusion-based models for image synthesis and editing, they are all one shot (i.e., no iterative editing capabilities) and do not naturally yield multi-granular control (i.e., covering the full spectrum of local-to-global edits). To overcome these drawbacks, we propose EMILIE: Iterative Multi-granular Image Editor. EMILIE introduces a novel latent iteration strategy, which re-purposes a pre-trained diffusion model to facilitate iterative editing. This is complemented by a gradient control operation for multi-granular control. We introduce a new benchmark dataset to evaluate our newly proposed setting. We conduct exhaustive quantitatively and qualitatively evaluation against recent state-of-the-art approaches adapted to our task, to being out the mettle of EMILIE. We hope our work would attract attention to this newly identified, pragmatic problem setting.
ISSN:2642-9381
DOI:10.1109/WACV57701.2024.00792