SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Images
DescriptionSingle-image novel view synthesis is a challenging and ongoing problem that aims to generate an infinite number of consistent views from a single input image. Although significant efforts have been made to advance the quality of generated novel views, less attention has been paid to the expansion of the underlying scene representation, which is crucial to the generation of realistic novel view images. This paper proposes SinMPI, a novel method that uses expanded multiplane images (MPIs) as the 3D scene representation to significantly expand the perspective range of MPIs and generate high-quality novel views from a large multiplane space. The key idea of our method is to use Stable Diffusion~\cite{rombach2021highresolution} to generate out-of-view contents, project all scene contents into expanded multiplane images, and then optimize the multiplane images under the supervision of pseudo-multi-view data generated by a depth-aware warping and inpainting module. Both qualitative and quantitative experiments have been conducted to validate the superiority of our method to the state of the art. Our code and data are available at https://github.com/TrickyGo/SinMPI.
Event Type
Technical Papers
TimeTuesday, 12 December 20239:30am - 12:45pm
LocationDarling Harbour Theatre, Level 2 (Convention Centre)
