Abstract
We present a novel method to synthesize novel view selfies from a mobile phone captured video. This is challenging due to the inconsistent geometry that is caused by the person’s unavoidable movement. Recent methods reconstruct the whole deformable scene implicitly with a deformation field. We argue that they are inefficient and hard to fit diverse real-world videos. In contrast, we use an explicit reconstruction for generalization and efficiency, where we separately track, reconstruct, and synthesize the foreground and background to overcome the geometry inconsistency. Several novel and effective modules are proposed for better performance and visual results. We demonstrate the advantage of the proposed method against the existing alternatives in a collection of our captured selfie videos with the support of quantitative and qualitative results.
Publication
Jia-Wang Bian*, Huangying Zhan*, Ian Reid, NVSS: High-quality Novel View Selfie Synthesis, International Conference on 3D Vision (3DV), 2021 [PDF]
@inproceedings{bian2021nvss,
title={NVSS: High-quality Novel View Selfie Synthesis},
author={Bian, Jia-Wang and Zhan, Huangying and Reid, Ian},
booktitle= {International Conference on 3D Vision (3DV)},
year={2021}
}Rendered Images
