In the literature, 3D reconstruction from 2D image has been extensively addressed but often still requires geometrical supervision. In this paper, we propose a self-supervised monocular scene reconstruction method with neural radiance fields (NeRF) learned from multiple image sequences with pose. To improve geometry prediction, we introduce new geometry constraints and a novel probabilistic sampling strategy that efficiently update radiance fields. As the latter are conditioned on a single frame, scene reconstruction is achieved from the fusion of multiple synthetized novel depth views. This is enabled by our spherical-decoder which allows hallucination beyond the input frame field of view. Thorough experiments demonstrate that we outperform all baselines on all metrics for novel depth views synthesis and scene reconstruction.
Input | ||||
![]() |
![]() |
![]() |
![]() |
|
Trajectory | Novel depths (and views) | |||
![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
3D Reconstruction | ||||
![]() |
![]() |
![]() |
![]() |
@InProceedings{cao2022scenerf, author = {Cao, Anh-Quan and de Charette, Raoul}, title = {SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields}, publisher = {arxiv}, year = {2022}, }