GoMVS: Geometrically Consistent Cost Aggregation for Multi-View Stereo

Jiang Wu1*, Rui Li1,2*, Haofei Xu2,3, Wenxun Zhao1, Yu Zhu1, Jinqiu Sun1, Yanning Zhang1
Northwestern Polytechnical University, ETH Zürich, University of Tübingen, Tübingen AI Center
CVPR 2024
*Indicates Equal Contribution
arXiv Code

TL;DR

GoMVS proposes a geometrically consistent cost aggregation approach under local planar assumption using surface normals.

Architecture

Architecture

Given a reference image and a set of source images, we use FPN to extract multi-scale features for cost volume reconstruction. To conduct geometrically consistent aggregation within the local window, we collect adjacent geometric cues and send them to the proposed geometrically consistent propagation (GCP) module, which computes the correspondence from the adjacent depth hypothesis space to the reference depth space. The resulting costs are endowed with geometric consistency, which facilitates better utilization of adjacent geometry and can be aggregated by the convolution.

Results on DTU dataset

results

Comparison on Tanks and Temples Benchmark

results

We show precision and recall error maps for the "Horse" scan. Our method demonstrates notable improvements over existing methods in challenging areas and ranks 1st on the official TNT Advanced Benchmark.

Empowering existing methods

results

Existing methods empowered by our approach achieve consistently improved performance.

Point clouds visualization

BibTeX

@inproceedings{wu2024gomvs,
        title={GoMVS: Geometrically Consistent Cost Aggregation for Multi-View Stereo},
        author={Wu, Jiang and Li, Rui and Xu, Haofei and Zhao, Wenxun and Zhu, Yu and Sun, Jinqiu and Zhang, Yanning},
        booktitle={CVPR},
        year={2024}
      }