CAD-Deform: Deformable Fitting of CAD Models to 3D Scans

Vladislav Ishimtsev*1 Alexey Bokhovkin*1 Alexey Artemov1 Savva Ignatyev1 Matthias Niessner2 Denis Zorin3,1 Evgeny Burnaev1
1Skolkovo Institute of Science and Technology
2Technical University of Munich
3New York University
*denotes equal contribution
ECCV 2020
Paper arXiv Code

CAD-Deform takes as input a set of 3D CAD models aligned on a RGB-D scan (left). In order to achieve tight fits (middle), we propose a novel part-based deformation formulation that maintains the desired CAD properties such as sharp features.

Abstract

Shape retrieval and alignment are a promising avenue towards turning 3D scans into lightweight CAD representations that can be used for content creation such as mobile or AR/VR gaming scenarios. Unfortunately, CAD model retrieval is limited by the availability of models in standard 3D shape collections (e.g., ShapeNet).

In this work, we address this shortcoming by introducing CAD-Deform, a method which obtains more accurate CAD-to-scan fits by non-rigidly deforming retrieved CAD models.

Our key contribution is a new non-rigid deformation model incorporating smooth transformations and preservation of sharp features, that simultaneously achieves very tight fits from CAD models to the 3D scan and maintains the clean, high-quality surface properties of hand-modeled CAD objects. A series of thorough experiments demonstrate that our method achieves significantly tighter scan-to-CAD fits, allowing a more accurate digital replica of the scanned real-world environment while preserving important geometric features present in synthetic CAD environments.

Shape Fitting to 3D Scans

Deformations obtained using our method and the baselines, with mesh colored according to the Euclidean distance to its nearest point in the scan. We note that high accuracy scores for Harmonic and ARAP deformations are achieved at the cost of destroying the initial structure of the mesh, particularly in regions where scan is missing (note that back side and armrests are gone for chairs in the first and second rows). In contrast, our method is better able to preserve smooth surfaces, sharp features, and overall mesh integrity, while keeping accurate local alignment.

BibTeX


      @inproceedings{ishimtsev2020caddeform,
          title     = {CAD-Deform: Deformable Fitting of CAD Models to 3D Scans},
          author    = {Ishimtsev, Vladislav and Bokhovkin, Alexey and Artemov, Alexey and Ignatyev, Savva and Niessner, Matthias and Zorin, Denis and Burnaev, Evgeny},
          journal   = {European Conference on Computer Vision (ECCV)},
          year      = {2020}
      }