Helisa Dhamo * Fabian Manhardt * Nassir Navab Federico Tombari
Technical University of Munich Google
* The first two authors contributed equally.
We propose the first fully-learned model to synthesize novel 3D scenes from a scene graph. The same model is capable of scene manipulations. Our method essentially predicts object-level 3D bounding boxes together with appropriate 3D shapes, which are then combined to create a full 3D scene.
@inproceedings{graph2scene2021,
title={Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes using Scene Graphs},
author={Dhamo, Helisa and Manhardt, Fabian and Navab, Nassir and Tombari, Federico},
booktitle={IEEE International Conference on Computer Vision (ICCV)},
year={2021}
}
As one of our contributions, we annotate a real world dataset (3RScan) with directed 3D bounding boxes for objects. The json files contain namely a dictionary of the scan ids of the scenes in the training set and validation set of 3RScan. Each scan contains a dictionary of object instances.
Every object node contains the fields ‘param7’, ‘8points’ and ‘direction’:
-- param7: Oriented 3D bounding box represented via 7 DoF parameters, namely length, width, height, 3D centroid, and an angle normalized from 0 to 90 degrees.
-- direction: specifies the facing direction of the object. It can take valid values of 1-4. Other values denote ambiguous cases.
-- 8points: Oriented 3D bounding box represented via its eight corners. Note that these points do not consider the facing direction of the object.
To obtain the final angle of an object, that considers the facing direction, you need to transform the last parameter (normalized angle) as follows:
param7[6] += (direction - 1) * 90
Our Pytorch implementation for this work is available here.
For questions regarding the method, code, or the canonical pose data, please contact us.