Reconstruction system (Tensor)#

This tutorial demonstrates volumetric RGB-D reconstruction and dense RGB-D SLAM with the Open3D Tensor interface and the Open3D Hash map backend.

It is possible to run the tutorial with the minimalistic dataset SampleRedwoodRGBDImages, but it is recommended to run the tutorial with real-world datasets with longer sequences to demonstrate its capability. Please refer to RGBD images for more available datasets. The Redwood dataset can be a good starting point.

If you use any part of the tensor-based reconstruction system or the hash map backend in Open3D, please cite [Dong2021]:

@article{Dong2021,
    author    = {Wei Dong, Yixing Lao, Michael Kaess, and Vladlen Koltun}
    title     = {{ASH}: A Modern Framework for Parallel Spatial Hashing in {3D} Perception},
    journal   = {arXiv:2110.00511},
    year      = {2021},
}

Note

As of now the tutorial is only for online dense SLAM, and offline integration with provided poses. The tutorials for tensor-based offline reconstruction system, Simultaneous localization and calibration (SLAC), and shape from shading (SfS) tutorials as mentioned in [Dong2021] are still under construction. At current, please refer to Reconstruction system for the legacy versions.

Quick start#

Getting the example code

# Activate your conda environment, where you have installed open3d pip package.
# Clone the Open3D github repository and go to the example.
cd examples/python/t_reconstruction_system/

# Show CLI help for ``dense_slam_gui.py``
python dense_slam_gui.py --help

Running the example with default dataset.

# The following command, will download and use the default dataset,
# which is ``lounge`` dataset from stanford.
python dense_slam_gui.py

The tensor reconstruction system supports CPU, CUDA, and SYCL devices. CUDA and SYCL require an Open3D build with the corresponding backend enabled.

# Use CUDA when available.
python dense_slam_gui.py --device 'CUDA:0'

# Use an Intel GPU through SYCL when available.
python dense_slam_gui.py --device 'SYCL:0'

Changing the default dataset. One may change the default dataset to other available datasets. Currently the following datasets are available:

  1. Lounge (keyword: lounge) (Default)

  2. Bedroom (keyword: bedroom)

  3. Jack Jack (keyword: jack_jack)

# Using jack_jack as the default dataset.
python dense_slam_gui.py --default_dataset 'bedroom'

Running the example with custom dataset using config file. Manually download or store the data in a folder and store all the color images in the image sub-folder, and all the depth images in the depth sub-folder. Create a config.yml file and set the path_dataset to the data directory. Override the parameters for which you want to change the default values.

Example config file for online reconstruction system has been provided in examples/python/t_reconstruction_system/default_config.yml, which looks like the following:

 1name: Default reconstruction system config
 2fragment_size: 100
 3# Set to CUDA:0 or SYCL:0 when the corresponding backend is available.
 4device: CUDA:0
 5engine: tensor
 6multiprocessing: false
 7path_dataset: ''
 8depth_folder: depth
 9color_folder: color
10path_intrinsic: ''
11path_color_intrinsic: ''
12depth_min: 0.1
13depth_max: 3.0
14depth_scale: 1000.0
15odometry_method: hybrid
16odometry_loop_interval: 10
17odometry_loop_weight: 0.1
18odometry_distance_thr: 0.07
19icp_method: colored
20icp_voxelsize: 0.05
21icp_distance_thr: 0.07
22global_registration_method: ransac
23registration_loop_weight: 0.1
24integrate_color: true
25voxel_size: 0.0058
26trunc_voxel_multiplier: 8.0
27block_count: 40000
28est_point_count: 6000000
29surface_weight_thr: 3.0

Capture your own dataset#

This tutorial provides an example that can record synchronized and aligned RGBD images using the Intel RealSense camera. For more details, please see Capture your own dataset.

Getting started with online reconstruction system#