Open3D 0.14 Release Notes
We are excited to present the new Open3D version 0.14!
In this release, you will find:
- TensorBoard visualization
- Upgraded GUI API
- Improved Tensor performance
- Improved I/O performance
- New 3D learning models in Open3D-ML: Point Transformer and PVCNN
- Improved interoperability with Numpy
Changes in Installation and Build system
- Open3D now works with Python 3.9. We release Open3D pre-compiled Python packages in Python 3.6, 3.7 3.8, and 3.9.
- Open3D 0.14 is the last version that supports conda installation. Starting from version 0.15, users will need to install Open3D with
pip install open3d. We recommend installing Open3D with
pipinside a conda virtual environment.
- Git submodules are no longer required in Open3D. You can now clone Open3D with
git clone https://github.com/isl-org/Open3D.gitwithout the
--recursiveflag. Also please note the updated Github URL.
- Open3D will now build in
Releasemode by default if
CMAKE_BUILD_TYPEis not specified.
Pythonis no longer required for building Open3D for C++ users.
- Open3D-ML is now recommended to be used along with PyTorch 1.8.2 and/or Tensorflow 2.5.2. Checkout Open3D-ML for more information.
Now you can use Open3D within Tensorboard for interactive 3D visualization! At a glance, you can:
- Save and visualize geometry sequences and their properties. This enables interactive visualization and debugging of 3D data and 3DML model training.
- Visualize 3D semantic segmentation and object detection with input data, ground truth, and predictions. In addition, any custom properties for a
PointCloud, from scalar to vector, can be easily visualized.
- Synchronize time steps and viewpoints during different runs. This helps debug and monitor the effect of parameter tuning.
Rich PBR materials
To get started, write some sample geometry data to a TensorBoard summary with this snippet:
from torch.utils.tensorboard import SummaryWriter # TensorFlow also works, see docs. import open3d as o3d from open3d.visualization.tensorboard_plugin import summary from open3d.visualization.tensorboard_plugin.util import to_dict_batch writer = SummaryWriter("demo_logs/") cube = o3d.geometry.TriangleMesh.create_box(1, 2, 4) cube.compute_vertex_normals() colors = [(1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0)] for step in range(3): cube.paint_uniform_color(colors[step]) writer.add_3d('cube', to_dict_batch([cube]), step=step)
Now you can visualize this in TensorBoard with
tensorboard --logdir demo_logs. For more details on how to use TensorBoard with Open3D, check out this tutorial.
Further enhancements have been added to the GUI viewer. Now you can:
- Directly visualize tensor-based geometry classes including
- Use physically based rendering (PBR) materials that deliver appealing appearance.
- New default lighting environment and skybox improves visual appeal
Use all the functionality in Tensorboard!
import open3d as o3d import open3d.visualization as vis a_sphere = o3d.geometry.TriangleMesh.create_sphere(2.5, create_uv_map=True) a_sphere.compute_vertex_normals() a_sphere = o3d.t.geometry.TriangleMesh.from_legacy(a_sphere) # Compare this... vis.draw(a_sphere) a_sphere.material = vis.Material('defaultLit') a_sphere.material.texture_maps['albedo'] = o3d.t.io.read_image('examples/test_data/demo_scene_assets/Tiles074_Color.jpg') a_sphere.material.texture_maps['roughness'] = o3d.t.io.read_image('examples/test_data/demo_scene_assets/Tiles074_Roughness.jpg') a_sphere.material.texture_maps['normal'] = o3d.t.io.read_image('examples/test_data/demo_scene_assets/Tiles074_NormalDX.jpg') # With this! vis.draw(a_sphere)
A complete, complex demo scene can be found at
- The Open3D
Tensorclass received a major performance boost with the help of Intel ISPC compiler and optimization for the contiguous code path.
python/benchmarks/corefor the benchmark scripts. For each operation, the geometric mean of run times with different data types is reported. The time is measured with an Intel i9-10980XE CPU.)
- A major upgrade of Parallel
HashMapis done. Now you can choose from multi-valued
HashSetdepending your value types. A comprehensive tutorial is also available.
- Linear Algebra performance have been optimized for small matrices, especially on 4x4 transformations.
- Semantics for tensor and tensor-based geometry have been improved, especially on device selection.
Functions expecting a Tensor now accept Numpy arrays and Python lists. For example:
import open3d as o3d import numpy as np mesh = o3d.t.geometry.TriangleMesh() mesh.vertex['positions'] = np.array([[0, 0, 0], [1, 0, 0], [1, 1, 0]], dtype=np.float32) mesh.vertex['colors'] = np.array([[1, 0, 0], [0, 1, 0], [0, 0, 1]], dtype=np.float32) mesh.triangle['indices'] = [[0, 1, 2]] o3d.visualization.draw(mesh)
- We now support I/O from/to Numpy compatible
.npyformats for Open3D tensors and tensor maps. It is now easier to convert between Open3D geometry classes and Numpy properties.
- We have improved I/O performance for tensor-based point cloud and triangle-mesh file-formats, including
.pts. Geometry loading time is hence improved for the stand-alone visualizer app.
- We added support for material data to the MessagePack serialization format of the RPC module.
We introduce a new class
RaycastingScenewith basic ray intersections functions and distance transform for meshes, utilizing the award-winning Intel Embree library.
Example code for rendering a depth map:
import open3d as o3d import matplotlib.pyplot as plt # Create scene and add a cube cube = o3d.t.geometry.TriangleMesh.from_legacy(o3d.geometry.TriangleMesh.create_box()) scene = o3d.t.geometry.RaycastingScene() scene.add_triangles(cube) # Use a helper function to create rays for a pinhole camera. rays = scene.create_rays_pinhole(fov_deg=60, center=[0.5,0.5,0.5], eye=[-1,-1,-1], up=[0,0,1], width_px=320, height_px=240) # Compute the ray intersections and visualize the hit distance (depth) ans = scene.cast_rays(rays) plt.imshow(ans['t_hit'].numpy())
Distance transform generated with
Normal estimation for tensor
PointCloudis supported with the tensor-compatible nearest neighbor search modules.
LineSetare implemented that allow user-defined properties. For example:
import open3d as o3d import open3d.core as o3c mesh = o3d.t.geometry.TriangleMesh() mesh.vertex["positions"] = o3c.Tensor([[0.0, 0.0, 1.0], [0.0, 1.0, 0.0], [1.0, 0.0, 0.0], [1.0, 1.0, 1.0]], dtype=o3c.float32) mesh.vertex["my_custom_labels"] = o3c.Tensor([0, 1, 2, 4], dtype=o3c.int32) mesh.triangle["indices"] = o3c.Tensor([[0, 1, 2], [1, 2, 3]], dtype=o3c.int32)
- We have enhanced point cloud registration (ICP) with a tensor interface:
- Float64 (double) precision point cloud is supported for a higher numerical stability
- Robust Kernels, including Huber, Tukey, and GM losses are supported for robust registration.
- Colored-ICP is now supported in the unified tensor-based API.
- See this tutorial for more details.
- We also provide an initial tensor-based reconstruction system in Python, including
- Customizable volumetric RGB-D integration;
- Dense RGB-D SLAM with a GUI;
- See this tutorial for more details.
The Open3D-ML library welcomes more state-of-the-art models and operators that are ready to use for advanced 3D perception, especially semantic segmentation, including
- New state-of-the-art Point Transformer for Semantic Segmentation.
- Highly Efficient Point-Voxel Convolution for Semantic Segmentation.
- RaggedTensor integration that enables batch
SparseConvolutionTransposealong with PyTorch.
- Batched voxelization for fast point-voxel conversions.
Expreimental ARM64 builds
- Experimental Ubuntu ARM64 build
- Python 3.6: open3d-0.14.1-cp36-cp36m-linux_aarch64.whl
- Python 3.7: open3d-0.14.1-cp37-cp37m-linux_aarch64.whl
- Python 3.8: open3d-0.14.1-cp38-cp38-linux_aarch64.whl
- Python 3.9: open3d-0.14.1-cp39-cp39-linux_aarch64.whl
- Visualizer does not work on Raspberry Pi due to the lack of full OpenGL
- Tested on Nvidia Jetson boards with Ubuntu 18.04
- You may use pyenv to setup your virtualenvs
- Experimental Apple Silicon M1 ARM64 build:
- Test your installation
pip install open3d-xxx.whl # Test the new vidualizer python -c "import open3d as o3d; c = o3d.geometry.TriangleMesh.create_box(); o3d.visualization.draw([c])" # Test the traditional visualizer python -c "import open3d as o3d; c = o3d.geometry.TriangleMesh.create_box(); o3d.visualization.draw_geometries([c])"
We hope you find Open3D 0.14.0 exciting and useful. Happy coding!
Remember that you can reach out with questions, requests, or feedback through the following channels:
We thank all the community contributors for this release!