Open3D 0.16 Release Notes

The fall brings a new "tock" release of Open3D, packed with new features and updates! Here are the highlights:

• Open3D visualization now works on Google Colab, and CPU rendering is now easy to use on all Linux machines.
• The visualizer has a new wireframe mode.
• Many new geometry processing algorithms were added, including mesh clipping, hole filling, extrusion and generation of text meshes.
• Added support for texture baking to create great looking 3D models.
• The Open3D standalone viewer is now available for Apple Silicon.
• Dependency version updates: Python 3.10, Tensorflow 2.8, PyTorch 1.12.

Google Summer of Code

Open3D had a successful Google Season of Code 2022 with many new features added, and more in the works for the next release. Here are the features that are part of this release:

Geometry processing algorithms for tensor-based geometry

[Contributed by @ZhengyuDiao]

• [new] Geometry creation functions: create_arrow(), create_box(), create_cone(), create_coordinate_frame(), create_cylinder(), create_icosahedron(), create_mobius(), create_octahedron(), create_sphere(), create_tetrahedron(), create_torus().
• [update] Improved support for reading / writing text format point cloud files in XYZ, XYZN, XYZI and XYZRGB formats.
• [fix] Fix the png/jpg textures loading in glb (binary glTF) file.
• [update] Tensor point cloud IO now uses float instead of double by default.

Open3D visualization on Google Colab

[Contributed by @FromBei-JingWithLove]

• [new] draw_plotly method brings interactive 3D visualization for Open3D to Jupyter notebooks and cloud environments (e.g., Colab).
• [new] Tutorial notebook demonstrating the various ways to use draw_plotly
  

Build System

• [new] Open3D is now available for Python 3.10. Python 3.6 is no longer supported.
• [new] C++ users can now link to the Open3D dynamic library through pkg-config files. There are available for Linux and macOS in the Open3D binary packages.
• [new] Alpha support for SYCL devices (AlderLake integrated GPUs, Arc GPUs).
• [fix] Windows and Visual Studio build fixes (contributed by @nigels-com, @jmherzog-de, @yuecideng)
• [fix] Fixed build Error on Fedora 36 with GCC 12 (contributed by @NobuoTsukamoto)
• [fix] Fix link error during build due to invalid LIBDIR in newer Debian systems (contributed by @bchretien)
• [update] Improved support for building on Apple Silicon.
• [update] Separate Python build (requirements_build.txt) and runtime (requirements.txt) dependencies. (with help from @johnthagen).
• [update] Documentation can be built faster with the make_docs.py --parallel option.

Core

• [new] Enable pickling for tensor and tensor based geometry (contributed by @yuecideng).
• [new] Random seed Open3D globally. Utility random number generation functions to ensure the global Open3D random seed is used.

  import open3d as o3d
  o3d.utility.random.seed(42)

  #include "open3d/utility/Random.h"
  int main() {
      using namespace open3d;
      utility::random::Seed(42);  // Globally seed.
      std::cout << utility::random::RandUint32() << std::endl;  // Simply get a random number.
      return 0;
  }

• [update] Faster creation of many small Tensors by using statically allocated memory for SizeVector.
• [new] Add tensor minimum(), maximum() ops (contributed by @yuecideng).
• [new] Add tensor Any(), All(), RemoveNonFinite()
• [update] Support for int64 index dtype in NearestNeighborSearch (contributed bu @chrockey)
• [update] The new datasets API is used throughout Open3D, including pipelines, examples and unit tests. New models with PBR materials (Helmet, Jesse monkey, crate and sword) are available. Add redwood indoor datasets (Augmented ICL-NUIM Dataset).
• [update] Support RadiusSearch for EstimateNormals() for Tensor PointCloud (contributed by @yuecideng).

Geometry

• [update] Tensor geometry attributes can now be accessed by the easier dot notation, in addition to the previous dictionary notation. For example:

  pcd.point["colors"] is pcd.point.colors
  tmesh.triangle["normals"] is tmesh.triangle.normals

Geometry classes of the Tensor API also add new functionality based on VTK:

• Mesh clipping along a plane
• Extraction of slices from triangle meshes as line sets
• Hole filling
• Boolean operations between watertight meshes
• A new algorithm for quadric decimation of triangle meshes
• Linear and rotational extrusion
• Generation of text meshes

Further, we add functionality for parametrising meshes with the UVAtlas library and added functions for baking vertex and triangle attributes to textures.

New operations for Tensor PointCloud:

• ClusterDBScan, ConvexHull

• Radius Outlier Filter, RemoveDuplicatedPoints, PaintUniformColor

• FarthestPointDownSample, HiddenPointRemoval and SegmentPlane

• Uniform and random down sample methods (contributed by @yuecideng).

• FPFH features. (contributed by @yuecideng)

• PointCloud boundary detection (contributed by @yuecideng)

  PointCloud	Boundaries

• [new] RemoveDuplicatedPoints() for PointCloud (Eigen API). (contributed by @scimad)

• [new] Farthest point downsampling (Eigen API) (contributed by @yuecideng).

• [update] Plane segmentation in point clouds is now multi-threaded. (Eigen API) (contributed by @yuecideng).

• [update] Add SelectByIndex and minor improvement to SelectByMask (contributed by @yuecideng).

• [new] Add Tensor AxisAlignedBoundingBox (contributed by @yuecideng).

• [update] SelectPolygonVolume.CropInPolygon returns point indices.

• [fix] Blocky / incomplete raycast output for larger voxel sizes fixed with EstimateRange() update (contributed by @jdavidberger).

• [update] Allow setting of number of threads that are used for building a raycasting scene (contributed by @jjabo)

Visualization

• [new] Wireframe mode added to Open3D visualizer
  
• [new] Support for Filament's bloom post-processing effects
• [fix] Make sure float and uint16 texture formats work correctly
• [fix] Fix bugs with point_width and line_width parameters in Material class

CPU rendering

• [update] Easy to use out of the box CPU rendering for Linux. Pre-built Mesa CPU rendering library is provided in the Python wheel if the system Mesa library is too old or not in use. You can enable CPU rendering with the environment variable OPEN3D_CPU_RENDERING=true before importing Open3D in Python or running a C++ program. See the tutorial for full details.
• [new] Updated CPU rendering works in Jupyter and Colab environments.
  

GUI

• [update] Vertical and horizontal radio button groups. (contributed by @forrestjgq)
• [new] Keypress event callback for windows. (contributed by @forrestjgq)
• [fix] Fix mouse event filter issue in popup menu covered area (contributed by @forrestjgq)
• [new] Added extra VisualizerWithVertexSelection point picking functions to python API (contributed by @d-walsh and @cansik)
• [update] Support double click mouse event in WebRTC. (contributed by @forrestjgq)
• [fix] Fix Point Clouds Getting Culled Prematurely.
• [fix] Set correct default material (lit with flat shading) for triangle mesh with triangle normals.
• [update] Add interface for get cropped geometry from VisualizerWithEditing class (contributed by @yuecideng).

IO

• [new] Open3D now implements a distortion and noise model from the Redwood dataset. This will enable you to simulate the output of realistic depth sensors.

Original	With noise + distortion	Difference

• [fix] Fix gltf PBR model load regression when loading through ASSIMP.
• [new] Read ply, stl, obj, off, gltf, glb, fbx file formats directly into a Tensor TriangleMesh.

Reconstruction

• [update] Update ICP interface to get iteration wise loss information.
• [new] Online SLAM Example with Realsense sensor support, bagfile support, and saved dataset support.
• [new] Legacy offline reconstruction C++ pipeline (contributed by @yuecideng)
• [update] InitializePointCloudForColoredICP efficiency improvement. (contributed by @Xiang-Zeng)

Open3D-ML

• [update] Upgrade PyTorch to v1.12 and TensorFlow to v2.8.2. With the new PyTorch version, Open3D PyTorch ops work directly with standard PyTorch wheels and custom wheels are not required any more.
• [fix] Avoid logging.basicConfig in non-scripts (contributed by @ajprax)
• [fix] filter_valid_label: force scores tensor onto the selected device (contributed by @ntw-au)
• [fix] Ensure that a checkpoint is saved on the final training epoch (contributed by @ntw-au)
• [fix] Handle nested dictionaries when merging configs (contributed by @ntw-au)
• [fix] Fix moving zip files (contributed by @tejaswid)
• [fix] Update RandLANet weights link (contributed by @Krupal09)
• [fix] Getting rid of torch._six (for future PyTorch versions) (contributed by @krshrimali)

Acknowledgments

We would like to thank all of our community contributors for their true labor of love for this release!

@ntw-au, @jdavidberger, @Xiang-Zeng, @jamesdi1993, @brentyi, @jjabo, @jbotsch-fy, @scimad, @cansik, @NobuoTsukamoto, @theNded, @chunibyo-wly, @jmherzog-de, @luzpaz, @code-review-doctor, @d-walsh, @johnthagen, @pmokeev, @erbensley, @hanzheteng, @chrockey, @agrellRepli, @bchretien, @nigels-com, @forrestjgq, @equant, @naruarjun, @ajprax, @INF800, @ntw-au, @tejaswid, @Krupal09, @krshrimali

Also thanks to the many others who helped the Open3D community by reporting as well as resolving issues.