Open3D 0.16 Release Notes

The fall brings a new "tock" release of Open3D, packed with new features and updates! Here are the highlights:

• Open3D visualization now works on Google Colab, and CPU rendering is now easy to use on all Linux machines.
• The visualizer has a new wireframe mode.
• Many new geometry processing algorithms were added, including mesh clipping, hole filling, extrusion and generation of text meshes.
• Added support for texture baking to create great looking 3D models.
• The Open3D standalone viewer is now available for Apple Silicon.
• Dependency version updates: Python 3.10, Tensorflow 2.8, PyTorch 1.12.

Google Summer of Code

Open3D had a successful Google Season of Code 2022 with many new features added, and more in the works for the next release. Here are the features that are part of this release:

Geometry processing algorithms for tensor-based geometry

[Contributed by @ZhengyuDiao]

• [new] Geometry creation functions: create_arrow(), create_box(), create_cone(), create_coordinate_frame(), create_cylinder(), create_icosahedron(), create_mobius(), create_octahedron(), create_sphere(), create_tetrahedron(), create_torus().
• [update] Improved support for reading / writing text format point cloud files in XYZ, XYZN, XYZI and XYZRGB formats.
• [fix] Fix the png/jpg textures loading in glb (binary glTF) file.
• [update] Tensor point cloud IO now uses float instead of double by default.

Open3D visualization on Google Colab

[Contributed by @FromBei-JingWithLove]

• [new] draw_plotly method brings interactive 3D visualization for Open3D to Jupyter notebooks and cloud environments (e.g., Colab).
• [new] Tutorial notebook demonstrating the various ways to use draw_plotly
  

Build System

• [new] Open3D is now available for Python 3.10. Python 3.6 is no longer supported.
• [new] C++ users can now link to the Open3D dynamic library through pkg-config files. There are available for Linux and macOS in the Open3D binary packages.
• [new] Alpha support for SYCL devices (AlderLake integrated GPUs, Arc GPUs).
• [fix] Windows and Visual Studio build fixes (contributed by @nigels-com, @jmherzog-de, @yuecideng)
• [fix] Fixed build Error on Fedora 36 with GCC 12 (contributed by @NobuoTsukamoto)
• [fix] Fix link error during build due to invalid LIBDIR in newer Debian systems (contributed by @bchretien)
• [update] Improved support for building on Apple Silicon.
• [update] Separate Python build (requirements_build.txt) and runtime (requirements.txt) dependencies. (with help from @johnthagen).
• [update] Documentation can be built faster with the make_docs.py --parallel option.

Core

• [new] Enable pickling for tensor and tensor based geometry (contributed by @yuecideng).
• [new] Random seed Open3D globally. Utility random number generation functions to ensure the global Open3D random seed is used.

  import open3d as o3d
  o3d.utility.random.seed(42)

  #include "open3d/utility/Random.h"
  int main() {
      using namespace open3d;
      utility::random::Seed(42);  // Globally seed.
      std::cout << utility::random::RandUint32() << std::endl;  // Simply get a random number.
      return 0;
  }

• [update] Faster creation of many small Tensors by using statically allocated memory for SizeVector.
• [new] Add tensor minimum(), maximum() ops (contributed by @yuecideng).
• [new] Add tensor Any(), All(), RemoveNonFinite()
• [update] Support for int64 index dtype in NearestNeighborSearch (contributed bu @chrockey)
• [update] The new datasets API is used throughout Open3D, including pipelines, examples and unit tests. New models with PBR materials (Helmet, Jesse monkey, crate and sword) are available. Add redwood indoor datasets (Augmented ICL-NUIM Dataset).
• [update] Support RadiusSearch for EstimateNormals() for Tensor PointCloud (contributed by @yuecideng).

Geometry

• [update] Tensor geometry attributes can now be accessed by the easier dot notation, in addition to the previous dictionary notation. For example:

  pcd.point["colors"] is pcd.point.colors
  tmesh.triangle["normals"] is tmesh.triangle.normals

Geometry classes of the Tensor API also add new functionality based on VTK:

• Mesh clipping along a plane
• Extraction of slices from triangle meshes as line sets
• Hole filling
• Boolean operations between watertight meshes
• A new algorithm for quadric decimation of triangle meshes
• Linear and rotational extrusion
• Generation of text meshes

Further, we add functionality for parametrising meshes with the UVAtlas library and added functions for baking vertex and triangle attributes to textures.

New operations for Tensor PointCloud:

• ClusterDBScan, ConvexHull

• Radius Outlier Filter, RemoveDuplicatedPoints, PaintUniformColor

• FarthestPointDownSample, HiddenPointRemoval and SegmentPlane

• Uniform and random down sample methods (contributed by @yuecideng).

• FPFH features. (contributed by @yuecideng)

• PointCloud boundary detection (contributed by @yuecideng)

  PointCloud	Boundaries

• [new] RemoveDuplicatedPoints() for PointCloud (Eigen API). (contributed by @scimad)

• [new] Farthest point downsampling (Eigen API) (contributed by @yuecideng).

• [update] Plane segmentation in point clouds is now multi-threaded. (Eigen API) (contributed by @yuecideng).

• [update] Add SelectByIndex and minor improvement to SelectByMask (contributed by @yuecideng).

• [new] Add Tensor AxisAlignedBoundingBox (contributed by @yuecideng).

• [update] SelectPolygonVolume.CropInPolygon returns point indices.

• [fix] Blocky / incomplete raycast output for larger voxel sizes fixed with EstimateRange() update (contributed by @jdavidberger).

• [update] Allow setting of number of threads that are used for building a raycasting scene (contributed by @jjabo)

Visualization

• [new] Wireframe mode added to Open3D visualizer
  
• [new] Support for Filament's bloom post-processing effects
• [fix] Make sure float and uint16 texture formats work correctly
• [fix] Fix bugs with point_width and line_width parameters in Material class

CPU rendering

• [update] Easy to use out of the box CPU rendering for Linux. Pre-built Mesa CPU rendering library is provided in the Python wheel if the system Mesa library is too old or not in use. You can enable CPU rendering with the environment variable OPEN3D_CPU_RENDERING=true before importing Open3D in Python or running a C++ program. See the tutorial for full details.
• [new] Updated CPU rendering works in Jupyter and Colab environments.
  

GUI

• [update] Vertical and horizontal radio button groups. (contributed by @forrestjgq)
• [new] Keypress event callback for windows. (contributed by @forrestjgq)
• [fix] Fix mouse event filter issue in popup menu covered area (contributed by @forrestjgq)
• [new] Added extra VisualizerWithVertexSelection point picking functions to python API (contributed by @d-walsh and @cansik)
• [update] Support double click mouse event in WebRTC. (contributed by @forrestjgq)
• [fix] Fix Point Clouds Getting Culled Prematurely.
• [fix] Set correct default material (lit with flat shading) for triangle mesh with triangle normals.
• [update] Add interface for get cropped geometry from VisualizerWithEditing class (contributed by @yuecideng).

IO

• [new] Open3D now implements a distortion and noise model from the Redwood dataset. This will enable you to simulate the output of realistic depth sensors.

Original	With noise + distortion	Difference

• [fix] Fix gltf PBR model load regression when loading through ASSIMP.
• [new] Read ply, stl, obj, off, gltf, glb, fbx file formats directly into a Tensor TriangleMesh.

Reconstruction

• [update] Update ICP interface to get iteration wise loss information.
• [new] Online SLAM Example with Realsense sensor support, bagfile support, and saved dataset support.
• [new] Legacy offline reconstruction C++ pipeline (contributed by @yuecideng)
• [update] InitializePointCloudForColoredICP efficiency improvement. (contributed by @Xiang-Zeng)

Open3D-ML

• [update] Upgrade PyTorch to v1.12 and TensorFlow to v2.8.2. With the new PyTorch version, Open3D PyTorch ops work directly with standard PyTorch wheels and custom wheels are not required any more.
• [fix] Avoid logging.basicConfig in non-scripts (contributed by @ajprax)
• [fix] filter_valid_label: force scores tensor onto the selected device (contributed by @ntw-au)
• [fix] Ensure that a checkpoint is saved on the final training epoch (contributed by @ntw-au)
• [fix] Handle nested dictionaries when merging configs (contributed by @ntw-au)
• [fix] Fix moving zip files (contributed by @tejaswid)
• [fix] Update RandLANet weights link (contributed by @Krupal09)
• [fix] Getting rid of torch._six (for future PyTorch versions) (contributed by @krshrimali)

Acknowledgments

We would like to thank all of our community contributors for their true labor of love for this release!

@ntw-au, @jdavidberger, @Xiang-Zeng, @jamesdi1993, @brentyi, @jjabo, @jbotsch-fy, @scimad, @cansik, @NobuoTsukamoto, @theNded, @chunibyo-wly, @jmherzog-de, @luzpaz, @code-review-doctor, @d-walsh, @johnthagen, @pmokeev, @erbensley, @hanzheteng, @chrockey, @agrellRepli, @bchretien, @nigels-com, @forrestjgq, @equant, @naruarjun, @ajprax, @INF800, @ntw-au, @tejaswid, @Krupal09, @krshrimali

Also thanks to the many others who helped the Open3D community by reporting as well as resolving issues.

Open3D 0.15 Release Notes

We are excited to bring you the best Open3D yet - version 0.15. Take a look below:

Starting from this release, we adopt a "tick-tock" model for balancing resolving issues vs. adding new features. In a nutshell, the "tick" releases are focused on resolving existing issues and eliminating bugs, while the "tock" releases mainly focus on developing new features. Open3D 0.15 is a "tick" release. We resolved over 500 issues for Open3D and Open3D-ML, as the infographic below illustrates.

Google Summer of Code 2022

Open3D has applied for the Google Summer of Code 2022 to increase community participation. Check out details and our project ideas here. Please help in making Open3D better for all.

Build System

• [New] We now provide Open3D binary packages for C++ users. No need to “Build from Source” - just download a binary package for your Operating System and use it “out of the box”. See GitHub releases for v0.15 and getting started guides for the latest development package.
• [New] Docker build tools to build ARM64 Linux wheels and multiple Python versions. See ARM64 build guide for more details.
• [New] Pre-compiled Open3D wheel for ARM64 Linux and macOS. Improved Apple Silicon support. Install Open3D on ARM64 Linux and macOS with pip install open3d.
• [Update] Open3D now builds with the new CXX11 ABI by default on Linux. Set -DGLIBCXX_USE_CXX11_ABI=OFF in cmake if you need the old ABI, e.g. to work with PyTorch / TensorFlow libraries.
• [Update] Starting with version 0.15, Open3D Conda packages are no longer supported. Install Open3D with pip install open3d inside a Conda virtual environment.

Core

Datasets

• [New] Dataset module for automatically downloading and managing example data. The following example demonstrates how to create a Dataset object, extract its path, and display it in the Open3D Visualizer:
  

  import open3d as o3d
  
  if __name__ == "__main__":
      dataset = o3d.data.EaglePointCloud()
      pcd = o3d.io.read_point_cloud(dataset.path)
      o3d.visualization.draw(pcd)

  #include 
  #include 
  #include "open3d/Open3D.h"
  
  int main() {
      using namespace open3d;
  
      data::EaglePointCloud dataset;
      auto pcd = io::CreatePointCloudFromFile(dataset.GetPath());
      visualization::Draw({pcd});
  
      return 0;
  }

Command Line Interface (CLI)

• [New] Open3D-dedicated Command Line Interface (CLI) for visualization and running Python examples. Below is a code snippet to get started with Open3D and its examples.

  # Install Open3D pip package
  pip install open3d
  
  # Print help
  open3d --help
  
  # List all runnable examples
  open3d example --list
  
  # Print source code of an example
  open3d example --show [category]/[example_name]
  
  # Run an example
  open3d example [category]/[example_name]
  
  # Run Open3D Viewer
  open3d draw
  
  # Open a mesh or point cloud file in Open3D Viewer
  open3d draw [filename]

  

• [Update] Python examples directory has been refactored for better namespace consistency and new examples have been added.

Neighbor search

• [Update] Updated neighbor search module. The neighbor search module is now equipped with highly optimized built-in implementations of all search methods (Knn, Radius, and Hybrid) supporting both CPU and GPU. Faiss build dependency is removed.

Visualization and GUI

• [New] Introducing raw mode visualization. The raw mode automatically simplifies the lighting environment and object materials to make it easier to inspect the underlying geometry of point clouds and triangle meshes.
  

• [New] Open3D new visualizer now features CPU Rendering based on Mesa’s LLVMpipe software OpenGL implementation. Interactive applications, demos, and Python scripts are all supported as well as off-screen rendering. This feature can also be used within Docker.

  • Method 1: LD_PRELOAD from the command line

    LD_PRELOAD=/home/open3d/development/mesa-21.3.4/libGL.so python examples/python/visualization/draw.py

  • Method 2: Preload library in Python

    import ctypes
    ctypes.cdll.LoadLibrary('/home/open3d/development/mesa-21.3.4/libGL.so')
    import open3d as o3d
    
    mesh = o3d.io.read_triangle_model('/home/open3d/development/FlightHelmet/FlightHelmet.gltf')
    o3d.visualization.draw(mesh)

    

• [New] WidgetProxy and WidgetStack widgets allow the creation of user interfaces on the fly (contributed by @forrestjgq).
• [New] Background color option for button widgets (contributed by @forrestjgq).
• [New] Set maximum visible items in a list widget to prevent lists from growing too large (contributed by @forrestjgq).
• [New] Function keys are now bindable (contributed by @forrestjgq).
• [New] Support for specifying intrinsic projection matrix in the new visualizer.
• [New] Add support for scaling 3D labels.
• [Fix] Open3D for TensorBoard plugin does not need Open3D-ML now.
• [Fix] Point picking, text input, and UI layout (contributed by @forrestjgq).

Geometry

• [Fix] Oriented Bounding Box
  • Fixed an issue where the orientation of the OrientedBoundingBox was mirrored.
  • [New] added a new parameter for robust oriented bounding box computation for degenerated point clouds.
• [Fix] Convex hull meshes created from point clouds now have outward-pointing triangles.
• [Update] Added a new parameter for robust convex hull computation.
• [Update] TriangleMesh GetSelfIntersectingTriangles() and related functions like IsWatertight(), GetVolume(), etc. are now more than 4 times faster.
• [Fix] Corrected an issue with io::AddTrianglesByEarClipping() where the algorithm could fail for concave polygons.
• [New] New Python examples for reconstruction and voxelization.
• [Fix] Improved logger performance.

Open3D-ML

• [New] MIT-licensed implementation of RandLANet.
  
• [New] Intel OpenVINO inference backend (contributed by @dkurt).
• [Fix] Fixed an issue with S3DIS where the loss gets NaN after a few epochs.
• [Fix] Fixed an issue with IoU calculation which fails for large point clouds while running inference in patches.
• [Fix] Fixed an issue where the labels are not correctly ordered in the visualizer.
• [New] Support for Images in Dataset Visualizer (contributed by @ajinkyakhoche).
  

Acknowledgment

We would like to thank all of our community contributors for their true labor of love for this release!

@ajinkyakhoche @ceroytres @chunibyo-wly @dkurt @forrestjgq @Fuhrmann-sep @jeertmans @junha-l @mag-sruehl @maxim0815 @Nicholas-Mitchell @nigels-com @NobuoTsukamoto @ntw-au @roehling @theNded

Also thanks to the many others who helped the Open3D community by reporting as well as resolving issues.