Open3D 0.12.0, the last release of 2020

Open3D 0.12.0 Release Notes

Open3D 0.12.0 is out, and it comes with new 3D object detection pipelines and datasets, newest versions of some of your preferred classic tools, and many bug fixes.

Keep reading for a summary of the most relevant features introduced in this release:

Extensions to the Open3D-ML module

The previous release of Open3D introduced an exciting new module dedicated to 3D Machine Learning Open3D-ML, featuring support for 3D semantic segmentation workflows. In this release, we have extended Open3D-ML with the task of 3D object detection. This extension introduces support for new datasets, such as the Waymo Open dataset, Lyft level 5 open data, Argoverse, nuScenes, and KITTI. As always, all these datasets can be visualized out-of-the-box using our visualization tool, from Python or C++. The visualization tool is now equipped with the capability to render 3D bounding boxes along with all the previously existing modalities, e.g. semantic labels, XYZRGB, depth, normals, etc.

Open3D-ML features

PointPillars, the first of the many object detection models to come in the near future. To enable the implementation of PointPillars, we have added a set of new ML operators in Open3D, such as: grid_sampling, NMS, and IOU. These operators are available to the community and can be used to build new models, using our Python and C++ APIs.

import os
import open3d.ml as _ml3d
import open3d.ml.torch as ml3d

cfg_file = "ml3d/configs/pointpillars_kitti.yml"
cfg = _ml3d.utils.Config.load_from_file(cfg_file)

model = ml3d.models.PointPillars(**cfg.model)
cfg.dataset['dataset_path'] = "/path/to/your/dataset"
dataset = ml3d.datasets.KITTI(cfg.dataset.pop('dataset_path', None), **cfg.dataset)
pipeline = ml3d.pipelines.ObjectDetection(model, dataset=dataset, device="gpu", **cfg.pipeline)

... 
# run inference on a single example.
result = pipeline.run_inference(data)

We have also updated our model zoo, providing new pretrained models on KITTI for the task of 3D object detection, and new semantic segmentation models on Paris-Lille3D and Semantic3D.

Remember that all the tools provided in Open3D-ML are compatible with PyTorch and TensorFlow!

Support for RealSense SDK v2

RealSense sensors’ support has been upgraded to leverage the RealSense SDK v2. Users can now capture crisp 3D data from L515 devices. As part of this upgrade, we include support for Bag files format (RSBagReader), and direct streaming from sensors. These operations can now be done through a new sensor class: RealSenseSensor, offering a simple and intuitive way to control your sensors.

import open3d as o3d
bag_reader = o3d.t.io.RSBagReader()
bag_reader.open(bag_filename)
while not bag_reader.is_eof():
    im_rgbd = bag_reader.next_frame()
    # process im_rgbd.depth and im_rgbd.color

bag_reader.close()
import json
import open3d as o3d
with open(config_filename) as cf:
    rs_cfg = o3d.t.io.RealSenseSensorConfig(json.load(cf))

rs = o3d.t.io.RealSenseSensor()
rs.init_sensor(rs_cfg, 0, bag_filename)
rs.start_capture(True)  # true: start recording with capture
for fid in range(150):
    im_rgbd = rs.capture_frame(True, True)  # wait for frames and align them
    # process im_rgbd.depth and im_rgbd.color

rs.stop_capture()

realsense_open3d

For further information, check this tutorial.

CORE and 3D reconstruction

Open3D 0.12 brings exciting CORE upgrades, including a new Neighbor Search module. This module supports typical neighbor search methods, such as KNN, radius search, and hybrid search, on both CPUs and GPUs, under a common interface!

Furthermore, we have created a new version of the TSDF integration algorithm accelerated on GPU. This version is able to achieve an outstanding computational performance, requiring between 2 and 4 ms to integrate a pair of frames.

New rendering functionalities

We have done an important effort over the last months to put out a modern, real-time, rendering API. This effort is still ongoing, and we are committed to bring top-tier rendering capabilities with a strong emphasis in performance, versatility, ease of use, and beauty. As part of our commitment, in this release we have added relevant extensions to this API:

  • Support for Screen-space reflections

monkey2

  • Full programmatic support for headless rendering in Filament (for real)
box = o3d.geometry.TriangleMesh.create_box(2, 2, 1)
render = rendering.OffscreenRenderer(640, 480)
render.scene.add_geometry("box", box, grey)
render.scene.camera.look_at([0, 0, 0], [0, 10, 0], [0, 0, 1])
img = render.render_to_image()
  • Support for arbitrary camera intrinsic matrices: A small step for the Camera class; a very anticipated step by the Open3D community
Camera::SetProjection(const Eigen::Matrix3d& intrinsics,
                               double near,
                               double far,
                               double width,
                               double height)
  • Support for text rendering: Render text in 3D
Label3D::Label3D(const Eigen::Vector3f& pos, const char* text)
  • Full control over the color grading pipeline
class ColorGradingParams {
public:
    ColorGradingParams(Quality q, ToneMapping algorithm);

    void SetTemperature(float temperature);
    void SetTint(float tint);
    void SetContrast(float contrast);
    void SetVibrance(float vibrance);
    void SetSaturation(float saturation);
    void SetChannelMixer(const Eigen::Vector3f& red,
                         const Eigen::Vector3f& green,
                         const Eigen::Vector3f& blue);
    void SetShadowMidtoneHighlights(const Eigen::Vector4f& shadows,
                                    const Eigen::Vector4f& midtones,
                                    const Eigen::Vector4f& highlights,
                                    const Eigen::Vector4f& ranges);
    void SetSlopeOffsetPower(const Eigen::Vector3f& slope,
                             const Eigen::Vector3f& offset,
                             const Eigen::Vector3f& power);

    void SetCurves(const Eigen::Vector3f& shadow_gamma,
                   const Eigen::Vector3f& midpoint,
                   const Eigen::Vector3f& highlight_scale);
}

Control shadow behaviors and post-processing effects:

class View
{
    void SetPostProcessing(bool enabled);
    void SetAmbientOcclusion(bool enabled, bool ssct_enabled);
    void SetAntiAliasing(bool enabled, bool temporal);
    void SetShadowing(bool enabled, ShadowType type);
}

Visualization and GUI: O3DViewer (beta)

The visualization module has been extended, using the new rendering capabilities and the GUI API, to create a unified visualizer displaying all the features contained in previous Open3D visualizers, e.g., camera animation, data selection, support for callbacks, and multiple shading modes.

fireflies

This new visualizer, codename O3DViewer, will be the official visualization tool in Open3D starting in Open3D 0.14. At that time, previous visualizers will be deprecated.

o3dviewer

We hope you find Open3D 0.12.0 exciting and useful. Happy coding!

Remember that you can reach out with questions, requests, or feedback through the following channels:

Find the full change log keep reading.

The Open3D team

Deprecating

  • All visualization tools, such as draw_geometries will be deprecated in Open3D 0.14 in favor of the new O3DViewer.
  • The Open3D 0.12 packages will be the last to support TensorFlow 2.3 and PyTorch 1.6 with CUDA 10.1. We will update to newer versions of these toolkits in the Open3D 0.13 packages. Note that this does not affect binary compatibility if Open3D is built from source.

Changes to Open3D-ML

  • Fixed infinte dataset iteration (#184)
  • Update readme and config files for parislille3d; align points for parislille3d (#180)
  • Disable data augmentation while testing. (#181)
  • Fixed absolute path bug (#182)
  • Add Data Augmentation (#178)
  • PointPillars bug fixes (#179)
  • Filter KITTI point cloud (#177)
  • Point pillars metrics (#172)
  • Add wide lines (#176)
  • Fix for changes in t::geometry (#173)
  • Added ShapeNet dataset (#157)
  • Added weight initialization (#174)
  • Update model zoo (#175)
  • New validation for torch (#169)
  • Point pillars train tf (#171)
  • Point pillars train (#170)
  • Readme randlanet semantic3d (#167)
  • Fixes for changes from TensorList to Tesor for t.geometry objects (#161)
  • Add Tensorflow model and inference pipeline. (#159)
  • Fix broken link to torch RandLA-Net Toronto 3d model (#163)
  • Add comments for visualize predictions (#151)
  • PointPillars inference pipeline (#153)
  • Link to kpconv parislille3d models in readme (#160)
  • Added Agroverse 3D Dataset (#155)
  • Change Bounding box class. (#149)
  • Add bounding boxes to visualizer (#140)
  • Add Download scripts (#145)
  • Add Lyft Dataset (#138)
  • Create sampler class for sampling point cloud idx and points idx (#135)
  • Add NuScenes dataset (#137)
  • Add Waymo Dataset (#136)
  • Add Kitti object detection dataset (#128)

Changes to Open3D

  • Added python bindings, improved descriptiveness of repr for bounding boxes, and added BG options to draw() (#2785)
  • Fix aspect ratios of background images in certain situations (#2783)
  • Add 3D Labels to SceneWidget (#2781)
  • ICP Point to Plane and Point to Point Registration and Example (#2743)
  • Make sure projection flags are always properly initialized (#2782)
  • RealSense sensor configuration, live capture and recording (#2748)
  • CUDA/CPU compatible TSDF integration (#2710)
  • Added animation callbacks (also some pybinds for rendering::Scene) (#2769)
  • Add ability to add mouse and key callbacks to SceneWidget in Python (#2776)
  • Expose Additional Filament features via View class (#2755)
  • Add support for SSR (via refraction material) (#2768)
  • Added compatible draw() function for external visualizer (#2772)
  • Add light entities to filament scene (#2771)
  • Fix memcpy issue for nms cuda (#2770)
  • Fix background not being drawn in some camera orientations (#2764)
  • Add shader for unlit with transparency (#2760)
  • RealSense SDK v2 (#2646)
  • Better picking, fixes crash removing last selection set (#2746)
  • Support multiple non-sun directional lights (#2759)
  • Add rpc interface to the new draw function (#2734)
  • Fix out of bounds memory access in ragged_to_dense op (#2751)
  • Enable HybridSearch on GPU (#2752)
  • Fix undefined reference in Debug build with gcc (#2744)
  • Expose Filament's Color Grading Controls (#2737)
  • O3DVisualizer can accept background images (#2735)
  • Fix build for BUILD_GUI=OFF SHARED_LIBRARIES=ON (#2745)
  • IoU op for BEV and 3D (#2742)
  • Add replacement for draw_geometries() (#2585)
  • Included missing header files (#2137)
  • Tensor transform pointcloud (#2704)
  • Added ability to set background image (#2701)
  • Modernize Hashmap (#2676)
  • Moved Line3D and Segment3D Transform implementation to cpp file (#2690)
  • Docs: Minor tweaks (README, bib ref, notebook instructions, some directives) (#2534)
  • Alt-Enter to toggle Fullscreen mode (#1939)
  • Faster mesh and pcd conversion (#2719)
  • Remove 16.04 kinect workaround (#2718)
  • Use assimp to load STL, enable ASCII read (#2506)
  • Fixes a second button with same name not working (#2696)
  • Add shader and supporting code for variable line width line sets (#2678)
  • Remove filament-only arm64 build (#2708)
  • Connect FixedRadiusSearch to NearestNeighborSearch (#2680)
  • Geometry stores Tensor instead of TensorList (#2692)
  • Update pybind11 and tinyobjloader version (#2703)
  • FixedRadiusSearch in core::nns (#2582)
  • Simplify macos dependencies (#2666)
  • Fix windows warning (as error) on master (#2691)
  • Fix warnings and link problems on windows for rpc interface (#2661)
  • Integration of Custom Filament Headless Backend (#2572)
  • RPC receiver for visualizer (#2182)
  • Added support for international characters in the GUI (#2655)
  • Cmake-3.17 required (#2665)
  • Build wheel with Kinect support on CI (#2648)
  • Write point cloud with custom attributes (#2594)
  • Convert half_edge_mesh.py to jupyter notebook (#2387)
  • Integrate Faiss to NearestNeighborSearch (#2400)
  • Update RANSAC (#2636)
  • Fix for implicit capture of this in cpp20 (#2638)
  • Nms op for pytorch and tensorflow (#2615)
  • Fixes for LGTM warnings (#2629)
  • Substitute set-env with environment files (#2631)
  • Modify shader to work with Filament's new inverse z buffer (#2626)
  • Voxelize op for torch and tensorflow (#2607)
  • Fix crash on exit if Python variable is assigned a Open3DScene (#2618)
  • Fixed shape check for faces and lines in SetMeshData (#2623)
  • Add function to Open3DScene to allow modifying one geometry's material, and to see if the scene has a geometry (#2612)
  • Create axis geometry on-demand when shown rather than with each geometry change (#2617)
  • Allow arbitrary pinhole camera (#2564)
  • Automatically cast int extents to the right type (#2590)
  • Log non-fatal errors in Visualizer as warnings instead of errors (#2579)
  • Update to match material handling of old OBJ file loader (#2576)
  • Add support for 4 channel (RGBA) color images to CreateFromRGBDImage (#2577)
  • Add pybind for triangle_material_ids (#2573)
  • Fix reading float from PCD, instead of float_t (#2563)
  • Fix string to boolean (#2574)
  • Fix reconstruction system (#2567)
  • Fixed incomplete function call. (#2562)
  • Change variable TriangleMeshSimplification.cpp (#2542)
  • Minor documentation fix (#2554)
  • Faiss Build Test (#2382)
  • Tensor ply read (#2377)
  • Give sheen a default value (i.e., don't leave it unitialized) (#2532)
  • Replace the non utf-8 char “” with "" (#2535)
  • Simplified install instructions for Open3D-ML (#2513)
  • Fix Open3DScene with no background color set not clearing on draw on macOS (#2530)
  • Added example to add geometry to a scene using the new GUI. (#2515)
  • Fixes bug for preloading libc++ and libc++abi in Python (#2527)
  • Allow Scene::AddGeometry to accept bounding boxes (#2520)
  • Add Filament 1.9.5 (#2517)
  • Remove accidental O(n^2) adding of geometries in draw_geometries resulting from a bad merge conflict resolution in d5b95454b (#2523)
  • Add robust kernel colorized icp (#2497) (Nacho)
  • Add miniconda build (#2477)
  • Add dummy texture coordinates for triangle meshes with no texture coordinates to avoid error messages. (#2514)
  • Fix z-buffering problem with labels/gradients in ml3d visualizer (#2512)
  • Better error msg for cuda mismatch (#2503)
  • Fix ambiguous transform (#2491)
  • Workflow dispatch fixes (#2500)
  • Test import of torch and tensorflow (#2496)
  • Fix deadlock with ml3d visualizer on macOS (#2502)
  • Removed shared_ptr as the pybind holder of GUI objects to prevent crashes caused by Python keeping the object alive after Filament resources are destroyed on the Python side. (#2501)
  • Change pybind's run() to not cleanup at the end. (#2499)
  • Added code to fix 'from' import statements in the ml namespaces (#2492)
  • Add MK docs pointer in readme and rst (#2486)
  • Rework Python API for creating windows to avoid crash on exit in some simple situations. (#2485)
  • Documentation build fix (Jupyter + Python visualization.{gui,rendering}) (#2484)

Open3D 0.11.0 is full of features

Open3D 0.11.0 Release Notes

We are excited to present Open3D 0.11.0!

Open3D 0.11.0 introduces a brand new 3D Machine Learning module, nicknamed Open3D-ML. Open3D-ML is an extension of your favorite library to bring support for 3D domain-specific operators, models, algorithms, and datasets. In a nutshell, users can now create new applications combining the power of 3D data and state-of-the-art neural networks! Open3D-ML is included in all the binary releases of Open3D 0.11.0.

Open3D-ML comes with support for Pytorch +1.4 and TensorFlow +2.2, the two most popular machine learning frameworks. The first iteration of this module features a 3D semantic segmentation toolset, including training and inference capabilities for RandlaNet and KPConv. The toolset supports popular datasets such as SemanticKITTI, Semantic3D, 3D Semantic Parsing of Large-Scale Indoor Spaces S3DIS, Toronto3D, and Paris-Lille-3D. Open3D-ML also provides a new model zoo compatible with Pytorch and TensorFlow, so that users can enjoy state-of-the-art semantic segmentation models without hassles.

We have endowed the new Open3D-ML module with a new data viewer tool. Users can now inspect their datasets and model’s predictions in an intuitive and simple way. This visualization tool includes support for Pytorch and TensorFlow frameworks and is fully customizable due to its Pythonic nature.

This viewer has been built upon the new visualization API, integrating the new Rendering and GUI modules. Thanks to the new visualization API, users can perform advanced rendering, fully programmatically from Python and C++. Users can also create slick GUIs with a few lines of Python code. Check out how to do this here.

The Open3D app has also been extended to include the following features:

  • Support for FBX and glTF2 assets
  • Full support for PBR models.

Open3D 0.11 includes for the first time support for Linux ARM (64-bit) platforms. This has been a long-time requested feature that finally made it into the release. You can now enjoy all Open3D features, including our new rendering and visualization pipelines in OpenGL-enabled ARM platform.

[Breaking] Please, notice that the API and the structure of Open3D have changed considerably after an intense refactoring process. You will need to update your code to use the new namespaces. Please, check the full changelog and the documentation for further information.

We hope you find Open3D 0.11.0 exciting and useful. Happy coding!

Remember that you can reach out with questions, requests, or feedback through the following channels:

The Open3D team

Legend:

  • [Added]: Used to indicate the addition of new features
  • [Changed]: Updates of existing functionalities
  • [Deprecated]: Functionalities / features removed in future releases
  • [Removed]: Functionalities/features removed in this release
  • [Fixed]: For any bug fixes
  • [Breaking] This functionality breaks the previous API and you need to check your code

Installation and project structure

  • [Added] fetch Filament with CMake FetchContent (#2085)
  • [Added] speeds up compilation by caching 3rdparty downloads (#2155)
  • [Added] Show STATIC_WINDOWS_RUNTIME in cmake summary when configuring for Windows (#2176)
  • [Added] Move master releases to new bucket for lifecycle management (#2453)
  • [Added] added missing license header in ml module py files (#2333)
  • [Added] add vis namespace (#2394)
  • [Added] Devel wheels for users (#2429)
  • [Added] Build Filament on ARM64 (#2415)
  • [Changed] cmake: pickup python from PYTHON_EXECUTABLE or from PATH (#1923)
  • [Changed] avoid deduplication of the -gencode option (#1960)
  • [Changed] do not link main library when building the tf ops lib because of (#1981)
  • [Changed] do not use system jsoncpp (#2005)
  • [Changed] update Eigen to use the GitLab commit id (#2030)
  • [Changed] update formatter: clang-format-10, yapf 0.30.0 (#2149)
  • [Changed] disable CMP0104 warning (#2175)
  • [Changed] Build examples iteratively on Windows CI (#2199)
  • [Changed] simplify filament build-from-source (#2303)
  • [Changed] set cmake minimum to 3.15 to support generator expressions in (#2392)
  • [Changed] cmake 3.18 required for windows (#2435)
  • [Changed] ubuntu 20.04 build filament from source CI (#2438)
  • [Fixed] turobojpeg windows static runtime (#1876)
  • [Fixed] fix auto & warning (as error) for clang 10 (#1924)
  • [Fixed] Fix Eigen warning when using CUDA (#1926)
  • [Fixed] fix bug in import_3rdparty_library for paths without trailing '/' (#2084)
  • [Fixed] Fix tbb build (#2311)
  • [Fixed] Fix for cmake externalproject_add patch_command (#2431)
  • [Breaking] restructure project directory and namespace (#1970)
  • [Breaking] reorg: opend3d::gui -> open3d::visualization::gui (#1979)
  • [Breaking] change folder case (#1993)
  • [Breaking] Reorg: Added namespace 'rendering' for visualization/rendering (#2002)
  • [Breaking] remove voxel_pooling namespace (#2014)
  • [Breaking] reorg: remove hash_* namespaces (#2025)
  • [Breaking] Rename GLHelper namespace (#2024)
  • [Breaking] Removed visualization::gui::util namespace (#2013)
  • [Breaking] lower case "open3d/3rdparty" intall dir (#2083)
  • [Breaking] refactor pybind namespace (#2249)
  • [Breaking] renamed torch python namespaces (#2330)
  • [Breaking] Move geometry to open3d::t (#2403)
  • [Breaking] move io to open3d::t (#2406)

CORE features and applications

  • [Added] Add cleanup flag in TriangleMesh::select_by_index (#1883)
  • [Added] Orient normals using propagation on MST of Riemannian graph (#1895)
  • [Added] PointCloudIO: UnitTests and Benchmarks (#1891)
  • [Added] expose UniformTSDFVolume's origin in Python API (#1762)
  • [Added] open3d_show_and_abort_on_warning(Core) (#1959)
  • [Added] ml-module (#1958)
  • [Added] add make check-cpp-style, apply-cpp-style (#2016)
  • [Added] ml op test code and torch reduce_subarrays_sum op (#2050)
  • [Added] CUDA header as system header for CMake 3.16 (#2058)
  • [Added] scalar support to more binary ops (#2093)
  • [Added] Tensor api demo (#2067)
  • [Added] Initial tensor-based pointcloud (#2074)
  • [Added] support Tensor.item() in pybind (#2130)
  • [Added] MKL integration with tests (#2128)
  • [Added] Linear algebra module (#2103)
  • [Added] rpc visualization interface (#2090)
  • [Added] Pick the color for all voxels when creating a dense VoxelGrid. (#2150)
  • [Added] Assimp Base integration (#2132)
  • [Added] ISS(Intrinsic Shape Signature) Keypoint Detection Module (#1966)
  • [Added] ask assimp to build zlib (#2188)
  • [Added] initial tensor-based image class (#2161)
  • [Added] Enable CI on Ubuntu 20.04 (focal) with CUDA on GCE (#2308)
  • [Added] ARM CI (#2414)
  • [Added] initial support for tensor-based mesh (#2167)
  • [Added] pybind for tpoincloud (#2229)
  • [Added] Add pybind for Application::PostToMainThread, fix grammar error in comment (#2237)
  • [Added] Tensor for custom object data types (#2244)
  • [Added] Nearest Neighbor module (#2207)
  • [Added] torch wrapper for voxel pooling (#2256)
  • [Added] Support cuda memory cacher (#2212)
  • [Added] ml-contrib subsample library (#2254)
  • [Added] python binding of NN class (junha/nn pybind) (#2246)
  • [Added] contrib neighbor search for ML ops (#2270)
  • [Added] GCE GPU CI docker (PyTorch + cuDNN) (#2211)
  • [Added] Re-add multithreaded performance improvements to ClassIO (#2230)
  • [Added] torch continuous conv wrappers (#2287)
  • [Added] Support Float32 and Float64 neighbor search (#2241)
  • [Added] Layer interface for voxel_pooling op (#2322)
  • [Added] Fast compression mode for png writing (issue #846) (#2325)
  • [Added] Added pybinds for scene, fixed bug with Open3DScene and LOD (#2323)
  • [Added] NanoFlann Parallel Search (#2305)
  • [Added] XYZI format IO with tgeometry::PointCloud (#2356)
  • [Added] Support CPU/GPU hashmap (#2226)
  • [Added] OpenBLAS/LAPACK and support for ARM (#2205)
  • [Added] Added max error threshold (#2411)
  • [Added] Add function to compute the volume of a watertight mesh (#2407)
  • [Added] Ray/line to axis-aligned bounding box intersections (#2358)
  • [Added] IO wrapper for geometry::PointCloud -> t::geometry::PointCloud (#2462)
  • [Added] Nacho/robust kernels (#2425)
  • [Changed] test_color_map.py: adjust rtol to allow enough FP tolerance for OPENMP reordering; add .request to all import urllib (#1897)
  • [Changed] Refactor CMake buildsystem (#1782)
  • [Changed] Refactor pointcloud tests (#1925)
  • [Changed] expose poisson rec threads param (#2035)
  • [Changed] TensorList refactoring and comparison tensor ops (#2066)
  • [Changed] updated internal fields of conv layers to ease debugging (#2104)
  • [Changed] speeds up legacy pointcloud converter (#2216)
  • [Changed] Update TriangleMeshSimplification.cpp (#2192)
  • [Changed] object-oriented dtype (#2208)
  • [Changed] use pybind11's gil management (#2278)
  • [Changed] use link target torch_cpu for cpu builds (#2292)
  • [Changed] make 3rdparty_tbb depend on ext_tbb (#2297)
  • [Changed] Update camera.cpp (#2312)
  • [Changed] Delay cuSOLVER and cuBLAS init so exceptions are transferred to Python. (#2319)
  • [Changed] convert noncontiguous tensors instead of throwing an exception. (#2354)
  • [Changed] Refector some image tests failing on arm simulator (#2393)
  • [Changed] Ensure C++ and Python units tests always run (#2428)
  • [Removed] disable CreateFromPointCloudPoisson test for macos (#2054)
  • [Removed] Remove looking for X11 on macOS (#2334)
  • [Removed] Remove unused variable from SolveJacobianSystemAndObtainExtrinsicMatrix (#2398)
  • [Removed] Nacho/remove openmp guards (#2408)
  • [Fixed] fix coord frame origin bug (#2034)
  • [Fixed] fix utility::LogXX {} escape problem (#2072)
  • [Fixed] Release Python GIL for fast multithreaded IO (#1936)
  • [Fixed] PointToPlane and ColoredICP only require target normal vectors. Fix #2075 (#2118)
  • [Fixed] fixed radius search op for torch (#2101)
  • [Fixed] fix windows python dtype convert (#2277)
  • [Fixed] list pytorch device correctly for pytest (#2304)
  • [Fixed] Fix handling of 4 channel images on PNG export (#2326)
  • [Fixed] Fix for "symbol already registered" (#2324)
  • [Fixed] Slice out-of-range (#2317)
  • [Fixed] fix NanoFlannIndexHolderBase mem leak (#2340)
  • [Fixed] fix c_str() temp address warning (#2336)
  • [Fixed] fix required_gradient propagation for the offset parameter (#2350)
  • [Fixed] -bugfix for float extents for torch cconv layer (#2361)
  • [Fixed] Fix nvidia download error (#2423)
  • [Fixed] fix filament default CLANG_DEFAULT_CXX (#2424)

Rendering and visualization

  • [Added] App: add option to material combobox to restore to original values from file (#1873)
  • [Added] Add support for PNG with alpha channel (#1886)
  • [Added] GUI: implements image widget (#1881)
  • [Added] GUI: added threading and loading dialog to app (#1896)
  • [Added] Integrates point cloud I/O progress into app loading progress bar (#1913)
  • [Added] Adds menu option on macOS to make Open3D viewer default for file types (#2031)
  • [Added] Implements python bindings for gui namespace (#2042)
  • [Added] Added gui::TreeView widget (#2081)
  • [Added] Adds ability to set button padding (#2082)
  • [Added] gui::TreeView now supports arbitrary widgets for its cells (#2105)
  • [Added] Added FBX to Open3DViewer dialog (#2204)
  • [Added] GUI changes for Open3D-ML visualization (#2177)
  • [Added] Enable transparency for lit material (#2239)
  • [Added] Add shader for materials with transparency (#2258)
  • [Added] Unconditionally take base color from MATKEY_COLOR_DIFFUSE (#2265)
  • [Added] Added unlitGradient shader for colormaps and LUT (#2263)
  • [Added] Expose method to update vertex attributes (#2282)
  • [Added] Added ability to change downsample threshold in Open3DScene (#2349)
  • [Added] Faster Filament geometry creation for TPointCloud (sometimes up to 90%) (#2351)
  • [Added] Better algorithm for downsampling (#2355)
  • [Added] Add bounding-box-only mode for rotation. (#2371)
  • [Added] Add "__visualization_scalar" handling to FilamentScene::UpdateGeometry (#2376)
  • [Added] Enable python to render to image (#2413)
  • [Added] Added ability to set the preferred with of gui::NumberEdit (#2373)
  • [Added] Expose caching related functions in Python (#2409)
  • [Added] TPointCloud support for new Scene class (#2213)
  • [Added] Downsample point clouds with by using a different index array (#2318)
  • [Added] Added unlitSolidColor shader (#2352)
  • [Added] Added special name for TPointCloud rendering to create UV arrays from scalar on C++ side (#2363)
  • [Added] Add has_alpha to pybind for Material (#2383)
  • [Added Add alpha to baseColor of shader (and simplify some shader calculations) (#2396)
  • [Changed] update_progress callbacks for ReadPointCloud and WritePointCloud (#1907)
  • [Changed] only update single geometry in Visualizer::AddGeometry and Visualizer::RemoveGeometry (#1945)
  • [Changed] Updated Info.plist file for app to claim it edits and is the default type for the file associations. Also adds .pcd as a supported file type. (#2001)
  • [Changed] overload draw_geometries (#1997)
  • [Changed] Visualization refactor: Open3DViewer and rendering::Scene (#2125)
  • [Changed] Disable MTL file saving for OBJ files with no UVs and textures (issue #1974) (#2164)
  • [Changed] Upgrade Open3D to use Filament 1.8.1 (#2165)
  • [Changed] Improve UI responsiveness when viewing large models (#2384)
  • [Changed] Force scene redraw when layout changes (#2412)
  • [Fixed] Fix window showing buffer from last resize when a window is moved on macOS (#2076)
  • [Fixed] Fixed crash when create an autosizing window with no child windows (which would autosize to (0, 0), which doesn't go over well) (#2098)
  • [Fixed] Fixed hidpi on Linux (#2133)
  • [Fixed] Tell macOS that Python using GUI on macOS works like an app (#2143)
  • [Fixed] Fix GUI to build with Windows (#2153)
  • [Fixed] Model loading and rendering (#2194)
  • [Fixed] Fix rendering anomalies with ao/rough/metal texture (#2243)
  • [Fixed] Fix vis-gui.py not being able to load any geometry (#2273)
  • [Fixed] fix screen rendering in offscreen mode (#2257)
  • [Fixed] Fix 'All' option in file dialog in vis-gui.py (#2274)
  • [Fixed] Fix left side of checkboxes in a treeview on the left not being clickable (#2301)
  • [Fixed] Fix checkboxes in treeview not always redrawing when clicked (#2314)
  • [Fixed] Fix crash on Abandoned FBX model (#2339)
  • [Fixed] Fixes for UI anomalies caused by responsiveness changes (#2397)
  • [Fixed] Force redraw when menu visibility changes (#2416)
  • [Fixed] Fix Scene::SetBackgroundColor() not working if ShowSkybox() not previously called (#2452)
  • [Fixed] Caching Related Bug Fixes (#2451)
  • [Fixed] Clean up Filament to avoid crashes. (#2348)
  • [Removed] Eliminate union in MaterialParameter as it was being used incorrectly (#1879)

Documentation, tutorials, and examples

  • [Added] jupyter tutorial on compute_point_cloud_distance (#1884)
  • [Added] add cmake 3.12+ installation docs (#1914)
  • [Added] Docs updated for build with CUDA (#2055)
  • [Added] improved doc for ml layers and ops (#2296)
  • [Added] added documentation about the ml build options (#2401)
  • [Added] add docs for arm build (#2445)
  • [Changed] Update docker-gui.rst (#1868)
  • [Changed] Docs: disable multiprocessing for sphinx-build (#1951)
  • [Changed] doxygen: fix for reorg; cleanup namespaces (#1975)

Open3D 0.10.0 is out!

Open3D 0.10.0 Release Notes

We are proud to present the 0.10.0 release of Open3D!

For this release, the Open3D team set its focus on the theme of Visualization and Rendering. For starters, we upgraded Open3D rendering capabilities, adding a new real-time renderer based on Filament. This renderer brings support for spatially-varying BRDFs, the Cook-Torrance model, Image-Based Lighting, and Physically-based rendering, among many other improvements. Overall, this translates into a much better rendering quality, endowing 3D models of a higher realism and beauty.

[Warning] As a consequence, we are deprecating the traditional rendering system in favor of the new one. But do not panic, the new rendering system and the traditional system will live together until the 0.12.0 release of Open3D when it will be officially removed from the project.

In order to improve the process of 3D visualization, Open3D has incorporated a new GUI module. It was decided to base the new GUI module on the successful Dear ImGui project due to its compact size and the possibilities of its immediate mode. This new GUI module will help 3D developers to build tailored nice-looking graphical applications with minimum effort.

As an example of what can be done with the GUI module, the Open3D team has developed a new standalone application for 3D visualization, in combination with the new rendering engine. The Open3D 3D visualizer is the quickest and easiest way of making your models look outstanding! Try it yourself by downloading it here (MacOSX).

It is believed that one of the most critical aspects of an open-source project is the quality of its documentation. For this reason, in every release, the team makes a big effort to bring you documentation of the highest quality possible. On this occasion, we decided to upgrade our tutorials to make them interactive. This allows users to directly play and experiment with the concepts presented in each tutorial. The new tutorials, in Jupyter notebook format, can be found here.

Finally, the GPU support that many of you have been requesting has started to make its way into master. In this first step, Open3D brings experimental support for a new Tensor library, that can be transparently used in CPUs and GPUs. We will keep migrating the entire library step-by-step, so please be patient. Feel free to test it out and provide us with feedback.

We hope you find Open3D 0.10.0 useful. Happy coding!

Remember that you can reach out with questions, requests, or feedback through the following channels:

The Open3D team

Legend:

  • [Added]: Used to indicate the addition of new features
  • [Changed]: Updates of existing functionalities
  • [Deprecated]: Functionalities / features removed in future releases
  • [Removed]: Functionalities/features removed in this release
  • [Fixed]: For any bug fixes
  • [Breaking] This functionality breaks the previous API and you need to check your code

Installation and project structure

  • [Added] Enable /bigobj flag for Windows debug build (#1660)
  • [Fixed] Fix brew link problem for python/tbb on OSX for CI (#1440)
  • [Fixed] Windows MSVC compilation warnings (#1663)
  • [Fixed] Typo in make docs (#1806)

CORE features and applications

  • [Added] GPU-enabled Tensor and TensorList (#1399)
  • [Added] Python binding for Tensor and Numpy/Pytorch I/O (#1455)
  • [Added] Added cache variables and code for selecting the gpu arch (#1478)
  • [Added] project_valid_depth_only option to [PointCloudFactory/CreatePointCloud] (#1402)
  • [Added] TensorList pybind (#1505, #1557)
    Thanks, iory
  • [Added] [voxel_down_sample_and_trace] Always return point references (#1406). Thanks, iory
  • [Added] Tensor binary element-wise ops (#1460)
  • [Added] Enable building docs for pure python submodules (#1520)
  • [Added] Option for using the triangle normal for sampled points (#1539)
  • [Added] added get_surface_area to pybind module (#1540)
  • [Added] 3rdparty library cutlass (#1551)
  • [Added] Tensor unary ops sqrt, sin, cos, neg, exp ops (#1532)
  • [Added] Tensor::To for type casting (#1533)
  • [Added] Python slice for Tensor class (#1545)
  • [Added] Proper pybind for Vector2dVector (#1623)
  • [Added] Log error in PointCloud::SegmentPlane() if there are less than ransac n points (#1637)
  • [Added] Tensor::Abs and python bindings (#1620)
  • [Added] Seed argument to mesh sampling functions (#1682) (#1683). Thanks, rowoflo
  • [Added] unified regular and advanced indexing for Tensor, and pybind (#1613)
  • [Added] boolean dtype, logical ops, comparision ops (#1652)
  • [Added] microbenchmark framework (#1696)
  • [Added] Speeds up compilation (#1722)
  • [Added] reduction op support for cpu and gpu (#1730)
  • [Added] support for building TensorFlow ops (#1780)
  • [Added] Tensor creation shortcuts and scalar value op (#1779)
  • [Added] Tensor::Fill explicit specialization, reduce number of outputs (#1801)
  • [Added] Tensor::NonZero and boolean mask advanced indexing (#1681)
  • [Changed] Update travis configuration to support clang-7 and gcc 7.4. (#1471)
  • [Changed] Update VoxelGrid::CarveSilhouette and VoxelGrid::CarveDepthMap to support only carving voxels that project to image (#1441) (#1457). Thanks, jkerfs
  • [Changed] Avoid cstdlib random generators in ransac for global registration (#1486). Thanks, sambarluc
  • [Changed] Boolean expression in GlobalOptimization.cpp (#1509). Thanks, scimad
  • [Changed] Give sensible name for python argument in visualizer.cpp (#1518). Thanks,
  • [Changed] Moved 3rdparty packages down a level to not pollute the include space (#1499). Thanks, tykurtz
  • [Changed] Suppress developer warning for OpenGL preferred library with CMAKE_VERSION>= 3.11 (#1628). Thanks, akashsharma02
  • [Changed] Update tinygltf submodule to fix a bug with exported attributes (#1711). Thanks, mosra
  • [Changed] Binary op output must fit in the output tensor (#1777)
  • [Changed] Changed center parameter in Scale and Rotate (#1774)
  • [Changed] use lowest instead of min for smallest value (#1827)
  • [Fixed] Fix number of edge not showing on verbose message (#1498). Thanks, scimad
  • [Fixed] Bug in update_geometry (#1501). Thanks, Forest75
  • [Fixed] PyTorch import segfault, set CXX_ABI (#1262)
  • [Fixed] Function names remove_non_finite_points and select_by_index (#1630)
  • [Fixed] Alpha shape pybind (#1624)
  • [Fixed] Some cameras are flipped (in the z-axis) when the bounding_box center of the scene is not visible (#1447). Thanks, pablospe
  • [Fixed] SyntaxWarning: replace 'is' with '==' for literals. (#1664). Thanks, hzxie
  • [Fixed] Bug in open3d::geometry::TriangleMesh::ClusterConnectedTriangles (#1669). Thanks, Akella17
  • [Fixed] VoxelGrid.voxels bug #1535 (#1688). Thanks, kosuke55
  • [Fixed] Issue #1689 (#1690). Thanks, akashsharma02
  • [Fixed] Windows warnings (as errors) (#1739)
  • [Fixed] Typos in several Geometry classes (#1773). Thanks, roehling
  • [Fixed] Windows Kinect compile issue (#1796)
  • [Fixed] get_xyz_from_pts interpolation (#1799)
  • [Fixed] colormap segfault and refactor (#1813)
  • [Fixed] Oriented bounding box GetPointIndicesWithinBoundingBox (#1798)

Rendering and visualization

  • [Added] Key callback action event to handle PRESS, RELEASE, REPEAT (#1313). Thanks, sammo2828
  • [Added] Filament integration (#1331, #1512, #1514, #1519, )
  • [Added] GUI module (#1451, #1481, #1495, #1430, #1434, #1435, #1565)
  • [Added] Standalone Visualization application (#1470, #1562, #1572, #1573, #1569, #1583, #1584, #1586, #1585, #1587, #1588, #1591, #1590, #1595, #1593, #1596, #1602, #1601, #1604, #1607, #1608, #1609, #1610, #1611, etc.)
  • [Added] Supports multiple textures in .obj file (#1517)
  • [Fixed] Headless Rendering with traditional system (#1695)

Documentation, tutorials, and examples

  • [Added] Jupyter tutorial for point cloud (#1631, #1640)
  • [Added] Make documentation buildable without Git source tree (#1772). Thanks, roehling
  • [Changed] Contributions guidelines (#1465)
  • [Changed] CPP documentation (#1541)
  • [Fixed] Typos fixed in compilation.rst and visualization.rst (#1476, #1477). Thanks, pauljurczak
  • [Fixed] Minor typo in compilation.rst (#1513). Thanks, scimad
  • [Fixed] Consistency in documentation filenames (#1511). Thanks, scimad
  • [Fixed] filename error in capture_your_own_dataset.rst tutorial (#1525). Thanks, scimad
  • [Fixed] Pass correct parameter into function update_geometry()(#1528). Thanks, Forest75
  • [Fixed] Fix bug in non_blocking_visualization.rst example (#1537). Thanks, scimad
  • [Fixed] Incorrect file name in a ReconstructionSystem document (#1699). Thanks, muskie82

Open3D 0.9.0 is ready!

Open3D 0.9.0 Release Notes

The Open3D team and the Open Source Vision Foundation are proud to present the 0.9.0 release of the Open3D library.

This release brings two exciting algorithms for processing point clouds and meshes: Poisson surface reconstruction and as-rigid-as-possible deformation. You can find a reference for how to use these algorithms in the following examples:

The problems related to headless mode rendering have also been solved in this release, among other bug fixes!

This is the last release of Open3D with support for Python 2.7.

On behalf of the Open3D team, we would like to thank our technical writer, Rohan Rathi and the Google season of docs organization for their help in the critical task of improving Open3D documentation. Your contribution has served to make Open3D more accessible to new users.

We are also happy to announce our new Discourse forum, which becomes the official way to discuss issues related to Open3D.

In an effort to improve our communication with the Open3D community and all the developers that use this library, we have created a new mailing list: open3d-all@osvf.org. Here developers and contributors can obtain first-hand information about the development progress of Open3D, the library’s roadmap, and information on how to proceed to implement new features. You can also join the mailing list via Google groups

Take a look at our documentation Open3D docs to see all the details, and send us feedback at open3d-info@osvf.org. You can also join our Discourse forum or the Discord network to participate in the development discussions.

Enjoy!

The Open3D team


Legend:

  • [Added]: Used to indicate the addition of new features
  • [Changed]: Updates of existing functionalities
  • [Deprecated]: Functionalities / features removed in future releases
  • [Removed]: Functionalities/features removed in this release
  • [Fixed]: For any bug fixes
  • [Breaking] This functionality breaks the previous API and you need to check your code

Installation and project structure

  • [Added] Add optional flag to install-deps-ubuntu.sh script #1191
  • [Changed] Restructure to expose python package at top level #1251
  • [Changed] avoid duplicate -O3 flag at Release build #1280
  • [Changed] CMake Option to use system installed libjpeg-turbo #1294
  • [Fixed] fix link warning #1223
  • [Breaking] Upgrade compiler #1327
  • [Deprecated] Support for Python 2.7

CORE features and applications

  • [Added] Implement image flip functions #1186
  • [Added] Merge close vertices #1189
  • [Added] Support rendering textured triangle mesh #1194
  • [Added] Add some Python init functions to geometry classes #1228
  • [Added] Cluster TriangleMesh Connected Components #1272
  • [Added] Support for Large PTS File IO #1279
  • [Added] Deform TriangleMesh using as-rigid-as-possible #1312
  • [Added] Add geometry without resetting bounding box #1315 (Thanks sammo2828!)
  • [Added] Poisson Surface Reconstruction #1317
  • [Added] Implementation of Alpha Shapes #1320
  • [Added] New ClearGeometries for Visualizer #1338 (Thanks sammo2828!)
  • [Added] Added VisualizerWithVertexSelection #1390
  • [Changed] std::unordered_map for voxelgrid structure #1180
  • [Changed] New Mesh Class Structure #1184
  • [Changed] Warning if no normals in ColoredICP #1187
  • [Changed] Update type check for open3d PointCloud #1193
  • [Changed] TexturedTriangleMesh clean up #1208
  • [Changed] LogFatal to throw exception rather than exit() #1209
  • [Changed] OrientedBoundingBox parameter update and crop #1218
  • [Changed] Polish global optimization #1224
  • [Changed] Proper LogError and LogWarning #1237
  • [Changed] Refactor Image::Prepare() in RealSense.cpp #1277
  • [Changed] Expose get_rotation_matrixfrom* directly in open3d.geometry #1355
  • [Changed] Expose RenderOption.mesh_show_wireframe to Python API #1388 (Thanks sammo2828)
  • [Changed] Only update single geometry in Visualizer::AddGeometry and Visualizer::RemoveGeometry #1392 (Thanks sammo2828)
  • [Changed] Improve numeric stability of TriangleTriangle3d #1177
  • [Fixed] fopen wrapper to enable unicode paths in Windows #1190
  • [Fixed] Fix read_rgbd_image in make_fragments.py #1192
  • [Fixed] Fix operator+ of AxisAlignedBoundingBox #1205
  • [Fixed] fixed maybe-uninitialized warning in TetraMesh.cpp #1212
  • [Fixed] Fast Global Registration does not populate fitness, inlier_rmse, and `corres #1001 #1221
  • [Fixed] fix control may reach end warning for VFatal #1222
  • [Fixed] fix control may reach end warning for VError #1288
  • [Fixed] Quick fix to colored icp #1292
  • [Fixed] fix write ply color values clamped #1306
  • [Fixed] osMesa headless rendering fixed #1358
  • [Fixed] Fix BPA memory leak #1363

Documentation, tutorials, and examples

  • [Changed] Improve docs for color map #1270 (Thanks RohanRathi)
  • [Changed] Improve docs for camera #1271(Thanks RohanRathi)
  • [Changed] Improve docs on Bounding Boxes for CPP api #1282 (Thanks RohanRathi)
  • [Changed] Update python bindings & add docs for Bounding Box #1319 (Thanks RohanRathi)
  • [Changed] Add script details in docs #1342 (Thanks RohanRathi)

Open3D 0.4.0 is out!

Open3D Version 0.4.0 (released 2018-10-25)

The Open3D team and the Open Source Vision Foundation (http://www.osvf.org) are proud to announce the release of the 0.4 version of the Open3D library.

This release brings support for RealSense RGB-D sensors to Open3D, enabling functionalities such as real-time RGB-D capturing and a new point cloud viewer. We have also added new documentation and examples using RealSense sensors to create 3D reconstructions.

We are also excited to introduce support for Jupyter notebooks with a brand new WebGL widget to perform advanced 3D visualization from the comfort of your browser.

One of our main goals is to leverage Open3D as a tool to simplify the use of state-of-the-art 3D pipelines like those used in computer vision and machine learning. With this goal in mind, we are proud to introduce the Open3D Ecosystem, a set of repositories that make use of Open3D to create powerful applications. The first member of this ecosystem is Open3D-PointNet [link], a version of the famous machine learning architecture [link] for point cloud classification and semantic segmentation, which is now fully usable from the commodity of Open3D routines.

From a project infrastructure perspective, we just finalized the integration with TravisCI to perform automatic unit testing. This is a large step forward in terms of quality control for Open3D and it will help to make our software more reliable while expediting its development. The team has made an enormous effort to define UnitTests for all the functionalities present in the library, a task that will be concluded in future releases.

Check out our release video [link] to see these functionalities in action!

For a detailed description of all the features of Open3D 0.4, please keep reading. We hope you enjoy this release and hope to hear from you. Please send us feedback at info@open3d.org and join our Discord network [link] to participate in the discussions.

 

Thanks!

The Open3D team


Legend:

  • [Added]: Used to indicate addition of new features
  • [Changed]: Updates of existing functionalities
  • [Deprecated]: Functionalities / features removed in future releases
  • [Removed]: Functionalities / features removed in this release
  • [Fixed]: For any bug fixes
  • [Breaking] This functionality breaks the previous API and you need to check your code

 

Installation and project structure

  • [Added] Automated wheel creation with cmake
  • [Added] Automated make conda-package generation
  • [Added] Support for arm(jetson tx2) platform (Thanks kuonangzhe!)
  • [Fixed] Cmake install to solve problems with 3rd party header path
  • [Fixed] glfw linking issue when built from source
  • [Fixed] Bug in Ubuntu for the default install path

 

CORE features and applications

  • [Added] RealSense support for live synchronized RGB-D capture (Thanks baptiste-mnh and LLDavid)
  • [Added] Integration with Jupyter visualization WebGL widgets
  • [Added] New methods for outliers removal: statistical outlier removal and radius outlier removal (Thanks Nicolas Chaulet!)
  • [Added] Open3D-PointNet repository to Open3D Ecosystem
  • [Fixed] Problem with the ENABLE_HEADLESS_RENDERING flag
  • [Added] “Visible” parameter to Visualizer::CreateVisualizerWindow to enable off-screen windows
  • [Added] Stanford dataset for 3D reconstruction system
  • [Changed] Major update of the 3D reconstruction system to improve usability and robustness

Documentation and tutorials

  • [Added] RealSense support documentation
  • [Added] Jupyter examples to run PointNet using Open3D for point cloud classification

Testing and benchmarking

  • [Added] Consistent Random number generation initialization (compiler independent)
  • [Added] TravisCI support to automatic UnitTest evaluation
  • [Added] PointCloud test cases
  • [Added] TriangleMesh test cases
  • [Added] LineSet test cases
  • [Added] RGBDImage test cases
  • [Added] KDTreeFlann test cases

Open3D 0.3.0 is ready to go!

Open3D Version 0.3.0 (released 2018-09-13)

Open3D is being developed under the auspices of the Open Source Vision Foundation (http://www.osvf.org). The team has been working hard to make Open3D accessible and easy to use.

In this regard, version 0.3.0 brings major features related to library installation, including improved CMake installation for Linux, Mac and Windows with off-the-shelf systems; new installation options using PIP and CONDA for Linux, Mac and Windows; and overall an easier and cleaner installation experience.

We are also continuing to extend the 3D processing and visualization functionality. Among other features, version 0.3 brings support for enhanced 3D reconstruction; extension of TSDF volume integration to floating-point intensity images; and an improved non-blocking visualization tool.

This version also comes with extended and improved documentation. We have enhanced the tutorials on multiway registration, marching cubes, global registration, and headless rendering, among others.

Open3D 0.3.0 also includes our first set of tests to verify the integrity and correctness of the library. You can expect to see much more of this in future releases.

For a detailed description of all the features of Open3D 0.3, please keep reading. We hope you enjoy this release and hope to hear from you. Please send us feedback at info@open3d.org and join our Discord network (https://discord.gg/D35BGvn) to participate in the discussions.

Thanks!

The Open3D team


Legend:

  • [Added]: Used to indicate addition of new features
  • [Changed]: Updates of existing functionalities
  • [Deprecated]: Functionalities / features removed in future releases
  • [Removed]: Functionalities / features removed in this release
  • [Fixed]: For any bug fixes
  • [Breaking] This functionality breaks the previous API and you need to check your code

 

Installation and project structure

  • [Added] PIP installation under Conda environment for Linux, Mac and Windows for Python 2.7, 3.5 and 3.6 [Doc]
  • [Added] CONDA installation for Linux, Mac and Windows for Python 2.7, 3.5 and 3.6 [Doc]
  • [Changed] CMake installation to simplify installation in Linux, Mac and Windows
  • [Changed] Project folder structure
  • [Changed] Subgroups in CMakeLists.txt
  • [Breaking] Namespace three is now open3d, which brakes previous API
  • [Removed] Namespace three removed
  • [Removed] OpenCV dependency
  • [Fixed] Excessive warnings during compilation
  • [Fixed] Library not found problem for -lGLEW on OSX
  • [Fixed]  'jpeglib.h' file not found problem on OSX
  • [Fixed] MacOS libpng linking problem
  • [Fixed] XCode build failure
  • [Fixed] Installation problems on brand-new Ubuntu 16.04 systems
  • [Fixed] Visual Studio 2017 15.8 build problem
  • [Fixed] Problem reinstalling open3d
  • [Fixed] Xcode IDE not producing libOpen3D.a

CORE features and applications

  • [Added] Version file
  • [Added] Feature to change view during non-blocking visualization
  • [Added] Support of rendering 3D line segments by generalization of LineSet and adding a Python binding
  • [Changed] Enhance features for 3D reconstruction system
  • [Changed] Extend TSDF volume integration algorithm to use additional inputs: float intensity Image + depth map
  • [Fixed] Name conflict in CreateWindow
  • [Fixed] Removes warnings in color map optimization
  • [Fixed] Visualization error on Visualizer::CaptureScreenImage (VisualizerRender.cpp)

Documentation and tutorials

  • [Added] Tutorial on how to transform Numpy array into open3d Image
  • [Changed] Tutorial on how to perform multiway registration, showing how to combine multiple point clouds into single point cloud
  • [Changed] New version of the marching cubes mesh tutorial
  • [Changed] New documentation for color_map_optimization
  • [Changed]  Documentation for headless rendering and fast global registration
  • [Changed] Changed code rendering style in documents. The document is also directly linked to the tutorial code, to automatically reflect any future change in tutorial
  • [Fixed] Clarify how to retrieve estimated point cloud normal

Testing and benchmarking

  • [Added] New test for TestRGBDOdometry
  • [Changed] Updated rgbd_odometry test to include camera_primesense.json
  • [Fixed] Bug inTestRealSense.cpp

Misc

  • [Changed] Sort the file list with Alphanumeric order to create global mesh from files

Open3D release 0.2 is here!

Version 0.2 (Release date July 1st, 2018)

The first major update of Open3D, with various additional features and bug fixes.

Visualization

  1. Headless rendering: GLFW 3.3 dev + OSMesa combinations for supporting headless rendering. This feature is especially useful for users who want to get depth/normal/images from a remote server without physical monitors.
  2. Non-blocking visualization: draw_geometries() is a useful function for quick overview of static geometries. However, this function holds process until a visualization window is closed. Non-blocking visualization shows a live update of geometry while the window is open.
  3. Mesh cropping: In v0.1 VisualizerWithEditing only supports point cloud cropping. The new version supports mesh cropping as well.
  4. Point cloud picker with an application of manual point cloud registration

Docker for Open3D: a new Docker CE based solution for utilizing Open3D. With this update, you can:

  1. sandbox Open3D from other applications on a machine.
  2. operate Open3D on a headless machine using VNC or the terminal.
  3. edit the Open3D code on the host side but run it inside an Open3D container.

Additional features:

  1. Fast global registration: Open3D’s implementation of ‘fast global registration’ paper [Zhou et al 2016]. For the task of global registration, the single threaded fast global registration is about 20 times faster than RANSAC based implementation.
  2. Color map optimization: Another interesting application for copying seamless texture map to the reconstructed geometry. This is an implementation of ‘’Color Map Optimization for 3D Reconstruction with Consumer Depth Cameras” paper [Zhou and Koltun 2014].  The optimization pipeline creates sharp texture mapping on the geometry taken with color cameras.
  3. Basic operations for color and depth images: new function that can generate a depth discontinuity mask from a depth image. New dilation operators for making a thicker discontinuity mask.

Enhancement on build system:

  1. Refined CMake build system: polished CMake build system so that it can fully support make install or make uninstall. Once installed, Open3D library is searchable using find_package module in CMake. CMakeList.txt file for the new application is simplified.
  2. PyPi support: Newer version provides pip install which is a more convenient way to begin using Open3D. Try ‘pip install open3d-python’ and ‘import Open3D’ in Python.
  3. Basic test framework: Adding initial support for ‘Gtest’ examples to test Open3D’s functions and classes.

Miscellaneous

  1. Changing Python package name: python package name is changed from py3d to open3d.
  2. Many bug fixes:
    1. Bug fix on the Reconstruction system regarding defining and utilizing information matrix.
    2. Additional materials on Open3D documents.

Open3D release 0.1

Version 0.1 (Release date Feb 22nd, 2018)

First official release. Open3D is an open-source library that supports rapid development of software that deals with 3D data. Open3D has the following core features:

Core principle

  1. Started from scratch.
  2. Focus on implementation of basic widely-used 3D processing algorithms.
  3. Does not depend on heavyweight libraries.
  4. Minimum lines of code
  5. The library can be built and run from source using Ubuntu/MacOSX/Windows systems.

Basic 3D data structures

  1. Open3D provides point cloud, triangle mesh, image, and pose graph data structures.
  2. Each data structure has its own I/O interface with various files.
  3. Supported file formats:
    1. Image: JPEG, PNG.
    2. 3D geometry: Bin, PCD, PLY, PTS, XYZ, XYZN, XYZRGB.
    3. camera trajectory & pose graph: JSON, LOG.

Basic data processing algorithms

  1. Point cloud downsampling, normal estimation, and vertex coloring.
  2. Gaussian and Sobel filter for image processing.
  3. Scalable TSDF volume integration.

Scene reconstruction

  1. Provides basic scene reconstruction system specialized for RGBD sequence.
  2. The system consists of RGBD Odometry, pose graph optimization, and TSDF volume integration.
  3. The integration volume can be scalable.

Surface alignment

  1. Implementation of local geometric feature: ‘Fast Point Feature Histograms (FPFH)’ paper [Rusu and Beetz 2009].
  2. Point-to-point/point-to-plane/colored ICP implementations with OpenMP acceleration.
  3. Provides a basic, RANSAC-based global registration pipeline.
  4. Fast pose graph optimization: implementation of ‘Robust reconstruction of indoor scenes’ paper[Choi et al 2015].
  5. In house convex optimization: Gauss-Netwon and Levenberg-Marquardt methods.

3D visualization

  1. Quick overview of point cloud, mesh, image.
  2. Various options to customize camera path or camera intrinsics.
  3. Color/depth/normal rendering supported.
  4. Providing rendering buffer access to save rendered images.
  5. Can configure custom key callback functions.
  6. Can select a region and crop of crop point cloud.

Python binding

  1. The c++ functions/classes/definitions are exposed to the Python API.
  2. The Python API provides a quick debug cycle for development of novel algorithms.