open3d.visualization.tensorboard_plugin.summary.add_3d#
- open3d.visualization.tensorboard_plugin.summary.add_3d(name, data, step, logdir=None, max_outputs=1, label_to_names=None, description=None)#
Write 3D geometry data as TensorBoard summary for visualization with the Open3D for TensorBoard plugin.
- Parameters:
name (str) – A name or tag for this summary. The summary tag used for TensorBoard will be this name prefixed by any active name scopes.
data (dict) –
A dictionary of tensors representing 3D data. Tensorflow, PyTorch, Numpy and Open3D tensors are supported. The following keys are supported:
vertex_positions
: shape (B, N, 3) where B is the number of point clouds and must be same for each key. N is the number of 3D points. Will be cast tofloat32
.vertex_colors
: shape (B, N, 3) Will be converted touint8
.vertex_normals
: shape (B, N, 3) Will be cast tofloat32
.vertex_texture_uvs
: shape (B, N, 2) Per vertex UV coordinates for applying material texture maps. Will be cast tofloat32
. Only one of[vertex|triangle]_texture_uvs
should be provided.vertex_[FEATURE]
: shape (B, N, _). Store custom vertex features. Floats will be cast tofloat32
and integers toint32
.triangle_indices
: shape (B, Nf, 3). Will be cast touint32
.triangle_colors
: shape (B, Nf, 3) Will be converted touint8
.triangle_normals
: shape (B, Nf, 3) Will be cast tofloat32
.triangle_texture_uvs
: shape (B, Nf, 3, 2) Per triangle UV coordinates for applying material texture maps. Will be cast tofloat32
. Only one of[vertex|triangle]_texture_uvs
should be provided.line_indices
: shape (B, Nl, 2). Will be cast touint32
.bboxes
: shape (B, Nbb). The tensor dtype should be open3d.ml.vis.BoundingBox3D. The boxes will be colored according to their labels in tensorboard. Visualizing confidences is not yet supported. Property references are not supported. Use separate from other 3D data.material_name
: shape (B,) and dtypestr
. Base PBR material name is required to specify any material properties. Open3D built-in materials:defaultLit
,defaultUnlit
,unlitLine
,unlitGradient
,unlitSolidColor
.material_scalar_[PROPERTY]
: Any material scalar property with float values of shape (B,). e.g. To specify the property metallic, use the key material_scalar_metallic.material_vector_[PROPERTY]
: Any material 4-vector property with float values of shape (B, 4) e.g. To specify the property baseColor, use the key material_vector_base_color.material_texture_map_[PROPERTY]
: PBR material texture maps. e.g.material_texture_map_metallic
represents a texture map describing the metallic property for rendering. Values are Tensors with shape (B, Nr, Nc, C), corresponding to a batch of texture maps with C channels and shape (Nr, Nc). The geometry must have[vertex|triangle]_texture_uvs
coordinates to use any texture map.
For batch_size B=1, the tensors may drop a rank (e.g. (N,3)
vertex_positions
, (4,) material vector properties or float scalar () material scalar properties.). Variable sized elements in a batch are also supported. In this case, use a sequence of tensors. For example, to save a batch of 2 point clouds with 8 and 16 points each, data should contain {‘vertex_positions’: (pcd1, pcd2)} where pcd1.shape = (8, 3) and pcd2.shape = (16, 3).Floating point color and texture map data will be clipped to the range [0,1] and converted to
uint8
range [0,255].uint16
data will be compressed to the range [0,255].Any data tensor (with ndim>=2 including batch_size), may be replaced by an
int
scalar referring to a previous step. This allows reusing a previously written property in case that it does not change at different steps. This is not supported formaterial_name
,material_scalar_*PROPERTY*
and custom vertex features.Please see the Filament Materials Guide for a complete description of material properties.
step (int) – Explicit
int64
-castable monotonic step value for this summary. [TensorFlow: IfNone
, this defaults to tf.summary.experimental.get_step(), which must not beNone
.]logdir (str) – The logging directory used to create the SummaryWriter. [PyTorch: This will be automatically inferred if not provided or
None
.]max_outputs (int) – Optional integer. At most this many 3D elements will be emitted at each step. When more than max_outputs 3D elements are provided, the first
max_outputs
3D elements will be used and the rest silently discarded. Use0
to save everything.label_to_names (dict) – Optional mapping from labels (e.g. int used in labels for bboxes or vertices) to category names. Only data from the first step is saved for any tag during a run.
description (str) – Optional long-form description for this summary, as a constant
str
. Markdown is supported. Defaults to empty. Currently unused.
- Returns:
[TensorFlow] True on success, or false if no summary was emitted because no default summary writer was available.
- Raises:
ValueError – if a default writer exists, but no step was provided and tf.summary.experimental.get_step() is None. Also raised when used with Tensorflow and
logdir
is not provided orNone
.RuntimeError – Module level function is used without a TensorFlow installation. Use the PyTorch SummaryWriter.add_3d() bound method instead.
Examples
With Tensorflow:
import tensorflow as tf import open3d as o3d from open3d.visualization.tensorboard_plugin import summary from open3d.visualization.tensorboard_plugin.util import to_dict_batch logdir = "demo_logs/" writer = tf.summary.create_file_writer(logdir) cube = o3d.geometry.TriangleMesh.create_box(1, 2, 4) cube.compute_vertex_normals() colors = [(1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0)] with writer.as_default(): for step in range(3): cube.paint_uniform_color(colors[step]) summary.add_3d('cube', to_dict_batch([cube]), step=step, logdir=logdir)
With PyTorch:
(Note that the import summary is needed to make add_3d() available, even though summary is not used.)
from torch.utils.tensorboard import SummaryWriter import open3d as o3d from open3d.visualization.tensorboard_plugin import summary # noqa from open3d.visualization.tensorboard_plugin.util import to_dict_batch writer = SummaryWriter("demo_logs/") cube = o3d.geometry.TriangleMesh.create_box(1, 2, 4) cube.compute_vertex_normals() colors = [(1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0)] for step in range(3): cube.paint_uniform_color(colors[step]) writer.add_3d('cube', to_dict_batch([cube]), step=step)
Now use
tensorboard --logdir demo_logs
to visualize the 3D data.Note
Summary writing works on all platforms, and the visualization can be accessed from a browser on any platform. Running the tensorboard process is not supported on macOS as yet.