# open3d.ml.tf.layers.ContinuousConv¶

class open3d.ml.tf.layers.ContinuousConv(*args, **kwargs)

Continuous Convolution.

This convolution supports continuous input and output point positions. This layer implements the convolution defined in

B. Ummenhofer and V. Koltun, Lagrangian Fluid Simulation with Continuous Convolutions, ICLR 2020.

The convolution at position $$\mathbf x$$ is defined as

$(f*g)(\mathbf x) = \frac{1}{\psi(\mathbf x)} \sum_{i \in \mathcal N(\mathbf x, R)} a(\mathbf x_i, \mathbf x)\; f_i\; g(\Lambda(\mathbf x_i - \mathbf x)).$

With $$f$$ as the input feature function and $$g$$ as the filter function. The input points are $$\mathbf x_i$$ and the input features are $$f_i$$. The normalization $$\frac{1}{\psi(\mathbf x)}$$ can be turned on with the normalize parameter. The per neighbor value $$a(\mathbf x_i, \mathbf x)$$ can be used to implement window functions; see parameter window_function. The function $$\Lambda$$ for looking up filter values is defined by the parameters coordinate_mapping and interpolation.

Example

This shows a minimal example of how to use the layer:

import tensorflow as tf
import open3d.ml.tf as ml3d

inp_positions = tf.random.normal([20,3])
inp_features = tf.random.normal([20,8])
out_positions = tf.random.normal([10,3])

conv = ml3d.layers.ContinuousConv(filters=16, kernel_size=[3,3,3])
out_features = conv(inp_features, inp_positions, out_positions, extents=2.0)

Parameters
• filters – The number of filters/output channels.

• kernel_size – The spatial resolution of the filter, e.g. [3,3,3].

• activation – The activation function to use. None means no activation.

• kernel_initializer – Initializer for the kernel weights.

• bias_initializer – Initializer for the bias vector.

• kernel_regularizer – Regularizer for the kernel weights.

• bias_regularizer – Regularizer for the bias vector.

• align_corners – If true then the voxel centers of the outer voxels of the filter array are mapped to the boundary of the filter shape. If false then the boundary of the filter array is mapped to the boundary of the filter shape.

• coordinate_mapping

The mapping that is applied to the input coordinates. One of ‘ball_to_cube_radial’, ‘ball_to_cube_volume_preserving’, ‘identity’.

• ’ball_to_cube_radial’ uses radial stretching to map a sphere to a cube.

• ’ball_to_cube_volume_preserving’ is using a more expensive volume preserving mapping to map a sphere to a cube.

• ’identity’ no mapping is applied to the coordinates.

• interpolation

One of ‘linear’, ‘linear_border’, ‘nearest_neighbor’.

• ’linear’ is trilinear interpolation with coordinate clamping.

• ’linear_border’ uses a zero border if outside the range.

• ’nearest_neighbor’ uses the nearest neighbor instead of interpolation.

• normalize – If true then the result is normalized either by the number of points (neighbors_importance is null) or by the sum of the respective values in neighbors_importance.

• radius_search_ignore_query_points – If true the points that coincide with the center of the search window will be ignored. This excludes the query point if ‘queries’ and ‘points’ are the same point cloud.

• radius_search_metric – Either L1, L2 or Linf. Default is L2

• offset – A single 3D vector used in the filter coordinate computation. The shape is .

• window_function

Optional radial window function to steer the importance of points based on their distance to the center. The input to the function is a 1D tensor of distances (squared distances if radius_search_metric is ‘L2’). The output must be a tensor of the same shape. Example:

def window_fn(r_sqr):
return tf.clip_by_value((1 - r_sqr)**3, 0, 1)


• use_dense_layer_for_center – If True a linear dense layer is used to process the input features for each point. The result is added to the result of the convolution before adding the bias. This option is useful when using even kernel sizes that have no center element and input and output point sets are the same and ‘radius_search_ignore_query_points’ has been set to True.

• dense_kernel_initializer – Initializer for the kernel weights of the linear layer used for the center if ‘use_dense_layer_for_center’ is True.

• dense_kernel_regularizer – Regularizer for the kernel weights of the linear layer used for the center if ‘use_dense_layer_for_center’ is True.

• in_channels – This keyword argument is for compatibility with PyTorch. It is not used and in_channels will be inferred at the first execution of the layer.

__init__(filters, kernel_size, activation=None, use_bias=True, kernel_initializer='uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, align_corners=True, coordinate_mapping='ball_to_cube_radial', interpolation='linear', normalize=True, radius_search_ignore_query_points=False, radius_search_metric='L2', offset=None, window_function=None, use_dense_layer_for_center=False, dense_kernel_initializer='glorot_uniform', dense_kernel_regularizer=None, in_channels=None, **kwargs)
build(inp_features_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call. It is invoked automatically before the first execution of call().

This is typically used to create the weights of Layer subclasses (at the discretion of the subclass implementer).

Parameters

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call(inp_features, inp_positions, out_positions, extents, inp_importance=None, fixed_radius_search_hash_table=None, user_neighbors_index=None, user_neighbors_row_splits=None, user_neighbors_importance=None)

This function computes the output features.

Parameters
• inp_features – A 2D tensor which stores a feature vector for each input point. This argument must be given as a positional argument!

• inp_positions – A 2D tensor with the 3D point positions of each input point. The coordinates for each point is a vector with format [x,y,z].

• out_positions – A 2D tensor with the 3D point positions of each output point. The coordinates for each point is a vector with format [x,y,z].

• extents – The extent defines the spatial size of the filter for each output point. For ‘ball to cube’ coordinate mappings the extent defines the bounding box of the ball. The shape of the tensor is either  or [num output points].

• inp_importance – Optional scalar importance value for each input point.

• fixed_radius_search_hash_table – A precomputed hash table generated with build_spatial_hash_table(). This input can be used to explicitly force the reuse of a hash table in special cases and is usually not needed. Note that the hash table must have been generated with the same ‘points’ array. Note that this parameter is only used if ‘extents’ is a scalar.

• user_neighbors_index – This parameter together with ‘user_neighbors_row_splits’ and ‘user_neighbors_importance’ allows to override the automatic neighbor search. This is the list of neighbor indices for each output point. This is a nested list for which the start and end of each sublist is defined by ‘user_neighbors_row_splits’.

• user_neighbors_row_splits – Defines the start and end of each neighbors list in ‘user_neighbors_index’.

• user_neighbors_importance – Defines a scalar importance value for each element in ‘user_neighbors_index’.

Returns

A tensor of shape [num output points, filters] with the output features.

compute_output_shape(inp_features_shape)

Computes the output shape of the layer.

This method will cause the layer’s state to be built, if that has not happened before. This requires that the layer will later be used with inputs that match the input shape provided here.

Parameters

input_shape – Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns

An input shape tuple.