# Ray Casting¶

The RaycastingScene class in Open3D provides basic ray casting functionality. In this tutorial we show how to create a scene and do ray intersection tests.

Initialization

As the first step we initialize a RaycastingScene with one or more triangle meshes.

:

# Load mesh and convert to open3d.t.geometry.TriangleMesh
cube = o3d.geometry.TriangleMesh.create_box().translate([0, 0, 0])
cube = o3d.t.geometry.TriangleMesh.from_legacy(cube)

:

# Create a scene and add the triangle mesh
scene = o3d.t.geometry.RaycastingScene()


add_triangles() returns the ID for the added geometry. This ID can be used to identify which mesh is hit by a ray.

:

print(cube_id)

0


Casting rays

We can now generate rays which are 6D vectors with origin and direction.

:

# We create two rays:
# The first ray starts at (0.5,0.5,10) and has direction (0,0,-1).
# The second ray start at (-1,-1,-1) and has direction (0,0,-1).
rays = o3d.core.Tensor([[0.5, 0.5, 10, 0, 0, -1], [-1, -1, -1, 0, 0, -1]],
dtype=o3d.core.Dtype.Float32)

ans = scene.cast_rays(rays)


The result contains information about a possible intersection with the geometry in the scene.

:

print(ans.keys())

dict_keys(['primitive_uvs', 'primitive_ids', 'geometry_ids', 'primitive_normals', 't_hit'])

• t_hit is the distance to the intersection. The unit is defined by the length of the ray direction. If there is no intersection this is inf

• geometry_ids gives the id of the geometry hit by the ray. If no geometry was hit this is RaycastingScene.INVALID_ID

• primitive_ids is the triangle index of the triangle that was hit or RaycastingScene.INVALID_ID

• primitive_uvs is the barycentric coordinates of the intersection point within the triangle.

• primitive_normals is the normal of the hit triangle.

We can see from t_hit and geometry_ids that the first ray did hit the mesh but the second ray missed.

:

print(ans['t_hit'].numpy(), ans['geometry_ids'].numpy())

[8.999999      inf] [         0 4294967295]


Creating images

We now create a scene with multiple objects

:

# Create meshes and convert to open3d.t.geometry.TriangleMesh
cube = o3d.geometry.TriangleMesh.create_box().translate([0, 0, 0])
cube = o3d.t.geometry.TriangleMesh.from_legacy(cube)
torus = o3d.geometry.TriangleMesh.create_torus().translate([0, 0, 2])
torus = o3d.t.geometry.TriangleMesh.from_legacy(torus)
[1, 2, 3])
sphere = o3d.t.geometry.TriangleMesh.from_legacy(sphere)

scene = o3d.t.geometry.RaycastingScene()


RaycastingScene allows to organize rays with an arbitrary number of leading dimensions. For instance we can generate an array with shape [h,w,6] to organize rays for creating an image. The class also provides helper functions for creating rays for a pinhole camera. The following creates rays Tensor with shape [240,320,6].

:

rays = o3d.t.geometry.RaycastingScene.create_rays_pinhole(
fov_deg=90,
center=[0, 0, 2],
eye=[2, 3, 0],
up=[0, 1, 0],
width_px=640,
height_px=480,
)
# Ee can directly pass the rays tensor to the cast_rays function.
ans = scene.cast_rays(rays)


The output tensors preserve the shape of the rays and we can directly visualize the hit distance with matplotlib to get a depth map.

:

import matplotlib.pyplot as plt
plt.imshow(ans['t_hit'].numpy())

:

<matplotlib.image.AxesImage at 0x7f4e202eddd8> Furhter we can plot the other results to visualize the primitive normals, ..

:

# use abs to avoid negative values
plt.imshow(np.abs(ans['primitive_normals'].numpy()))

:

<matplotlib.image.AxesImage at 0x7f4e201eb0f0> :

plt.imshow(ans['geometry_ids'].numpy(), vmax=3)

:

<matplotlib.image.AxesImage at 0x7f4e20155048> 