Web visualizer and Jupyter¶
Open3D web visualizer enables advanced rendering and visualization of 3D models in modern browsers. You first start the visualization server on a local or remote machine, and then the 3D models can be viewed from any device with a modern browser. The web visualizer server and client communicate via the WebRTC (Web Real-Time Communication) protocol.
In this tutorial, we cover two common use cases:
Standalone mode: Run Open3D web visualizer server as a standalone application and view the 3D models in a browser. It supports both C++ and Python. You’ll only need add one line to your code to enable the server.
Jupyter mode: Run Open3D web visualizer as embedded Jupyter cells. It supports both the Python Jupyter notebook and Jupyter lab environments. You’ll need to modify your code to use the non-blocking APIs and wrappers.
As of the current release, Open3D web visualizer is supported on the following platforms:
Standalone mode server (C++/Python)
Jupyter mode server (Python)
Standalone/Jupyter mode client (web browser)
Additional notes on compatibility:
On Ubuntu, web visualizer server runs on EGL. This is supported in most machines as long as the it has graphics capabilities (either via integrated graphics or discrete GPU). A discrete GPU will be required if the CPU does not come with integrated graphics. For example, AWS cloud instances typically use Intel XEON processors without integrated graphics. You’ll need to add a discrete GPU to the instance and install the graphics driver to enable the web visualizer.
Web visualizer servers are not supported on the ARM platform yet. To get the web visualizer working on ARM, you’ll need to compile the WebRTC library from source for ARM. See the
3rdparty/webrtcfolder for more details.
Google Colab and Kaggle notebook are not supported. You’ll need to run you own Jupyter or JupyterLab server.
In standalone mode, Open3D web visualizer server runs as a standalone application. It supports both C++ and Python. You’ll only need to make very minimal changes to your code to enable the server.
-DBUILD_WEBRTC=ON are set in the CMake config
step (they are set to
ON by default). In your C++ code, enable the WebRTC
server backend by calling:
mkdir build && cd build cmake .. make DrawWebRTC -j$(nproc) cd bin/example ./DrawWebRTC google-chrome http://localhost:8888 # Or, open the address in your browser
Checkout examples/cpp/DrawWebRTC.cpp for the complete source code.
In your Python code, enable the WebRTC server backend by calling:
import open3d as o3d o3d.visualization.webrtc_server.enable_webrtc()
python examples/python/visualization/draw_webrtc.py google-chrome http://localhost:8888 # Or, open the address in your browser
Checkout examples/python/visualization/draw_webrtc.py for the complete source code.
Similar to other web servers, the Open3D web visualizer server allows binding to custom IP address and port. This is typically required for setting up remote servers.
To set custom IP address, set the
WEBRTC_IP environment variable. To set
custom port number, set the
WEBRTC_PORT environment variable. For instance:
# Bind to localhost:8888 (default) python examples/python/visualization/draw_webrtc.py # Bind to 127.0.0.1:8889 WEBRTC_IP=127.0.0.1 WEBRTC_PORT=8889 python draw_webrtc.py
To enable remote visualization, we typically bind to the internal IP and
internal port as seen by the server. The router may translate the internal IP
and internal port to external IP and port. For instance, on Google cloud, we
can get the internal and external IP.
of the instance. We then set the
WEBRTC_PORT to the
internal IP and port, enable the corresponding port in the firewall rules and
enable HTTP traffic. Finally, we start the web visualizer server and visit the
external IP address from the browser.
Although the WebRTC traffic is encrypted, the web server uses HTTP by default which is not encrypted and is suitable only for local access. If used for remote visualization, the web page will be visible and controllable by any one with network access. For sensitive or confidential data, please rebuild with appropriate web server configuration and SSL certificates.
Install Jupyter with:
pip install jupyter jupyter notebook
or, you may also install JupyterLab instead:
pip install jupyterlab jupyter-lab
Then, run the example notebook docs/jupyter/visualization/jupyter_visualization.ipynb.
Unlike standalone mode,
WEBRTC_PORT are not used in
Jupyter mode. However, you might want to
specify IP and port for Jupyter.
For remote visualization, please use a secure Jupyter server.
In Jupyter mode, we need to avoid blocking visualization API calls.
Otherwise, running visualization in one cell will block the execution of the
next cell. We provide Jupyter-specific helper functions to achieve non-blocking
visualization. For instance,
open3d.web_visualizer.draw is used instead of
open3d.visualization.draw in the example notebook above.
draw, you may also create your own non-blocking visualization helper
functions with the
_AsyncEventLoop class. See
python/open3d/web_visualizer.py for the
_AsyncEventLoop class and
the implementation of
open3d.web_visualizer.draw as an example.
Build Jupyter package from source¶
Open3D official Python wheels come with Jupyter web visualizer support. To build Open3D Python package from source with Jupyter web visualizer, you’ll need to :
yarn. For example, you may install npm via a package manager, and install
npm install -g yarn. Make sure they can be called from the terminal.
Install Jupyter build dependencies:
pip install -r python/requirements_jupyter_build.txt
Build Open3D Python package:
mkdir build && cd build cmake -DBUILD_JUPYTER_EXTENSION .. make install-pip-package -j$(nproc)
Advanced topic: local server in airplane mode¶
When the computer has no active network interfaces (e.g. Wi-Fi is turned off and
ethernet is unplugged, the machine only has the loopback
WebRTC may fail to work. In this case, we need to create a dummy interface.
The workaround is tested on Ubuntu.
# Setup sudo ip link add dummy0 type dummy sudo ip addr add 220.127.116.11/24 dev dummy0 sudo ip link set dummy0 up # Check interfaces ip addr # Do WebRTC things here python examples/python/visualization/draw_webrtc.py google-chrome http://localhost:8888 # Or, open the address in your browser # Clean up sudo ip link set dummy0 down sudo ip link delete dummy0
Advanced topic: TURN server¶
WebRTC’s ICE (Interactive Connectivity Establishment) framework resolves client-server connection via STUN or TURN servers.
In most scenarios, a STUN server is sufficient to figure out the traffic
routing. In certain network configurations (e.g. behind a NAT or firewall),
a TURN server is required to forward WebRTC traffic. You may add your custom
TURN server (video traffic relay) by setting the
environment variable. If you have more than one TURN servers, separate them
;. For instance:
# UDP only WEBRTC_STUN_SERVER="turn:user:password@my_turn_server.com:3478" # TCP only WEBRTC_STUN_SERVER="turn:user:password@my_tcp_turn_server.com:3478?transport=tcp" # UDP and TCP (more than one TURN servers) WEBRTC_STUN_SERVER="turn:user:password@my_turn_server.com:3478;turn:user:password@my_tcp_turn_server.com:3478?transport=tcp"
Advanced topic: debugging network issues¶
You may face issues such as failure to connect, lag or low visualization quality in specific network configurations (e.g. NAT, network proxy, VPN configurations). Here are some ideas to debug network issues:
Make sure localhost works before trying to host a remote server. Make sure standalone mode works before trying the Jupyter mode.
Test both client and server machines for connectivity and throughput at https://test.webrtc.org/.
If you face control lag, try switching from TCP to UDP connection or switching to a closer TURN (video traffic relay) server.
If the visualization video has compression artifacts, you may be using a TURN server with insufficient bandwidth. Try switching to a TURN server with higher bandwidth or closer to your location.