point_viz.h

Class 

class Camera

Controls the camera view and projection.

Public Functions

Camera()

Todo:: document me

impl::CameraData matrices(double aspect) const

Calculate camera matrices.

Parameters:: aspect – [in] aspect ratio of the viewport
Returns:: projection, view matrices and target location.

void reset(): Reset the camera view and fov.

void birds_eye_view(): Set camera view as looking from the top as a bird (Birds Eye View).

void roll(float degrees)

Roll the camera with respect to its current roll angle.

Parameters:: degrees – [in] offset to the current roll angle

void set_roll(float degrees)

Set roll in degrees.

Parameters:: degrees – [in] roll angle

float get_roll() const

Get the roll in degrees.

Returns:: roll in degrees

void yaw(float degrees)

Orbit the camera left or right about the camera target.

Parameters:: degrees – [in] offset to the current yaw angle

void set_yaw(float degrees)

Set yaw in degrees.

Parameters:: degrees – [in] yaw angle

float get_yaw() const

Get the yaw in degrees.

Returns:: yaw in degrees

void pitch(float degrees)

Pitch the camera up or down.

Parameters:: degrees – [in] offset to the current pitch angle

void set_pitch(float degrees)

Set pitch in degrees.

Parameters:: degrees – [in] pitch angle

float get_pitch() const

Get the camera pitch in degrees.

Returns:: pitch in degrees

void dolly(double amount)

Move the camera towards or away from the target.

Parameters:: amount – [in] offset to the current camera distance from the target

void set_dolly(double log_distance)

Set dolly (i.e. log distance) from the target to the camera.

Parameters:: log_distance – [in] log of the distance from the target

double get_dolly() const

Get the log distance from the target to the camera.

Returns:: log_distance

void dolly_xy(double x, double y)

Move the camera in the XY plane of the camera view.

Coordinates are normalized so that 1 is the length of the diagonal of the view plane at the target. This is useful for implementing controls that work intuitively regardless of the camera distance.

Parameters:

x – [in] horizontal offset
y – [in] vertical offset

void set_view_offset(const vec3d &view_offset)

Set view offset.

Parameters:: view_offset – [in] view offset of the camera

vec3d get_view_offset() const

Get view offset.

Returns:: view offset of the camera

void set_fov(float degrees)

Set the diagonal field of view.

Parameters:: degrees – [in] the diagonal field of view, in degrees

float get_fov() const

Get field of fiew of a camera in degrees

Returns:: fov in degrees

void set_orthographic(bool state)

Use an orthographic or perspective projection.

Parameters:: state – [in] true for orthographic, false for perspective

bool is_orthographic() const

Get orthographic state.

Returns:: true if orthographic, false if perspective

void set_proj_offset(float x, float y)

Set the 2d position of camera target in the viewport.

Parameters:

x – [in] horizontal position in in normalized coordinates [-1, 1]
y – [in] vertical position in in normalized coordinates [-1, 1]

vec2d get_proj_offset() const

Get the 2d position of camera target in the viewport.

Returns:: (x, y) position of a camera target in the viewport

void set_target(const mat4d &target)

Directly set camera target object pose

Parameters:: target – [in] target where camera is looking at

mat4d get_target() const

Get the pose of a camera target.

Returns:: target camera pose

class Cloud

Manages the state of a point cloud.

Each point cloud consists of n points with w poses. The ith point will be transformed by the (i % w)th pose. For example for 2048 x 64 Ouster lidar point cloud, we may have w = 2048 poses and n = 2048 * 64 = 131072 points.

We also keep track of a per-cloud pose to efficiently transform the whole point cloud without having to update all ~2048 poses.

Public Functions

explicit Cloud(size_t n, const mat4d &extrinsic = identity4d)

Unstructured point cloud for visualization.

Call set_xyz() to update

Parameters:

n – [in] number of points
extrinsic – [in] sensor extrinsic calibration. 4x4 column-major homogeneous transformation matrix

Cloud(size_t w, size_t h, const float *dir, const float *off, const mat4d &extrinsic = identity4d)

Structured point cloud for visualization.

Call set_range() to update

Parameters:

w – [in] number of columns
h – [in] number of pixels per column
dir – [in] unit vectors for projection
off – [in] offsets for xyz projection
extrinsic – [in] sensor extrinsic calibration. 4x4 column-major homogeneous transformation matrix

void update_from(const Cloud &other)

Updates this cloud’s state with the state of other, accounting for prior changes to this objects’s state.

Parameters:: other – [in] the object to update the state from.

void clear()

Clear dirty flags.

Resets any changes since the last call to PointViz::update()

void dirty()

Set all dirty flags.

Re-sets everything so the object is always redrawn.

inline size_t get_size() const

Get the size of the point cloud.

Todo:: document me

Returns:

inline size_t get_cols() const

Get the columns of the point cloud.

Returns:: number of columns in point cloud. (1 - for unstructured)

void set_range(const uint32_t *range)

Set the range values.

Parameters:: range – [in] pointer to array of at least as many elements as there are points, representing the range of the points

void set_key(const float *key)

Set the key values, used for coloring.

Parameters:: key – [in] pointer to array of at least as many elements as there are points, preferably normalized between 0 and 1

void set_key_alpha(const float *key_alpha)

Set the key alpha values, leaving the color the same.

Parameters:: key_alpha – [in] pointer to array of at least as many elements as there are points, normalized between 0 and 1

void set_key_rgb(const float *key_rgb)

Set the key values in RGB format, used for coloring.

Parameters:: key_rgb – [in] pointer to array of at least 3x as many elements as there are points, normalized between 0 and 1

void set_key_rgba(const float *key_rgba)

Set the key values in RGBA format, used for coloring.

Parameters:: key_rgba – [in] pointer to array of at least 4x as many elements as there are points, normalized between 0 and 1

void set_mask(const float *mask)

Set the RGBA mask values, used as an overlay on top of the key.

Parameters:: mask – [in] pointer to array of at least 4x as many elements as there are points, preferably normalized between 0 and 1

void set_xyz(const float *xyz)

Set the XYZ values.

Parameters:: xyz – [in] pointer to array of exactly 3n where n is number of points, so that the xyz position of the ith point is i, i + n, i + 2n

void set_offset(const float *offset)

Set the offset values.

TODO: no real reason to have this. Set in constructor, if at all

Parameters:: offset – [in] pointer to array of exactly 3n where n is number of points, so that the xyz position of the ith point is i, i + n, i + 2n

void set_pose(const mat4d &pose)

Set the ith point cloud pose.

Parameters:: pose – [in] 4x4 column-major homogeneous transformation matrix

void set_point_size(float size)

Set point size.

Parameters:: size – [in] point size

void set_column_poses(const float *rotation, const float *translation)

Set the per-column poses, so that the point corresponding to the pixel at row u, column v in the staggered lidar scan is transformed by the vth pose, given as a homogeneous transformation matrix.

Parameters:

rotation – [in] array of rotation matrices, total size 9 * w, where the vth rotation matrix is: r[v], r[w + v], r[2 * w + v], r[3 * w + v], r[4 * w + v], r[5 * w + v], r[6 * w + v], r[7 * w + v], r[8 * w + v]
translation – [in] translation vector array, column major, where each row is a translation vector. That is, the vth translation is t[v], t[w + v], t[2 * w + v]

void set_column_poses(const float *column_poses)

Set the per-column poses, so that the point corresponding to the pixel at row u, column v in the staggered lidar scan is transformed by the vth pose, given as a homogeneous transformation matrix.

Parameters:: column_poses – [in] array of 4x4 pose elements and length w (i.e. [wx4x4]) column-storage

void set_palette(const float *palette, size_t palette_size)

Set the point cloud color palette.

Parameters:

palette – [in] the new palette to use, must have size 3*palette_size
palette_size – [in] the number of colors in the new palette

class Cuboid

Manages the state of a single cuboid.

Public Functions

Cuboid(const mat4d &transform, const vec4f &rgba)

Constructor to initialize a cuboid

Parameters:

transform – [in] 4x4 matrix representing a transform defining the cuboid
rgba – [in] 4x1 float matrix representing cuboid color in RGBA format

void update_from(const Cuboid &other)

Updates this cuboid’s state with the state of other, accounting for prior changes to this objects’s state.

Parameters:: other – [in] the object to update the state from.

void clear()

Clear dirty flags.

Resets any changes since the last call to PointViz::update()

void set_transform(const mat4d &pose)

Set the transform defining the cuboid.

Applied to a unit cube centered at the origin.

Todo:: document me

Parameters:: pose – [in]

void set_rgba(const vec4f &rgba)

Set the color of the cuboid.

Todo:: document me

Parameters:: rgba – [in]

class Image

Manages the state of an image.

Public Functions

Image()

Todo:: document me

void update_from(const Image &other)

Updates this image’s state with the state of other, accounting for prior changes to this objects’s state.

Parameters:: other – [in] the object to update the state from.

void clear()

Clear dirty flags.

Resets any changes since the last call to PointViz::update()

void set_image(size_t width, size_t height, const float *image_data)

Set the image data.

Parameters:

width – [in] width of the image data in pixels
height – [in] height of the image data in pixels
image_data – [in] pointer to an array of width * height elements interpreted as a row-major monochrome image

void set_image_rgb(size_t width, size_t height, const float *image_data_rgb)

Set the image data (RGB).

Parameters:

width – [in] width of the image data in pixels
height – [in] height of the image data in pixels
image_data_rgb – [in] pointer to an array of width * height elements interpreted as a row-major RGB image

void set_image_rgba(size_t width, size_t height, const float *image_data_rgba)

Set the image data (RGBA).

Parameters:

width – [in] width of the image data in pixels
height – [in] height of the image data in pixels
image_data_rgba – [in] pointer to an array of width * height elements interpreted as a row-major RGBA image

void set_mask(size_t width, size_t height, const float *mask_data)

Set the RGBA mask.

Not required to be the same resolution as the image data

Parameters:

width – [in] width of the image data in pixels
height – [in] height of the image data in pixels
mask_data – [in] pointer to array of 4 * width * height elements interpreted as a row-major rgba image

void set_position(float x_min, float x_max, float y_min, float y_max)

Set the display position of the image.

Todo:: : this is super weird. Coordinates are {x_min, x_max, y_max, y_min} in sort-of normalized screen coordinates: y is in [-1, 1], and x uses the same scale (i.e. window width is ignored). This is currently just the same method the previous hard-coded ‘image_frac’ logic was using; I believe it was done this way to allow scaling with the window while maintaining the aspect ratio.

Parameters:

x_min – [in]
x_max – [in]
y_min – [in]
y_max –

void set_hshift(float hshift)

Set horizontal shift in normalized viewport screen width coordinate.

This may be used to “snap” images to the left/right screen edges.

Some example values: 0 - default, image is centered horizontally on the screen -0.5 - image moved to the left for the 1/4 of a horizontal viewport -1 - image moved to the left for the 1/2 of a horizontal viewport +1 - image moved to the right for the 1/2 of a horizontal viewport +0.5 - image moved to the right for the 1/4 of a horizontal viewport

Parameters:: hshift – [in] shift in normalized by width coordinates from 0 at the center [-1.0..1.0]

void set_palette(const float *palette, size_t palette_size)

Set the image color palette.

Parameters:

palette – [in] the new palette to use, must have size 3*palette_size
palette_size – [in] the number of colors in the new palette

void clear_palette(): Removes the image color palette.

std::pair<int, int> viewport_coordinates_to_image_pixel(const WindowCtx &ctx, double x, double y) const

Returns the image pixel corresponding to the provided viewport coordinates.

Parameters:

ctx – [in] the WindowCtx.
x – [in] X coordinate for the viewport
y – [in] Y coordinate for the viewport

Throws:

std::runtime_error – if the image data has not been set, or if the image size (set by set_position) is zero in either dimension.

Returns:

Image pixel corresponding to the provided viewport coordinates. IMPORTANT: the pixel is outside the image if either the (x, y) are less than zero or greater than the image (width, height).

std::pair<double, double> image_pixel_to_viewport_coordinates(const WindowCtx &ctx, int px, int py) const

Returns the inverse of “viewport_coordinates_to_image_pixel” This is useful for computing positions relative to an image pixel.

Parameters:

ctx – [in] the WindowCtx.
px – [in] X pixel coordinate
py – [in] Y pixel coordinate

Returns:

a coordinate normalized to -1, 1 in the window’s Y axis

std::pair<double, double> pixel_size(const WindowCtx &ctx) const

Returns the pixel size as a pair representing width and height in window pixels.

Parameters:: ctx – [in] the WindowCtx.
Returns:: a pair representing the image pixel size in window pixels.

class Label

Manages the state of a text label.

Public Functions

Label(const std::string &text, const vec3d &position)

Constructs a Label object

Parameters:

text – [in] Text to display
position – [in] Set 3D position of label as x,y,z coordinates of type double

Label(const std::string &text, float x, float y, bool align_right = false, bool align_top = false)

Sets the 2D position of the lavel

Parameters:

text – [in] Label text to display
x – [in] horizontal position [0, 1]
y – [in] vertical position [0, 1]
align_right – [in] interpret position as right of the label
align_top – [in] interpret position as top of the label

void update_from(const Label &other)

Updates this label’s state with the state of other, accounting for prior changes to this objects’s state.

Parameters:: other – [in] the object to update the state from.

void clear()

Clear dirty flags.

Resets any changes since the last call to PointViz::update()

void dirty()

Set all dirty flags.

Re-sets everything so the object is always redrawn.

void set_text(const std::string &text)

Update label text.

Parameters:: text – [in] new text to display

void set_position(const vec3d &position)

Set label position.

Parameters:: position – [in] 3d position of the label

void set_position(float x, float y, bool align_right = false, bool align_top = false)

Set position of the 2D label.

Parameters:

x – [in] horizontal position [0, 1]
y – [in] vertical position [0, 1]
align_right – [in] interpret position as right of the label
align_top – [in] interpret position as top of the label

void set_scale(float scale)

Set scaling factor of the label.

Parameters:: scale – [in] text scaling factor

void set_rgba(const vec4f &rgba)

Set the color of the label.

Parameters:: rgba – [in] color in RGBA format

class Lines

Manages the state of a single set of lines.

Public Functions

Lines(const mat4d &transform, const vec4f &rgba)

Constructor to initialize a set of lines

Parameters:

transform – [in] 4x4 matrix representing a transform for the points
rgba – [in] 4x1 float matrix representing lines color in RGBA format

void update_from(const Lines &other)

Updates this objects’s state with the state of other, accounting for prior changes to this objects’s state.

Parameters:: other – [in] the object to update the state from.

void clear()

Clear dirty flags.

Resets any changes since the last call to PointViz::update()

void set_points(size_t num_points, const float *points)

Set the 3D points defining the lines. Each line is defined by 2 points, a start and an end.

Parameters:

num_points – [in] number of points
points – [in] num_points*3 array of doubles representing the xyz of each point

void set_transform(const mat4d &pose)

Set the transform for the lines.

Parameters:: pose – [in] new pose for the lines

void set_rgba(const vec4f &rgba)

Set the color of the lines.

Parameters:: rgba – [in] color

class PointViz

A basic visualizer for sensor data.

Displays a set of point clouds, images, cuboids, and text labels with a few options for coloring and handling input.

All operations are thread safe when running rendering (run() or run_once()) in a separate thread. This is the intended way to use the visualizer library when a nontrivial amount of processing needs to run concurrently with rendering (e.g. when streaming data from a running sensor).

Public Functions

void add_default_controls(std::mutex *mx)

Add default keyboard and mouse bindings to a visualizer instance

Controls will modify the camera from the thread that calls run() or run_once(), which will require synchronization when using multiple threads.

Parameters:: mx – [in] mutex to lock while modifying camera

explicit PointViz(const std::string &name, bool fix_aspect = false, int window_width = default_window_width, int window_height = default_window_height, bool maximized = false)

Creates a window and initializes the rendering context

Parameters:

name – [in] name of the visualizer, shown in the title bar
fix_aspect – [in] Window aspect to set
window_width – [in] Window width to set, else uses the default_window_width
window_height – [in] Window height to set, else uses the default_window_height
maximized – [in] If true, the window will be maximized to fit the screen.

~PointViz(): Tears down the rendering context and closes the viz window

void run()

Main drawing loop, keeps drawing things until running(false)

Should be called from the main thread for macos compatibility

void run_once()

Run one iteration of the main loop for rendering and input handling

Should be called from the main thread for macos compatibility

bool running()

Check if the run() has been signaled to exit

Returns:: true if the run() loop is currently executing

void running(bool state)

Set the running flag. Will signal run() to exit

Parameters:: state – [in] new value of the flag

void visible(bool state)

Show or hide the visualizer window

Parameters:: state – [in] true to show

void update()

Update visualization state

Send state updates to be rendered on the next frame.

void push_key_handler(std::function<bool(const WindowCtx&, int, int)> &&callback)

Add a callback for handling keyboard input

Parameters:: callback – [in] the callback. The second argument is the ascii value of the key pressed. Third argument is a bitmask of the modifier keys The callback’s return value determines whether the remaining key callbacks should be called.

void push_mouse_button_handler(std::function<bool(const WindowCtx &ctx, ouster::viz::MouseButton button, ouster::viz::MouseButtonEvent event, ouster::viz::EventModifierKeys mods)> &&callback)

Add a callback for handling mouse button input

Parameters:: callback – [in] the callback. The callback’s arguments are ctx: the context containing information about the buttons pressed, the mouse position, and the viewport; button: the mouse button pressed; mods: representing which modifier keys are pressed during the mouse click. The callback’s return value determines whether the remaining mouse button callbacks should be called.

void push_scroll_handler(std::function<bool(const WindowCtx&, double x, double y)> &&callback)

Add a callback for handling mouse scrolling input

Parameters:: callback – [in] the callback. The callback’s arguments are ctx: the context containing information about the buttons pressed, the mouse position, and the viewport; x: the amount of scrolling in the x direction; y: the amount of scrolling in the y direction. The callback’s return value determines whether the remaining mouse scroll callbacks should be called.

void push_mouse_pos_handler(std::function<bool(const WindowCtx&, double x, double y)> &&callback)

Add a callback for handling mouse movement

Parameters:: callback – [in] the callback. The callback’s arguments are ctx: the context containing information about the buttons pressed, the mouse position, and the viewport; x: the mouse position in the x direction; y: the mouse position in the y direction. The callback’s return value determines whether the remaining mouse position callbacks should be called.

void push_frame_buffer_handler(std::function<bool(const std::vector<uint8_t>&, int, int)> &&callback)

Add a callback for processing every new draw frame buffer.

NOTE: Any processing in the callback slows the renderer update loop dramatically. Primary use to store frame buffer images to disk for further processing.

Parameters:

callback – [in]

function callback of a form f(fb_data, fb_width,

fb_height) The callback’s return value determines whether the remaining frame buffer callbacks should be called.

void pop_key_handler(): Remove the last added callback for handling keyboard events

void pop_mouse_button_handler(): Remove the last added callback for handling mouse button events

void pop_scroll_handler(): Remove the last added callback for handling mouse scroll events

void pop_mouse_pos_handler(): Remove the last added callback for handling mouse position events

void pop_frame_buffer_handler(): Remove the last added callback for handling frame buffer events

void push_frame_buffer_resize_handler(std::function<bool(const WindowCtx&)> &&callback)

Add a callback for handling frame buffer resize events.

Parameters:: callback – [in] function callback of the form f(const WindowCtx&). The callback’s return value determines whether the remaining frame buffer resize callbacks should be called.

void pop_frame_buffer_resize_handler(): Remove the last added callback for handling frame buffer resize events.

Camera &camera()

Get a reference to the camera controls

Returns:: Handler to the camera object

TargetDisplay &target_display()

Get a reference to the target display controls

Returns:: Handler to the target display controls

void add(const std::shared_ptr<Cloud> &cloud)

Add an object to the scene

Parameters:: cloud – [in] Adds a point cloud to the scene

void add(const std::shared_ptr<Image> &image)

Add an object to the scene

Parameters:: image – [in] Adds an image to the scene

void add(const std::shared_ptr<Cuboid> &cuboid)

Add an object to the scene

Parameters:: cuboid – [in] Adds a cuboid to the scene

void add(const std::shared_ptr<Lines> &lines)

Add an object to the scene

Parameters:: lines – [in] Adds a lines to the scene

void add(const std::shared_ptr<Label> &label)

Add an object to the scene

Parameters:: label – [in] Adds a label to the scene

bool remove(const std::shared_ptr<Cloud> &cloud)