 |
0.1.33
|
|
0.1.33
|
| Dependencies | X11/xorg (Mac/Linux) |
|---|---|
| Python Import | from pyhelios import Visualizer |
| Main Class | Visualizer |
| Dependencies | X11/xorg (Mac/Linux), OpenGL 3.3+ |
|---|---|
| Platforms | Windows, Linux, macOS |
| GPU | OpenGL 3.3 compatible graphics card required |
| Constructors |
|---|
| Visualizer(width, height, antialiasing_samples=4, headless=False) |
The class associated with the visualization is called Visualizer. The class constructor takes width and height arguments that specify the size of the graphics window in pixels. Additional optional parameters control antialiasing quality and headless rendering mode. Below is an example:
antialiasing_samples defaults to 4. Pass 0 to disable antialiasing, which is required for exact color reproduction — see Exact Color Mode.
| Installation Method | Dependency | 
| 
| 
|
|---|---|---|---|---|
| Runtime (pip install) | X11 Server | Required:$ brew install --cask xquartz | Usually pre-installed | None |
| OpenGL 3.3+ | Compatible graphics card and drivers | |||
| Build from Source (additional deps) | Development Libraries | Same as runtime | $ sudo apt-get install libx11-dev xorg-dev libgl1-mesa-dev libglu1-mesa-dev libxrandr-dev | Visual Studio |
LIGHTING_PHONG_SHADOWED, where it was not benign: it indicated that the shadow map had never been created, and shadows were silently missing from the rendered output.This plugin facilitates visualization of model geometry and data. It can visualize a number of different geometric shapes, including all of the primitive types supported by the Helios context. Individual geometric objects can be added though API commands, and there is a command to automatically add all geometric primitives from the Helios context.
The visualizer uses two types of coordinate systems to specify the locations of points in space during visualization:
In order to actually display a window for visualization, we must issue a command to plot the geometry. There are different commands to produce a visualization window depending on the intended output:
The command printWindow() outputs the current visualization to file (JPEG or PNG). As of helios-core 1.3.85 it renders the frame itself when the one on the GPU is stale, so calling plotUpdate() first is harmless but no longer necessary: buildContextGeometry() followed directly by printWindow() produces a correct image.
The current window can be closed using the closeWindow() command.
Below is an example of opening a window (blank), exporting its contents to file, then closing the window:
The Visualizer supports headless rendering for server environments or automated workflows where no display is available. Headless mode enables all visualization and image export functionality without requiring a graphics display or user interaction.
Enabling Headless Mode:
Headless mode is enabled through the constructor by setting the headless parameter to True:
Key Features:
LIGHTING_PHONG_SHADOWED)Note: Shadowed lighting in headless mode requires helios-core 1.3.80 or later. Earlier versions created the shadow-map framebuffer only in windowed mode, so a headless render using
LIGHTING_PHONG_SHADOWEDcame out with the entire scene rendered at the fully-shadowed brightness — not merely missing its shadows.
Note: As of helios-core 1.3.80, a headless render that never calls setLightDirection() uses the same default (1,1,1) light direction as windowed mode. In earlier versions the default light direction was never sent to the shader in headless mode, so headless and windowed renders shaded the same scene differently.
Anti-aliasing:
Headless rendering is anti-aliased: geometry is rendered into a multisampled framebuffer and resolved before readback, using the sample count given to the constructor. The requested count is clamped to the driver maximum, and a driver that refuses the multisampled attachments falls back silently — isHeadlessMultisamplingActive() is how to detect that.
Note: macOS drives OpenGL through a translation layer that accepts the multisampled attachments but does not rasterize into them, so
isHeadlessMultisamplingActive()can reportTruethere while the saved image is not actually anti-aliased.
Important Notes:
SyntheticAnnotation, which is built on it) usable on a batch compute node. EGL is tried first and the hidden-window GLFW path is used when it is unavailable. This requires EGL at build time; the Linux wheels are built with it, so they need libegl1 installed alongside libgl1 (see the README)Use Cases:
The Visualizer provides multiple options for customizing the window background:
Default Gradient Background
By default, the Visualizer displays a subtle vertical gradient background that transitions from a neutral blue at the bottom to white at the top. This provides a clean, professional appearance without requiring any configuration.
Solid Color Background
The window background can be set to a constant color using the setBackgroundColor() command:
Transparent Background
For creating visualizations with transparent backgrounds (useful for overlaying on other images or presentations), use the setBackgroundTransparent() command. When rendering to the screen, a checkerboard pattern indicates transparency. When exporting to PNG format via printWindow(), the background will have true alpha channel transparency:
Note: Transparent backgrounds only work with PNG output format. JPEG output will have an opaque background.
Custom Image Background
You can set a custom texture image as the background using the setBackgroundImage() command:
The image will be stretched to fill the window while maintaining the aspect ratio to avoid distortion. Both JPEG and PNG formats are supported.
Sky Texture Background
For outdoor scenes, a three-dimensional sky background can be added using the setBackgroundSkyTexture() command. This creates a dynamically scaling sky sphere that uses shader-based transformations to keep the camera always inside, eliminating the need to manually specify a radius. The sky appears infinitely distant regardless of camera position or zoom level:
The method automatically uses the default sky texture (plugins/visualizer/textures/SkyDome_clouds.jpg) if no texture file is specified. The sky can be toggled on/off like other background modes using setBackgroundColor(), setBackgroundTransparent(), or setBackgroundImage().
PyHelios Visualizer supports geometry visualization through two primary methods:
Note: Direct geometry addition methods (addRectangle, addTriangle, etc.) available in the native C++ Helios library are not yet exposed in PyHelios. Use Context primitives and buildContextGeometry() instead.
The visualizer can automatically import some or all geometry from the Context. This is accomplished using the buildContextGeometry() command. To add all primitives in the Context, the buildContextGeometry() command would be issued, which is passed the Context. We can add a subset of the Context geometry through an additional argument which takes a list of UUIDs.
| Command | Description |
|---|---|
| buildContextGeometry(context) | Add all primitives in the Context to the Visualizer. |
| buildContextGeometry(context, uuids=[...]) | Add a subset of primitives in the Context to the Visualizer. |
The example below shows how to add all Context geometry to the visualizer.
PyHelios Visualizer provides methods to add reference objects:
| Method | Description |
|---|---|
| addCoordinateAxes() | Add XYZ coordinate axes at the origin |
| addCoordinateAxesCustom(origin, length, sign) | Add coordinate axes at custom position with custom length |
| addGridWireFrame(center, size, subdivisions) | Add a 3D grid wireframe for spatial reference |
Example:
For large point clouds (tens of millions of points), the visualizer includes automatic culling optimization to maintain interactive performance. This feature is enabled by default when the number of points exceeds a configurable threshold.
Note: Point cloud culling configuration functions are available in the native C++ Helios library but are not yet exposed in PyHelios. The automatic culling optimization is still active at the C++ level.
To this point, we have not actually plotted anything in the Visualizer window. A final command is needed to display all of the geometry added to the Visualizer in the window we have opened. There are two functions for doing this, which are detailed below.
The plotInteractive() function can be used to generate an interactive plot of the geometry in the Visualizer. This means that the code will pause to produce the plot/visualization until the window is closed by the user. The user can interact with the plot by issuing keystrokes to, e.g., zoom. An example is given below to generate an interactive plot.
In an interactive plot, the view can be modified via the mouse or keyboard. Note that for very large scenes, mouse-based controls may be laggy.
Mouse camera controls are given by:
| Mouse Button | Action |
|---|---|
| scroll wheel | zoom in/out relative to look-at position |
| left mouse button (+drag) | rotate camera view about look-at position |
| right mouse button (+drag) | move look-at position |
Keyboard camera controls are given by:
| Key | Action |
|---|---|
| up arrow | increase the viewing elevation angle |
| down arrow | decrease the viewing elevation angle |
| left arrow | rotate camera left (clockwise) |
| right arrow | rotate camera right (counter-clockwise) |
| spacebar+up arrow | move the camera position upward |
| spacebar+down arrow | move the camera position downward |
| + | zoom in |
| - | zoom out |
The plotUpdate() function simply updates the plot window based on current geometry, and continues on to the next lines of code. This can be useful when a window should show the scene while the program carries on. Writing a still image to file no longer requires it (helios-core 1.3.85+): printWindow() renders the frame itself, so the plotUpdate() call in the example below is redundant but harmless.
Rendered pixels can be read into Python rather than written to file with getWindowPixelsRGB(). Called with no argument, it allocates the buffer itself and returns it along with the image dimensions:
Data is stored as r-g-b per column per row, so pixels[0:3] is the RGB value of row 0 column 0 and len(pixels) == 3 * width * height.
A pre-allocated list may be passed instead, in which case it is filled in place. It must hold exactly 3 * width * height elements, where the dimensions come from getFramebufferSize() — not getWindowSize() and not the dimensions passed to the Visualizer constructor. On a high-DPI (Retina) display the framebuffer is larger than the window, typically by a factor of two per axis, so a buffer sized from the window dimensions is four times too small. PyHelios rejects a wrongly-sized buffer with a ValueError; the no-argument form avoids the question entirely and is preferred.
The surfaces of primitives are most commonly colored by specifying an r-g-b/r-g-b-a color code when adding the geometry, and thus this is the default behavior.
Rectangles and disks have the capability of coloring their surface according to a texture map. A texture map can be specified by providing the path to either a JPEG or PNG image file. The image will be mapped onto the surface of the primitive element.
Primitives can be colored by mapping associated data values to a color table. Given some range of data values, each value is normalized by this range and used to look up an associated color in the color table. The available predefined color tables are shown below. There is also the capability of defining custom color tables.
| Colormap Name | Example |
|---|---|
| COLORMAP_HOT | 
|
| COLORMAP_COOL | 
|
| COLORMAP_RAINBOW | 
|
| COLORMAP_LAVA | 
|
| COLORMAP_PARULA | 
|
| COLORMAP_GRAY | 
|
| COLORMAP_CUSTOM | N/A |
To color primitives by data values, use the colorContextPrimitivesByData() method:
The colorbar is the legend showing how values are mapped to the color table. The table below gives functions for customizing colorbar behavior, including for example its position, size, and visibility.
| Function | Description |
|---|---|
| enableColorbar() | Make the colorbar visible. |
| disableColorbar() | Make the colorbar invisible. |
| setColorbarPosition(position) | Set the position of the colorbar. Note that position.z gives the depth of the colorbar. |
| setColorbarSize(size) | Set the size of the colorbar. |
| setColorbarRange(cmin, cmax) | Set the range of data values for the colorbar/colormap. Setting a range explicitly stops the colorbar from auto-ranging over the data, including for a degenerate range such as (0, 0). An inverted range is rejected. |
| setColorbarTicks(ticks) | Set locations of data tick along colorbar. If a tick lies outside the colorbar range, the range is expanded to fit it, which moves the colormap limits as well as the labels; call setColorbarRange() afterwards to restore an explicit range. |
| setColorbarTitle(title) | Set the title text displayed above the colorbar. |
| setColorbarFontColor(color) | Set the color of text in the colorbar. |
| setColorbarFontSize(font_size) | Set the size of the colorbar text in points. |
When custom ticks are not specified using setColorbarTicks, the visualizer automatically generates tick positions and labels using Heckbert's "nice numbers" algorithm. This algorithm selects tick values that are "clean" multiples of 1, 2, or 5 times a power of 10, making the colorbar easier to read.
Adaptive Tick Count: The number of ticks is automatically determined based on the colorbar size and font size to prevent overcrowding. Smaller colorbars will have fewer ticks to maintain readability.
For example:
The tick label formatting automatically adapts based on the data type and range:
The data type (integer vs. float) is automatically detected when calling colorContextPrimitivesByData().
To allow for more realistic visualizations, the Phong shading model can be enabled. The Phong shading model can be enabled with or without shadows. The shading options are detailed in the table below. The appropriate value is passed to the setLightingModel() command to enable the specified shading model.
| Shading Model | Value | Description |
|---|---|---|
| No shading | 0 or "none" | No shading is applied. Objects are colored only according to the r-g-b(-a) color code or texture map. |
| Phong shading | 1 or "phong" | Phong shading model |
| Phong with shadows | 2 or "phong_shadowed" | Phong shading model with shadows. |
If the Phong shading model is used, the position of the light source should be specified. This is accomplished through the setLightDirection() command, which takes a unit vector pointing toward the light source. The example below shows how to enable the Phong lighting model with shadows, with the light position set according to the position of the sun.
Rendering uses a physically-based linear-light pipeline by default. Albedo is decoded from sRGB to linear light before shading, and the shaded radiance is tone-mapped through an ACES filmic curve and re-encoded to sRGB. Compared with shading directly on non-linear sRGB values, mid-tones are brighter, shadow terminators are smoother, and bright surfaces roll off rather than clipping flat against the 8-bit framebuffer.
Rendered images therefore differ from PyHelios versions before this one. The previous appearance is available with disableLinearPipeline(), and enableLinearPipeline() restores the default. isLinearPipelineEnabled() reports the current state.
setExposure(exposure) controls the linear exposure multiplier applied before tone mapping; it must be positive. getExposure() reads it back.
Under a Phong lighting model, surfaces are shaded as ambient*A + diffuse*max(0, N.L) + specular*max(0, N.H)^shininess, where A is the hemispheric ambient term. The four parameters are set with setPhongMaterial(ambient, diffuse, specular, shininess) and read back with getPhongMaterial(), which returns them as a tuple. The defaults are (1.0, 0.8, 0.2, 32.0).
The specular highlight is new: it was previously absent entirely despite the model being described as Phong. Setting specular to zero recovers surfaces without a highlight.
Individual materials can override any of these parameters by attaching phong_ambient, phong_diffuse, phong_specular or phong_shininess material data with Context.setMaterialDataFloat(). Each parameter falls back individually to the global value, so a material can override only its specular lobe.
Ambient light is modeled hemispherically, blending a sky color from above with a ground-bounce color from below according to surface orientation, rather than applying one constant term to every surface. The two colors are set with setAmbientColors(sky_color, ground_color) and read with getAmbientSkyColor() and getAmbientGroundColor(). The defaults are a sky of (0.5, 0.6, 0.75) and a ground of (0.35, 0.3, 0.22); setting both to the same value recovers the old constant ambient term.
Surfaces are shaded with smooth, interpolated per-vertex normals by default, removing the faceted appearance of tessellated curved geometry such as stems, trunks and fruit. disableSmoothShading() selects flat shading and enableSmoothShading() restores the default; isSmoothShadingEnabled() reports the current state.
Only geometry that supplies distinct vertex normals is affected:
Sphere, Tube and Cone objects, which use the analytic surface normal of the shape they approximate.Polymesh objects loaded from an OBJ or PLY file that carries per-vertex normals, or given them with Context.computePolymeshObjectVertexNormals().Patches, triangles and voxels added directly to the Context carry the face normal replicated across their vertices and render identically either way.
Flat shading is sometimes preferable for alpha-masked cutout geometry such as leaf textures, where interpolated normals can look worse than a single flat normal per face.
The default camera position is at an elevation angle of 20 degrees and to the North, with the camera looking toward the origin. The distance of the camera from the origin is automatically adjusted to fit all primitives in view.
There are multiple ways of specifying custom camera views. One method involves specifying the (x,y,z) position of the camera, and the (x,y,z) position that the camera is looking at. This is accomplished using the command setCameraPosition(position, lookAt).
The other method involves specifying the spherical coordinates of the camera with respect to the (x,y,z) position the camera is looking at. This is accomplished using the setCameraPositionSpherical(angle, lookAt).
By default the fragment shader multiplies primitive colors by 1.5, which brightens ordinary renders but means the color read back out of the framebuffer is not the color that was set on the primitive. enableExactColorMode() disables that multiplier so a color set with Context.setPrimitiveColor() is reproduced exactly in the rendered image. disableExactColorMode() restores the default.
This matters when the framebuffer carries data rather than an image — for example when object IDs are encoded as RGB values and decoded back from the rendered pixels.
Enabling exact color mode also disables the linear-light pipeline (see Rendering Pipeline), since tone mapping would likewise alter the values read back; disabling it restores the pipeline. The two modes are mutually exclusive.
Exact reproduction has three requirements, and missing any one of them silently corrupts the result:
enableExactColorMode() must be called, or colors come back 1.5x too bright.antialiasing_samples=0. Antialiasing blends colors at primitive edges, and a blended edge pixel decodes to an object ID that was never in the scene.LIGHTING_NONE is the default, so this holds unless setLightingModel() changed it.getTextboxSize(textstring, fontsize, fontname) returns the extent that addTextboxByCenter() would occupy for the same string, font and size, as a vec2 in window-normalized units. Nothing is added to the visualizer.
The width is the sum of the glyph advances, so it includes the side bearings; the height is that of the tallest glyph in the string and therefore depends on which characters it contains. The _ and ^ subscript and superscript markers are handled as addTextboxByCenter() handles them: they occupy no width themselves and halve the size of the character that follows. The result depends on the current framebuffer dimensions and DPI scale, so it changes when the window is resized.
Two methods display an image with machine-learning annotations drawn over it, reading the formats written by the radiation model's camera annotation output.
| Method | Reads |
|---|---|
displayImageWithBoundingBoxes(image_file, bbox_file, classes_file="", line_width=2.0, fontsize=12) | Ultralytics YOLO bounding boxes, as written by RadiationModel.writeImageBoundingBoxes() |
displayImageWithSegmentationMasks(image_file, mask_file, fill_opacity=0.4, line_width=2.0, fontsize=12, show_labels=True) | COCO JSON segmentation masks, as written by RadiationModel.writeImageSegmentationMasks() |
Boxes are drawn as colored outlines with the class name on a filled chip, colored by class ID from a fixed palette of seven colors, so classes whose IDs differ by a multiple of seven share a color. If classes_file is empty, a classes.txt beside the annotation file is used when one exists; otherwise boxes are labeled with their numeric class ID.
Masks are drawn as translucent filled polygons with solid outlines, colored by position in the file rather than by class, so two touching objects of the same class stay distinguishable. Pass fill_opacity=0 for outlines only, or show_labels=False when many overlapping chips would cover the image.
Like displayImageFromFile(), both methods clear any existing geometry and do not return until the window is closed.
This plug-in uses all or parts of the following open-sourced software libraries:
The OpenGL Extension Wrangler Library: Copyright (C) 2008-2016, Nigel Stewart <nigels[]users sourceforge net> Copyright (C) 2002-2008, Milan Ikits <milan ikits[]ieee org> Copyright (C) 2002-2008, Marcelo E. Magallon <mmagallo[]debian org> Copyright (C) 2002, Lev Povalahev All rights reserved.
The FreeType Project: Portions of this software are copyright © 2019 The FreeType Project (www.freetype.org). All rights reserved.
GLFW: Copyright © 2002-2006 Marcus Geelnard Copyright © 2006-2019 Camilla Löwy
OpenGL Mathematics (GLM): Copyright (c) 2005 - 2014 G-Truc Creation
libjpeg: This software is copyright (C) 1991-2016, Thomas G. Lane, Guido Vollbeding. All Rights Reserved except as specified below.
libpng: Copyright (c) 1995-2019 The PNG Reference Library Authors. Copyright (c) 2018-2019 Cosmin Truta. Copyright (c) 2000-2002, 2004, 2006-2018 Glenn Randers-Pehrson. Copyright (c) 1996-1997 Andreas Dilger. Copyright (c) 1995-1996 Guy Eric Schalnat, Group 42, Inc.
1.13.2