Scene and Driver Management

This section gives an overview on how scenes in ILNumerics Visualization Engine are managed. Knowing about these concepts is important for an efficient design of complex user interfaces.

Separation of Concerns

One important concept for the design of user interfaces says: those parts dealing with the presentation of the GUI to the user should be strictly separated from other parts, dealing with other functionality like business logic, data access a.s.o. For years, this 'separation of concerns' has been significantly helping to master complexity, decreasing the potential for bugs and leading to a better overall application design. It can be found in many technologies, like html/css, WPF and MVC.

ILNumerics' Visualization Engine realizes this concept by allowing the programmer to define the visual output in terms of abstract 'scene objects'. A scene can be seen as a 'model' which is able to fully describe all visual aspects and even some details of the dynamic behavior of the scene. Such Scenes describe completely what is being rendered. For sophisticated GUIs such scene objects can certainly become fairly complex. However, by no means they describe how the rendering itself happens.

ILNumerics Visualization Engine Scene Management: abstract scene descriptions are rendered by arbitrary drivers for specific targets: interactive targets (Windows.Forms; Panel) and static file targets.

Compatibility

The rendering is done by means of individual 'drivers'. Each driver is responsible for rendering the abstract scene model by one single and very specific viewing technology. ILNumerics scenes can currently equally be rendered to bitmap targets (png, jpg, bmp a.t.l.), vector graphics (SVD) and dynamically changing, interactive Windows.Forms controls (Panel control, leveraging OpenGL or GDI+). Since the scene is completely abstracted from the rendering API, the list of supported drivers can be extended in a very clean manner by simply adding another driver for a new technology. This being said, we currently are in the process of adding DirectX drivers to the list.

The driver is solely responsible for the list of supported features. Obviously, not all aspects of a scene can be realized by every driver. A SVG driver for example, rendering the scene to a static representation of SVG instructions is obviously not able to support interactivity to the same extent an OpenGL driver does.

Rendering Model

For interactive rendering it is important to take the threading model into account: dynamic changes to what a scene looks like must be synchronized with the actual rendering process. Precautions are necessary, in order to prevent changes to, let's say a triangle in the scene description from conflicting with simultaneous access to the same data by the rendering driver. Otherwise, rendering artifacts may result.

Two popular concepts exist for this synchronization challenge: locking and copying.

1. Locking prevents from conflicts due to concurrent access to the scene definition by help of common thread synchronization methods: mutexes and semaphores (locks). What may seem to be the most obvious solution at first glance turns out to bring several disadvantages in practice: locking causes increased contention between threads. For large objects a recomputation as well as the rendering can take a significant amount of time. During that period either the updates or the rendering must wait and can prevent frames from getting updated soon enough. The application becomes unresponsive. One can try to decrease these effects by limiting the locking to only subparts of the whole scene. However, this is not helpful in case of large single objects, which are common in technical visualizations. Also, often enough one cannot limit the locking to a single node due to existing interdependencies between multiple nodes. Finally, this scheme can drastically increase the complexity of the implementation.
2. Copying as the other option is utilized by ILNumerics Visualization Engine. Here, every driver maintains an individual copy of the scene. This copy is always immediately ready to get rendered. Changes to the original scene get populated to the rendering copy. The user triggers that synchronization process explicitly by calling Configure() on the affected part of the scene. This synchronization involves a simple copy of the changed data only and can be implemented very efficiently, using common synchronization strategies. In ILNumerics, scenes are copied by copying all scalar data, like camera positions, single color settings, text for labels and so on. However, for all buffer data, i.e. array data which potentially can be large - a reference counting based 'copy on write' scheme is implemented.

Having the ability to efficiently copy scenes bring further advantages: complete (sub)scenes can be reused this way simply by adding a copy of it. At the same time such copies do not increase the memory footprint significantly, since all buffers are still shared as long as no changes to any side are made. Furthermore, since each driver maintains its own copy of the whole scene, multiple drivers can be used simulteaneously to render the same scene without conflicting or interacting with each other.

One example is the creation of a snapshot of the current state of the scene by sending the scene to a GDI driver and continuing its use in an interactive OpenGL panel. Another example is the ability to use several Panel controls in the same GUI surface, all of them may or may not render and interact with the same scene definition or individual scenes or arbitrary combinations of them.

Scene Rendering Details

Global Scenes

Let's recap how ILNumerics manages scenes for rendering with the help of a simple example. The user commonly starts by defining one 'global' scene. Here, all visual objects are assembled from individual ILNumerics.Drawing objects. The configuration of these objects is also completely done in this global scene.

The same global scene definition is given to the IILDriver.Scene property of multiple drivers. Here 4 drivers share and render the same global scene: three Panel and one GDI driver. Each driver holds a private copy of the global scene, hence the drivers are independent of each other.

 

Note, that a scene does not store any information about the size of its rendered result! It does store, however, lengths, distances and angles as positioning information of objects contained. But those information carry abstract units only. The true size of the rendering is determined by the renderer alone. The same is true for the background color. If you want to render a scene by use of, let's say SVGDriver, these properties are the minimum, which have to be provided.

Drivers

If it comes to rendering, the scene must be given to a driver. All drivers in ILNumerics implement the IILDriver interface. Typically, scenes are displayed interactively inside Panel control, which is a Windows.Forms control which again implements the IILDriver interface. The `IILDriver.Scene` property provides access to the global scene rendered by the driver and allows the assignment of new scenes as well as the manipulation of the existing global scene.

Panel is a straightforward solution for rendering scenes onto Windows.Forms. For offscreen rendering, i.e.: the rendering without a visible form surface, any of the available drivers can be utilized programmatically:

SVGDriver - creates a vector graphic definition out of the scenes content in SVG format.

GDIDriver - full featured software renderer, used for scene snapshots and for offscreen rendering.

PickingDriver - internal usage: helps for picking details from arbitrary scene objects.

Synchronized Scene Copies

Before the driver renders a scene it creates a synchronized copy. 'Synchronized' because the copy is not detached from the global scene: once the global scene changes, the driver will recognize the change and update its synchronized copy accordingly.

While not intended as target for common update operations, access to the synchronized copy maintained by each driver is provided by the IILDriver.SceneSynchRoot property. This is useful in certain situations as we will see soon.

The main advantage of the synchronized copy is the fact that the driver will always be able to render its synchronized copy without conflicting with any updates happening to the global scene. However, changes to a scene exist, which are targeting the scene hold by a specific driver only. The most common example are changes by dynamic user interaction, like mouse interaction. 

Mouse Interaction

A user may interact with the visual representation of a scene - created and displayed by a specific driver. Most the time, this driver will be an Panel control and the interaction is done by utilizing a mouse. Such interactions effect the scene hold by the driver only. They do not by default alter the global scene. That effect is by design. Consider the following program which utilizes two Panel controls in order to present two different views of the same global scene.

 

The full example displays a form like the following:

In order to interact with the scene in one view, the user choses the corresponding panel and modifies the view, possibly by rotating the scene. This changes the camera position of that view. The camera of the global scene is not affected - only the synchronized copy is altered! This prevents the other view of the global scene from changing as well.

One fact which is important to internalize: In order to retrieve the current state of an object changed by user interaction, one needs to retrieve the object directly from the synchronized copy via the drivers IILDriver.SceneSynchRoot property:

Note that mouse and other user interaction commonly happen in the same UI thread which also does the rendering of a control. Therefore, no concurrency issues exist when changing the synchronized scene directly from the main UI thread. In fact, this is what is done inside ILNumerics as well: common mouse event handlers are catched and used to manipulate the synchronized scene for a panel / driver.

Concurrent Changes to Global and Synchronized Scenes

One issue may arise when such interaction conflict with concurrent changes to the global scene. The program might implement an animation to the global scene and allow the user to rotate the synchronized scene at the same time. ILNumerics will always give changes introduced by the global scene preceedance over those, configured on the synchronized scene. This is done in every rendering frame where changes in the global scene are identified and used in order to overwrite the current state of the synchronized scene.

Synchronized Scenes and Eventing