This document explains the interaction techniques available in the MR (tracker based) version of SFA.
This document is broken down into two major sections, one describing basic tracker interaction, one describing subsetting with the trackers and one describing the volume cutting. There is also a brief section describing plate thumbnails.
Using the trackers, the user can interact with the three dimensional scene in three dimensions, which is much more natural than interacting with a three-dimensional scene with the two-dimensional mouse.
Each tracker has three buttons attached. From top to bottom, these will be referred to as the nose button, the middle button, and the tail button. There are two trackers, one for each hand. Since the trackers are interchangable (either can be used in either hand) I will refer to one tracker as the DH tracker (Dominant Hand) and the other as the NDH (Non-Dominant Hand) tracker. In SFA, as with other MR applications, interaction tasks are split amongst the hands based on required level of control. Tasks requiring coarse grained level of control (such as menu selection, where each menu item is large) is delegated to the NDH, while fine grained tasks (such as 3D selection) is delegated to the DH.
As mentioned above, the trackers support a form of menuing to control
state oriented items as well as function activation. This is accomplished
using a technique called sundial menus. Sundials are described in
more detail here.
Since we will often want exact placement of the subsets, we consider subsetting to be a fine grained control interaction, and is therefore performed by the DH. The nose button on the DH tracker has been reserved for subsetting functionality.
Pressing the DH nose button begins a new subset, with the first of the two defining corners at the current tracker location (or a projection of the current tracker location into the main volume if the tracker is currently outside the volume). While holding the nose button down, movement of the tracker moves the position of the second defining vertex. This movement continues until the user releases the nose button, at which time the subset is saved, and the list of glyphs to be drawn is recalculated.
While a subset is being swept out, a partially transparent gray cube is drawn, indicating to the user the size of the subset they are creating. As soon as the subset has been completed, this cube is no longer drawn, to prevent the cube getting in the way of the data being visualized.
Multiple subsets are controlled through the use of thumbnail windows.
For a discussion on SFA subsetting and an explanation of thumbnails, please
see this
page.
Since placing and orienting the plate is also a fine control task, it has been mapped to the middle button on the DH tracker. Upon pressing the middle buton, a 'plate' appears, attached to the DH cursor. The user can then position and orient the plate as he or she desires.
There are currently three plate visualization modes:
How it works: As the plate is moved around in the dataset, glyphs that are near the plate are 'folded into' the plate. This means glyphs are brought into the plate's 2D coordinate system and mapped to a grid on the plate. The value at each gridpoint is dependent on the existance of a glyph that folds into that point, and if one does fold into that point, the value of the attribute being visualized for the glyph that was folded in. A basic (real-time) contouring algorithm is applied to generate contour lines using the current colour-ramp.
The user can easily discern patterns in the attribute being visualized
by following the contour diagram. Clusters of glyphs with abnormal values
stand out from the rest of the set by a large cluster of contour lines
grouped together. Since the contouring algorithm runs in real-time, the
plate can be dragged through the set to search for interesting areas.
With the single slice visualization, only a single 'slice' of the texture is displayed at any given time. The default value (ie: no glyph near the current location on the plate surface) is completely black. As the plate intersects a glyph, the intersection location will start glowing brighter, with brighter values indicating higher values of the attribute being visualized. The user can easily spot single glyphs or groups of glyphs in the field which are much brighter or dimmer than the rest of the field.
The main problem with single slice visualization is the lack of resolution
in the 3D texture. Due to the limitation of texture memory found on most
modern graphics workstations, texture size is extremely limited. Since
we have a much lower resolution texture than we have in actual glyph placement
in the scene, OpenGL performs texture blending to make up for the lack
of texture resolution. The problem with this is that texture blending causes
fading of spots as the intersection point moves away from the plane of
the plate. This can cause incorrect conclusions being drawn from the image.
As such, single slice lens visualization is not generally recommended for
use.
Unlike single slice lens visualization, multi-slice lens visualization uses the lens to look into the whole scene behind it, as opposed to just one location of the scene. Semantically, it acts truly like a lens would. Instead of having a black background, it uses no background (a completely transparent background is actually used). Instead of white luminance spots (which would now be invisible) it uses inverted luminance spots -- that is, the higher the value of the attribute being visualized, the darker the spot will appear. Looking through the lens into the glyph field, the user will see the regular glyph field (as the lens is transparent, the lens itself does not occlude the field behind it), but with inverted luminance spots superimposed over each glyph. Due to the lack of texture resolution the spots may actually be slightly off the glyph. This is actually a good feature, since it allows viewing of both the glyph and its' inverted luminance spot simulataneously. Use of higher texture resolutions on machines in the future will correct this discrepency.
The primary application of x-ray texturing is to find patterns in a single attribute which would not otherwise be visible. For example, it is very difficult to find general patterns in an alpha attribute. Using the lens on this attribute can show many interesting patterns not visible without the lens.
When applied to dense glyph fields, x-ray texturing suffers the same
problem as the dense glyph field itself: the grouping of many inverted
luminance spots tends to blend together down the perspective projection
and it becomes difficult to pick out individual sections. However, this
problem does not happen in less dense fields or when the line of sight
does not run down a line of glyphs.
Each thumbnail also provides a sundial menu to control various aspects of the plate. The sundial menu is accessed by place the DH tracker so it's cursor is overtop of the appropriate thumbnail, and pressing the plate button (the middle button on the DH tracker). This will pop up a sundial menu for that particular plate with the following options: New, Move, Resize, and Visualization Method.
Unlike subsets, contour plates by default expect only one plate to be active at a time. With a subset, pressing the subset button a second time will begin sweeping out another subset. With plates, pressing the plate button will cause the current plate to move to the current tracker location. By selecting the 'New' menu option (on any plate sundial) a new plate is added the next time you press the plate button.
If you select to add a new plate and then decide not to, select the 'Move' option. This will revert back to the original state of moving the current plate on button presses. Selecting the 'Move' menu item on a plate thumbnail makes that plate the currently active plate.
Resize allows basic resizing of the current plate. At this time, resizing is fairly clumsy. The basic interaction (upon the next button press) selects the vertex of the plate currently closest to the DH cursor. As the user moves the cursor around, the plate resizes accordingly. Releasing the plate button locks the plate in to its' new size.
Visualization method allows you to switch between the three available
visualization methods described above: Contour plates, single slice lens,
and multi-slice lens texturing. This only affects the current plate.