Autostereo Interleaver

Autostereo Interleaver is a generalized implementation of the Combiner algorithm described in the IEEE VR 07 paper A GPU Sub-pixel Algorithm for Autostereoscopic Virtual Reality.

The interleaver expands upon the combiner in several ways. First and foremost, it supports an arbitrary number of view points, which broadens its capability beyond Varrier to the creation of PHSColograms and lenticular panoramagrams. Second, it encapsulates its internal state, allowing multiple instances to be in use simultaneously by applications with multiple OpenGL contexts. Third, it is implemented using a subset of OpenGL 2.0 conforming to OpenGL ES 2.0, allowing for greater portability. Finally, it is licensed under the GNU Library GPL allowing it to be used by non-Free applications and frameworks.

Source code, along with an OBJ-viewer example application, may be downloaded here. This code has been tested under Linux, OSX, and Windows XP. A Makefile is included for Linux and OSX, and a Visual Studio 7 project is included for Windows.

The interleaver library depends upon The OpenGL Extension Wrangler Library In addition to GLEW, the OBJ viewer example depends upon SDL, PNG, zlib, and jpeg.

Startup and Shutdown

struct il_context *il_init_context(int n, int w, int h, const char *vert_name, const char *frag_name)

Initialize and return an interleaver context.

nThe number of view points
wOff-screen render buffer width
hOff-screen render buffer height
vert_nameVertex shader file name (NULL)
frag_nameFragment shader file name (NULL)

The n argument gives the number of view points. That is, the number of off-screen render buffers to be interleaved. w and h give the size of the display, allowing the interleaver to preallocate these off-screen render buffers. vert_name and frag_name allow custom interleaving shaders to be introduced. If either is NULL then the default shader interleaver.vert or interleaver.frag will be used. Because OpenGL state is initialized, the application must call this function after acquiring its OpenGL context, and initializing GLEW.

void il_fini_context(struct il_context *context)

Finalize the interleaver context and release all OpenGL state associated with it.

Configuration

Display, line screen, and lenticular configuration is communicated to the interleaver via the following structure. All interleaver functions receive a constant pointer to this structure. Structure values may change as often as the application requires. Multi-tile display systems will require multiple display configuration structures.

struct il_display
{
    int viewport_x;
    int viewport_y;
    int viewport_w;
    int viewport_h;

    float quality;
    float debug;

    float BL[3];
    float TL[3];
    float BR[3];

    float pitch;
    float angle;
    float thick;
    float shift;

    float *cycle;
    float *step0;
    float *step1;
    float *step2;
    float *step3;
    float *depth;
};
viewport_x
viewport_y
viewport_w
viewport_h

Viewport position and size. The interleaver manipulates viewport parameters to control the off-screen rendering quality.

quality

Rendering quality. This must be a floating point value between 0.0 and 1.0. It determines the degree of decimation of the off-screen render buffers, and thus balances off-screen render performance against on-screen visual quality. Because spatially-multiplexed autostereo display fundamentally introduces a reduction in resolution, a reduced quality value can bring a significant increase in performance with no apparent degradation of output.

debug

Debug scaling. This value is used to scale the line screen pitch and thick values (defined below) in order to exaggerate the scale of the input line screen and output interleaving. This is useful when visually inspecting the output for correctness. The effective pitch is pitch / debug and the effective optical thickness is thick * debug. The default value is one, and a reasonable debugging value is 20 to 100.

BL
BR
TL

The 3D world-space positions of the bottom-left, bottom-right, and top-left corners of the display. These values must be given in the same coordinate system as the eye positions submitted to il_draw and il_perspective.

The remaining fields require a more detailed description. The values pitch, angle, thick, and shift describe the basic configuration of a parallax barrier or lenticular array:

pitch

The line screen pitch is the physical width of each lenticule or barrier strip, measured in screen units: the same units as BL, BR, and TL above.

angle

The line screen angle gives the rotation of the linescreen from vertical, in degrees.

thick

The line screen optical thickness describes the physical distance between a display surface and a parallax barrier, taking into account the optical properties of the barrier medium. Equivalently, it gives the focal length of a lenticular array.

shift

The line screen shift is a screen-space distance allowing the primary direction of the projected pattern to be adjusted left or right.

cycle

The line screen duty cycle is a number between zero and one giving the opaque fraction of each period of the line screen. This field is an array of values, allowing each channel to have an independantly specified duty cycle.

The following image shows the waveform used to modulate each channel on a simple display with no channel crosstalk mitigation. The shaded area denotes one repetition of the wave. The wavelength is determined by the line screen pitch (a screen-space measurement), while the opaque portion of the channel is determined by the line screen cycle (a fraction). The physical width of the opaque region is simply pitch * cycle and the physical width of transparent region is pitch * (1.0 - cycle).

The following image shows the waveform used to modulate each channel using crosswalk mitigation. The step0, step1, step2, and step3 fields give fractional values between zero and one (just as cycle) that soften the transitions between opaque and transparent.

The depth is a positive value determining the strength of any destructive interference between the current channel and its adjacent channels.

As with the cycle, each of these fields is an array of values, allowing the shape of the waveform to be specified independantly for each channel. Note that the cycle fraction includes the step fractions. Thus, when the step and depth fields have their default zero value, then the waveform behaves in the basic mode, as shown in the previous image.

Given the dynamically sized fields of the display configuration structure, a few convenience functions are provided to simplify its allocation and deallocation.

struct il_display *il_init_display(int n)

Allocate and return an interleaver display. The n argument gives the number of view points.

void il_fini_display(struct il_display *display)

Finalize the interleaver display and release all allocated memory associated with it.

Rendering

void il_prep(const struct il_context *context, const struct il_display *display, int view)

Prepare to render a view of the scene. Applications should call this function just before rendering each view. The view argument selects the off-screen buffer to be used, between 0 and n, the value passed to il_init_context.

contextInterleaver context
displayDisplay configuration
viewView index

void il_draw(const struct il_context *context, const struct il_display *display, const float *pos)

Interleave the rendered views on the display. Applications should call this function only after rendering all view points.

contextInterleaver context
displayDisplay configuration
posView positions

The pos array should contain 3 * n floats giving n 3D floating point position vectors.

Utility

void il_perspective(const struct il_display *display, const float *pos, float near, float far, float *M)

Compute a perspective projection matrix. This projection will be correct for the given view position pos and the screen corners defined by the given display configuration.

displayDisplay configuration
posView position
nearNear clipping distance
farFar clipping distance
MProjection matrix (return)

For a detailed examination of how this function works, see this document.

CAVELib applications need not call this function, as the proper projection is computed by the CAVELib. In general however, this projection is not trivially computed. Standalone OpenGL applications based on SDL or GLUT will find this function useful.

float il_viewpoints(const struct il_display *display, float px, float ipd, float *pos, int n)

Generate n view points in pos, properly positioned for the given display.

displayDisplay configuration
pxPixels per channel
ipdInterpupillary distance
posView position return
nNumber of views

The px argument gives the desired interleaving channel size in pixels. The minimum feasible value is 0.333, which implies the tightest possible LCD sub-pixel interleaving. Using the display configuration given by display, this function computes an optimal view distance. Using ipd, the interpupillary distance, an array of n eye positions are computed and positioned at the optimal distance.

View positions are returned in pos as a list of 3 * n floating point values giving n 3D positions, just as expected by il_draw. The float return value is the computed optimal view distance.

Example

The following code demonstrates the steps an application must take during a screen update. Assume here that context was received from a call to il_init_context, that display is a pointer to a correctly initialized il_display structure, that pos is an array of floating point 3D positions as returned by il_viewpoints, and that draw_scene() performs a normal OpenGL rendering of the scene.

    /* Render all views to off-screen buffers. */

    for (i = 0; i < views; ++i)
    {
        float M[16];

        il_prep(context, display, i);
        il_perspective(display, pos, 1.0f, 100.0f, M);

        glMatrixMode(GL_PROJECTION);
        glLoadMatrixf(M);
        glMatrixMode(GL_MODELVIEW);

        glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
        draw_scene();
    }

    /* Interleave all views to the screen. */

    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
    glDepthMask(GL_FALSE);

    il_draw(context, display, pos);

    glDepthMask(GL_TRUE);

Note here that the depth mask is disabled in order to ensure that the full-screen rectangle rendered during the interleaving pass is not occluded by the depth buffer.

Notes

Applications should take a few simple precautions to avoid conflicting with the interleaver. The vast majority of applications need not be concerned with these issues. These special cases do not impose significant restrictions on the application. Conflict avoidance is merely a matter of clean OpenGL programming style.

Applications must initialize GLEW before calling il_init_context.

Applications should not assume that their destination frame buffer is frame buffer object 0. Applications that generate their own frame buffer objects should query the currently bound frame buffer object before binding their own. This value should then be rebound when the application's off-screen rendering is complete. Unfortunately, the frame buffer object binding is not a pushable attribute. An explicit query is required.

Care should be taken when manipulating the viewport during scene rendering. Depending on the quality setting, the viewport rectangle may differ from the expected value during off-screen rendering. Applications that wish to manipulate the viewport should first query the current rectangle and make changes relative to the returned value. Keep in mind also that the scissor rectangle may need to be set to match the viewport rectangle. Viewport and scissor state are both pushable attributes.

kooima@csc.lsu.edu