The Virtual Showcase

An innovative augmented reality display system

 

Introduction

The Virtual Showcase allows for three-dimensional graphical augmentation of real objects placed inside a glass housing similar to a traditional showcase. The Virtual Showcase has the same form factor as a real showcase, making it compatible with traditional museum displays. Scientific or cultural artefacts are placed inside the Virtual Showcase, allowing to add additional virtual objects, explanations, reconstructions or virtual characters used for storytelling. The virtual (and the real) part of the showcase can react in various ways to a visitor, enabling intuitive interaction with the content displayed.

Working Principle

The Virtual Showcase project introduces a new type of stereoscopic, multi-user display system, which allows the presentation of stereoscopic images overlaid on top of real objects to multiple users. The basis for these displays are semi-transparent, half-silvered mirrors, which allow visitors to look through them to see an illuminated, real object inside the showcase. At the same time the mirrors reflect a computer-generated, stereoscopic image, which is overlaid and registered with the real object.

In general the Virtual Showcases consist of two main parts: an assembly of half-silvered mirrors and a stereo-capable graphics display. The showcases' contents are illuminated with a controllable light source, while view-dependent stereoscopic graphics are presented to the observer(s). For our current prototypes, stereo separation and graphics synchronization are achieved with active shutter glasses and infra-red emitters. Since the displayed graphics is view-dependent, the actual positions of the viewers have to be measured while run-time. This is called head-tracking and is accomplished with a tracking device. A large number of different tracking devices are commercial available, today.

The mirror assemblies are placed in such a way that they reflect the projection screen. Users can see real objects inside the showcase through the half-silvered mirrors merged with the graphics displayed on the projection screen. The corresponding images are rendered on a single or multiple PCs with appropriate hardware accelerated graphics boards.

Hardware Prototypes developed at Vienna University of Technology

So far we developed and built two Prototypes of Virtual Showcases: a projector-based single-user prototype, and a monitor-based prototype supporting up to four concurrent users.

Fig. 2: Single-User Prototype with real chessboard inside. On the top, the projector, the mirror, and the cameras used for head-tracking can be seen.

The single-user prototype

A DLP-projector (InFocus LP 350) mounted at top of showcase is used for indirect rear projection onto a horizontally mounted projection screen. A mirror mounted with an angle of 45 degrees deflects the horizontal beam of the projector down to the rear-projection screen. The rear projection screen consist of a thin film which is fixed between two panes of glass obtaining a projection to a preferably plane screen. The rear projection screen (51cmx68cm) is mounted horizontal above a half silvered mirror. The half silvered mirror takes an angle of 33 degrees to the horizontal and reflects the image to the viewer in front of the showcase. Five halogen lamps are mounted beneath the front end of the half silverd mirror for illumination. The dimension of the whole box is: length=73cm, width=56cm, height=175cm.

Active shuttering, where shutterglasses are synchronized with the graphics card, is used for enabling stereo vision. For head-tracking, ARToolkit-based optical tracking is used, which gives results accurate enough for experiments, but not for real-world usage. A firewire camera is attached to the top of the showcase observing the area in front of the showcase. Additionally, an optical marker is attached to the shutterglasses which are worn by the viewer. This marker is then being recongized by the software, which allows for determining the position of the viewers head from the markers.

Interaction with the showcase is possible in two ways. On the one hand by means of a touchpad, similar to those built into laptop computers, attached to the front of the showcase, on the other hand by using a "laser" pointer device. The laserpointer consists of a simple wand with two ARToolkit markers, one attached to the front and one attached to the end of the wand. From the recognized position of the markers, the direction of a virtual laserbeam can be calculated. This beam may then be used for interacting with the content.

This hardware prototype may be seen as a first basic approach to the Turkish Chess Player scenario (see below). Similar to this scenario, a single user resides at a rather fixed position in front of the showcase. The field of view is restricted to the front of the showcase. Due to these spatial and optical restrictions, the prototype supports only a single user at a time. According to the definition of the Turkish Chess Player scenario, the movement of the users head may be assumed to be rather marginal, but is nevertheless tracked.

The size of the area behind the half-silvered mirror, which may contain a physical exhibit, is 50cm x 50cm. The rather small angle between mirror and showcase floor (33 degrees) restricts the height of the exhibit to about 20 cm. In the example scenario the showcase contains a physical chessboard, which is augmented with virtual chess figures. The closed and compact construction of the prototype provides a reasonable visual quality of the augmentation also in bright environments and allows for a certain degree of mobility.

Fig. 3: Monitor-based four user prototype. At the top, you see the emitter for magnetic tracking and the projector used to illuminate the object.

Four-user prototype

Our latest prototype user four CRT monitors as image-geerating devices, which gives us a comparably cheap way to generate high-resolution high-frequency stereo graphics for up to four users. The rendered images are reflectedfrom acrylic glass plates covered with a semi-reflective film, assembled as a pyramid stub as shown in the image.

For head- and interaction tracking we use our old ascension flock of birds system, which gives reasonable tracking quality as long as the user doesn't come too close to the metal frames of the monitors.

A DLP projector is used to illuminate the contents of the showcase, which gives us pixelwise control over the lighting of the real object and provides the basis for some special lighting effects (see below). The control of the light projector has been integrated into our OpenInventor-based augmented reality framework Studierstube.

This prototype comes closest to a virtual showcase how it might be standardized as a generic showcase for museums or other users. By using monitors and other off-the-shelf hardware (like PC with state-of-the-art graphics acceleration cards), the cost and availability of the components reached a level reasonable even for smaller museums and companies.

Rendering the Showcases Content

Because of the nature of the showcase's display, special image-generation techniques have been developed to create the impression of a seamless transition from the real world to virtual content and vice versa. The virtual image has to be rendered in a way that, after mirroring, it appears to be aligned with the real content of the showcase. The real objects inside the showcase can be illuminated by a projector, allowing for pixelwise illumination effects to create mor realistic results.

Occlusion shadows

Occluding virtual objects by the real objects in the showcase requires rendering a virtual representation of the real object as an occluder object into the z-buffer of the graphics card. Even more interesting is if we try to occlude the real objects inside the showcase by virtual content. If the real object is uniformely lit across its surface, because of the half-sivered mirrors, the light reflected from the surface will always shine through to the user and therefore all virtual objects will appear semi-transparent. If we use a projector to illuminate the real object, we can calculate the pixels on the surface of the real object that are occluded by virtual content, and render these pixels black. Therefore, the virtual objects appear solid and clearly in front of the real objects. The algorithm for calculating occlusion shadows in OpenGL is published by Oliver Bimber.

Going even further, we could use the light projector to render radiosity effects onto the real content, i.e. illuminate the real objects with virtual (colored) light sources or calculate reflections of virtual objects onto the real ones.

Fig. 4: The virtual object rendered in front of an uniformly lit real object (l.), the real object illuminated by a projector, using occlusion shadows (m.), and the virtual object in front of the partially-lit real object, appearing to be solid and opaque (r.) Fotos: Oliver Bimber.

Authoring Presentations

Presentations for the virtual showcase should be interactive, but have to use simple and robust interaction tools due to the requirements of a public installation (robustness of the devices and untrained audience). A good presentation should also include the real objects inside the showcase as key objects of the story that is being told.

To support users in creating presentations that fullfill these requirements, we develop APRIL - the Augmented Reality Presentation and Interaction Language. See the APRIL project pages for more information.

Applications

Fig. 5: Architectural model inside the showcase, overlayed with a hypothetical (virtual) roof construction.	Fig. 6: The Turkish Chess Player inside the Virtual Showcase. A similar installation will be shown at the Technical Museum Vienna.	Fig. 7: Detailed view of the chess-application, showing virtual chess figures on a real chessboard.

Further Information

Contact: Florian Ledermann ledermann@ims.tuwien.ac.at

Project Partners - Links to the partners involved in the Virtual Showcases project.

Acknowledgements

The project "Virtual Showcases - Presenting hybrid exhibits" is a collaborative research project of ten research partners, industrial partners and museums from Austria, Belgium, Germany and Portugal funded by the European Union (IST-2000-28610). 3d-scanning of objects was supported by the innovative project '3D technology' of Vienna University of Technology.