FROM CAD MODEL TO HUMAN-SCALE MULTIMODAL

INTERACTION WITH VIRTUAL MOCK-UP

An Automotive Application

Damien Chamaret, Paul Richard and Jean-Louis Ferrier

Laboratoire d’Ingénierie des Systèmes Automatisés (LISA), Université d’Angers

62 Avenue Notre Dame du Lac, 49000 Angers, France

Keywords: Virtual environment, human-scale, multimodal interaction, CAD model, virtual mock-up.

Abstract: This paper presents and validates a new methodology for the efficient integration of CAD models in a

physical-based virtual reality simulation. User interacts with virtual mock-up using a string-based haptic

interface that may provides haptic sensation to both hands in a large workspace. Visual and tactile displays

provide users with sensory feedback and improve both user performance and immersion. Stereoscopic

images are displayed on a 2m x 2.5m retro-projected screen and viewed using polarized glasses. The

proposed methodology implemented in a low-cost system, has been tested with an automotive application

task. However, the presented approach is general enough to be applicable to a large variety of industrial

applications.

1 INTRODUCTION

Most research on virtual environments dedicated to

Computer Aided Design (CAD) application are

confronted to difficult problems related for instance

to real-time 3D simulation including physics,

integration of multisensorial feedbacks, etc. Another

problem to overcome is the transformation CAD

models to virtual reality (VR) models.

In this paper we present and validate a new

methodology for an efficient integration of CAD

models in a physical-based virtual reality simulation

that provides the user with multimodal feedback.

Haptic interaction is based on the SPIDAR system

illustrated in Figure 1 (Bouguila et al. 2000). The

methodology has been tested with an automotive

application task. However, the presented approach is

general enough to be applicable to other tasks and

industrial applications requiring realistic interaction.

2 RELATED WORK

2.1 Visuo-haptic VR Configuration

Projection-based Virtual Environments such as

CAVEs™ (Cruz-Neira et al., 1993) Workbenches

(Krueger and Froehlich, 1994) or immersive wall

(Richard et al., 2006) are the most popular VR

configurations. They provide a large number of

performance/immersion factors like stereoscopic

visualization, large screens, large manipulation

space, etc. However, adding force feedback to these

configurations without degrading their

performance/immersion factors is not an easy task.

Most general purpose haptic devices, like the

PHANToM (Massie and Salisbury, 1994) are often

used with desktop visualization configurations. Most

of the time, they are not able to adapt to VR

configurations, leading to a degradation of some of

the performance/immersion factors of the VR

configuration.

Some general purpose haptic systems have been

integrated within large screen projection-based VR

configurations (Brederson et al., 2000), (Grant and

A. Helser, 1998), (Garrec et al., 2004).

The only large screen projection based VR

configurations equipped with non-intrusive haptic

system involves the SPIDAR system (Bouguila et

al., 2000), (Tarrin et al., 2003), (Ishii and Sato,

1994) (Richard et al., 2006).

307

Chamaret D., Richard P. and Ferrier J. (2008).

FROM CAD MODEL TO HUMAN-SCALE MULTIMODAL INTERACTION WITH VIRTUAL MOCK-UP - An Automotive Application.

In Proceedings of the Fifth International Conference on Informatics in Control, Automation and Robotics - RA, pages 307-310

DOI: 10.5220/0001506403070310

 SciTePress

Figure 1: Schematics of the human-scale SPIDAR system.

This non-intrusive haptic device provides force feedback

to user’s both hands while moving in a large-scale

workspace.

2.2 Grasp Feedback

Grasp feedback includes both tactile and shape

feedback. Haptic systems exist for both but they are

different and rarely integrated.

The best known solution for providing a realistic

grasp feedback consists of using props. Props are

physical objects held in hand by the user. Props have

been proposed for tasks such as application control

(Coquillart and Wesche, 1999), 3D objects

manipulation (Hinckley, 1994), (Tarrin et al., 2003),

and design. Several psychophysics experiments

demonstrate the benefits of props (Hinckley, 1994).

Props provide stable grasp feedback, intuitive

manipulation as well as realistic shape and texture

rendering.

Props do not allow sensation of the collision

with a surface touched by the prop itself. Combining

props with force feedback is again a difficult task

because most force feedback systems can’t attach

props in a flexible way.

The next section of the paper presents a CAD to

VR methodology. Section 4 describes the developed

prop-based stringed haptic configuration. An

industrial application is presented in Section 5.

Section 6 concludes the paper and describes future

work.

3 CAD TO VR METHODOLOGY

The CAD to VR methodology is illustrated in Figure

2. Our methodology involves different steps such as

model tessellation (1), model integration (2-3) and

sensorial feedback (4-5-6). The CAD to VR model

transformation is illustrated in Figure 3.

3.1 Model Tessellation (1)

The first step was to choose an appropriate common

exchange file format between standard CAD

software (such as CATIA) and 3D general purpose

modelling software (such as 3D Studio Max). The

tessellation procedure consists in decreasing the

number of faces without degrading the 3D shape of

the model.

Figure 2: Schematics of the CAD to VR model

transformation.

3.2 Model Integration (2-3)

Many loaders may be used to integrate 3D models in

a real time C/C++ 3D application. However, this

leads to graphic simulation in which the virtual

objects do not have physical properties. In order to

give physical properties and behaviour to the model,

we developed a single procedure that allows to

automatically obtain both graphical and physical

models of loaded objects. Moreover, the physical

model on which is based the real time simulation,

exactly corresponds to the graphical one.

The physical model is built using PhysX

efficient well-known open source physic engine

from AGEIA (http://www.ageia.com).

3.3 Sensorial Feedbacks (4-5-6)

In order to increase both realism of the simulation

and operator performance during interaction with the

virtual mock-up, different sensorial feedbacks are

provided. Simulated forces are calculated by the

physic engine and displayed on user hand using the

SPIDAR system.

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

308

Figure 3: illustration of a CAD model transformation to a

physicalized VR model.

4 PROP-BASED STRINGED

HAPTIC INTERACTION

In the context of the automotive application

described in the following section, a prop has been

integrated into the SPIDAR system in order to

provide both realistic and low-cost grasp sensation

while performing the task. Figure 4 shows the

attachments of the prop to the SPIDAR (Figure 4a).

As shown in Figure 4b, the user grasps the prop (a

real car lamp is integrated into a plastic part). In

order to increase accuracy, both position and

orientation of the prop are obtained using a Patriot

3D tracking system (http://www.polhemus.com).

(a) (b)

Figure 4: Top view of the prop attached to the SPIDAR

system (a), grasping of the prop by a user (b).

5 AUTOMOTIVE APPLICATION

Our VR human-scale platform opens the door to

many CAD applications requiring realistic

integration modalities such as visualization, audio,

force and tactile feedback. One such application,

from the automotive industry, is described in this

section. It concerns accessibility and maintenance of

car lamps.

Figure 5: Illustration of the final stage of the task: the

application provides both haptic and visual force feedback

to the operator.

5.1 Description of the Application

During the conception stage, car designers have to

make sure that anyone will easily be able to achieve

maintenance task concerning car lamps. In this

context, special attention has to be paid to the

following aspects:

• Accessibility: to be able to reach and pick-up car

lamps,

• Replacement: to be able to remove broken lamps

and replace them by new ones.

Currently, the only solution is to build a mock-

up of the car. The process is of course slow and

expensive. A cheaper and faster solution consists in

realizing the tests on virtual mock-ups. An

additional advantage is that it can be done earlier in

the conception process, which eases modifications.

Our methodology associated to the human-scale

haptic virtual environment highly contributes to the

widespread use of virtual mock-up in a realistic

simulation.

This methodology has been validated in the

previously mentioned context (car lamps

maintenance). The Figure 5 shows screenshots of the

virtual mock-up during the maintenance task. The

FROM CAD MODEL TO HUMAN-SCALE MULTIMODAL INTERACTION WITH VIRTUAL MOCK-UP - An

Automotive Application

309

final stage of this task involves the correct

placement of a car lamp. As illustrated on the

bottom screenshot, both collision detection and force

feedback are visually displayed respectively using a

red colour and clear blue line (orientation of the

force).

5.2 Hardware and Software

Architecture

As opposed to most of the existing virtual reality

human-scale platforms that are based on clusters,

our hardware architecture is based on only one

Personal Computer (bi-Xeon 5150, 4Go RAM and

8800 GTX Graphic board).

The frame rate is however, in the described

application (600 000 Faces), maintained to about 30

frames per second. Thus, the use of a physical

Processing Unit is not necessary.

6 CONCLUSIONS AND FUTURE

WORK

We presented and validated a new methodology for

the efficient integration of CAD models in a

physical-based virtual reality simulation. User

interacts with virtual mock-up using a string-based

haptic interface that may provides haptic sensation

two both hands in a large workspace. Visual and

haptic displays provide users with sensory feedback

and improve both user performance and immersion.

Stereoscopic images are displayed on a 2m x 2.5m

retro-projected screen and viewed using polarized

glasses. The proposed methodology has been tested

with an automotive application task. However, the

presented approach is general enough to be

applicable to other tasks and industrial applications.

In the next future we plan to add a virtual hand

with physical properties to allow dexterous

manipulation of 3D objects. We will also replace the

magnetic tracking system by an optical MOCAP

solution. We will also use our methodology for other

CAD applications.

ACKNOWLEDGEMENTS

The authors would like to thanks the representative

of Valeo Lighting System (Angers - france) involved

in the project, especially Sébastien DENIS and

Xavier GALLARD.

REFERENCES

L. Bouguila, M. Ishii and M. Sato, “Multi-Modal Haptic

Device for Large-Scale Virtual Environment”, 8 th

ACM International Conference on Multimedia, 2000.

C. Cruz-Neira, D. Sandin, and T. Defanti, “Surround

Screen Projection Based Virtual Reality”. Proceedings

of SIGGRAPH, 1993.

W. Krueger and B. Froehlich, “The Responsive

Workbench”, IEEE Computer Graphics and

Applications, 1994.

P. Richard, D. Chamaret, F.-.X, Inglese, P. Lucidarme J.-L

Ferrier: Human-Scale Haptic Virtual Environment for

Product Design: Effect of Sensory Substitution.

International Journal of Virtual Reality, 2006, 5(2):

pp. 37-44.

T. H. Massie and J. K. Salisbury, “The PHANTOM Haptic

Interface: A Device for Probing Virtual Objects”,

Proceedings of the ASME Winter Annual Meeting,

Symposium on Haptic Interfaces for Virtual

Environment and Teleoperator Systems , 1994.

J.D. Brederson, M. Ikits, C.R. Johnson, C.D. Hansen, and

J.M. Hollerbach, “The Visual Haptic Workbench”,

Proceedings of the Fifth PHANToM Users Group

Workshop, 2000.

B. Grant and A. Helser, “Adding Force Display to a

Stereoscopic Head-Tracked Projection Display”,

VRAIS, 1998.

P. Garrec, J.P. Friconneau and F. Louveau, “Virtuose 6d:

A New Industrial Master Arm using Innovative Ball-

Screw Actuators”, Proc. of International Symposium

on Robotics, 2004.

N. Tarrin, S. Coquillart, S. Hasegawa, L. Bouguila, and M.

Sato, “The Stringed Haptic Workbench : a New Haptic

Workbench Solution”, EUROGRAPHICS Proceedings,

2003.

M. Ishii and M. Sato. “A 3d Spatial Interface Device using

Tensed Strings”, Presence, 3(1), 1994.

S. Coquillart and G.Wesche, “The Virtual Palette and the

Virtual Remote Control Panel: A Device and an

Interaction Paradigm for the Responsive Workbench.

Proc IEEE VR, 1999.

K. Hinckley, R. Pausch, J. Goble, and N. Kassell, “Passive

Real-World Interface Props for Neurosurgical

Visualization”, CHI, 1994.

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

310