USING WIRELESS TECHNOLOGY TO DEVELOP A VIRTUAL

REALITY COMMAND AND CONTROL CENTRE

Damian Green, Neville Stanton, Guy Walker, Paul Salmon

BITLAB, Brunel University, Uxbridge, Middx, UB8 3PH, England

Keywords: Wireless Technology, Applications, Command & Control, Human Factors

Abstract: This paper investigates the applicability of wireless communication systems for use in a command and

control environment. Human positional data is transmitted over a wireless network. This data is then used to

update a highly accurately modelled real-time 3D environment of the surroundings, with avatars positioned

at the transmitted points. The data is displayed on a stereoscopic 3D screen enabling novel automatic

tracking of human movement and allowing for more rapid and informed tactical decision making. This

system has applicability in a variety of situations including Command and Control in essential services –

fire, rescue and military.

1 INTRODUCTION

Contemporary studies of command and control in a

variety of domains show significant shortcomings in

human performance.

A study by Baber et al (Baber, 2004) observed

an exercise on the Hazardous Materials course at the

Fire Service Training College in the UK. The

incident management comprises five phases: initiate

response to incident (1), perform initial incident

assessment (2), chemical identification (3), chemical

assessment (4), and resolve incident (5). From

Baber et al’s analysis, it is possible to see that the

information that could be drawn upon during the

incident becomes available to the performers at

different phases during the unfolding of the incident.

Thus, the fire-fighters are not aware that the child

has respiratory problems until around phase 4, or

that the farm is deserted until well into phase 3, i.e.

until arrival at the scene. With this knowledge, the

definition of response is made easier, the question of

which Personal Protective Equipment to wear can be

solved and the search strategy simplified. Thus,

Situation Awareness, from a system perspective, can

be viewed as the sum of knowledge relating to

specific topics within the system. The challenge in

incident management becomes one of ensuring that

the appropriate agents have access to appropriate

knowledge of topics at the right time.

With a view to military applications, friendly-

fire remains a problem for even the most wired

military in history, with aerial and artillery attacks

the most serious. So far, 18 of the 149 fatalities

suffered by U.S. and British soldiers in Gulf War II

were the result of errant strikes by coalition forces.

These tragic numbers appear to be an improvement

over Gulf War I, which recorded 35 deaths from

friendly fire - 24% of all combat fatalities, vs. 12%

in Gulf War II. But the U.S. and its allies deployed

twice as many troops in the first Gulf conflict. And

as the Pentagon gathers the final data from the war,

the number of friendly-fire deaths could increase.

Although the risks of armed conflict on a Cold

Ware scale may be lower, there is increasing

turbulence world-wide, with persistent mid to low-

intensity threats, a trend that is likely to (Robinson,

2003) continue. Throughout the Cold War, nations’

military forces prepared for conventional war – tank

against tank, ship against ship and plane against

plane. Within the past decade, the use of military

force characteristically has not been state-on-state

but more typically exercised in regional

engagements in peacekeeping, peace enforcement

and disaster relief. Armed conflict continues to

become more complex and to require even greater

precision (Hubbard, 2003). Providing armed forces

with new capabilities is essential (Hubbard, 2003).

This work’s primary purpose is to investigate the

applicability of modern technology to save lives in

essential services and to reduce unnecessary friendly

fire incidents.

Green D., Stanton N., Walker G. and Salmon P. (2004).

USING WIRELESS TECHNOLOGY TO DEVELOP A VIRTUAL REALITY COMMAND AND CONTROL CENTRE.

In Proceedings of the First International Conference on E-Business and Telecommunication Networks, pages 52-59

DOI: 10.5220/0001393500520059

 SciTePress

1.1 Related Work

One technology developed to reduce friendly fire is

the Battlefield Target Identification (BTID), an

updated version of the Battlefield Combat

Identification System (BCIS). BTID uses

transmitters (as did BCIS) that aim very shortwave

bursts of communication data at a target just before

letting go a salvo. If the target responds with the

correct encrypted reply indicating that it's a friend

then the gunner's sight will turn red. The system is

accurate at distances of up to three miles (Science,

2003).

A second program is called Individual Combat

Identification System (ICIS). It comes in two parts: a

specialized laser sight mounted on a rifle and a

combination laser-detection device and radio

transponder mounted on a soldier's helmet. When a

soldier takes aim with the ICIS laser, the beam will

activate the detector if the person in the sights wears

the helmet of a U.S. soldier. The transponder will

then send an encrypted signal warning the shooter

not to fire. ICIS uses off-the-shelf components and

weighs less than two pounds. It can run for 30 days

on a single camera battery.

Soldiers who already carry 70 pounds of

equipment on their backs cannot carry a heavy

broadcast device powerful enough to reach fighter

bombers. And those devices would need to generate

a signal in a radius of 360 degrees to be able to warn

not only planes but tanks and other soldiers.

Better networks for the military should make it

easier to avoid friendly fire incidents. Many support

units in Iraq lacked standard radios, let alone high-

tech communications devices. Key questions remain

as to how to fit friendly-fire information into a

decision that usually involves but a few seconds.

The right choice can make the difference between

blowing up allies or taking fatal fire and casualties

from an enemy. Friendly-fire deaths could remain an

inevitability of battle for years to come.

This paper sets out to investigate the various

technologies applicable for carrying out wireless

communication of human positional data with a

view to the communication of more detailed data

such as audio and video. 3D data is communicated

via a suitable network for the use in a command and

control environment which is carefully designed

around the human controller’s needs. This needs-

analysis is determined using human factors methods.

Such methods include task analysis techniques,

observational studies, verbal protocol analysis,

critical decision method, social network analysis and

coordination demand analysis (Stanton, 2004).

2 THE TECHNOLOGY

To provide front line forces with world class

systems these have to be developed from the outset

with the human operator in mind. The purpose of

this research is to investigate the possibility for the

wireless transmission and receiving of data to and

from a person in the field (PINF) to a command and

control centre (CC). The purpose of this work is to

create processes and methods which will be

integrated as part of future systems.

Figure 1: PINF Hardware

The data transmitted from the person in the field

may include Global Positioning System data (GPS),

Differential GPS (DPGS), audio and video (AV).

The data received back to the PINF from the CC

may include GPS data of other personnel on the

field, and relevant video data,

Figure 1. If GPS data of ally positions is received by

the PINF, a variety of methods can be used for the

displaying of this data on a headset, (radar notation

etc).

Figure 1 illustrates example hardware technology

that may be adorned by the PINF.

USING WIRELESS TECHNOLOGY TO DEVELOP A VIRTUAL REALITY COMMAND AND CONTROL CENTRE

Internet

WLAN /

GPRS

MFC IVF (C++ )

Stereo

Application

Winsock

component

Server

Satellite Satellite Satellite

Laptop

GPS

Receiver

PDA

Large Screen X 3

CRT

projector

X 3

Router

High-Power

Rendering Machines with

Dual-Head Graphic Card

COMMAND & CONTROL

Figure. 2: Overall System Concept

2.1 Hardware Options for Wireless

Position Communication Using

GPRS

To demonstrate a proof of concept, various

commercially available technologies were

investigated for their applicability. It should be

noted that for real-world military application, these

technologies may not be secure, fast or reliable

enough. The main drive of this research is to

demonstrate the 3D technology; however, much of

the technology which may be suitable for this kind

of application is still under development. The

following sub-sections suggest various commercial

off the shelf (COTS) hardware configurations.

2.1.1 Laptop and GPS Receiver with WLAN

Card

An initial proposal for the PINF hardware may

include a compact laptop, for example the ultra

lightweight (4.39 lbs) and compact design Sony

VAIO® V505DC1 Notebook, connected to a GPS

receiver via an RS232 cable, which is in turn

connected to a wireless LAN card internally or

through the PCMCIA slot, Figure 3a. With the

option of external amplified antenna and PCMCIA

option the NavRoute WAAS enabled GPS HP-60

device can be used in a notebook computer. Further

investigation is needed to determine whether the

GPS position can be constantly streamed from this

device. The advantage to this configuration is high

processing power, which may be necessary when

streaming and compressing video data. The

downside is the lack of compactness and the

inherent superfluous technology – keyboard, LCD

screen.

2.1.2 PDA With Inbuilt GPRS With GPS

Compact Flash Device

A second possibility requires using a more compact

device, such as a Personal Digital Assistant (PDA)

with inbuilt GPRS and a Compact Flash GPS

Receiver device plugged in, as shown in Figure 3b.

The disadvantage to using PDAs is the lack of

processing power needed to perform video

compression (which may be required on future

prototypes). A PDA that includes a relevant

Software Development Kit (SDK) for developing

streaming applications would need to be chosen and

evaluated. The NavRoute WAAS enabled GPS HP-

600 is compatible with PDAs with a Compact Flash

socket.

2.1.3 PDA with Inbuilt GPRS and GPS

A third possibility may be through the use of a PDA

with inbuilt GPS and GPRS as shown in Figure 3c.

ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS

A study is needed to determine the suitability of any

available SDK exist for each specific PDA model.

The model shown is the Garmin iQue 3600 with

built in GPS PDA.

Disadvantages to the possibilities identified in

sections 2.1.2 and 2.1.3 include the lack of sockets

for the attachment of devices such as webcams and

microphones for further communication. The

webcam on the PDA itself may not be sufficient

because it would need to be held near the PINF’s

head. It is more advantageous to have the PDA

separate from acquisition technology.

2.1.4 Local Positioning Systems (LPS)

LPS originated from the operating concepts of the

global positioning system (GPS) satellite network,

which takes signals from orbiting satellites to track

large out-door assets, such as automobiles, trucks

and railway cars.

GPS signal is too weak to penetrate walls and

other obstacles and its commercial resolution is too

imprecise to provide meaningful location data for

indoor applications. Using these technologies and

methods, LPS uses electronic tags, discreet antennas,

and low-power radio signals to instantly locate,

track, secure and inventory assets. It is ideal for

stationary assets, such as large physical inventories,

or for mobile assets, including people and portable

equipment.

PDAs can be used in conjunction with custom-

built hardware devices which receive VHF/UHF

singals and triangulate. This position can then be

sent using the PDAs over a wireless LAN or using

GPRS.

2.1.5 WLAN Triangulation

A laptop wireless can be used in conjunction with

triangulating software. Ekahau (Ekahau, 2003) have

developed enhanced position determination software

to allow positional information to be derived from

triangulation by WLAN access points. The Ekahau

Positioning Engine™ (EPE) is a positioning server

that provides PC, PDA, and asset TAG location

coordinates (x, y, floor) and tracking features to

client applications. EPE uses a stand-alone manager

application for drawing the Ekahau Tracking

Rails™, recording site calibration data, tracking

wireless devices on map, and statistically analyzing

the positioning accuracy.

Ekahau's positioning technology has an average

accuracy of up to 1 meter (3½ ft), enabling people

and asset tracking both indoors and outdoors. This

opens up the possibility of WLAN networks being

deployed purely for indoor tagging and tracking

purposes – an application area currently dominated

by proprietary tagging solutions. Ekahau Manager is

used to create a positional model – A floor-plan or

map image (BMP, JPG, PNG) is opened and Ekahau

Tracking Rails are drawn on the map to increase

positioning accuracy and stability. The Positioning

Model is created by moving around the area while

clicking the map to record sample points containing

received signal strength intensity (RSSI) samples.

No information about the access point locations is

required.

2.1.6 Command and Control

Communication Hardware

If GPRS is used as the communication medium, at

the CC end, one GPRS phone with contract is all

that is required, and the CC connects to the mobile

device through a standard network connection. If the

CC is non-mobile and network capable then this is

no problem. The O

company offers such an ISP

connection. However if the CC is mobile yet is still

in the vicinity of mobile phone base-stations then a

GPRS card and dedicated phone line is necessary.

For the CC environment, a large Stewart stereo wall

was used (Figure 4) (Corporation, 2003).

If WLAN is used as the transmission medium, a

connection to the Internet is all that is needed at the

CC end.

2.1.7 Hardware Options Discussion

Some of the PDAs described have GPRS

transmission capability. In absence of a WLAN, this

may be a viable communication medium. The

emerging standard of Universal Mobile

Telecommunications System (UMTS) may be more

appropriate. UMTS represents an evolution in terms

of services and data speeds from today's "second

generation" mobile networks. UMTS is the natural

evolutionary choice for operators of GSM networks,

which currently represent a customer base of more

than 850 million end users in 195 countries and

represents over 70% of today's digital wireless

market

The Ekahau solution sounds like a good

approach for indoor triangulation or in cases where

WLAN coverage is great and GPS is unavailable.

The accuracy is not enormous however and it

requires significant pre-calibration. It is difficult to

imagine how the deployment of WLAN devices and

the creation of positional models would be useful for

anything other than the development of aesthetically

pleasing prototypes for a system which would later

use alternative positioning technology. The fact that

the software requires 5 MB of RAM may hinder its

use on PDAs.

USING WIRELESS TECHNOLOGY TO DEVELOP A VIRTUAL REALITY COMMAND AND CONTROL CENTRE

PDAs using the Pocket PC operating system

(OS) capable of GPRS are available on most of the

HP modern HP series; Jornada, iPaq 5500, 5550 and

so on. Combined with a Pretec Compact GPS card, a

small portable wireless programmable GPS solution

can be realised. This combination is shown in Figure

3. However if the device is in the vicinity of a

WLAN then a PDA which has integrated WLAN

802.11b such as the HP 5500 would allow free

transmission of position in areas with WLAN

coverage, such as the environment mentioned. One

which requires a plug-in WLAN is unsatisfactory

because there is then no space to include a GPS

receiver.

Outdoors, GPS devices are easily hampered by

adverse weather conditions – cloud cover etc.

Bluetooth wireless technology is designed to

replace cables between cell phones, laptops, and

other computing and communication devices within

a 10-meter range. As such it is not useful for the

transmission of GPS data to a remote CC, although

it may have useful applications for local PINF

communications (comms). Wi-Fi is wireless

network; it provides an extension or replacement of

wired networks for dozens of computing devices.

Figure 3: Candidate Hardware Configurations for Client

Position Communication Device a) Compact PC with GPS

receiver connected via GPRS Wireless PCMCIA card. b)

PDAs with GPS. c) PDA with inbuilt GPS. d) PDA with

GPRS and CF GPS

Figure 4: Command and Control Centre shown on a stereo

Stewart Screen at the Bitlab, Brunel University, UK

2.2 Developing a GPS Receiving

GPRS PDA Application

A variety of PDAs were acquired for development

purposes, including the HP Jornada which has GPRS

and a CF slot for a GPS device, as well as numerous

IPaq 5550s which have both WLAN capability and

CF slot for GPS receivers. The Microsoft Visual

Studio .NET Application Wizard was used to create

a Visual Basic Smart Device Application. The

.NET Compact Framework is a hardware-

independent environment which can run programs

on resource-constrained computing devices,

encompassing personal data assistants (PDAs) such

as the Pocket PC, mobile phones, set-top boxes,

automotive computing devices, and custom-designed

embedded devices built with the Windows CE .NET

operating system.

The number and members of classes differ from

what is available for developing desktop

applications. Fewer classes are available for devices

using the .NET Compact Framework (for example,

Web forms, remoting, messaging, management, and

printing are not supported), and the enumeration of

classes typically differs among platforms.

ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS

Figure 5: An Early GPS Radar Application

Running on an HP Jornada PDA

2.2.1 Receiving GPS

Programs which allow communication with GPS

devices were investigated. An example VB

application which communicates between a PDA

and GPS device was found on MSDN (MSDN,

2003). However it required the use of a COMM

control. Visual Studio .NET does not have this

control as standard and Visual Studio 6, or VB 6.0

needs to be used, or through a technique known as

Interop. Interop is hidden from the programmer by

Visual Studio so that it looks like another .NET

control. Using Interop adds an unneeded level of

complexity and performance penalty to programs.

An early prototype of a client application which uses

GPS positioning and a radar type interface is shown

in the photograph in Figure 5.

2.2.2 Communication

An initial protocol was developed for the

transmission of multiple person positions to test the

principle.

where..

P is the position of the person (x,y,z)

id is a unique number identifying the

person who’s position is being tracked

etx is x03h (a unique header)

stx is x02h

chk is a checksum of the previous bits

for error detection

2.3 Command And Control

To develop a Command and Control application

which communicates with wireless technology and

displays this data in 3D stereo in real-time, careful

consideration needs to go to the choice of software

platform.

Visual Basic provides rapid application

development (RAD) but is lacking in fast real-time

3D graphical display. The use of the OpenInventor

(OIV) IVF interface library for gluing Inventor to

Microsoft Foundation Classes (MFC) works well

using the C++ platform (TGS, 2003). The purpose of

the IVF class library is to simplify integration of

functionality provided by the Open Inventor Win32

Classes with MFC-based applications. IVF builds

upon the application framework provided by MFC to

deliver the additional functionality of Open Inventor

in a manner familiar to MFC developers.

A "socket" is an endpoint of communication: an

object through which an application communicates

with other Windows Sockets applications across a

network.

MFC contains two different classes for

communicating via sockets (WinSock) (Microsoft,

2003). TGS’s OIV allows for rapid development

through its Visual Studio AppWizard extensions. It

also allows for easy stereo extensibility to

applications, which is necessary for command and

control visualization on the screen shown in Figure

A preliminary application has been developed

which uses the aforementioned technology as shown

in Figure 6. The application imports a 3D model in

the VRML format (.iv) of the environment over

which data is wirelessly communicated. This has

been modelled from AutoCAD survey data from the

estates department of the university and using the

theodolite measured point and line data to extrude

areas of grass as a baseline for the modelling of

buildings. Some buildings have been left as

wireframe extrusions of survey data to show the

different representational schemes possible to the

commander. Further investigations will determine

the most useful representation of the situation. A

schematic diagram of the software configuration is

shown in Figure. 2. Initial tests of the software have

realized impressive update speeds, even on a

Alienware 3.2 GHz, 1GB RAM desktop machine.

2.4 Human Factors

Rather than suggest that one form of representation

suits all types of task, our proposal is to investigate

the appropriateness of each representation for each

aspect of the scenario and the decisions being made.

It is anticipated that more strategic, tactical and

operational tasks might require different types of

representation. For example, formulation of

strategies might best be supported by more abstract,

goal-oriented displays of information. Operational

tasks might be better supported by a physical

USING WIRELESS TECHNOLOGY TO DEVELOP A VIRTUAL REALITY COMMAND AND CONTROL CENTRE

analogue representation of the real world. Tactical

tasks might require both sorts of representation, or a

merging of the two forms of data.

Matching the team role to the representation

might be an overly constrictive approach to interface

design, but allowing members of the team to have

access to different representations of the scenario

relevant to the task that they are controlling may be

more fruitful.

Operational testing of the prototype environments

will be undertaken within an experimental paradigm.

First, we need to judge exactly what performance

gains can be made over conventional command and

control methods. Second, we need to discover what

process improvements there are to be made with

these new forms of command and control

technology. There may be additional performance

improvements to be realized by using the technology

to perform tasks more effectively.

General research questions into multiple agent-

based computing interaction needs to be addressed

It is inevitable that network enabled computing will

lead to new forms of interaction. As the task-

artefact cycle products, new artefacts produce new

tasks and new tasks produce new artefacts. Network

enabled technologies are likely to have a

considerable social component to the interaction,

supporting human-human interaction via computing.

3 CONCLUSIONS

Robinson et al. write that for sound legal and

operational reasons in our pluralistic society, we

shall require an audit trail of operational decisions

and consequences (Robinson, 2003). By virtue of the

fact that all movements are streamed over a network

in the approach described, human paths can be

recorded and later replayed with a minimum of

effort in a 3D environment. This allows for a better

criticism of tactics employed with the aim of

improving future tactics. This could provide useful

input to movement behavioral models such as those

developed by Reece (Reece, 2003). This research

project is still in its early stages and with the

intended Human factors investigations; detailed

testing of these technologies will likely unfold some

interesting results.

The accuracy of GPS signal, urban density,

weather conditions and wireless LAN coverage of

the environment are all limiting factors to the

approach’s usefulness. In a new, unmodelled and

unwired environment the full potential of this

technology would not be realized to the detail

shown. Although it is envisaged that the use of

photogrammetric approaches could be employed.

For example, an aircraft could be flown at a constant

altitude over the environment taking a series of

images; the images would then use feature matching

to create a coarse point cloud.

Future work with VR headsets and haptic devices

for more immersive visualization and interactivity

both at the CC and at the PINF side is a topic set to

be investigated.

There are many potential applications of the

technology in civil, military and emergency services

command and control. Any domain that has a

central command and distributed workforce has the

potential to benefit from the insights resulting from

this research.

ICETE 2004 - WIRELESS COMMUNICATION SYSTEMS AND NETWORKS

Figure 6: Proprietary OpenInventor Application Showing 3D Model of Environment Used for Wireless Positional

Communication

ACKNOWLEDGEMENT

This research was made possible by funds provided

by the UK Ministry of Defence through DSTL to

support the Human Factors Integration Defence

Technology Centre.

REFERENCES

Baber, C., Walker, G., Salmon, P., Stanton, N A. 2004.

Observation Study Conducted at the Fire Service

Training College. Human Factors Integration -

Defence Technology Centre Report. Fifth Conference

of Human Performance, Situation Awareness and

Automation Technology, Daytona Beach, Florida.

Corporation, Stewart Film Screen. Stewart Screens.

Accessed: 2003. http://www.stewartfilm.com

Ekahau. Triangulation Software. Accessed: 2003.

http://www.ekahau.com

Hubbard, M., Fentone, Col G . 2003. "Non-lethal

capabilities and the road ahead." Journal of Defence

Science 8(3): 210-214.

Microsoft. MSDN. Accessed: 2003.

http://msdn.microsoft.com

MSDN. Migrating a GPS App from the Desktop to

eMbedded Visual Basic 3.0. Accessed: 2003.

http://msdn.microsoft.com/msdnmag/issues/01/01/GP

S/default.aspx

Reece, Douglas A. 2003. "Movement Behavior for Soldier

Agents on a Virtual Battlefield." Presence

(Teleoperators and Virtual Environments) 12(4): 387 -

410.

Robinson, P., Pickard, Lt Col I. 2003. "The UK approach

to future Command and Inform (C4ISR)." Journal of

Defence Science 8(3): 179-186.

Science, Federation of Military. Battlefield Combat

Identification System (BCIS). Accessed: 2003.

http://www.fas.org/man/dod-101/sys/land/bcis.htm

Stanton, N. A., Hedge, A.; Brookhuis, K.; Salas, E. and

Hendrick, H 2004. The Handbook of Human Factors

and Ergonomics., London: Taylor and Francis.

TGS. TGS Open Inventor. Accessed: 2003.

http://www.tgs.com

USING WIRELESS TECHNOLOGY TO DEVELOP A VIRTUAL REALITY COMMAND AND CONTROL CENTRE