MULTI-AGENT AND EMBEDDED SYSTEM TECHNOLOGIES

FOR AUTOMATIC SURVEILLANCE

M. C. Romero, F. Sivianes, A. Carrasco

Departamento Tecnología Electrónica, University of Seville, Avda. Reina Mercedes s/n 41012, Seville, Spain

M. D. Hernández, J. I. Escudero

Departamento Tecnología Electrónica, University of Seville, Avda. Reina Mercedes s/n 41012, Seville, Spain

Keywords: Multi-agent system, embedded system, surveillance, telecontrol, SCADA.

Abstract: Supervisory Control and Data Acquisition (SCADA) systems have traditionally used text-based Human

Machine Interfaces (HMI). We propose a system which integrates multimedia information in SCADA

systems in order to improve and support the telecontrol tasks. This system has been deployed in a real

environment and we have obtained satisfactory results. Then, we also propose an improvement for this

system. This improvement allows telecontrol operators to use the system without needing any experience

with computers and also it allows an automatic surveillance of the elements in the utility environment. The

development of this improved system is accomplished by using the main advantages of embedded and

multi-agent system technologies.

1 INTRODUCTION

The main goal of every SCADA (Boyer, 1999)

system is the remote supervision and control of

devices –mostly sensors and actuators– which are

located at remote facilities.

The information provided by these devices, for

example the measurement of the current in a power

line is short, not longer than a few dozen bytes. As a

result, the traditionally used technologies are still

suitable for the delivery of devices data, and

probably will be in the future.

However, transmission and networking

technologies are not the only fields that have

dramatically been improved over the last years.

Multimedia compression techniques have also

evolved in such a way that digital video and audio

can now be processed by cheap PCs, obtaining good

quality displays using relative low bit rates. Modern

codecs such as MPEG-4 (ISO/IEC, 1991) allow for

the compression of high resolution video signals so

they can be meaningfully represented using less than

1 Mbps.

Human Machine Interfaces, used to make the

management of the SCADA network operators

easier, typically display the status data as text

strings, and in some cases, in the form of symbols

representing the state of the device. Complementing

that information with video or audio can be of much

help when operating wide SCADA networks, where

there is usually no staff at remote stations.

Automatic surveillance or videoconference

(Romero, 2004), are other bonus features that

multimedia information could bring to SCADA

systems.

2 MULTIMEDIA IN SCADA

SYSTEMS

Rather than talking about integrating multimedia

data into SCADA systems, it would be better to

think about integrating SCADA data into modern

network systems.

Typical transmission links and protocols used on

SCADA networks do not fulfil the bandwidth

requirements that multimedia data deployment

impose, as data acquired from SCADA devices is

usually transmitted over low-bandwidth links using

serial protocols.

Another important key factor is that temporal

requirements on the reception of the acquired data

are very tight on SCADA systems; cycle time on

391

C. Romero M., Sivianes F., Carrasco A., D. Hernández M. and I. Escudero J. (2008).

MULTI-AGENT AND EMBEDDED SYSTEM TECHNOLOGIES FOR AUTOMATIC SURVEILLANCE.

In Proceedings of the Tenth International Conference on Enterprise Information Systems - AIDSS, pages 391-396

DOI: 10.5220/0001670803910396

 SciTePress

SCADA buses generally ranges between 10 ms and

100 ms. Furthermore, transmission reliability is also

very important as loss of data might cause a big

impact to the integrity of the system.

We have successfully injected SCADA data into

an Ethernet system, using a custom SCADA bridge (

Figure

1). Data transmitted using the IEC870

(IEC, 1990) protocol from a Multitrans PLC device

–which gives data about the voltage and intensity of

a power line, among other features– is injected into

an Ethernet network using bidirectional serial-to-

Ethernet converters.

The converter we have used, named the IS-Server

device, takes serial data from the Multitrans device

and converts it into TCP packets. It sends them to a

predefined IP network address. If the control system

needs to receive data in its original serial form, the

inverse process may be applied using another IS-

Server device.

In our tests, status data, which is periodically

polled from a Multitrans device, travels between two

Ethernet-based LANs connected via a 100Mbps

fiber optic link, resulting in the SCADA data being

received in less than 10 ms, even when a raw

multimedia stream, which needs nearly all the

available bandwidth, was also sharing the fiber optic

link.

Figure 1: Ethernet SCADA bridge.

As proven above, from the time point of view,

sharing the physical medium between SCADA and

multimedia data is possible in Ethernet networks,

but transmission reliability must also be considered.

Ethernet hub devices use CSMA-based medium

sharing techniques (ISO/IEC, 1993), which can lead

to packet loss in a high traffic scenario, due to

packet collision. Therefore, switches should be the

starting point on the design of any multimedia

SCADA system.

Network management techniques such as Quality

of Service (Arindam, 1999) might also be used to

ensure SCADA data always gets the bandwidth it

needs. QoS allows packet prioritization and

bandwidth provision, based on one or more factors,

such as the IP address of the sender.

3 IDOLO SYSTEM STRUCTURE

3.1 Interface

We have built IDOLO’s interface as a web-based

software system, supported by a database system,

which manages all needed system configuration and

allows operators to easily have access to all

multimedia information of the SCADA system.

Being web-based means it can be independently

run from an existing SCADA/HMI system, and at

the same time it can be integrated into HMI systems

supporting web navigation. HMI software

supporting ActiveX technology can easily integrate

the IDOLO software system via an embedded

ActiveX web browser.

Furthermore, some web browsers, such as

Internet Explorer, also support ActiveX components;

so software components specific to the IDOLO

system have been developed as ActiveX controls.

The IDOLO navigation system has been

developed using PHP programming language on the

server side, and Javascript on the client side. PHP is

used to control the content and layout of served

HTML pages, and to serve as the interface between

the IDOLO system and a MySQL database server

that holds all system configurations, such as data

about cameras or the design of multimedia synoptics

of each station. Using Javascript permits interfacing

with web-browser, allowing the IDOLO system to

be informed of user input events (e.g. mouse clicks)

or control specific browser properties.

3.2 Camera Management

From a logical point of view, the IDOLO system is

formed by 2 elements: stations and cameras. It’s

hierarchically organized so that stations own one or

more cameras.

We have worked with 3 types of cameras. Each

one has special features, which have made us take

different approaches to manage each camera type.

First, network cameras, which do not include

internal storage systems, need an external system to

store their video. The network camera we have

chosen, AXIS 230, uses MJPEG codec, which is no

more than a sequence of JPEG images. MJPEG is

not an efficient video codec as it only performs

spatial compression (based on Discrete Cosine

Transformation), as opposed to MPEG-4 that

performs both spatial and inter-frame compression.

ICEIS 2008 - International Conference on Enterprise Information Systems

392

As it needs an external FTP server with enough

storage space to stock the video, this can make for

an excessive use of network bandwidth.

Webcams and analog cameras need dedicated

PCs to performing the needed compression and

streaming. We have chosen Windows Media

software platform to deal with these processes,

which allows full control over nearly all aspects

regarding the previously mentioned procedures.

 Windows Media Encoder is a software

component that takes video and audio from

USB cameras and frame grabbers and

compresses them with Windows Media codecs,

variants of MPEG-4 codec in the case of video

and MP3 in the case of audio. It also allows for

storing of compressed content in the fly on

both local and remote storage.

 Although WME also allows streaming using

HTTP/TCP protocols, Windows Media

Services is a specialized component that takes

data compressed with WME and pushes it to

the network using RTSP/UDP protocols. Apart

from using more efficient network protocols

for streaming, WMS is the needed solution for

allowing external stations to receive

multimedia content from WME encoders, in

case IP addresses of encoder stations are inside

the local scope.

3.3 Multimedia Synoptics

Synoptics are the common way that HMI systems

use to show, in a single screen, all representative

data about SCADA elements. The IDOLO system

takes this approach and adapts it to the multimedia

field (Figure 2) by:

 Showing video windows and audio coming

from cameras located at a given station.

 Having graphics of all representative elements

in a station, which, when clicked, command

PTZ cameras to move to a preset position so it

focuses on the selected element.

 Allowing the activation of manual control on

PTZ cameras, which are able to control them

using the mouse or even a joystick.

Figure 2: Screenshots comparing multimedia synoptics

and classic synoptics.

The concept of multimedia synoptics is also a

solution to the inherent delay introduced by MPEG-

family codecs, which makes manual control of PTZ

devices a bit confusing as images run late in respect

to camera movements.

4 DEPLOYMENT ON AN

ELECTRICAL FACILITY

The final goal of the IDOLO project has been the

integration of our system into a real, working

SCADA system. For this reason, we have garnered

the cooperation of the Medina Garvey electrical

facility, a regional electrical provider located in

Seville.

The existing facilities owned by Medina Garvey

have been controlled and supervised using SCADA

hardware and software developed by Team Arteche.

An example of those SCADA components has been

the Multitrans device we have used in our

integration tests.

As we have not wanted to compromise the

integrity of existing system, we have decided to use

separate links for sending multimedia and SCADA

information, from remote stations to the supervision

station. The modern facilities in Medina Garvey

were equipped with fiber optic links to communicate

with supervision station, so we have used spare fiber

cabling for our purposes. These fiber optic links are

more than adequate for transmitting high-quality

multimedia streams.

There are some older facilities that do not use

such high-speed links. In such cases, they have used

serial radio links to transmit SCADA status

information, which do not have enough bandwidth

to send multimedia information over them. But these

facilities had also been equipped with telephonic

lines, so we have used ADSL technology for sending

our multimedia streams. However, using ADSL

limited the transmission to fewer, and lower-quality

streams; we have tested the transmission of one 512

Kbps stream, and alternatively, two 256 Kbps

streams, successfully.

In respect to the SCADA/HMI system used by

Medina Garvey, SIPCON HMI system, it’s a

proprietary, MS-DOS based system, which does not

allow the integration of external components.

Therefore, IDOLO software has been deployed

separately, by using a PC with a Internet Information

Services web-server to provide web content to the

intranet.

MULTI-AGENT AND EMBEDDED SYSTEM TECHNOLOGIES FOR AUTOMATIC SURVEILLANCE

393

5 WHY USE EMBEDDED

TECHNOLOGY?

When our system has been installed in that real-life

environment, we have interviewed operators in order

to find out their degree of satisfaction. One of the

improvements that they have proposed has been that

the IDOLO system is like a black box for them. That

is a system which is easy to maintain and configure

without needing any experience with computers (just

by pressing a button). That have made us think about

the possibility of developing the system by using

embedded technology.

Embedded systems shouldn’t be thought of as

PCs with general characteristics. Although, more

and more, they could be replaced in some

application fields thanks to the development of

microprocessors and microcontrollers.

They could be considered as hard – soft

applications which are more interesting in the

immediate future of the Information and

Communications Technology sector. The main idea

is to use systems which are practically invisible and

made up of microprocessors and software in very

small systems. These systems allow us to obtain

information and accomplish its processing

everywhere, in a quick and easy way (ubiquitous

computation). If we are able to create a network

composed of these embedded systems, which are

almost invisible, and we provide them with

communication and action abilities, we can create a

spatially distributed hardware structure that is

available at all times. It makes user anticipate

(pervasive computation) (Marwedel, 2003) (Sutter,

2003). All of this can be accomplished in a more

secure and reliable way and with a quick and easy

maintenance with an economic costs and electric

consumption lower than those of traditional PCs.

By introducing these systems in some

hierarchical schemes of “surveillance” distributed in

substations or transformation centers, we can

accomplish an automatic surveillance which allows

preventive maintenance operations. In this way we

can anticipate possible anomalies that could produce

blackouts, which are quite problematic for the end

users and, as a result, have significant economic

impact for the electric utilities.

6 WHY USE MULTI-AGENT

TECHNOLOGY?

When we are talking about automatic surveillance

we re referring to not only proactive but also

reactive surveillance. This implies that the elements

which carry out the surveillance have human

capacities and are able to make decisions and act by

themselves. This human behavior has to be

programmed, and we are working to achieve this by

using multi-agent system (MAS) technology.

MAS are systems composed of multiple

interacting computing elements, known as agents.

Agents are computer systems with two important

capabilities (Wooldridge, 2002):

• They are capable of autonomous actions and

can decide by themselves.

• They are capable of interacting with another

agents in a social way (cooperation,

coordination, negotiation…)

So, first we are modeling the different

interactions among the different kinds of agents.

Each defined agent is responsible for the

surveillance in a specific way because each of them

has a fixed intelligent level. Some of them have less

processing capacity, so they can carry out a basic

analysis of captured situations, whereas, others have

more processing capacity, so they can accomplish an

advanced analysis.

What kind of information is going to be

analyzed? We are working with several types of

sensors (senses in Figure 3) which capture image

(visible and infrared), smoke and volumetric

measurements. Depending on the kind of data, the

analysis is carried out in a different way. After

modeling the interactions, we are going to design the

MAS. We have a main goal in mind to cover the

maximum amount of elements in the electrical

facility environment (surveillance targets) with the

minimum agents and at the same time by keeping

agents cooperating. Autonomy and learning

capacities are very important for our system and

both of them are great advantages of MAS.

7 ARCHITECTURE FOR THE

NEW SYSTEM

We use two levels of surveillance. In the lowest

hierarchical level there is a network composed by

guard agents which don’t need to have an excessive

processing capacity. These vigilant systems are

always collecting information (through sensors) and

they have to control their environment by generating

alarms or corrective actions (actuators) if

appropriated. They have to interact with their

environment in a quick and easy way, processing in

real time. They have certain mobility in a limited

area.

ICEIS 2008 - International Conference on Enterprise Information Systems

394

Traditionally, these reactive systems, which are

in continuous interaction with the environment,

execute certain steps according to that environment,

and wait for events to happen. When this occurs,

they carry out the process operations by generating

out data and changing it to a new state. In this way,

anticipation or prevision capacity is limited, since it

is restricted to a series of events from a series of

known states, leading the system to another waiting

state.

However, if we make these guards interact, we

can create a knowledge network. This network

allows guards to request information by searching

for the most adequate solution. If it is not possible to

find a solution, our system works by acceding to the

highest level, where there is a supervisor with a

higher computation capacity. This can transmit the

solution to the lower level, or request information

from other supervisors.

The high–level architecture is shown in Figure 3.

Figure 3: Scheme for the new system.

The guard agent on the lower level is

implemented with an embedded system, based on an

Atmel’s AV32 microcontroller (Atmel Home, 2007),

specifically the AT32AP7000. It includes a data

memory of 32Kbytes on-chip SRAM and external

memory interfaces (Secure Digital); 16 KB

instructions and 16 KB data caches; and MMU and

DMA controllers. Peripherals include a 16-bit stereo

audio DAC, 2048x2048 pixel TFT/STN LCD

controllers, 480 Mbps USB 2.0 with on chip

transceivers and two 10/100 Ethernet MACs. Serial

interfaces include RS232, USART, I2S, AC97,

TWI/I2C, SPI, PS/2 and several synchronous serial

modules which support most serial communication

protocols. The board is preloaded with Linux and

shipped with I/O interface drivers that can be called

from your own code.

These characteristics allow for us to

communicate this system with any other sensors

systems, obtaining information on real time about

temperature, humidity, movement and IR images

from transformers centres or substations.

Supervisor agent hardware is a system based on a

VIA Luke CoreFusion™ Processor (VIA Home,

2007), x86 consumer electronics platform compliant.

It includes typical PC interfaces which are x86

compliant: out video signal AGP with MPEG-2

decoder/MPEG-4 Acceleration, interfaces

ULTRADMA y SATA, 10 /100 Base -T Ethernet

connection and two USB ports. The supervisor agent

works on a preloaded Linux system, which is

customized to this architecture, and always keeps the

x86 compliance.

8 CONCLUSIONS

Within this paper, we have described our IDOLO

system which integrates multimedia information in a

HMI/SCADA system for power system telecontrol

and then we have described our current work to

carry out an embedded version of that system in

order to improve it. This system has already been

successfully deployed on a real electrical facility,

and uses multimedia information in order to support

the telecontrol of the power facility, so operators

benefit from the advantages introduced by the

displaying of video and audio signals coming from

remote stations. They can get live views of

maintenance works, or inspect devices located on

remote facilities, from the supervision station, or

even from their home though internet.

For the new improved system, we are developing

an automatic surveillance function and advanced

image analysis and processing techniques from IR

images to improve the telecontrol tasks for operators

in control centers and make them easier. We think

that function can be achieved thanks to the use of the

autonomous and cooperative capacities of agents in

MAS technology.

ACKNOWLEDGEMENTS

The work described in this paper has been funded by

the Ministerio de Ciencia y Tecnología within I+D+I

National Program through the project with reference

number TEC2006-08430.

We would also like to thank ISIS Engineering

(Seville) for providing us with IS-SERVER

Surveillance

targets

Surveillance

targets

Guards

Senses

SENSORS

RTU

Supervisor

Vision

Hearing

Smelling

...

Trainning agent

Actions

MULTI-AGENT AND EMBEDDED SYSTEM TECHNOLOGIES FOR AUTOMATIC SURVEILLANCE

395

prototypes, and Medina-Garvey electrical company

for letting us use their facilities.

REFERENCES

Boyer, S. A., 1999. SCADA: Supervisory Control And

Data Acquisition, 2

Edition, ISA – The

Instrumentation, Systems and Automation Society,

New York (United States).

ISO/IEC, 1999. Information technology – Coding of

audio-visual objects – MPEG-4, ISO/IEC 14496.

Romero, M. C., Díaz, S., Sánchez, G., Escudero, J. I.,

2004. SIP Application To Multimedia Telecontrol Of

Power Systems. In World Automation Congress

Proceedings, pp. 131 – 137.

IEC, 1990. Telecontrol and equipment systems - Part 5:

Transmission protocols, IEC 60870-5.

ISO/IEC, 1993. Information technology – Local area

networks – Part 3: Carrier sense multiple access with

collision detection, ISO/IEC 8802-3.

Arindam, P., 1999. QoS in Data Networks: Protocols and

Standards. Ohio State University, http://www.cs.wustl.

edu/~jain/cis788-9/ftp/qos_protocols/index.html

Marwedel, P., 2003. Embedded System Design. Springer,

ISBN: 978-1-4020-7690-9.

Sutter, E., 2002. Embedded System Firmware Demystified.

Publishers Group West, ISBN 9781578200993.

Atmel Home, http://www.atmel.com/products/avr32/

VIA Home, http://www.via.com.tw/en/products/mainboar

ds/motherboards.jsp?motherboard_id=221

Wooldridge, M., 2002. An Introduction to MultiAgent

Systems. Wiley, ISBN 047149691.

ICEIS 2008 - International Conference on Enterprise Information Systems

396