SmaCCS: Smart Camera Cloud Services

Towards an Intelligent Cloud-based Surveillance System

Sven Tomforde

, Uwe J

anen

, J

org H

ahner

and Martin Hoffmann

Organic Computing Group, University of Augsburg, Eichleitnerstr. 30, 86159 Augsburg, Germany

Volavis GmbH, Schuckenteichweg. 31, 33818 Leopoldsh

ohe, Germany

Keywords:

Intelligent Surveillance, Cloud Computing, Automated Machine Learning, Smart Camera, Organic Comput-

ing.

Abstract:

Today, high performance and feature rich surveillance systems are very costly as they require an expensive

set of infrastructure components. As a consequence, such systems including, e.g., complex automatic video

content analysis, are restricted to large scale applications, such as airports or train stations. In smaller settings,

e.g. in shop surveillance, mostly low-cost display or record-only systems are in use. In this position paper we

propose to combine two well-known approaches in order to make Intelligent Video Surveillance applicable

and affordable in small to medium-scale scenarios. The proposal includes to combine the concept of Smart

Cameras, i.e. cameras equipped with local processing resources, with the ideas of Cloud Computing, i.e. the

on-demand provisioning of computing and storage services for complex calculations, and the management of

large amounts of data, i.e. video storage. The former allows for the cost effective pre-processing of video

data close to the sensor, while using the latter concept does not require large initial investments into expen-

sive infrastructure components such as powerful compute servers. The paper presents research issues of the

necessary system design, including precise system goal and system model aspects. Based on this, we discuss

several research issues required to be addressed for solving the overall goals.

1 INTRODUCTION

Within the last decade, a strongly increasing usage

of video-based surveillance has been observed driven

from both, the industry- and academia-side. The

application spectrum reaches from analyses of cus-

tomers’ ﬂows in retail business to semi-automated

surveillance of safety-critical areas like airports and

railway stations. Most of the currently used systems

are proprietary and isolated applications serving just

one speciﬁc purpose. Typically, they consist of a pre-

deﬁned set of cameras and a central point of oper-

ation, where all video streams are combined, stored,

and observed by human operators. This system model

results in non-scalable, hardware-intensive, and con-

sequently cost-inefﬁcient solutions, see e.g. (Javed

and Shah, 2008).

One promising solution to alleviate these unde-

sired system properties is to combine Cloud Comput-

ing (Vaquero et al., 2008) concepts with more auton-

omy for the cameras: Smart Camera Cloud Services

(SmaCCS). The term Smart Camera (SC) refers to

a standard camera that is equipped with an on-board

computation unit which is able to fulﬁl video prepro-

cessing tasks (e.g. feature extraction and annotation of

video ﬁles with meta-data). In addition, a variable set

of SCs is able to self-organise a surveillance network.

Thereby, the Cloud-solution serves as interface to the

user and as basis for computation-intensive tasks like

video analysis (e.g. detection of persons and move-

ments, see (J

anen et al., 2012)), pattern recognition,

and learning of conspicuous and abnormal behaviour.

A possibly large set of SCs will most probably result

in a huge amount of data that needs speciﬁc analy-

ses methods following the Big Data principle (Bryant

et al., 2008), which is most promisingly tackled using

a Cloud-based approach.

This paper outlines the general SmaCCS system

and names the upcoming research challenges to be ad-

dressed. The paper is structured as follows. Section 2

describes the state of the art regarding Smart Cam-

era and Intelligent Surveillance Systems, followed by

concepts for moving parts of the functionality into the

Cloud. Afterwards, Section 3 introduces the system

design of SmaCCS and highlights the most impor-

tant research issues. Finally, Section ?? summarises

the paper and gives an outlook to current and future

work.

288

Tomforde S., Jänen U., Hähner J. and Hoffmann M..

SmaCCS: Smart Camera Cloud Services - Towards an Intelligent Cloud-based Surveillance System.

DOI: 10.5220/0004590902880293

In Proceedings of the 10th International Conference on Informatics in Control, Automation and Robotics (ICINCO-2013), pages 288-293

ISBN: 978-989-8565-70-9

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

2 RELATED WORK

Intelligent Surveillance Systems and Smart Cam-

eras. According to (Velastin and Remagnino, 2006),

intelligent surveillance systems can be classiﬁed by

their degree of autonomy and capability to sat-

isfy self-X properties like self-organisation and self-

conﬁguration. The ﬁrst class consists of analogues

CCTV (Closed Circuit Television) techniques for im-

age distribution and storage in a single control room.

In contrast, systems of the second class already com-

bine computer vision and CCTV which results in

semi-automatic visual surveillance. Finally, fully au-

tomated wide-area surveillance systems represent the

third class. These systems are characterised by the

distribution of intelligence among a possibly large set

of collaborative cameras.

The targeted SmaCCS observation system con-

sists of several pan-tilt-zoom (PTZ) capable Smart

Cameras (SC), see Fig. 1. A SC is an automated sys-

tem and combines an optical sensor with a computa-

tion unit for preprocessing and on-board analysis of

video data, see (Schneiderman, 1975). The output of

the optical sensor is preprocessed by the local com-

putation unit; hence, only limited image data has to

be transferred using communication and the network

trafﬁc can be mostly reduced to event- and status-

messages.

Figure 1: Schematic overview of a SC network.

Surveillance systems have captured increasing in-

terest of both, the research and the industrial worlds,

in recent years (Javed and Shah, 2008). In the last

decade, surveillance systems of the third class, self-X

properties like self-organisation and the distribution

of hardware components have been the focus of in-

vestigations (Collins et al., 2001; Lipton et al., 2004;

Monari, 2012). The gap between research and real

robust industrial systems is wide. The initial costs

to build up a commercial full-automated surveillance

system are very high. Even large cities like Lon-

don run semi-automated systems, e.g. a static cam-

era is able to detect persons entering forbidden ar-

eas, (Cernium, 2013). Nonetheless, great parts of

the initially demanded research (Chu et al., 2004) has

been successfully performed for pure video surveil-

lance systems. The SmaCCS project aims at closing

the remaining gap by reducing the initial costs with a

pay-per-use concept and an outsourcing of hard- and

software components in the Cloud.

Cloud Computing – Privacy Issues and the Ap-

plication to Surveillance or SC Systems. Cloud

Computing (Vaquero et al., 2008) emerged as a pay-

per-use approach for accessing computing resources

(hardware and software) that are delivered as a service

over a network (i.e. the Internet). In this context, the

term “Cloud” describes the virtualisation and abstrac-

tion of the potentially complex infrastructure and the

remote access by customers. Today, a variety of com-

mercial Cloud-solutions are available focussing on

data centre (e.g. Amazon’s EC2), storage (e.g. Drop-

box), or Software (e.g. Microsoft’s Ofﬁce 365). The

focus of the SmaCCS approach is to make use of the

data centre functionality as this is an economical (i.e.

reduce initial hardware cost) and highly scalable al-

ternative to centralised system architectures of current

surveillance systems.

Due to the potentially computation-extensive pro-

cessing tasks caused by continuous video data from

a large set of sources, a scalable Cloud-based com-

putation unit is necessary. Similar ideas have already

been presented in the literature. For instance, (Zhang

et al., 2009) describe a quality monitoring process in

industrial automation that is based on visual inspec-

tion (i.e. to detect surface defects). Here, the standard

approach is to make use of extensive image logging

and off-line human interpretation. In contrast, the au-

thors introduce their concept of measuring microm-

eter defects from a distance using a cloud of cheap

vision sensors.

Besides this ﬁrst industrial-based video analysis

scenario, other researchers worked on a “Cloud-based

algorithmic framework which is scalable and adap-

tive to online smart city video sensing system” (Wen

et al., 2010). Here, spatio-temporal relationships are

derived from large-scale camera networks and simu-

lated using a Cloud-setup in order to build the best

possible topological structure. Hence, the focus is

more on design time decision support than on runtime

usage of the Cloud as access and analysis platform.

Finally, privacy is an important issue if sensible

data is transferred to the Cloud (i.e. in terms of le-

gal compliance and user trust). Therefore, (Pear-

son, 2009) state that this has to be covered at every

phase of designing the system. Corresponding aspects

concerning the privacy challenges will be considered

within the SmaCCS system.

SmaCCS:SmartCameraCloudServices-TowardsanIntelligentCloud-basedSurveillanceSystem

289

3 SYSTEM DESIGN

The following section deﬁnes the goal and describes

the system design of SmaCCS. Afterwards, the corre-

sponding research challenges are outlined.

3.1 System Goal

The goal of SmaCCS is to investigate a scalable

system allowing for an affordable semi- or full-

automated intelligent video surveillance system. In

this context, affordable means that the user in-

stalls one gateway and as many out-of-the-box SCs

as he needs for the speciﬁc tasks, let these SCs

automatically conﬁgure themselves based on the

bought license-model, and use a scalable access- and

processing-solution in the Cloud. The user will be

taxed for the Cloud-part on a pay-per-use approach.

The Cloud-based solution reduces the initial invest-

ment cost and increases the scalability of the system

and the possibility to automatise image-analysis and

recording management tasks. The system can consist

of arbitrary SCs, whereas the user is taxed on the par-

ticularly used number. The system aims especially

at small- and medium-sized surveillance tasks, like

the observation of business areas, large private house-

holds, or railway stations and airﬁelds.

3.2 System Model

The system model consists of a user-side and a Cloud-

side. Details are illustrated by Fig. 2. The Cloud’s

server components are an application-, a Nagios-

(Nagios Enterprises, 2013), and a message-queueing

(MQ) server. The image-database as well as the

Cloud-analysis unit are the core components to per-

form exhausted image-analysis on Cloud-side. On

user-side, the SCs and a gateway to connect to the

Cloud have to be installed. The user accesses the ob-

servation system as well as the image-analysis results

via an application-interface (GUI).

3.2.1 User-Side

As mentioned before, a SC consists of an optical sen-

sor (e.g. a pan-tilt-zoom capable camera), a light-

weighted computation unit (SC-analysis), and a com-

munication interface. Each of the SCs is able to per-

form parts of the image analysis like movement de-

tection and detection of persons. In addition, the

SC can observe suspicious events (like persons ap-

proaching forbidden areas). Based on this informa-

tion, the SC decides autonomously about the frame-

rate and the image quality to be stored at server-side

(within the Cloud) and annotates the video data with

the derived meta-information. This reduction of ir-

relevant information is an important step for image-

data-compression which is essential for Cloud-based

storing of data.

3.2.2 Cloud-Side

The pre-processed image-data is streamed to the

image-database for storing and the Cloud-analysis

for post-processing. Based on the annotated data

stored in the Cloud’s database, the user can perform

computational-intensive analysis tasks or search for

speciﬁc situations (e.g. besides a standard search like

“all video data between time a and b”, he might look

for persons with yellow clothes). The application

server is the connection-interface to the user. It pro-

cesses the image analysis results and presents it to the

user. Single SCs communicate via XML messages

likewise to the Sensor Model Language (SML, 2013).

Therefore, the MQ-server is the transferring commu-

nication component, as well as the communication

between application server and SC network. As the

whole system consists of Cloud- and distributed com-

ponents, it is necessary to capture system states for

maintenance services. The Nagios-Server is an ade-

quate solution to handle component malfunctions.

3.3 System Components and Research

Challenges

The overall system as outlined in Fig. 2 consists of

a variable set of SCs, a Cloud-based server, and a

web-based user-access system. The following part of

this paper describes the corresponding system compo-

nents to be developed and the resulting research chal-

lenges related to these components.

1. Smart Camera System. SmaCCS builds upon

the results of the CamInSens system (D’Angelo et al.,

2012) and QTrajectories project (J

anen et al., 2012).

Fig. 3 depicts the data stream within a single SC. The

image-data will be transferred from the optical sen-

sor (chip) to the internal-processing unit (PROC) of

the camera – this provides a video stream via its com-

munication interface (COMM). Typically, this opti-

cal sensor is a standard pan-tilt-zoom camera. In

addition, the sensor annotates the current timestamp

and current pan-tilt-zoom conﬁguration to the JPEG-

header. For hard real-time constraints, this informa-

tion should be annotated as close to the data source

as possible. The SC-analysis unit is responsible for

the ﬁrst image analysis (like person detection). The

enriched and annotated image data is accessible via a

COMM Interface by the Cloud services.

ICINCO2013-10thInternationalConferenceonInformaticsinControl,AutomationandRobotics

290

Figure 2: System Model of the SmaCCS system.

Figure 3: Architectural view of a single SC.

Typical observation jobs (like counting people

in an entrance area of a building) are usually ﬁxed

assigned to dedicated SCs located at speciﬁc posi-

tions (e.g. above a door). More sophisticated surveil-

lance jobs like “look at position X”-commands initi-

ated by a user to get an image of a special location,

need a more sophisticated job scheduling method.

Therefore, each observation job is represented by a

software-agent. This agent is responsible for the ful-

ﬁlment of the user command. Such an observation

job can be initiated by the user or by the SC sys-

tem itself as a reaction to the observation result of

another surveillance job. The agent itself searches

self-organised for an adequate SC resource to fulﬁl

its job. Therefore, the agents trade camera resources

within the system. Another possibility is that more

important agents simply suppress other agents from a

camera resource.

In addition, the control of each SC can be im-

proved. Thereby, the Multi-level Observer/Controller

(MLOC) framework (Tomforde, 2012) will be ap-

plied to each SC in order to develop a self-organising

camera alignment solution. Furthermore, the MLOC

framework serves as classiﬁcation system for the cur-

rently detected situation of the SC and therefore de-

cides about the data-quality to be submitted to the

Cloud, similar to the classiﬁcation of network-trafﬁc

situations in (Tomforde et al., 2011).

2. Cloud System. Besides the “normal” techni-

cal Cloud-side tasks as outlined before, the main re-

search challenges refer to a) an efﬁcient data-storage

solution for video and image data and b) to intel-

ligent data-analysis method (e.g. based on machine

learning techniques). Storing video data efﬁciently

and accessing the contents based on a database con-

cept is an interesting task and has attracted researches

for years, see e.g. (Collins et al., 2001) for a good

overview. Here, the challenge is to provide an efﬁ-

cient technique for searches and queries that scales

well with possibly huge data – i.e. as provided by re-

lational databases. Video and image data are not suit-

able for relational databases – hence, a data partition-

ing concept is needed. The second part – the Cloud-

analysis – has to ﬁnd best-possible answers to user

queries. Thereby, the query should be more powerful

than standard solutions.

In addition, machine learning and data mining

techniques are needed to detect re-occurring complex

situations that might induce a security-threat to the

surveillance system. Therefore, video-data from sev-

eral SCs needs to be aggregated and suspicious move-

ments (in terms of trajectories, cf. the QTrajectories

project that serves as input (J

anen et al., 2012)) have

to be detected. Additionally, the movements and sus-

picious events are categorised by a pre-trained alarm-

classiﬁcation system. This alarm system informs the

user in case of events that have been classiﬁed as sus-

picious and reduces the false-alarm-rate signiﬁcantly.

SmaCCS:SmartCameraCloudServices-TowardsanIntelligentCloud-basedSurveillanceSystem

291

3. User Interface. The system is designed to

be used by both professional security operators and

private users. We therefore aim at developing a user

interface that is easy to use but also comes with all

features needed to be used in safety critical environ-

ments. The notiﬁcation of users in case of alarms is

an important task. We will extend our existing work

as described in (Hoffmann et al., 2010) by carrying

our work on the user interface design further. In (Yee

et al., 2012), practical solutions are given to adapt a

GUI to most effectively suite the needs of different

user groups under psychological aspects. Among oth-

ers, this approach may help to improve the useability

and acceptance of the system.

A ﬁrst draft of the GUI is depicted in Fig. 4. A

web browser is used on the clients in order to display

the GUI whereas the business logic is running on the

server side. By offering a web application users may

access the system from any location with any device

available (notebook, smartphone etc.). Some func-

tionalities of the system are described in the following

on the basis of Fig. 4: A screen displaying video data

is taking most of the space to the right side. On the left

side, the menu is displayed that contains links in order

to browse a camera network and select cameras. In or-

der to browse historical video data, a calendar will be

included (in the middle of screen). Further menus will

give access to further screens. E.g. reports generated

by an algorithm for people counting may be accessed.

In order to administrate the system, various param-

eters can be adjusted. As mentioned before, a shop

system will be integrated in order to sell services to

the user. This services can be algorithm for analysing

video data or the enhancement of the system (more

cameras, storage upgrade etc.).

Figure 4: Possible GUI design for SmaCCS’ start page.

By carefully designing the user interface we aim

at reaching a broad range of users and usage scenar-

ios that require different levels of security, privacy an

the consideration of legal aspects as discussed in the

following section.

4. Security and Legal Issues. The consideration

of privacy is of utmost importance for social accep-

tance of surveillance technology. We aim at develop-

ing a trustworthy video surveillance systems by prop-

erly combining different aspects that current systems

do not manage. In particular, we propose the com-

bination of the following issues into the SMACCS

framework.

First of all, we will develop a properly deﬁned in-

terface to detection algorithms mainly based on com-

puter vision techniques. Thereby we make sure, the

user knows exactly the data that the system gener-

ates and stores. For example, an algorithm for peo-

ple counting only delivers (and stores) the number of

persons moving in or out a certain area. In contrast

to this, an algorithm for people detection and recog-

nition might store data containing biometric informa-

tion. By offering a well designed interface, the op-

erators can customise the system to ﬁt their speciﬁc

needs (and laws of their respective countries (Zick,

2007)). The content protection relies not only on us-

ing convenient cryptographic techniques, but also law

enforcement and user cooperation in order guarantee

the legal compliance of the system. For example, we

propose user pairing for accessing stored video data.

In order to access data that has been saved on the

system, two user have to agree on this request. Af-

ter both of them logged into the system, a speciﬁed

part (with time of beginning and end) of the stream

is made available for viewing. Some of the ideas

are based on a model for large-scale smartphone-

based sensor networks that has to cope with simi-

lar challenges (Kapadia et al., 2010). In addition,

the SmaCCS approach can re-use insights from the

predecessor project CamInSens (Hornung and Desoi,

2011).

4 CONCLUSIONS

This paper presented a concept for a scalable and

cost-efﬁcient Intelligent Video Surveillance system

for small- and medium-sized surveillance tasks, like

the observation of business areas, large private house-

holds, or railway stations and airﬁelds. The ap-

proach combines the advantages concept of Smart

Cameras, i.e. cameras equipped with local process-

ing resources, with the those of Cloud Computing, i.e.

the on-demand provisioning of computing and stor-

age services for complex calculations. Thereby, ex-

haustive image analysis and the management of large

amounts of data, i.e. for video storage, becomes pos-

sible. The Smart Camera approach allows for the cost

effective pre-processing of video data close to the sen-

sor, while using the Cloud-concept does not require

ICINCO2013-10thInternationalConferenceonInformaticsinControl,AutomationandRobotics

292

large initial investments into expensive infrastructure

components such as powerful compute servers.

Current and future work will try to ﬁnd answers

on the research challenges outlined in this paper. Es-

pecially, the research issues of the necessary sys-

tem design, including precise system goal and sys-

tem model aspects, the machine learning and efﬁcient

video/image data storage and querying are focused.

REFERENCES

Bryant, R. E., Katz, R. H., and Lazowska, E. D. (2008).

Big-data computing: Creating revolutionary break-

throughs in commerce, science, and society. Com-

puting Research Initiatives for the 21st Century. Com-

puting Research Association.

Cernium (2013). Perceptrac System webpage. Online,

http://www.cernium.com.

Chu, M., Reich, J., and Zhao, F. (2004). Distributed atten-

tion in large scale video sensor networks. IEE Seminar

Digests, 2004:61–65.

Collins, R. T., Lipton, A. J., Fujiyoshi, H., and Kanade,

T. (2001). Algorithms for cooperative multisensor

surveillance. In Surveillance, Proceedings of the

IEEE.

D’Angelo, D., Grenz, C., Kuntzsch, C., and Bogen, M.

(2012). CamInSens – An Intelligent in-situ Security

System for Public Spaces. In Proceedings of the 2012

International Conference on Security & Management

(SAM), 2012, 16.-19. Jul, Las Vegas, USA. CSREA

Press.

Hoffmann, M., J

anen, U., Fares, A., and H

ahner, J. (2010).

Amidivin: basic algorithms for alarm management in

distributed vision networks. In Proceedings of the

Fourth ACM/IEEE International Conference on Dis-

tributed Smart Cameras, ICDSC ’10, pages 150–157,

New York, NY, USA. ACM.

Hornung, G. and Desoi, M. (2011). Smart Cameras und

automatische Verhaltensanalyse. Kommunikation und

Recht, pages 153–158.

anen, U., Feuerhake, U., Klinger, T., Muhle, D., H

ahner,

J., Sester, M., and Heipke, C. (2012). QTrajectories:

Improving the Quality of Object Tracking Using Self-

Organizing Camera Networks. In ISPRS Annals of

the Photogrammetry, Remote Sensing and Spatial In-

formation Sciences, Volume I-4, pages 269 – 274.

Javed, O. and Shah, M. (2008). Automated Multi-Camera

Surveillance: Algorithms and Practice. Springer Pub-

lishing Company, Incorporated, 1 edition.

Kapadia, A., Myers, S., Wang, X., and Fox, G. C. (2010).

Secure cloud computing with brokered trusted sensor

networks. In Collaborative Technologies and Systems

(CTS), 2010 International Symposium on, pages 581–

592.

Lipton, A. J., Clark, J. I., Brewer, P., Venetianer, P. L., and

Chosak, A. J. (2004). Objectvideo forensics: activity-

based video indexing and retrieval for physical secu-

rity applications. In Intelligent Distributed Surveil-

liance Systems, IEEE, pages 56–60.

Monari, E. (2012). Dynamische Sensorselektion zur

auftragsbasierten Objektverfolgung in Kameranetzw-

erken. KIT Scientiﬁc Publishing. ISBN: ISBN 978-

3866447295.

Nagios Enterprises (2013). Nagios webpage. Online,

http://www.nagios.org/.

Pearson, S. (2009). Taking account of privacy when design-

ing cloud computing services. In Software Engineer-

ing Challenges of Cloud Computing, 2009. CLOUD

’09. ICSE Workshop on, pages 44–52.

Schneiderman, R. (1975). Smart cameras clicking with

electronic functions. Electronics, pages 74 – 81.

SML (2013). Sensor Model Language webpage. Online,

http://www.opengeospatial.org.

Tomforde, S. (2012). Runtime adaptation of tech-

nical systems: An architectural framework for

self-conﬁguration and self-improvement at runtime.

udwestdeutscher Verlag f

ur Hochschulschriften.

ISBN: 978-3838131337.

Tomforde, S., Hurling, B., and H

ahner, J. (2011). Dis-

tributed Network Protocol Parameter Adaptation in

Mobile Ad-Hoc Networks. In Informatics in Control,

Automation and Robotics, volume 89 of LNEE, pages

91 – 104. Springer.

Vaquero, L. M., Rodero-Merino, L., Caceres, J., and Lind-

ner, M. (2008). A break in the clouds: towards a

cloud deﬁnition. SIGCOMM Comput. Commun. Rev.,

39(1):50–55.

Velastin, S. and Remagnino, P. (2006). Intelligent Dis-

tributed Video Surveillance Systems. Professional Ap-

plications of Computing. Institution of Engineering

and Technology. ISBN: 978-0863415043.

Wen, Y., Yang, X., and Xu, Y. (2010). Cloud-computing-

based framework for multi-camera topology inference

in smart city sensing system. In Proceedings of the

2010 ACM multimedia workshop on Mobile cloud me-

dia computing, MCMC ’10, pages 65–70, New York,

NY, USA. ACM.

Yee, C. K., Ling, C. S., Yee, W. S., and Zainon, W.

(2012). Gui design based on cognitive psychology:

Theoretical, empirical and practical approaches. In

Computing Technology and Information Management

(ICCM), 2012 8th International Conference on, vol-

ume 2, pages 836–841.

Zhang, L., Malki, S., and Spaanenburg, L. (2009). Intel-

ligent camera cloud computing. In Circuits and Sys-

tems, 2009. ISCAS 2009. IEEE International Sympo-

sium on, pages 1209–1212.

Zick, T. (2007). Clouds, Cameras, and Computers:

The First Amendment and Networked Public Places.

Florida Law Review, 69(St. John’s Legal Studies Re-

search Paper No. 06-0062):1 – 66. Available at SSRN:

http://ssrn.com/abstract=956160.

SmaCCS:SmartCameraCloudServices-TowardsanIntelligentCloud-basedSurveillanceSystem

293