The Survey of Big Data Problems in the Video Surveillance System

Xin Wang, Zheng Xu and Jie Dai*

The Third Research Institute of the Ministry of Public Security, Shanghai, China

Keywords: Big data, Video intelligent analysis, Video structured description, Repository database, Cloud computing.

Abstract: Video surveillance has become the main tool due to its rich, intuitive and accurate information. However,

with the large-scale construction of video surveillance systems all over the world, problems such as “useful

information and clues cannot be found immediately with video big data” decrease detecting efficiency

during crime prediction and public security governance. This paper examines the current techniques

including video intelligent analysis and video structured description (VSD), knowledge discovery in

database, and cloud computing including virtualization, distributed computing and storage, and proposes a

framework of the next generation video surveillance system to explain how to discovery knowledge from

video big data, organize and manage massive heterogeneous resources, and provide operating environment

and resources for tasks, for the purpose of supporting police to predict crime quickly and efficiently.

1 INTRODUCTION

Recently, the worldwide terrorist incidents and

crime events occur frequently, and it is urgent for

governments and police to pay serious attention to

the public security governance, the traffic accidents,

criminal prediction and other incidents. With the

help of cloud computing (Liu et al., 2010; Liu et al.,

2011), internet of things (Hu et al., 2014; Luo et al.,

2011), and Big Data (Xu et al., 2014; Xu et al.,

2015), video surveillance has become the main tool

due to its rich, intuitive and accurate information. A

great amount of video surveillance systems have

been built all over the world. China has built more

than 23 million video surveillance cameras till 2013,

of which 3 million are utilized by police, and the

video surveillance are entering the big data era with

its 4V properties. That is, the video data has very

huge volume, taking one city for example, thousands

of cameras are built of which each collects high-

definition video over 24 to 48GB every day with the

rapidly growth; secondly, data collected includes

variety of formats involving multimedia, images and

other unstructured data; furthermore the valuable

information contains in only a few frames called key

frames of massive video data; and the last problem

caused is how to improve the processing velocity of

a large amount of original video with computers, so

* The corresponding author: Jie Dai

as to enhance the crime prediction and detection

effectiveness of police and users.

To solve those problems, technologies such as

knowledge mining and deduction, pattern

recognition and cloud computing are widely utilized

in the next generation video surveillance system, to

assist police to discover valuable information and

predict crime from large amount data. China

National Laboratory of Pattern Recognition (NLPR)

has developed the distributed video surveillance

system (Zh et al., 2012), which is applied to discover

unusual behaviour and traffic violations with pattern

recognition. The Industrial Technology Research

Institute of Taiwan set up the Surveillance Video

Analysis Center and built the ” Cloud Intelligent

Video Analysis and Retrieval System”, which

provides video retrieval and other video analysis

services to aid police to discover crime efficiently.

Pacific Northwest National Laboratory (PNNL)

founded National Visualization and visual analysis

techniques and tools Analytics Center (NVAC)

providing sorts of video analysis tools

(http://vis.pnnl.gov/). British ADVISOR project

(Annotated Digital Video for Intelligent Surveillance

and Optimized Retrieval) could estimate crowd

density, analyse human behaviour in and around

subway and predict potential danger and crime early.

IBM has developed a “Smart Vision Suite” with the

ability of automatic event detection, and distributed

technology is used to large-scale system deployment

and internet business.

126

Dai J., Xu Z. and Wang X.

The Survey of Big Data Problems in the Video Surveillance System.

DOI: 10.5220/0006020301260129

In Proceedings of the Information Science and Management Engineering III (ISME 2015), pages 126-129

ISBN: 978-989-758-163-2

Based on the research above, this paper proposes

a framework including video intelligent analysis and

video structured description (VSD) (

et al., 2010),

knowledge discovery in database, and cloud

computing, and discusses how to analyse, organize,

manage and store video big data. The rest of the

paper is arranged as follows. Section 2 introduces

the problems. The architecture is detailed in section

3. In section 4, recent work and research are given.

In the end, some conclusions are drawn and future

work is given in section 5.

2 PROBLEMS DESCRIPTION

Surveillance video data has the 4V properties of big

volume, variety of data format, low value and slow

processing velocity, resulting in several problems in

application especially in crime detection and public

security management for police:

(1) Little video analysis technology is utilized to

recognize vehicle information (license-plate, logo,

and colour etc.) and simple applications, however

crime prediction and clue discovery from massive

video data are most rely on human detection, and it

is still hard to discover deep information and

complex content by computers, also lack of the

standardized description of analysed content.

(2) Due to lack of effective resource

management and organization, a great amount of

computing and storage resources could not be

utilized effectively when analyse and process video

big data.

(3) Without the policing repository database, it is

hard to mine the more complex relationship and

deeper semantics from a great amount of data, also

unnecessary to recommend police available

information, clues, case trend.

3 FRAMEWORK FOR THE NEXT

GENERATION VIDEO

SURVEILLANCE SYSTEM

We propose a new framework to show how to

process, organize, manage and store massive video

data. As shown in Fig.1.

The framework has three parts: video intelligent

analysis (Object detection, target tracking, behaviour

analysis and event analysis) and video structured

description (VSD) are utilized to mine valuable

information (persons, cars, unusual behaviours etc.)

from large scale video data, which then is expressed

in standard format. The second part is construction

of policing repository database, which is used to data

mining, information describing, moreover

knowledge reasoning as the domain knowledge, and

provide real cases to assist crime prediction.

Furthermore virtualization and cloud computing

provide efficient computing environment for

techniques all above, and storage environment for

various types of structured and unstructured data.

Figure 1: The framework for the next generation video

surveillance system.

3.1 Video Intelligent Analysis and

Video Structured Description

(VSD)

Video intelligent analysis and structured description

are applied to deal with original video, of which the

results are frames containing persons and cars, and

their structured description with standard format.

All the data are packaged with unified standard

format and transferred to the distributed cloud

platform which provides greatly efficient storing and

computing ability. Due to the limited bandwidth, the

“front + back” pattern is adopted, that is: simple

video analysis algorithms are carried out in the

cameras, and results are sent back to “the cloud” to

support more complex computing and applications.

The pattern could avoid network congestion caused

by large-scale video big data.

3.2 Repository Database Construction

Repository database could be constructed as follow

steps: first, knowledge collection, that is collecting

and analyzing existed cases, policies and

regulations, and make them as knowledge repository

The Survey of Big Data Problems in the Video Surveillance System

127

The Survey of Big Data Problems in the Video Surveillance System

127

sample set; secondly is knowledge discovery, that

domain knowledge are mined, clustered and

analyzed from the collected cases and rules, with

machine learning such as support vector machines

(SVM), or expert guidance; thirdly, knowledge

representation, domain knowledge and rules should

be represented with unified form such as RDFS,

OWL and SWRL , and stored in repository database

and model database, from which the information

would be utilized to support training models,

semantic retrieval, reasoning and crime prediction.

3.3 Virtualization, Distributed

Computing and Storage

Technology

For types of tasks such as video content analysis,

semantic modeling and reasoning, Mapreduce,

Spark, Storm and other distributed processing model

are applied to deal with corresponding task. Take

video retrieval for example, Mapreduce would be

used to support the task, of which the key is

represented by the time in video, and video data are

divided into several parts by the key, then all tasks

execute simultaneously.

Moreover, to enhance the efficiency of data

storage, the structured description data, images and

video data during video analyzing, processing, and

retrieval would be classified to optimize the storage

management and satisfy a variety of requests for the

end-users.

Virtualization is adopted to support IT resource

consolidation and optimum use.

4 PREVIOUS WORK

During 2008-2009, the third research institute of

Ministry of Public Security introduced video

structured description technology for the demand in

video surveillance applications, and undertakes a

series national science and technology major

projects including the Ministry of National Science

and Technology Support project, 863 smart city

project and the Core Electronic Devices, High-end

Generic Chips and Basic Software project. Numbers

of public security intelligent video surveillance

systems are carried out successfully, including VSD

based road surveillance video retrieval system in

Shuangliu in Chengdu, Taicang in Jiangsu Province

and so on.

5 CONCLUSIONS

In this paper, we propose a novel framework for the

next generation video surveillance system, which

addresses the problems video big data cause during

public security governance and crime predicting. In

this framework, Video intelligent analysis and video

structured description (VSD), knowledge discovery

in database, and cloud computing are introduced,

and video intelligent analysis and VSD discover

targets and express them with standard format.

Knowledge discovery is utilized to repository

database construction, since the repository is the

“material basis” of domain and supports models

training, semantic retrieval, crime prediction and

reasoning. Cloud computing techniques such as

Virtualization, distributed computing and storage

technology provide efficient operating environment,

and optimize the allocation of computing, storage

and network resources for tasks.

The above techniques provide the basic tools

and environment from the point of video big data

mining, organization, and management. However,

some other problems still exist: it still cannot satisfy

the routine detection and application for police. For

example, combining crime prediction results with

visualization methods is necessary for users during

detection. These unsolved problems particularly

merit our further study.

ACKNOWLEDGEMENTS

This work was supported in part by the National

Science and Technology Major Project under Grant

2013ZX01033002-003, in part by the National High

Technology Research and Development Program of

China (863 Program) under Grant 2013AA014601,

in part by the National Science Foundation of China

under Grant 61300028, in part by the Project of the

Ministry of Public Security under Grant

2014JSYJB009.

REFERENCES

Liu,Y., Zhu,Y., Ni, Lionel M., and Xue, G. 2011. A

Reliability-Oriented Transmission Service in Wireless

Sensor Networks. IEEE Transactions on Parallel and

Distributed Systems, 22(12): 2100-2107.

Liu, Y., Zhang, Q., and Ni, Lionel M. 2010. Opportunity-

Based Topology Control in Wireless Sensor Networks.

IEEE Transactions on Parallel and Distributed

Systems, 21(3): 405-416.

ISME 2015 - Information Science and Management Engineering III

128

ISME 2015 - International Conference on Information System and Management Engineering

128

Hu, C., Xu, Z. et al. 2014. Semantic Link Network based

Model for Organizing Multimedia Big Data. IEEE

Transactions on Emerging Topics in Computing,

2(3):376-387.

Luo, X., Xu, Z., Yu, J., and Chen, X. 2011. Building

Association Link Network for Semantic Link on Web

Resources. IEEE transactions on automation science

and engineering, 8(3):482-494.

Xu, Z. et al. 2015. Knowle: a Semantic Link Network

based System for Organizing Large Scale Online

News Events. Future Generation Computer Systems,

43-44:40-50.

Xu, Z., Luo, X., Zhang, S., Wei, X., Mei, L., and Hu, C.

2014. Mining Temporal Explicit and Implicit

Semantic Relations between Entities using Web

Search Engines. Future Generation Computer Systems,

37:468-477.

Zh, H., et al. 2010. Video Structured Description: A Novel

Solution for Visual Surveillance. Lecture Notes in

Computer Science, Volume 6298:629-636.

Zh, T., Liu, S., Xu, C., Lu, H. 2012. Mining Semantic

Context Information for Intelligent Video Surveillance

of Traffic Scenes Industrial Informatics. IEEE

Transactions on Volume:9 , Issue: 1 :149 – 160.

http://vis.pnnl.gov/ .

The Survey of Big Data Problems in the Video Surveillance System

129

The Survey of Big Data Problems in the Video Surveillance System

129