Evaluation of Command Effectiveness of Anti-Terrorism Drill

Based on Analytic Hierarchy Process

Na Gao

1,2,* a

Department of Public Security Management,

Liaoning Police College,

Dalian, Liaoning, 116036, China

Liaoning Research Base for Social Governance and Legal System Construction, China

Keywords: Anti-terrorism Drill, Command Effectiveness, Analytic Hierarchy Process, Evaluation Model.

Abstract: Anti-terrorism drill is an effective means to improve the anti-terrorism combat capability, and the command

in the drill is its key link. In order to ensure the quality of anti-terrorism drill and improve the command

efficiency, Delphi method is adopted to establish the evaluation of five core command capabilities,

including intelligence mastery capability, situation analysis capability, operational control capability, combat

coordination capability and organizational support capability index system. The analytic hierarchy process

is introduced, combined with the scoring of the expert group.A comprehensive evaluation model is

established, and the application evaluation is carried out for the anti-terrorism drill in D city. The results

show that the model comprehensively considers the functionality and balance of each index, effectively tests

the command effectiveness in anti-terrorism drill, and plays an obvious guiding role in the improvement of

command ability.

1 INTRODUCTION

The anti-terrorism drill is an effective means to deal

with extreme violent terrorist events. Cities in China

have successively carried out large-scale

anti-terrorism drills, which has improved the city's

ability to respond to terrorist attacks. The

effectiveness of anti-terrorism drills should be tested

through scientific evaluation methods. As for the

evaluation of emergency drills, scholars conducted

research from different perspectives and methods.

In terms of content, Gao Xiao and other scholars

conducted drills on the coordination and emergency

response capabilities of road emergency

commanders and rescuers for various road

emergencies (Gao, 2022); Zhao kaigong and other

scholars constructed a reasonable scenario for mine

https://orcid.org/0000-0002-2980-2133

Gao Na (1981 - ), female, born in Jinzhou, Liaoning

Province, holds a master's degree and is an associate

professor. She is mainly engaged in research on

emergency management, emergency policing, etc. E-mail:

314903384@qq.com. 15502625732.

emergency drill (Zhao, 2022). At the method level,

scholars adopt a variety of methods from the overall

perspective. Ren longlong and others used BP neural

network to build a high-speed railway emergency

drill evaluation model (Ren, 2020). Wang lei and

Chen guohua formed a quantitative drill

performance evaluation method (Wang, 2008) from

the use of time resources and the demonstration of

emergency capacity. Li rong adopted a combination

of AHP and TOPSIS (Li, 2020).

Anti-terrorism is a systematic project that

requires the active cooperation of relevant industries,

departments and the masses. The key for

coordination is the command and deployment in

emergency response (Sunnie, 2020). At present,

there is still a lot of research space for the academic

community to conduct a detailed evaluation from the

micro level, especially the command and

deployment in the drill. Therefore, this paper selects

the crucial decision-making command in the

anti-terrorism drill for evaluation, and tries to find a

scientific and reasonable command decision-making

evaluation method by using the analytic hierarchy

process.

Gao, N.

Evaluation of Command Effectiveness of Anti-Terrorism Drill Based on Analytic Hierarchy Process.

DOI: 10.5220/0012072600003624

In Proceedings of the 2nd International Conference on Public Management and Big Data Analysis (PMBDA 2022), pages 233-238

ISBN: 978-989-758-658-3

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

233

2 CONSTRUCTION OF THE

EVALUATION INDEX SYSTEM

OF ANTI-TERRORISM DRILL

COMMAND EFFECTIVENESS

2.1 Analysis of Key Factors

The command ability of anti-terrorism drill is the

comprehensive embodiment of the quality and

efficiency of the command subject's using

knowledge and wisdom to solve the command

problems in the anti-terrorism drill (Hu, 2022). It

reflects the cognition of the command subject to the

anti-terrorism operation and the level of command

action. include:

Quick understanding ability: through listening to

sporadic and fragmentary situations, quickly grasp

the basic situation of the events (Sun, 2018).

Accurate judgment ability: judge the nature and

situation of the event, and predict the possible trend

(Shang, 2016).

Decisive decision-making ability: according to

the judgment made on the nature and situation of the

event, select an applicable plan or formulate a

disposal plan on the spot (Shang, 2016).

Thorough operational research capability: deploy

the disposal actions, including dividing the police

forces under their jurisdiction into groups, defining

their respective responsibilities and tasks, action

plans, police equipment and cooperation, and

determining the command position and mode (Li,

2017).

Flexible adaptability: timely adjust the police

deployment and action plan according to the

changing situation (Yang, 2019).

2.2 Selection Principle of Evaluation

Indicators

Guided by practical improvement. The selection of

evaluation indicators is guided by the actual combat,

highlighting the actual combat standards in terms of

evaluation design, evaluation content, evaluation

standards, etc.

The ability test shall be taken as the standard.

Because it is difficult to determine the command

decision of anti-terrorism drill, we should focus on

the results rather than the process when designing

and selecting indicators, and take the inspection of

various abilities that the command should have as

the standard.

Supported by quantitative analysis. In the design

and selection of indicators, we should try to rely on

quantitative analysis to analyze the role of command

effectiveness in anti-terrorism drills.

It is essential to be comprehensive and accurate.

First of all, it is necessary to highlight the

comprehensive consideration of command ability,

comprehensively analyze various influencing factors,

and build an integrated evaluation index. Secondly,

we should respect the objective facts, and the

selected indicators should objectively reflect the

command ability needs. Third, ensure that the

indicators are derived from first-hand information,

that is, accessibility.

2.3 Construction Method of Evaluation

Index

Considering the influencing factors of anti-terrorism

simulation training and the quantifiable ability of

indicators, it should be reasonable and

comprehensive. The Delphi method is used to

determine the evaluation indicators at all levels. The

steps are as follows:

Determine the evaluation content. Prepare

anti-terrorism simulation training materials and

various instructions for relevant research.

Establish an expert group. Invite experts with

senior research in anti-terrorism simulation training

to provide experts with research contents, relevant

requirements, etc.

Through relevant materials and their own

cognition, experts filled in the evaluation index

questionnaire of anti-terrorism simulation training

command effect.

Analyze and sort out the returned questionnaires.

The second expert questionnaire shall be prepared

according to the experts' opinions and fed back to

the experts together with the previous opinions.

Again, analyze and sort out the collected

questionnaire. Repeat until the expert opinion does

not change.

2.4 Evaluation Index of

Anti-Terrorism Drill Command

Effectiveness

Establish the evaluation index system of

anti-terrorism drill command effectiveness, as

shown in Figure 1. The indicator system has four

layers: the first layer is the criterion layer, which is

the general goal of anti-terrorism drill command.

The second is the middle tier, which requires five

core capabilities for anti-terrorism drill command.

The third layer is the element layer needed to

achieve core competence, which is represented by

PMBDA 2022 - International Conference on Public Management and Big Data Analysis

234

......C

. As the element indicators under the

control of operational capability cannot be

quantified, the fourth layer of quantifiable element

indicators are added to this indicator. The third layer

has two indicators corresponding to

the accuracy and timeliness of instructions issued

under each indicator. C

311

312

......C

351

352

respectively.

Figure 1: Anti-terrorism drill command effectiveness evaluation indicator system.

3 DESIGN OF EVALUATION

MODEL FOR

ANTI-TERRORISM DRILL

Obtaining the value of the anti-terrorism drill

command effect index basically depends on the

subjective judgment of the evaluation experts. When

establishing the evaluation model, try to adopt the

strategy of eliminating the subjective factors in the

evaluation.

3.1 Evaluation of End Indicators

3.1.1 Acquisition of End Indicators

(1) Fuzzy value

The fuzzy value of expert judgment is described in

the form of interval number. The value range of

interval number is defined as:

[]

RaaaaaaQ ∈≤≤≤=

212121

, and 1000,,

.To limit the

degree of fuzziness in evaluation, the width of

interval number is specified as

()

150 ≤≤ Qw

Evaluation of Command Effectiveness of Anti-Terrorism Drill Based on Analytic Hierarchy Process

235

(2) Linguistic scale

The set of language scales is generally the level of

"excellent, good, medium, poor, and extremely

poor". In order to minimize the fuzziness in the

evaluation, each large level is subdivided into three

sub levels. See Table 1 for the corresponding

relationship between language scale and interval

number.

Table 1: Correspondence between Language Scales and Interval Numbers.

level

interval

numbe

level

interval

numbe

level

interval

numbe

excellent

Excellent+

[95,100]

medium

medium+ [65,70

)

extremely

poor

extremely

poor+

[20,30)

Excellent [90,95

)

mediu

[60,65

)

Excellent- [85,90

)

medium- [55,60

)

extremely

poor

[10,20)

good

good+ [80,85)

poor

oor+ [47,55)

goo

[75,80)

[39,47)

extremely

poor-

[0,10)

- [70,75

)

- [30,39

)

3.1.2 Valuation Criteria of Indicator Value

The evaluation criteria are the basis for evaluation

by evaluation experts and the premise to ensure the

objectivity of the evaluation. See Table 2 for the

evaluation criteria of anti-terrorism drill command

effectiveness indicator

Each index adopts a four level scoring standard,

the first level is 85-100； the second level is 70-84;

The third level is 55-69; The fourth level is 0-54.

3.2 Calculation of Index Weight

The weight is the importance of the indicator to the

system effectiveness, which reflects the evaluation

experts’ recognition of the importance of the

indicator. The effectiveness indicators of

anti-terrorism simulation training are numerous and

have a complex hierarchical structure. The weight of

indicators at each level is determined by using the

analytic hierarchy process.

3.2.1 Construction of Judgment Matrix

The indicator system shown in Figure 1 is used to

construct a judgment matrix by comparing the

elements of each layer in pairs. Taking the criterion

layer as an example, the constructed judgment

matrix is as follows:













14/17/13/15/1

413/132/1

74153

33/15/113/1

523/131

3.2.2 Calculation of Relative Weight of

Indicators

For the constructed judgment matrix, the relative

weight of each index is calculated. Taking matrix P

as an example, the relative weight of each index A=

(0.234,0.0859,0.4665,0.1696,0.0441). The

calculation results are shown in Table 2. The

maximum characteristic root is 5.1755. According to

the RI table, the corresponding RI value is 1.11, so

CR=CI/RI=0.0395<0.1, passing the one-time test.

Similarly, the relative weight of other indicators can

be obtained:

: 0.51, C

: 0.0636, C

: 0.0329, C

: 0.1296,

: 0.2638; C

:0 .1194, C

: 0.1336, C

: 0.7471;

: 0.4754, C

: 0.3203, C

: 0.1068, C

: 0.0516,

: 0.0459; C

: 0.637, C

: 0.2583, C

: 0.1047,

: 0.125, C

: 0.875; C

311

: 0.5, C

312

: 0.5; C

321

: 0.33,

322

: 0.67; C

331

: 0.67, C

332

: 0.33; C

341

: 0.67, C

342

0.33; C

351

: 0.5, C

352

: 0.5.

Table 2: Calculation of relative weight of indicators.

Indicators Feature vecto

Wei

ht value The maximum characteristic CI

1.5849 0.234

5.1755 0.0439

0.5818 0.0859

3.1598 0.4665

1.1487 0.1696

0.2988 0.0441

PMBDA 2022 - International Conference on Public Management and Big Data Analysis

236

3.2.3 Calculation of Composite Weight of

Indicators

Composite weight of indicators, that is, the weight

of end indicators relative to the overall goal. The

calculation method is: the relative weight value of

the indicator multiplied by the weight value of the

upper indicator. The calculation results are shown in

Table 3.

Table 3: Calculation of composite weight of indicators.

indic

ators

composite

weight

indic

ators

composit

e weight

indi

cato

composit

e weight

0.11934 C

0.01488 C

0.07699

0.03033 C

0.06173 C

0.01026

0.01148 C

0.06418 C

311

0.11089

312

0.11089 C

321

0.04931 C

322

0.10011

331

0.03338 C

332

0.01644 C

341

0.01613

342

0.00794 C

351

0.01071 C

352

0.01071

0.10804 C

0.04381 C

0.01776

0.00551 C

0.03859

3.2.4 Evaluation Model

The linear weighted sum model has a greater

compensation effect on smaller index values, while

the geometric weighted average model can highlight

the balance of each index. Comprehensively

consider the functionality and balance, and use the

comprehensive model to calculate, 𝐸



=𝜆



⋅

∑

𝑒



∗



⋅𝜔



+𝜆



⋅

∏

𝑒



∗











where is the weight of

the functional factors and balance factors in the final

evaluation value, and 𝜆



,𝜆



≥0,𝜆



+𝜆



=1 𝑒



∗

represents the comprehensive evaluation of the

indicators 𝑖 by the evaluation experts, and

𝜔



represents the weight of the indicators

. The

comprehensive model not only considers the

functional contribution of each indicator value to the

final evaluation value, but also considers the

balanced contribution of the end indicators.

4 APPLICATION EXAMPLES OF

COMMAND EFFECTIVENESS

EVALUATION IN

ANTI-TERRORISM DRILLS

4.1 Data Acquisition

The evaluation object is a anti-terrorism drills

command led by the Anti Terrorism Office in 2021

in City D. A total of 10 experts from the Anti

Terrorism Office of City D, the Anti Terrorism

Detachment of the Municipal Public Security

Bureau and the anti-terrorism training tactical

command and training experts were invited to score

the indicators by using two methods: fuzzy value

and language scale according to the evaluation

criteria.

4.2 Data Processing

10 experts score independently, and take the average

value as the expert score value. The results of linear

weighting and geometric weighting are shown in

Table 4. The final evaluation value is 82.6039 when

the comprehensive model is used to calculate

4.06.0



, taking into account the

functionality and balance.

Table 4: Evaluation data of anti-terrorism drill command effectiveness.

Evaluation

indicator

index

weight

Expert

score

Linear

weighted

score

Geometrically

weighted score

Evaluation

indicator

index

weight

Expert

score

Linear

weighted

score

Geometrically

weighted score

C11 0.11934

8.9505 1.674055342

C331 0.03338

2.5035 1.155020108

C12 0.01488

0.99696 1.064564533

C332 0.01644

0.9864 1.069628107

C13 0.07699

6.62114 1.409084494

C341 0.01613

0.4839 1.056394097

C14 0.03033

2.15343 1.13801678

C342 0.00794

0.5558 1.034308464

C15 0.06173

4.01245 1.293931071

C351 0.01071

0.81396 1.047474631

C21 0.01026

0.8721 1.046636409

C352 0.01071

0.87822 1.048327422

C22 0.01148

0.8036 1.049981689

C41 0.10804

7.5628 1.582502454

C23 0.06418

4.1717 1.307232327

C42 0.04381

3.32956 1.208922452

C311 0.11089

8.53853 1.618799108

C43 0.01776

1.332 1.079694995

C312 0.11089

8.31675 1.614081811

C51 0.00551

0.4408 1.024438817

C321 0.04931

3.35308 1.231291877

C52 0.03859

2.7013 1.178154736

C322 0.10011

5.0055 1.479394116

Evaluation of Command Effectiveness of Anti-Terrorism Drill Based on Analytic Hierarchy Process

237

4.3 Result Analysis

Based on the above results, the expert group's

evaluation on the command effectiveness of the

anti-terrorism drill in City D is good+, and is

satisfied with the overall command of the

anti-terrorism drill. However, there is still room for

improvement in individual indicators, such as the

timeliness of correct use of reserve police force

instructions. In the future training, we should focus

on strengthening the training of weak indicators and

high weight indicators.

5 CONCLUSION

As an emergency preparedness link, anti-terrorism

drill is an effective response to extreme violent

terrorist events and a powerful means to deter

terrorist acts. Emergency command and

decision-making is the key link in anti-terrorism

drill. This study adopts the Delphi method, takes the

actual combat improvement as the guidance, the

ability test as the standard, the quantitative analysis

as the support, and takes the comprehensive and

accurate as the basis, and relies on these indicator

systems to establish principles. The evaluation index

system of five core command capabilities, including

intelligence mastery capability, situation analysis

capability, operational control capability, combat

coordination capability and organizational support

capability, has been constructed. The indicator

system includes four levels of indicators, namely,

criterion level, intermediate level and element level.

At the same time, the comprehensive evaluation

model is established by using the analytic hierarchy

process combined with the expert group scoring.

Conduct application evaluation for anti-terrorism

drill in D city. The comprehensive model not only

considers that the linear weighted average model has

a greater compensation effect on the smaller index

value, but also considers that the geometric

weighted average model is more prominent in the

balance of each index. Therefore, the model can

effectively test the command effectiveness in the

anti-terrorism drill, and has a targeted guiding role

in improving the command ability.

FUND PROJECT

The phased research results of the general project of

Liaoning Social Science Planning Fund

(L21BGL032).

REFERENCES

Gao X., Zhang J., Zou R., Li J., Cao Z.. Multi-User

Collaborative Virtual Emergency Drill System for

Urban Road Emergencies[J]. ISPRS Annals of the

Photogrammetry, Remote Sensing and Spatial

Information Sciences, 2022, X-3/W2-2022:9-15.

Guo Yongliang. The Transition of Counter-Terrorism

Model in China--From Elite Model to Participative

Model [J]. The Jurist,2016(02):146-161.

Hu xiaohui, Establishment of anti-terrorism and

Cooperation Mechanism for the Intermodal Transit

Hub of Airport and Railway Station--Taking Haikou’s

Meilan International Airport and High Speed Railway

Station as an Example[J]. Journal of China Peoole’s

Police University.2022,38(10):62-66.

Li Heng. Construction of China’s Anti-Terrorist

Intelligence Early Warning and Response System[J].

Journal of Intelligence,2017,36(07):3-9.

Li Rong, Liu Zhigang, Pan Hanchuan, Wang Huasheng.

Comparison of relative importance of indicators. [J].

AILWAY Transport and Economy, 2020,

42(01):110-115.

Ren Longlong, Guo Song, Sun Pengju, Luo Yuping, Hu

Daxin. Evaluation of High-Speed Railway Emergency

Drills based on BP Neural Network. [J]. Railway

Transport and Economy,2020,42(12):83-88.

Shang You, Jiang Wenqi, Wang Chenchen. Evaluation on

Regional Counter-Terrorism Intelligence System

Based on Prospect Theory and TOPSIS Method[J].

Journal of Intelligence,2016,35(09):1-5.

Sunnie Haam, Jun Lee, Sanghwa Lee. Research on

necessity and applicability of virtual training for

effective police counterterrorism drills[J].

International Journal of Highway Engineering,

2020,22(2):69-78.

Sun Yueying, Du Wenya, Yang Yuhai, Gao Feng.

Strength and Conditioning Training System of SWAT

Under the Background of Modern Anti-Terrorism

Practice[J].Journal of Shenyang Sport

University,2018,37(03):118-123.

Wang Lei, Chen Guohua. Demonstration Analysis on

Emergency Drilling Performance Assessment Based

on Time Constraint Model[J]. China Safety Science

Journal,2008(02):34-39.

Yang Xinhe, Qu Daoyuan. Research on prenention of

railway safety and terrorist risks[J]. China Safety

Science Journal,2019,29(S1):145-149.

Zhao Kaigong, Wang Haiyan, Zheng Dengfeng. Research

on Structural Similarity Design Emergency Exercise’s

Scenario[J]. Geofluids,2022,2022:2-9.

PMBDA 2022 - International Conference on Public Management and Big Data Analysis

238