Leveraging Event Marketing Performance using AI in Facial Recognition

Peter Khallouf

and Christine Markarian

Data Science - Data and IT, International University of Applied Sciences, Germany

Department of Engineering and Information Technology, University of Dubai, U.A.E.

Keywords:

Artiﬁcial Intelligence, Facial Recognition Algorithm, Machine Learning, Engagement Measurement, Event

Marketing.

Abstract:

With the advances in technology and the rapid changes in human-technology interactions, the event mar-

keting ﬁeld has seen major developments over the past years. Despite its remarkable growth, many aspects

of event marketing do not yet align with the best available technologies. In this paper, we aim to leverage

event marketing performance using artiﬁcial intelligence techniques. We design a framework that optimizes

attendee-feedback generation using a facial-recognition algorithm. The framework measures attendees’ en-

gagement levels by periodically extracting attendee facial features during a session, categorizing them into

seven states of emotions (anger, disgust, fear, happiness, neutral, sadness, and surprise), and then analyzing

session engagements based on the obtained results. These measurements are then used to give insights about

an event’s performance during and after sessions, thus improving the overall performance of a given event.

The proposed framework is easy-to-implement, time-efﬁcient, and cost-effective.

1 INTRODUCTION

Over the past decades, marketing has become one of

the most prominent circulating topics, steering most

of our activities, decision-making, and inﬂuences.

It has played a major success role in most indus-

tries. With the advances in technology and the rapid

changes in human-technology interactions, the event

marketing ﬁeld has seen major developments (Hoyle,

2016; Gupta, 2003). A survey of 700 companies

conducted by Harvard Business Review Analytic Ser-

vices shows that 93 percent of companies put a lot of

effort into hosting events, 57 percent of which con-

sider it a very high priority (Services, 2018). Despite

its remarkable growth, many aspects of event market-

ing do not yet align with the best available technolo-

gies. While a lot of research has been dedicated to

studying the evolving role of artiﬁcial intelligence in

marketing in general (Sterne, 2017; De Bruyn et al.,

2020; Huang and Rust, 2021; Huang and Rust, 2021),

only few works have targeted event marketing in par-

ticular (Neuhofer et al., 2020).

In this paper, we focus our study on leveraging

event marketing performance using artiﬁcial intelli-

gence techniques.

Event marketing is known as three main goals:

generating brand awareness, measuring engagement

during an event, and educating the audience. Engage-

ment during an event is measured by observing atten-

dees in their sessions attended, meetings held, surveys

conducted, and social media activities. Generating at-

tendee feedback from these observations is often done

using traditional techniques which are mostly prone

to high level of inaccuracy and unreliability. In gen-

eral, people have a tendency, for many reasons, not

to reveal their true opinions about events or products.

They sometimes do not want to put effort into ex-

pressing their likes and dislikes - the reason why many

surveys are ﬁlled recklessly and inattentively. Gath-

ering proper feedback would require constantly fol-

lowing up with attendees, which not only incurs ad-

ditional costs, but does not give the correct insights,

since many people would simply not want to par-

ticipate in giving feedback. Such inaccuracies lead

to degradation in the Return On Investment (ROI).

Hence, a slight improvement in the techniques used to

generate customer feedback would imply higher suc-

cess rates in proﬁt maximization.

Motivated by the above, we develop in this pa-

per a novel framework that leverages event marketing

performance by improving attendee feedback gener-

ation using Artiﬁcial Intelligence (AI) techniques in

facial recognition. The idea is to observe attendees

during the sessions and identify, based on their fa-

cial expressions, whether they feel engaged or not.

568

Khallouf, P. and Markarian, C.

Leveraging Event Marketing Performance using AI in Facial Recognition.

DOI: 10.5220/0010864800003116

In Proceedings of the 14th International Conference on Agents and Artiﬁcial Intelligence (ICAART 2022) - Volume 3, pages 568-573

ISBN: 978-989-758-547-0; ISSN: 2184-433X

This gives companies insights about their presenta-

tions, trainings, and products, without the need to

reach out to potential customers to ask about their

feedback. Moreover, given that facial expressions are

known to be a direct translation of true emotions (Ek-

man, 1993; Niedenthal et al., 2000), the information

gathered in this framework would be way more accu-

rate compared to those gathered using the traditional

approaches.

Artiﬁcial Intelligence (AI) techniques have been

widely used in marketing. (Campbell et al., 2020)

studied the revolution of AI in business and society,

aiming to understand and engage customers. (Kietz-

mann et al., 2018) used AI to understand and guide

consumers with the help of machine learning tech-

niques, by collecting data from different sources and

combining them to gain on-the-spot consumer in-

sights. (Whitehill et al., 2014) calculated the relia-

bility of human observers’ judgments to learn about

students’ engagement from their faces.

To the best of our knowledge, there has been no

study about event engagement measurement using AI.

1.1 Our Contribution

In this paper, we propose a new engagement mea-

surement framework that aims to optimize attendee

feedback generation. The framework uses a facial-

recognition algorithm to classify attendees based on

seven states of emotions (anger, disgust, fear, hap-

piness, neutral, sadness, and surprise) and accord-

ingly calculates their average engagement level in a

given session. Our proposed approach is easy-to-

implement, time-efﬁcient, and cost-effective.

At the heart of the framework is a software that de-

ploys a high-deﬁnition camera to extract attendee fa-

cial features at speciﬁc session times (including time

spent in a session). The software inputs the extracted

pictures into a facial recognition algorithm, resulting

in the attendees’ states of emotions. The records are

saved in a database and are used to calculate the av-

erage session engagements. These engagements are

then used to give insights about the success of a given

presentation or an event in general. Moreover, what

makes the framework even more powerful is the fact

that the results of session engagements obtained from

the software could be displayed instantly to session

presenters. Hence, if during a session, the majority

of the states of emotions express confusion, the ses-

sion presenter can shift immediately to other means of

communication to get back attendees’ attention. The

software would give the proper alerts instantly during

a presentation so as to make the best out of a given

session.

To output attendees’ states of emotions, our

framework uses the facial recognition algorithm pro-

posed in (Richhariya and Gupta, 2019). This algo-

rithm is shown to outperform most facial-recognition

algorithms available so far in the literature, in terms

of speed, accuracy, and implementation cost.

1.2 Outline

The remainder of the paper is structured as follows.

In Section 2, we give an overview of works related to

AI-based event marketing and facial recognition. In

Section 3, we present the algorithmic, hardware, and

software requirements of the proposed framework. In

Section 4, we present the engagement measurement

framework and analyze it in Section 5. In Section 6,

we extend the scope of the framework to event orga-

nizers and attendees. We conclude in Section 7 with

a discussion and future works.

2 RELATED WORK

In this section, we give an overview of AI-based event

marketing and facial recognition studies related to our

work.

2.1 AI-based Event Marketing

AI has been intensively used to enhance marketing

strategies. A study in (Campbell et al., 2020) shows

the revolution of AI in business and society in terms of

predicting, understanding, and engaging customers.

The study is based on using nine marketing functions

in the marketing planning process to consolidate how

AI can enhance it. The nine components are inherent

to strategic marketing, providing an organizing frame-

work to assist marketing professionals with practical

uses of AI.

(Kietzmann et al., 2018) studies the importance

of AI in the way advertisers understand and guide

consumers. With the help of machine learning tech-

niques, advertisers are able to collect consumers’ data

from different sources, combine the collected data,

and mine them, to deliver on-the-spot consumer in-

sights. The study highlights on the role of AI in help-

ing consumer and advertisers at the same time, by giv-

ing insights that take into account the public’s privacy

rights.

2.2 Facial Recognition

Our framework deploys the facial recognition algo-

rithm proposed in (Richhariya and Gupta, 2019). The

Leveraging Event Marketing Performance using AI in Facial Recognition

569

algorithm is based on using the so-called Univer-

sum data to conduct multiclass face emotion catego-

rization from human facial pictures. Unlike previ-

ous Universum-based models which have high train-

ing costs, the proposed algorithm, known as, It-

erative Universum Twin Support Vector Machine

(IUTWSVM), outperforms the previous models. It

is characterized by high accuracy in detecting facial

expression, speed, and low implementation cost.

A study about automatic recognition of student

engagement from facial expressions in (Whitehill

et al., 2014) has been an inspiration for our work in

this paper. The study considers human observers’

judgement, studies the signals made from these judge-

ments, and uses machine learning techniques to auto-

mate the process. A comparison between human ob-

servers and AI-based observation is made. Low-cost

computers with a high-resolution camera are used to

monitor the engagement of students in a given class.

This information is used to understand when and why

students get disengaged and to accordingly shift to

other teaching methods.

3 TECHNICAL REQUIREMENTS

To implement the solution and integrate between the

infrastructure, the high-deﬁnition camera, and the

facial-recognition algorithm, a number of hardware

and software requirements are to be met. In this sec-

tion, we give an overview of the facial-recognition al-

gorithm and discuss these requirements.

3.1 Facial-Recognition Algorithm

Facial recognition is generally performed in ﬁve

steps (Woodward Jr et al., 2003):

1. An image taken by a camera is acquired.

2. The locations of faces are detected in the acquired

image.

3. Each detected face is analyzed based on the spa-

tial geometry of face features. There are different

ways to extract these features, the most common

method is known as Principle Components Anal-

ysis (PCA). This process yields a so-called tem-

plate for each face, which is a reduced set of data

representing the unique face features.

4. The generated template is compared to templates

of known faces stored in a database. This process

results in scores that indicate how similar the gen-

erated template is in comparison to those in the

database. In case of identity veriﬁcation, the gen-

erated template is compared to only one template

Figure 1: Findface Camera Interface Detecting Several

Faces (Findface, 2021).

in the database, that associated with the claimed

identity.

5. The last step determines whether the scores ac-

quired are high enough to declare a match.

We refer the reader to (Richhariya and Gupta,

2019) for a detailed description of how the algorithm

performs these steps.

3.2 Hardware Requirements

Unlike other IT solutions that normally require a

server and sometimes with high speciﬁcations, our

proposed solution does not deploy a server. Instead,

one computer with the following speciﬁcations is

used, to share the resources among the different parts

of the solution. The computer needed would run on

a Windows 7 operating system and have the follow-

ing speciﬁcations: 64-bit architecture, 3.20 GHz Intel

Core i7 2400 processor, 8 GB of RAM, Nvidia video

card 3Gb of video functionality. We can additionally

add a redundant computer to cover the high availabil-

ity plan and failover coverage. However, this is only

needed for robustness as a form of a contingency plan

associated with the event’s success.

A high-deﬁnition camera is used. We propose the

FindFace camera, which is recognized in many world-

wide projects and is highly reliable in terms of hard-

ware robustness and service support services (Find-

face, 2021). It is easily available and has relatively

low costs. The FindFace camera is able to simulta-

neously identify a signiﬁcantly large number of faces

and send notiﬁcations at a very high speed - it can

detect, identify, and notify about, all the images avail-

able within the same frame in 0.5 seconds. Figure 1

shows the camera’s interface detecting several faces

at a time. Our framework uses this feature to allow

a presenter in an event to shift between presentation

methods based on the notiﬁcations received.

ICAART 2022 - 14th International Conference on Agents and Artiﬁcial Intelligence

570

3.3 Software Requirements

The computer would host a database that saves all the

inward transactions from the camera using the man-

ufacturer’s Software Development Kit (SDK). These

transactions would contain the attendees’ images that

will be fed to the facial recognition algorithm. The

latter returns the state of emotion (anger, disgust, fear,

happiness, neutral, sadness, and surprise) for each im-

age, which will be saved in the database. At the end

of the event, the database will contain loads of data

that can be expressed in ﬁgures and translated into in-

sights. Depending on the investor’s database licensing

budget, any of the following databases could be used:

SQL Server, MySQL, or Oracle.

MATLAB R2008b will be installed on the com-

puter to implement the facial recognition algorithm.

The software can be developed using any .Net lan-

guage. Its functionalities are straightforward - it com-

municates with the camera, sends the images to the

facial recognition algorithm, saves the transactions in

the database, and reports tools to visualize the ﬁnal

results. We give a detailed software architecture in

Section 4.

4 ENGAGEMENT

MEASUREMENT

FRAMEWORK BASED ON

FACIAL RECOGNITION

The engagement measurement framework builds

upon the following steps.

1. The FindFace camera is installed in the exact lo-

cation (kiosk) where a given presentation is taking

place.

2. The FindFace camera will be connected to the

main network where the computer is deployed.

3. A number of software will be installed to the com-

puter, namely, the Database Management System

(DBMS), MATLAB R2008b, and the FindFace

driver.

4. A database with the required tables will be cre-

ated to hold the incoming information from the

camera and the facial-recognition algorithm. This

database will be installed to the Database Man-

agement System (DBMS).

5. The DBMS’s high availability will be conﬁgured

to the other computer to prevent any possible

failover. As described in Section 3, this will only

be added in case a robust solution is desirable.

Figure 2: Flowchart/ Process Diagram.

Figure 3: Software Design Structure.

6. The below software will be developed. Its job

will be to communicate with the different hard-

ware and software components of the solution.

The software will be developed using any .Net

language, preferably C-Sharp. The latter is a high-

level language and is easy to use. The implemented

software will be straightforward and hence easy to im-

plement. Its functionality is described as follows - we

refer the reader to Figure 2 for an illustration.

– integrate, via the SDK, with the FindFace camera

– receive the captured images

– send the received images to MATLAB for feature

extraction

– save the results from MATLAB in the database

– generate a report based on the treated data in the

database

Next, we describe the software’s design in terms

of its view, data, and logic layers - we refer the reader

to Figure 3 for an illustration.

View Layer: The software’s view layer includes

two tabs. The ﬁrst tab contains the FindFace cam-

era and the database controls to input the connectivity

link. Other controls display the connectivity status

and the number of images saved in the database, after

Leveraging Event Marketing Performance using AI in Facial Recognition

571

being processed by the facial recognition algorithm.

The database will contain a record for each attendee

with information about the time stamp, presentation

session, and each emotional status during the presen-

tation. These controls are needed to monitor and en-

sure that the system functions properly. The second

tab contains the dashboard that will display the results

associated with the event marketing evaluation in the

form of graphs and spreadsheets. These results will

contain the live data as well as the conclusive data

at the end of the event, of a given presentation. At

this point, one can measure the engagement level in

the presentation based on the insights concluded from

these data.

Data Layer: The data layer uses transactions in all

the database classes to ensure the transaction’s roll-

out during interruptions. We create the database con-

nection class, referenced in a separate class that will

contain the Data Manipulation Language (DML).

Logic Layer: The logic layer is coded in a scalable

way using classes and objects to ensure its adaptive-

ness with future solution enhancements and different

integrations. The facial recognition algorithm will

have a separate class, which will return the state of

emotion for the corresponding received image. The

main form will run a thread having the ordered func-

tion as follows:

– Create a record in the database for each attendee

once the presentation starts.

– Run the facial recognition algorithm periodically,

such as, every one minute, and update the results

associated with all attendees in the database.

– Update a ﬂag in the database, during the presen-

tation, about the session’s status (in progress, ﬁn-

ished)

– The presenter’s computer will be linked to the

same network as the software, and the software

will integrate with the presentation tools to record

the start/end of the presentation.

5 PERFORMANCE

Unlike traditional feedback generation approaches

that require extensive resources in terms of time, cost,

and processing, the proposed framework is able to

output large amounts of insightful information within

milliseconds.

The framework implements a facial-recognition

algorithm, rather than relying on the facial-

recognition features offered by the state-of-the-art

cameras. This not only implies lower costs, but it also

means that by only performing algorithm improve-

ments, one can improve the quality of the solution

provided by the framework. Hence, hardware main-

tainance and support efforts are only limited to the

main functionality of the camera, i.e., capturing the

faces.

As can be observed in Section 3, the implemen-

tation of the entire framework requires merely a high

level of integration between the hardware and soft-

ware. This simplicity makes the framework even

more practical and hence desirable.

6 EVENT ORGANIZERS &

ATTENDEES

In a broader context, our framework can be extended

to be deployed by large event organizers, such as

Expos. Event organizers can attract more organiza-

tions around the globe by providing features using our

proposed framework. Additions can be easily made

to allow the framework to collect and analyze the

attendee-feedback data related to a particular industry

and share this data with the related companies. This

helps companies get insights not only about their own

products and presentations but also about other com-

petitors. Companies who wish to share such data and

be given such features could beneﬁt from the experi-

ence of other companies and accordingly make their

future decisions.

Moreover, the framework could also be extended

to give insights to attendees. In huge events, it is com-

mon for attendees to visit several kiosks and keeping

track of the visits might not be easy. That is, an at-

tendee is very likely to forget his/her impression about

products or companies unless he/she makes an effort

to. The extended framework could provide each at-

tendee, upon his/her request, a summary of and an

overall impression about the kiosks visited, the pre-

sentations attended, and the interactions made during

the event. This makes the attendee’s visit more efﬁ-

cient and fruitful.

7 CONCLUDING REMARKS &

FUTURE WORK

We have proposed in this paper a new AI-based

framework that leverages event marketing perfor-

mance by optimizing attendee-feedback generation

using a facial-recognition algorithm. This framework

ICAART 2022 - 14th International Conference on Agents and Artiﬁcial Intelligence

572

helps optimize event marketing strategies made by

companies after an event and during it.

A next step would be to implement this framework

in a real-world event and compare its performance to

that of a traditional framework. As described earlier,

the approach used in the framework is time-efﬁcient

and can be easily implemented, making it a promising

event managing and marketing tool.

Like any other AI-based framework, it is impor-

tant to address the ethical concerns which accompany

the proposed framework (Liu et al., 2021; Zhu et al.,

2020; Oseni et al., 2021). While such events are gen-

erally public and it is common to have cameras in-

stalled for security purposes, it is still important to

get the approval of attendees when it comes to us-

ing their photos and analyzing them to get insights.

Hence, depending of the region where the event is

taking place, the proposed engagement measurement

framework will be accompanied with a number of

procedures, policies, and guidelines.

We have demonstrated in this study the role of AI

in leveraging event marketing performance, focusing

on attendee engagement as an indicator of the quality

of presentations/events. Our proposed platform ana-

lyzes the feedback of attendees by observing the at-

tendees’ facial expressions. The role of the platform

becomes even more vital in the case of presentations

in which there is no human presenter. In this case, the

system is expected to make an automatic intervention

as a reaction to the feedback gathered. To achieve

this, the system is fed with a number of presenting

formats such as image, video, or text. As a reaction

to the attendees’ feelings, the system would automat-

ically shift between these formats to produce the best

results.

REFERENCES

Campbell, C., Sands, S., Ferraro, C., Tsao, H.-Y. J.,

and Mavrommatis, A. (2020). From data to ac-

tion: How marketers can leverage ai. Business

Horizons, 63(2):227–243. ARTIFICIAL INTELLI-

GENCE AND MACHINE LEARNING.

De Bruyn, A., Viswanathan, V., Beh, Y. S., Brock, J. K.-

U., and von Wangenheim, F. (2020). Artiﬁcial intelli-

gence and marketing: Pitfalls and opportunities. Jour-

nal of Interactive Marketing, 51:91–105.

Ekman, P. (1993). Facial expression and emotion. American

psychologist, 48(4):384.

Findface. Face detection, veriﬁcation and recognition tech-

nology. Available at https://ﬁndface.pro/en/.

Gupta, S. (2003). Event marketing: Issues and challenges.

IIMB Management Review, 15(2):87–96.

Hoyle, L. H. (2016). Event marketing. John Wiley & Sons,

Inc.

Huang, M.-H. and Rust, R. T. (2021). A strategic frame-

work for artiﬁcial intelligence in marketing. Journal

of the Academy of Marketing Science, 49(1):30–50.

Kietzmann, J., Paschen, J., and Treen, E. (2018). Artiﬁcial

intelligence in advertising: How marketers can lever-

age artiﬁcial intelligence along the consumer journey.

Journal of Advertising Research, 58:263–267.

Liu, B., Ding, M., Shaham, S., Rahayu, W., Farokhi, F., and

Lin, Z. (2021). When machine learning meets privacy:

A survey and outlook. ACM Comput. Surv., 54(2).

Neuhofer, B., Magnus, B., and Celuch, K. (2020). The

impact of artiﬁcial intelligence on event experiences:

a scenario technique approach. Electronic Markets,

pages 1–17.

Niedenthal, P. M., Halberstadt, J. B., Margolin, J., and

Innes-Ker,

A. H. (2000). Emotional state and the de-

tection of change in facial expression of emotion. Eu-

ropean journal of social psychology, 30(2):211–222.

Oseni, A., Moustafa, N., Janicke, H., Liu, P., Tari, Z., and

Vasilakos, A. (2021). Security and privacy for artiﬁ-

cial intelligence: Opportunities and challenges. arXiv

preprint arXiv:2102.04661.

Richhariya, B. and Gupta, D. (2019). Facial expression

recognition using iterative universum twin support

vector machine. Applied Soft Computing, 76:53–67.

Services, H. B. R. A. (2018). The event marketing evolu-

tion. Technical report.

Sterne, J. (2017). Artiﬁcial intelligence for marketing:

practical applications. John Wiley & Sons.

Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., and Movel-

lan, J. (2014). The faces of engagement: Automatic

recognition of student engagement from facial expres-

sions.

Woodward Jr, J. D., Horn, C., Gatune, J., and Thomas,

A. (2003). Biometrics: A look at facial recognition.

Technical report, RAND CORP SANTA MONICA

CA.

Zhu, T., Ye, D., Wang, W., Zhou, W., and Yu, P. S. (2020).

More than privacy: Applying differential privacy in

key areas of artiﬁcial intelligence. arXiv preprint

arXiv:2008.01916.

Leveraging Event Marketing Performance using AI in Facial Recognition

573