Exploring the Relationship Between Emotions and Norms in

Decision-Making Processes of Intelligent Agents

Joaquin Taverner

1,2 a

, Carmengelys Cordova

1,2 b

, Elena Del Val

1,2 c

, Soledad Valero

1,2 d

and Estefania Argente

1,2 e

Valencian Research Institute for Artiﬁcial Intelligence (VRAIN), Universitat Polit

ecnica de Val

encia, Valencia, Spain

Valencian Graduate School and Research Network of Artiﬁcial Intelligence (valgrAI), Spain

Keywords:

Emotions, Norms, Intelligent Agents, Multiagent Systems.

Abstract:

In this paper, we explore the relationship between norms and emotions, examining the fundamental implica-

tions that they entail in the development of future models of reasoning and decision-making for intelligent

agents based on BDI. Our approach focuses on assessing the impact of anticipated emotions, self-image, and

social-image projection as well as utility factors on the complex decision-making processes that an agent faces

when deciding whether to comply with or violate a norm. To this end, we propose the use of two types of

anticipated emotions, self-conscious, which shape the personal self-image, and social emotions, which are

displayed by other agents in the environment and are used to estimate the social image. To represent the

agent’s emotional state, we present a new model based on the pleasure dimension in which we represent the

self-conscious emotions of Pride and Guilt. Using the language for intelligent agents AgentSpeak, we propose

a syntax for deﬁning the norms in the agent’s code. We show a new reasoning cycle based on the BDI model

in which we add new functionalities to accommodate affective and normative processes. Affective processes

support modifying the agent’s emotional state as well as estimating anticipated emotions and computing self-

image and social image. Normative processes allow the instantiation of active norms and normative reasoning.

1 INTRODUCTION

Traditionally, intelligent agent decision-making mod-

els have mainly focused on economic or practical rea-

soning. However, these approaches fall far short of

achieving an accurate simulation of human behavior.

Future simulation models will have to reach a higher

level of abstraction, both in knowledge representation

and in reasoning and decision-making processes, to

encompass concepts such as emotions, ethics, values,

or social norms (Zhang and Lu, 2021; Walton, 2019;

Dorri et al., 2018; da Costa Pereira et al., 2017).

Norms play an essential role in individual

decision-making and in structuring social behavior

(Daci et al., 2010; O’Neill, 2017). Social norms fa-

cilitate cohesion and cooperation by establishing a

shared set of rules that establish the boundaries be-

https://orcid.org/0000-0002-5612-8033

https://orcid.org/0009-0006-3130-3532

https://orcid.org/0000-0002-1279-3429

https://orcid.org/0000-0003-4565-326X

https://orcid.org/0000-0002-5431-3868

tween what is acceptable and inappropriate in differ-

ent situations, contributing to collective identity and

membership (Gross and Vostroknutov, 2022a; An-

drighetto et al., 2015). Beyond guiding decisions, the

internalization of these norms shapes personal values

and how the consequences of actions are evaluated

(Sterelny, 2019). Understanding the importance of

norms is crucial for accurately analyzing human be-

havior and is a key element in any attempt to authen-

tically simulate the complexities of social interactions

in artiﬁcial intelligence.

On the other hand, emotions also play a funda-

mental role in human behavior and decision-making

processes, inﬂuencing how we perceive and respond

to the world around us. Affective states have a di-

rect impact on the evaluation of situations, affecting

our choices and actions (Lerner et al., 2015; Barnes

and Thagard, 2019). In the context of social norms,

emotions are intrinsically linked to conformity and

compliance with established norms (Tangney et al.,

2007; Tracy and Weidman, 2021). Norms not only

act as ethical guides but also generate emotional re-

sponses associated with social acceptance or fear of

Taverner, J., Cordova, C., Val, E., Valero, S. and Argente, E.

Exploring the Relationship Between Emotions and Norms in Decision-Making Processes of Intelligent Agents.

DOI: 10.5220/0012594700003636

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 16th International Conference on Agents and Artiﬁcial Intelligence (ICAART 2024) - Volume 1, pages 471-479

ISBN: 978-989-758-680-4; ISSN: 2184-433X

471

rejection (Bagozzi et al., 2016; Demeulenaere, 2021).

This connection between emotions and norms is bidi-

rectional: emotions can drive compliance with norms,

and, in turn, adherence to norms can elicit emotional

responses (Argente et al., 2020). Understanding this

dynamic link between emotions and norms is essen-

tial to simulate human social behavior and the behav-

ior of human social organizations, both in individual

decision-making and in broader social interactions.

In this paper we study the relationship between

emotions and norms in depth, exploring the possi-

ble implications for future models of intelligent agent

reasoning and decision making. The main research

objective is to analyze the impact of anticipated emo-

tions, self-image, and social image on the decision-

making processes when an agent must decide whether

or not to comply with a norm. To address these com-

plex dynamics, we propose an innovative approach

that incorporates emotional components into a nor-

mative agent model based on BDI model (Beliefs-

Desires-Intentions) (Rao et al., 1995).

The rest of the paper is organized as follows: Sec-

tion 2, an initiation into the theoretical background is

presented. Section 3 offers an overview of pertinent

state-of-the-art research concerning affective and nor-

mative agents. The introduction of our novel norma-

tive affective agent model is detailed in Section 4. Fi-

nally, the concluding remarks and prospects for future

work are presented in Section 6.

2 EMOTIONS AND NORMS

In human society, there are moral and social-

conventional norms, but also legal, epistemic, aes-

thetic and personal norms (O’Neill, 2017). Nucci

and Turiel identiﬁed moral norms as those relating to

“justice, welfare, or rights of individuals or groups”

(Nucci and Turiel, 1978). Conventional norms are

deﬁned as a set of rules and guidelines that are es-

tablished by a society to regulate the behavior of

its members (Killen et al., 2013). Norms prescribe

the behavior that is considered acceptable or unac-

ceptable in various social situations (Demeulenaere,

2021). From a sociological perspective, norms serve

as a basis for maintaining order and social cohesion,

providing a shared framework of reference that gov-

erns the behavior of individuals within a group or

society and guides interactions and relationships be-

tween those individuals (Boytsun et al., 2011). The

establishment of behavioral expectations facilitated

by norms facilitates relationships, communication,

and cooperation among members of society.

We can fundamentally categorize norms into ﬁve

types (Argente et al., 2020):

• Institutional norms, which are established by

competent authorities (e.g., the government) and

are presumed to be followed by all individuals in

a society.

• Social norms (or conventions), which emerge

from repeated social interactions, establishing

practices that are socially accepted.

• Interaction norms, which refer to formal agree-

ments that affect speciﬁc groups for a limited pe-

riod (e.g., a commercial contract).

• Private norms, which are internal rules self-

imposed by individuals to guide their behavior.

Among the various theories on emotions,

appraisal-based perspectives have gained wide accep-

tance in the ﬁeld of emotional psychology (Lazarus,

1991). According to this approach, emotions are not

simply considered as automatic responses to external

stimuli but arise through a process of cognitive

evaluation. Therefore, emotions can be considered

as subjective and evaluative responses to events or

situations, shaped by each individual’s interpretation

of the personal relevance and emotional signiﬁcance

of such events (Lewis et al., 2010).

Different types of emotional experiences affect in-

dividuals’ decisions and behaviors in signiﬁcant ways

(Barnes and Thagard, 2019). For this paper, we focus

on anticipated emotions (Bagozzi et al., 2016). These

types of emotions represent to some degree emotional

expectations regarding future events, either in terms

of positive emotions ( i.e., satisfaction), negative emo-

tions ( i.e., fear or anxiety), or neutral emotions ( i.e.,

calm or indifference) (Carrera et al., 2011). Here we

can distinguish two types of anticipated emotions de-

pending on the target to which these emotions are di-

rected. Thus, anticipated emotions can be directed

towards oneself, reﬂecting how the individual inter-

nally perceives the emotional consequences of his/her

intentions, which is part of one’s self-image (Bodner

and Prelec, 2003; Gross and Vostroknutov, 2022a).

On the other hand, emotions can come from individu-

als in the social environment. The anticipation of so-

cial emotions is closely linked to concerns about how

individuals will be perceived by other individuals in

society (Simamora, 2021), which is known as social

image (Gross and Vostroknutov, 2022b). Social emo-

tions and social image provide guidance in decision-

making permitting subjects to anticipate and evaluate

future social outcomes (Grossman and Van der Weele,

2017; Gratch et al., 2006).

There is an evident inﬂuence between norms and

emotions that acts in a bidirectional way. Emotions

EAA 2024 - Special Session on Emotions and Affective Agents

472

often promote compliance with norms, while adher-

ence to norms can trigger emotional responses (Ar-

gente et al., 2020). Self-conscious emotions, like

Pride or Guilt, play a fundamental role in the self-

evaluation of norm compliance (Tangney et al., 2007).

Moreover, based on expectations, anticipated emo-

tions are used to estimate the emotional impact of

complying with a norm (Bagozzi et al., 2016).

In (Argente et al., 2020), four types of relation-

ships between norms and emotions are deﬁned (see

Figure 1):

• (i) Emotion is considered in the normative rea-

soning process: emotions are a factor to be con-

sidered during the stage of reasoning in which

it is determined whether to fulﬁll or violate a

norm. Speciﬁcally, anticipated self- and social

emotios play a crucial factor in the normative rea-

soning process by anticipating future emotional

outcomes of fulﬁlling or violating a norm. For

example, self- or social emotions (e.g., pride or

disgust) can be motivators for complying with a

norm.

• (ii) Norm fulﬁllment/violation generates emo-

tions: norm fulﬁllment/violation becomes a trig-

gering event for a cognitive-level appraisal pro-

cess that results in the elicitation of emotions.

Thus, the fulﬁllment of a norm can produce posi-

tive emotions in the individual (e.g., pride) and in

the individuals that compose his/her social envi-

ronment (e.g., gratitude). However, the violation

of a norm can produce negative emotions (e.g.,

guilt) in the individual and generate social rejec-

tion.

• (iii) Emotion enables compliance with social

norms: speciﬁcally, anticipated social emotions

have been proposed as facilitators of compliance

with social norms. When an individual anticipates

the emotional impact of fulﬁlling or violating a

norm (i.e., social image), he/she uses this infor-

mation as a factor in determining whether to fulﬁll

or violate a norm.

• (iv) Emotion helps to internalize private norms:

the continuous observation of the emotional and

social consequences of compliance or violation of

behavior in a given social environment facilitates

the internalization of private or internal norms. In

addition, it allows individuals to improve their ac-

curacy in establishing their own social emotions

in the face of a given behavior.

(i) Emotion is considered in the

normative reasoning process

(ii) Norm fulﬁllment/

violation generates emotions

(iii) Emotion enables

compliance with social norms

(iv) Emotion helps internalizing

private norms

Norms

Emotions

Figure 1: Relationships between emotions and norms.

3 EMOTIONS AND NORMS IN

AGENTS

In the ﬁeld of artiﬁcial intelligence, intelligent agents

are deﬁned as entities capable of perceiving their en-

vironment, making decisions autonomously, and per-

forming actions to achieve speciﬁc objectives. One of

the most widely used approaches for modeling agents

is the Beliefs-Desires-Intentions (BDI) model (Rao

et al., 1995). This model is based on the idea that

agents act according to their beliefs about the world,

their desires or goals to be achieved, and the inten-

tions that derive from the combination of beliefs and

desires.

In the literature, we can ﬁnd different proposals

that have covered to a greater or lesser extent the re-

lationships between emotions and norms established

in Section 2. For example, in (Staller et al., 2001)

TABASCO

JAM

, an architecture for normative BDI

agents, is proposed. That architecture encompasses

the relationship (i) (see Figure 1). In that architecture,

the normative reasoning of agents is modeled by a set

of conditional rules that include the intensity of emo-

tion as a decision factor in determining whether or not

to execute a given action.

In (Ahmad et al., 2012) the model for normative

agents OP-RND-E is presented. The authors propose

a model that uses the fulﬁllment or violation of norms

as a motivating element for the elicitation of emo-

tions. For this purpose, they use an emotional ap-

praisal process capable of eliciting positive or nega-

tive emotions. These emotions are subsequently used

by the agent to reason about the fulﬁllment of its ob-

jectives. Therefore, that proposal covers relationship

(ii) through the use of elicited emotions.

Another interesting proposal can be found in (Fer-

Exploring the Relationship Between Emotions and Norms in Decision-Making Processes of Intelligent Agents

473

reira et al., 2013). The authors use the architecture

for affective agents BDI FAtiMA (Mascarenhas et al.,

2022) to develop a normative agent model. That

model makes it possible to deﬁne the relationships be-

tween agents and to establish acceptable social behav-

iors. Agents react emotionally, with self-conscious

emotions such as pride or shame when they perceive

that an agent in the environment has performed a be-

havior that is acceptable or non-acceptable. Similarly,

in (Tzeng et al., 2021; Tzeng, 2022) a model is pro-

posed that uses social emotions to determine compli-

ance with norms. The agent has an affective appraisal

process based on a ﬁnite set of rules that deﬁne pre-

deﬁned emotions related to the fulﬁllment/violation

of each norm.

Anticipated emotions have also been used in nor-

mative agent models. For example, in (Kollmann

et al., 2016), the ECABA architecture is presented.

This architecture employs anticipated self-consciuos

emotions in the normative reasoning process of agents

to determine whether to fulﬁll or violate a norm.

As can be seen, there are different proposals of

agents considering emotions in normative reasoning

processes. However, these proposals generally ex-

plore only one of the relationships between emotions

and norms described in Section 2. Also, to our knowl-

edge, none of these proposals explore the combina-

tion of self-conscious and social anticipated emotions.

4 PROPOSAL

In this paper, we present an innovative approach to

agent modeling using a BDI model and the AgentS-

peak language (Bordini and H

ubner, 2005). Our

model incorporates emotions and norms comprehen-

sively into the agent’s decision-making process, ad-

dressing the complex relationships (i), (ii), and (iii)

between these elements described in Section 2. We

highlight the importance of emotional state and an-

ticipated emotions in the agent’s normative reason-

ing, especially when faced with decisions about norm

compliance or violation.

To develop a model of normative-affective agents,

we propose a normative agent model that considers

anticipated emotions at both the individual and social

levels. This article focuses on the deontic operator

of prohibition, but in the future, the proposal could

be extended to other operators. In our model, norms

are represented using the tuple

⟨

id, A, P, R, S

⟩

, where

id serves to identify the norm, A represents the set of

conditions activating the norm, P is the set of actions

affected by the norm, and R and S denote the reward

and sanction, respectively. For instance, to express

the prohibition of exceeding 60 kilometers per hour,

and following the prolog-style syntax commonly used

in AgentSpeak, the syntax of the norm is deﬁned as:

prohibition(ms, so 60, accelerate, 0, 1)

where, ms (max speed) is the norm’s identiﬁer,

so 60 (speed over 60) denotes the activation condi-

tion, in this example, it is activated when the agent is

driving in a zone limited to 60 kilometers per hour;

accelerate is the action of accelerating the vehicle,

and the reward for adhering to the speed limit is 0,

while the sanction is 1.

In this system, all agents are capable of perceiv-

ing the actions performed by other agents in their en-

vironment with a certain degree of probability. When

agents perceive that an action complies with or vio-

lates a norm, they can react emotionally.

Table 1 summarizes the proposed emotions for

our normative-affective agent model. We deﬁne two

social emotions, Gratitude, and Disgust, along with

three internal emotions, Pride, Guilt, and Calm. So-

cial emotions are triggered, with a certain probability,

when agents perceive that another agent has complied

with or violated a norm. These emotions have been

carefully chosen to encompass a broad spectrum of

emotional responses within both social and internal

contexts. Gratitude stands out for reinforcing posi-

tive social interactions, fostering connection, and ac-

knowledging the beneﬁcial actions of others (Bartlett

et al., 2012). Meanwhile, disgust plays a crucial role

in signaling aversion to undesirable social behaviors,

thus contributing to the regulation of social norms and

values (Inbar and Pizarro, 2022). On the other hand,

pride serves as a mechanism of positive reinforce-

ment, providing a sense of achievement and boost-

ing self-esteem. In contrast, guilt functions as a self-

regulation mechanism, prompting reﬂection on past

actions and contributing to more conscientious and

socially responsible behavior (Onwezen et al., 2013).

Finally, calm is presented as an essential element for

establishing emotional balance, offering a serene state

that facilitates thoughtful decision-making and effec-

tive stress management (Meshulam et al., 2012).

Internal emotions are deﬁned through the pleasure

dimension (see Figure 2). Through this representa-

tion, the agent can have different intensities for each

emotion, providing a more nuanced perspective in its

emotional response.

During the normative reasoning process, both self-

emotions and social emotions are anticipated to esti-

mate self-image and social image. Subsequently, self-

image and social image are used by the normative rea-

soning process to decide whether to fulﬁll or violate

the norm.

EAA 2024 - Special Session on Emotions and Affective Agents

474

Table 1: Emotions that can be elicited in our agent model.

Pleasure

Emotion

Social Self

Positive Gratitude Pride

Negative Disgust Guilt

Neutral Calm

-1 0 1-0.5 0.5

Guilt Pride

Calm

PleasureDispleasure

-0.25-0.75 0.750.25

Figure 2: Pleasure dimension.

The Algorithm 1 illustrates the agent’s behav-

ior (see Figure 3). The agent can initially have a

set of initial beliefs B

= {b

, b

, ...} stored in the

agent’s belief set B = {b

, b

, ...}; a set of initial in-

tentions I

= {ι

, ι

, ...} saved in the agent’s intention

set I = {ι

, ι

, ...}; a set of norms N

= {n

, n

, ...}

stored in the norm set N = {n

, n

, ...}, where n

⟨

id, A, P, R, S

⟩

; and the initial emotional knowledge E

stored in the variable E. Emotional knowledge con-

sists of a tuple

⟨

σ, ∆

, ∆

−

, ρ

−

⟩

, where σ repre-

sents the agent’s pleasure level; ∆

and ∆

−

denote the

agent’s tendency to experience positive (e.g., Pride) or

negative (e.g., Guilt) emotions, respectively; and ρ

and ρ

−

are vectors recording the number of positive

(e.g., Gratitude) and negative (e.g., Disgust) emotions

perceived in agents in the environment as a result of

certain actions.

Once the variables are initialized, the process be-

gins by obtaining the set of environment perceptions

β ← {θ

, θ

, ...} using the percept method. These

perceptions include both the agent’s observations in

the environment and any messages received from

other agents. Subsequently, the belief revision

method takes the agent’s beliefs B along with the set

of perceptions β as input and updates both the belief

set B and the emotional information E, i.e., the emo-

tions perceived in the environment.

Next, the agent’s desires D are estimated using

the options method, and intentions I are calculated

through the filter method. Once these estimated

belief, desire, and intention sets are in place, the agent

devises a plan π using the plan method. This method,

utilizing B, D, and I, along with the available action

catalog A, establishes the plan to be followed.

An iterative process then begins in which each

action α constituting the plan π is chosen one by

one. For each action α, the agent calculates the pos-

sible active norms using the norm

instantiation

method. This method evaluates the activation con-

ditions of the norms in N, considering the agent’s

beliefs B. Once the active norms are determined,

Algorithm 1: Agent’s reasoning cycle.

1: B ← B

2: I ← I

3: N ← N

4: E ← E

5: while True do

6: β ← percept()

7: B, E ← belief revision(B, β)

8: D ← options(B,I)

9: I ← ﬁlter(B,D,I)

10: π ← plan(B,D,A)

11: while not π = {} do

12: α ← ﬁrst element of π

13: π ← tail of π

14: µ ← norm instantiation(B,N)

15: σ

′

, ρ

′

←anticipated emotions(B,E,µ)

16: θ ← norm reasoning(B,E, µ, σ

′

, ρ

′

, α)

17: if θ = violate then

18: execute(α)

19: else

20: π ← {}

21: end if

22: β ← percept()

23: B, E ← belief revision(B, β)

24: D ← options(B, I)

25: I ← f ilter(B, D, I)

26: update mood(B,E,θ)

27: if succeeded(I,B) or impossible(I,B) then

28: Break

29: end if

30: end while

31: end while

the agent estimates anticipated emotions using the

anticipated emotions method. This method uti-

lizes B, E, and the active norms µ to calculate the pa-

rameters σ

′

and ρ

′

, representing self-image and social

image, respectively. σ

′

is a tuple

′

, σ

′

indicating

the anticipated emotions when complying with or vi-

olating the norm, respectively. These parameters are

estimated considering the current emotional state of

the agent by the equations:

′

= (σ + ∆

)

(1)

′

= 1 − (σ + ∆

−

)

(2)

Similarly, ρ

′

is a tuple

′

, ρ

′

representing the

anticipated social emotions when complying with or

violating the norm. These factors are estimated as a

sum of the positive and negative social emotions (i.e.,

Gratitude and Disgust) perceived on the environment

in each case, taking into account the number of agents

(nA) inside the environment:

Exploring the Relationship Between Emotions and Norms in Decision-Making Processes of Intelligent Agents

475

Belief revision

Options

Filter

Plan

Percept

Normative

reasoning

Update mood

Beliefs

Norms

Events

Actions

Intentions

Emotions

Desires

Anticipate

emotions

Norm

Instantiation

Figure 3: Process diagram of the proposed normative-affective BDI agent.

∑

|ρ

i=1

|ρ

| · (nA − 1)

(3)

∑

|ρ

−

i=1

−

|ρ

−

| · (nA − 1)

(4)

The normative reasoning process, represented by

the norm reasoning method, uses the calculated in-

formation to determine the decision to comply or vi-

olate the norm through an equation:

θ =



fulﬁll, if θ

≥ θ

violate, otherwise

(5)

= ω

· σ

′

+ ω

· ρ

′

+ ω

· ϕ

(6)

= ω

· σ

′

+ ω

· ρ

′

+ ω

· ϕ

(7)

where ω

represents the importance that the agent

gives to each factor and ϕ

and ϕ

are the expected

utility if the norm is fulﬁlled or violated, respectively.

Once the agent makes the decision θ, its emotional

state is updated using the update mood function. If

the agent decides to violate the norm (using the previ-

ous example, exceeding 60 kilometers per hour), the

action α is executed through the execute method. If

it chooses to comply with the prohibition, the plan is

discarded, and no actions are executed.

Subsequently, perceptions, beliefs, social emo-

tions, desires, and intentions are rechecked. Follow-

ing that, the agent’s emotional state is updated us-

ing the update mood function. Finally, if the plan

has achieved the goal (succeeded) or is unattainable

(impossible), the plan execution is halted.

5 EXAMPLE

Consider an environment with ﬁve coexisting agents

(nA = 5). After applying Algorithm 1 for t cycles,

one agent, named AgentA, is in a state where B and

I have the current set of beliefs and intentions; N in-

cludes the previous exceeding 60 kilometers per hour

norm (named maxSpeed); its emotional knowdlege



, ∆

−

, ρ

−



, where σ

= 0 (i.e, calm

emotional state), ∆

= 1 and ∆

−

= 1 (strong incli-

nation for both positive and negative emotions, that

is, it tends to take great pride in its ”good” actions

but also feels deep remorse for its ”bad” actions);

= [1, 2, 1] and ρ

−

= [2, 2, 1], where each compo-

nent indicates the number of positive (e.g., Gratitude)

or negative (e.g., Guilt) emotions, respectively, per-

ceived in the environment in each cycle, as a result

of its previous actions. Moreover, AgentA values its

social image (ω

= 0.9), has high importance on its

self-image (ω

= 0.75) and relative importance on the

norm utility (ω

= 0.5).

Suppose accelerate is the action to be evaluated

now, maxSpeed is active, and the expected utility of

the norm is higher if violated than fulﬁlled (e.g. ϕ

0.25 and ϕ

= 0.75).

In the anticipated emotions method, AgentA

calculates:

′

= (0.5 + 1)

= 2.25

′

= 1 − (0.5 + 1)

= −1, 25

′

(1 + 2 + 1)

3 · (5 − 1)

= 0.333

′

(2 + 2 + 1)

3 · (5 − 1)

= 0.417

In the norm reasoning method, AgentA calcu-

lates:

= 0.75 · 2.25 + 0.9 · 0.333 + 0.5 · 0.25 = 2.11

= 0.75 · −1.25 + 0.9 · 0.417 + 0.5 · 0.75 = −0.19

EAA 2024 - Special Session on Emotions and Affective Agents

476

AgentA fulﬁlls the maxSpeed norm, discarding the

accelerate action. Despite the preference for vio-

lating the norm (ϕ

> ϕ

), its preference for social-

image and self-image makes it fulﬁll the norm.

Now, consider an antagonist, AgentB, with high

pride and low guilt of its actions (∆

= 1 and ∆

−

0.1), low importance on its self-image (ω

= 0.1), low

importance on its social image (ω

= 0.1) and relative

importance on the norm utility (ω

= 0.5). In a simi-

lar situation as AgentA, the anticipated emotions

method of this agent AgentB would have calculated:

′

= (0.5 + 1)

= 2.25

′

= 1 − (0.5 + 0.1)

= 0.64

′

(1 + 2 + 1)

3 · (5 − 1)

= 0.333

′

(2 + 2 + 1)

3 · (5 − 1)

= 0.417

And the norm reasoning method of agent

AgentB would calculate:

= 0.1 · 2.25 + 0.1 · 0.333 + 0.5 · 0.25 = 0.383

= 0.1 · 0.64 + 0.1 · 0.417 + 0.5 · 0.75 = 0.48

In this case, AgentB violates the maxSpeed norm,

executing the accelerate action, as it does not care

about others’ feelings and its social image and prefers

violating the norm.

6 CONCLUSION

Considering emotional consequences (anticipated

emotions) in the cognitive reasoning of agents when

determining whether to comply with or violate a norm

is a step towards improving the simulation of human

social behavior. This paper has analyzed the inter-

play between norms and emotions, paying attention to

their implications in the decision-making process of

intelligent agents in normative environments. A novel

perspective has been introduced by considering the

inﬂuence of anticipated emotions as well as the util-

ity impact on the processes associated with decision-

making. We have differentiated two categories of

anticipated emotions: self-emotions, instrumental in

shaping the agent’s self-image, and social emotions,

emanating from other agents within the environment.

The emotional state of the agent is encapsulated in

a model grounded in the pleasure dimension, specif-

ically incorporating the self-conscious emotions of

Pride and Guilt. Furthermore, we proposed a syntax

within the framework of AgentSpeak for the deﬁni-

tion of norms in the agent’s code. We have also in-

troduced a novel reasoning cycle, extending the BDI

model to accommodate additional functionalities tai-

lored for affective and normative processes. Affec-

tive processes have been designed to facilitate alter-

ations in the agent’s emotional state, compute antici-

pated emotions, and assess self-image and social im-

age. Concurrently, normative processes empower the

instantiation of active norms and the execution of nor-

mative reasoning, thus enriching the landscape of in-

telligent agent decision-making. This proposal en-

hances the agent’s decision-making capabilities, en-

abling them to make more informed and socially ac-

ceptable decisions. This approach lays the foundation

for the development of more intelligent and socially

aligned agents capable of navigating complex social

environments.

In this work, anticipated emotions are estimated

from default values that do not differentiate between

different types of norms. In the future, it would be in-

teresting to evaluate anticipated emotions when com-

plying with or violating a norm by considering factors

such as the importance of the norm, both at the indi-

vidual level (e.g., evaluating personal ethical values)

and at the societal level (e.g., evaluating the ethical

behavior of society), or previous emotional experi-

ences when complying with or violating a particular

norm. In this way, agents could infer which types of

norms have a higher emotional and social cost.

ACKNOWLEDGEMENTS

Work partially supported by Generalitat Valenciana

CIPROM/2021/077, Spanish Government projects

PID2020-113416RB-I00 and TED2021-131295B-

C32; TAILOR project funded by EU Horizon 2020

under GA No 952215.

REFERENCES

Ahmad, A., Ahmad, M. S., Yusoff, M. Z. M., and Ahmed,

M. (2012). Formulating agent’s emotions in a norma-

tive environment. In Knowledge Technology, pages

82–92. Springer.

Andrighetto, G., Grieco, D., and Tummolini, L. (2015). Per-

ceived legitimacy of normative expectations motivates

compliance with social norms when nobody is watch-

ing. Frontiers in psychology, 6:1413.

Exploring the Relationship Between Emotions and Norms in Decision-Making Processes of Intelligent Agents

477

Argente, E., Del Val, E., Perez-Garcia, D., and Botti, V.

(2020). Normative emotional agents: a viewpoint

paper. IEEE Transactions on Affective Computing,

13(3):1254–1273.

Bagozzi, R. P., Belanche, D., Casal

o, L. V., and Flavi

an, C.

(2016). The role of anticipated emotions in purchase

intentions. Psychology & Marketing, 33(8):629–645.

Barnes, A. and Thagard, P. (2019). Emotional decisions.

In Proceedings of the eighteenth annual conference of

the cognitive science society, pages 426–429. Rout-

ledge.

Bartlett, M. Y., Condon, P., Cruz, J., Baumann, J., and

Desteno, D. (2012). Gratitude: Prompting be-

haviours that build relationships. Cognition & emo-

tion, 26(1):2–13.

Bodner, R. and Prelec, D. (2003). Self-signaling and diag-

nostic utility in everyday decision making. The psy-

chology of economic decisions, 1(105):26.

Bordini, R. H. and H

ubner, J. F. (2005). Bdi agent pro-

gramming in agentspeak using jason. In International

workshop on computational logic in multi-agent sys-

tems, pages 143–164. Springer.

Boytsun, A., Deloof, M., and Matthyssens, P. (2011). So-

cial norms, social cohesion, and corporate gover-

nance. Corporate Governance: An International Re-

view, 19(1):41–60.

Carrera, P., Caballero, A., Mu

noz, D., and Oceja, L. (2011).

Anticipated emotions and personal experience for pre-

dicting behavioral intentions and behavioral expecta-

tions. The Spanish journal of psychology, 14(2):535–

547.

da Costa Pereira, C., Tettamanzi, A. G., Villata, S., Liao,

B., Malerba, A., Rotolo, A., and van Der Torre, L.

(2017). Handling norms in multi-agent system by

means of formal argumentation. Journal of Applied

Logics-IfCoLoG Journal of Logics and their Applica-

tions, 4(9):1–35.

Daci, J. et al. (2010). Legal principles, legal values and legal

norms: are they the same or different? Academicus

International Scientiﬁc Journal, 1(02):109–115.

Demeulenaere, P. (2021). Norms. In Research Handbook on

Analytical Sociology, pages 233–248. Edward Elgar

Publishing.

Dorri, A., Kanhere, S. S., and Jurdak, R. (2018). Multi-

agent systems: A survey. Ieee Access, 6:28573–

28593.

Ferreira, N., Mascarenhas, S., Paiva, A., Di Tosto, G.,

Dignum, F., McBreen, J., Degens, N., Hofstede, G. J.,

Andrighetto, G., and Conte, R. (2013). An agent

model for the appraisal of normative events based in

in-group and out-group relations. In AAAI.

Gratch, J., Mao, W., and Marsella, S. (2006). Modeling

social emotions and social attributions. Cambridge

University Press Cambridge.

Gross, J. and Vostroknutov, A. (2022a). Why do people

follow social norms? Current Opinion in Psychology,

44:1–6.

Gross, J. and Vostroknutov, A. (2022b). Why do people

follow social norms? Current Opinion in Psychology,

44:1–6.

Grossman, Z. and Van der Weele, J. J. (2017). Self-image

and willful ignorance in social decisions. Journal of

the European Economic Association, 15(1):173–217.

Inbar, Y. and Pizarro, D. A. (2022). How disgust affects

social judgments. In Advances in Experimental Social

Psychology, volume 65, pages 109–166. Elsevier.

Killen, M., Rutland, A., Abrams, D., Mulvey, K. L., and

Hitti, A. (2013). Development of intra-and inter-

group judgments in the context of moral and social-

conventional norms. Child Development, 84(3):1063–

1080.

Kollmann, S., Siafara, L. C., Schaat, S., and Wendt, A.

(2016). Towards a cognitive multi-agent system

for building control. Procedia Computer Science,

88:191–197.

Lazarus, R. S. (1991). Emotion and adaptation. Oxford

University Press.

Lerner, J. S., Li, Y., Valdesolo, P., and Kassam, K. S. (2015).

Emotion and decision making. Annual review of psy-

chology, 66:799–823.

Lewis, M., Haviland-Jones, J. M., and Barrett, L. F. (2010).

Handbook of emotions. Guilford Press.

Mascarenhas, S., Guimar

aes, M., Prada, R., Santos, P. A.,

Dias, J., and Paiva, A. (2022). Fatima toolkit: To-

ward an accessible tool for the development of socio-

emotional agents. ACM Transactions on Interactive

Intelligent Systems (TiiS), 12(1):1–30.

Meshulam, M., Winter, E., Ben-Shakhar, G., and Aharon,

I. (2012). Rational emotions. Social Neuroscience,

7(1):11–17.

Nucci, L. P. and Turiel, E. (1978). Social interactions and

the development of social concepts in preschool chil-

dren. Child development, pages 400–407.

O’Neill, E. (2017). Kinds of norms. Philosophy Compass,

12(5):e12416.

Onwezen, M. C., Antonides, G., and Bartels, J. (2013).

The norm activation model: An exploration of

the functions of anticipated pride and guilt in pro-

environmental behaviour. Journal of economic psy-

chology, 39:141–153.

Rao, A. S., Georgeff, M. P., et al. (1995). Bdi agents: from

theory to practice. In Icmas, volume 95, pages 312–

319.

Simamora, B. (2021). How proponents and opponents in-

ﬂuence achievement motivation: The role of the antic-

ipated emotions of other people. Gadjah Mada Inter-

national Journal of Business, 23(1):1–35.

Staller, A., Petta, P., et al. (2001). Introducing emotions

into the computational study of social norms: A ﬁrst

evaluation. Journal of artiﬁcial societies and social

simulation, 4(1):U27–U60.

Sterelny, K. (2019). Norms and their evolution. In Hand-

book of Cognitive Archaeology, pages 375–397. Rout-

ledge.

Tangney, J. P., Stuewig, J., and Mashek, D. J. (2007).

What’s moral about the self-conscious emotions. The

self-conscious emotions: Theory and research, pages

21–37.

EAA 2024 - Special Session on Emotions and Affective Agents

478

Tracy, J. L. and Weidman, A. C. (2021). The self-conscious

and social emotions: A personality and social func-

tionalist account. The Guilford Press.

Tzeng, S.-T. (2022). Engineering normative and cogni-

tive agents with emotions and values. In Proceedings

of the 21st International Conference on Autonomous

Agents and Multiagent Systems, pages 1878–1880.

Tzeng, S.-T., Ajmeri, N., and Singh, M. P. (2021). Noe:

Norm emergence and robustness based on emotions in

multiagent systems. In International Workshop on Co-

ordination, Organizations, Institutions, Norms, and

Ethics for Governance of Multi-Agent Systems, pages

62–77. Springer.

Walton, D. (2019). Practical reasoning in the deliberations

of an intelligent autonomous agent. Blended Cogni-

tion: The Robotic Challenge, pages 53–84.

Zhang, C. and Lu, Y. (2021). Study on artiﬁcial intelligence:

The state of the art and future prospects. Journal of

Industrial Information Integration, 23:100224.

Exploring the Relationship Between Emotions and Norms in Decision-Making Processes of Intelligent Agents

479