![](bg2.png)
other suitable opportunities for this applicant. We will
see that some formal models of belief change will ac-
tually reject the first information, as if it had never
been obtained. On the other hand, some models will
actually keep the older information. In the present ex-
ample, this is appropriate. However, we argue that
this can also be a problem in different contexts.
The fundamental problem is that the well-known
postulates that describe rational belief change can be
satisfied by significantly different processes, in a way
that may introduce unintended bias into a system that
is intended to provide a normative model. While the
problems introduced through biased data sets in ML
approaches are now being increasingly recognized,
this kind of bias in an axiomatic reasoning system
needs to be put under the same scrutiny.
2.2 Belief Revision
Broadly, the study of belief revision is concerned with
the manner in which agents incorporate new informa-
tion with their pre-existing beliefs. The most influen-
tial approach to belief revision is the AGM approach,
in which a set of rationality postulates is specified to
define the essential features of the revision process
(Alchourr
´
on et al., 1985). This framework is defined
in the context of propositional logic. So we assume
a fixed vocabulary V , which is just a set of proposi-
tional variables. Formulas over V are defined using
the usual logical connectives ¬, ∧. The beliefs of an
agent are represented by a logically closed set of for-
mulas.
An AGM revision operator is a function ∗ that
maps a belief set K and a formula φ to a new belief
set K ∗ φ. The new belief set represents what an agent
should believe after incorporating φ. Informally, we
want to add φ to the belief set, and remove as little as
possible while maintaining consistency. An AGM re-
vision operator must satisfy a set of postulates, which
we do not list here in the interest of space. However,
as an example, one straightforward postulate is the so-
called success postulate:
K ∗ φ |= φ.
This simply says that new information should be be-
lieved; this postulate encapsulates the fundamental
assumption of the AGM approach that new infor-
mation is provided by a reliable source. It is well-
known that the semantics of AGM belief revision can
be defined with respect to plausibility orderings over
propositional interpretations (Katsuno and Mendel-
zon, 1992).
In this paper, we are not directly concerned with
AGM revision. It has a known limitation in that it
can not be used for iterated belief change. The most
influential approach to iterated belief change is an ex-
tension of AGM called DP revision (Darwiche and
Pearl, 1997). The main difference is that the beliefs
of an agent are now represented by an epistemic state
E, which is essentially a total pre-order over possible
interpretations. The minimal elements of this order-
ing are considered the most plausible states, and we
let B(E) denote the set of formulas that are true in all
of the minimal states. At the level of formulas, there
is a set of postulates for DP revision that essentially
guarantee that they behave like AGM revision opera-
tors. But then there are four additional postulates that
explicitly deal with iterated change:
Darwiche-Pearl Postulates
[DP1] If β |= α, then B(E ∗ α ∗ β) = B(E ∗ β).
[DP2] If β |= ¬α, then (κ ∗ β) ∗ α = κ ∗ α.
[DP3] If α ∈ B(E ∗ β), then α ∈ B(E ∗ α ∗ β).
[DP4] If ¬α 6∈ B(E ∗ β), then ¬α 6∈ B(E ∗ α ∗ β).
It turns out that there are many different DP operators
that satisfy all of the postulates. We mention the two
most extreme examples:
• Natural Revision: After revision by φ, B(E) be-
comes the set of states where φ is true. For all
other states the ordering defined by E is preserved.
• Lexicographic Revision: After revision by φ, all
states where φ is true are moved before the states
where φ is false. The relative ordering between
φ-states and ¬φ-states is preserved.
Figure 1 gives a schematic depiction of how these op-
erators work. We remark that there are known is-
sues with these operators, and improvements have
been proposed (Booth and Meyer, 2006; Jin and
Thielscher, 2007). However, for the purposes of this
paper, the significant feature is simply that these two
extreme examples satisfy the raitionality postulates.
3 ETHICAL CONSIDERATIONS
3.1 On Prescriptive Theories
For the ethical evaluation of formal belief change, we
will consider several approaches: Kant’s categorical
imperative, utilitarianism, and virtue ethics. These are
all well-known approaches, and we refer the reader to
(Hill, 2009; Rosen, 2003; Hursthouse, 2001) for a de-
scription of each approach, as well as comparisons
between them. For the purposes of this paper, the in-
tention is not to draw hard ethical conclusions; our
primary intention is simply to highlight where ethical
issues can creep into formal models of reasoning.
ICAART 2023 - 15th International Conference on Agents and Artificial Intelligence
566