![](bg2.png)
Perhaps not surprisingly, it turns out that this is
exactly the kind of resource management that is
required to enable AI systems to approach human-
level intelligence in real-world environments. Thus,
it makes perfect sense to investigate how AI systems
can be endowed with this cognitive function for the
purpose of improving their operation and making
them applicable to more open-ended and complex
tasks and environments. The goal need not be to
replicate any biological function in detail, but rather
to extract useful concepts and methods from the
biological side while leaving undesirable limitations
behind in order to facilitate the creation of AI
systems that can successfully operate in real-world
environments in realtime using limited resources.
While attention has been largely ignored in the
field to-date, there are notable exceptions. These
include cognitive architectures such as NARS
(Wang, 1995), LIDA (Baars, 2009) and Clarion
(Sun, 2006). However, the attentional functionality
implemented in these systems is incomplete in
various ways, such as focusing solely on data-
filtering (ignoring control issues, e.g. how
prioritization affects processing of selected data) and
external environmental information (ignoring
internal system states). The ASMO framework
(Novianto, 2009) is somewhat unique as it assumes a
tight coupling between attention and self-awareness
and includes focus on internal states. However, none
of this work addresses realtime processing, which is
one of the major reasons we desire attentional
functionality, in a vigorous fashion. Attention has
also been studied in relation to AI within the limited
scope of working memory (c.f. Phillips, 2005 and
Skubic, 2004). While attention and working memory
are closely related, this is a restrictive context to
study attention within as working memory can in
most cases be modelled as a cognitive function
rather than an architectural component.
This paper starts with a brief overview of human
attention and subsequently attempts to extract
principles that may be useful for AI systems. This is
followed by a discussion of how these principles
might be extended to levels beyond human attention
for meta-reasoning and introspection. We then
present a high-level design of an attention
mechanism intended for AI architectures.
2 HUMAN ATTENTION
Research of human attention has a long history
dating back to the beginnings of psychology. Back
in 1890, the American psychologist William James
wrote the following (James 1890):
“Everyone knows what attention is. It is the taking
possession by the mind, in clear and vivid form, of
one out of what seem several simultaneously
possible objects or trains of thought. Focalization,
concentration, of consciousness are of its essence. It
implies withdrawal from some things in order to
deal effectively with others, and is a condition which
has a real opposite in the confused, dazed,
scatterbrained state which in French is called
distraction, and Zerstreutheit in German.”
- William James
This elegant description indicates that the
importance of attention for the human mind was
identified as early as the 18
th
century. The beginning
of modern attention research is commonly tied to
Colin Cherry’s work on what has been called the
“cocktail party effect” (Cherry, 1953), which
addresses how we are able to focus on particular
sensory data in the presence of distracting
information and noise, such as following and
participating in a conversation at a cocktail party in
the presence of many other conversations and
background noise, and still be able to catch when
someone calls our name in the background. The
ability to be in a focused state of attention while
remaining reactive to unexpected events, seems to
call for a selective filtering mechanism of some sort
while at the same time requiring deliberate steering
of cognitive resources. The cocktail party scenario is
a good illustration of the dual nature of attention:
We will refer to the deliberate, goal-driven side as
top-down attention and the reactive, stimulus-driven
side as bottom-up attention.
A number of models for attention were
subsequently proposed, some of which were
considered early selection models as selection of
sensory information is assumed to occur early in the
sensory pipeline based on primitive physical features
of the information. This implies that the
determination of what is important and should be
selected is based on shallow, primitive processing
with very limited or non-existent analysis of
meaning. The Broadbent filter model (Broadbent,
1958) is the most prominent of these. A number of
late selection models have also been proposed, that
assume further analysis of incoming sensory
information must be performed in order to determine
its relevance and carry out efficient selection. The
Deutsch-Norman (Norman 1969) model is based on
the assumption that sensory information is not
actually filtered, but processed to the point of
activating representations stored in memory.
ICINCO2012-9thInternationalConferenceonInformaticsinControl,AutomationandRobotics
282