![](bg8.png)
ways to deal with current limitations of the frame-
work, namely:
• not being able to model or prove statements quan-
tifying over mental states,
• not modelling the environment by assuming that
the agent is the only actor,
• not accounting for multiple agents.
It is clear that neither of these limitations have an easy
fix. It is not obvious how to quantify over mental
states: we need to consider the temporal aspect of the
agent system and prove that a property holds across
several mental states. We have not encountered any
proposals in the literature. Modelling the environ-
ment seems less of a challenge but potentially adds
little immediate value beyond providing a better level
of abstraction. Even if we alleviate assumptions of
complete knowledge for agents, we still need to add
some domain knowledge into the environment to be
able to make any logical reasoning of interest. In a
way, it simply pushes part of the problem into another
structure, but could potentially be a step towards be-
ing able to state richer assumptions about interactions
in the environment. Lastly, we want to discuss mod-
elling of multiple agents and of communication. This
is very much at the core of the issue in the field of
verifying multi-agent systems. The nature of multi-
ple parallel processes quickly makes every model ex-
plode in complexity. We acknowledge its importance
but it is too early to share any thoughts on its role in
our continued work, and how to resolve the problems
that may arise.
6 CONCLUSIONS
We have argued that the use of theorem proving can
be a valuable tool for developing formal verification
techniques to ensure the reliability of multi-agent sys-
tems. The value comes from assuring that developed
techniques work as intended and does not introduce
new levels of possible errors. Such errors may com-
promise the reliability of the system even further by
providing a false sense of security.
We have described a verification framework for
the GOAL agent programming language that can be
used to prove properties of agents. We have proved
some of the results of the author’s original work us-
ing the Isabelle/HOL proof assistant, and deemed it
feasible to formalize all of the results.
Finally, we have discussed the many challenges
ahead and given ideas to address them. Our plans are
to continue work on the Isabelle formalization and si-
multaneously work on extending the theory.
ACKNOWLEDGEMENTS
I would like to thank Asta H. From for Isabelle re-
lated discussions and for comments on drafts of this
paper. I would also like to thank Jrgen Villadsen for
comments on drafts of this paper. I would also like to
thank Koen V. Hindriks for helpful insights.
REFERENCES
Alechina, N., Dastani, M., Khan, A. F., Logan, B., and
Meyer, J.-J. (2010). Using Theorem Proving to Verify
Properties of Agent Programs, pages 1–33. Springer.
Bordini, R. H., Fisher, M., Visser, W., and Wooldridge, M.
(2006). Verifying Multi-agent Programs by Model
Checking. Autonomous Agents and Multi-Agent Sys-
tems, 12:239–256.
Cossentino, M., Fortino, G., Garro, A., Mascillaro, S., and
Russo, W. (2008). PASSIM: A simulation-based pro-
cess for the development of multi-agent systems. In-
ternational Journal of Agent-Oriented Software Engi-
neering, 2:132–170.
de Boer, F. S., Hindriks, K. V., Hoek, W., and Meyer, J.-J.
(2007). A verification framework for agent program-
ming with declarative goals. Journal of Applied Logic,
5:277–302.
Dennis, L. A. and Fisher, M. (2009). Programming Verifi-
able Heterogeneous Agent Systems. In Programming
Multi-Agent Systems, pages 40–55. Springer.
Dix, J., Logan, B., and Winikoff, M. (2019). Engineer-
ing Reliable Multiagent Systems (Dagstuhl Seminar
19112). Dagstuhl Reports, 9(3):52–63.
Hindriks, K. V. (2009). Programming Rational Agents in
GOAL, pages 119–157. Springer US.
Hindriks, K. V., Boer, F., Hoek, W., and Meyer, J.-J. (1999).
Agent programming in 3APL. Autonomous Agents
and Multi-Agent Systems, 2:357–401.
Jensen, A. B. (2021). Towards Verifying a Blocks World for
Teams GOAL Agent. In ICAART 2021 - Proceedings
of the 13th International Conference on Agents and
Artificial Intelligence. SciTePress. To appear.
Jensen, A. B., Hindriks, K. V., and Villadsen, J. (2021). On
Using Theorem Proving for Cognitive Agent-Oriented
Programming. In ICAART 2021 - Proceedings of the
13th International Conference on Agents and Artifi-
cial Intelligence. SciTePress. To appear.
Misra, J. (1994). A Logic for Concurrent Programming.
Technical report, Formal Aspects of Computing.
Nipkow, T., Paulson, L., and Wenzel, M. (2002).
Isabelle/HOL — A Proof Assistant for Higher-Order
Logic, volume 2283 of LNCS. Springer.
Paulson, L. C. (2000). Mechanizing UNITY in Isabelle.
ACM Trans. Comput. Logic, 1(1):332.
Poutakidis, D., Winikoff, M., Padgham, L., and Zhang,
Z. (2009). Debugging and Testing of Multi-Agent
Systems using Design Artefacts, pages 215–258.
Springer.
ICAART 2021 - 13th International Conference on Agents and Artificial Intelligence
352