![](bg6.png)
a decision-theoretic approach. The advantage of us-
ing such an approach is to achieve optimality. An-
other contribution consists of handling uncertainty on
resource consumption combined with uncertainty on
execution time.
In the MDP we present, actions are not instanta-
neous as in the previous planners and can deal with
complex time constraints such as a temporal window
of execution and temporal precedence constraints. We
also show that our approach can solve large problems
with a hundred operations, many of which are uncer-
tain. Another requirement mentioned for the rover
applications is concurrent actions. This problem is
under development, taking advantage of some multi-
agent concepts : as soon as the rover needs to execute
two actions, we consider those actions are concurrent
agents. This new line of research allows for bridg-
ing the gap between the multiagent systems and the
distributed MDP.
6 CONCLUSION AND FUTURE
WORK
In this paper we presented an MDP planning technique
that allows for a plan where operations have complex
dependencies, time and resource constraints. The op-
erations are organized in an acyclic graph where each
operation has a temporal window during which it can
be executed and an uncertain resource consumption
and execution time. This approach is based on an
MDP using a rich model of time and resources with
dependencies between tasks. This technique allows
us to deal with the variable duration of actions. We
presented experimental results showing that our ap-
proach can scale to large robotic problems (a hundred
of operations). Our approach also overcomes some of
the limitations described in (Bresina et al., 2002). In-
deed, our model is able to handle more complex time
constraints and uncertainty on durations and resource
consumptions. Moreover, as required in (Bresina
et al., 2002), our system can consider plans of more
than a hundred tasks.
In this paper, we have focused on planetary rover
applications. Nonetheless, our approach is not re-
stricted to space applications. It can be applied to
any scenario formalized by an acyclic graph with tem-
poral constraints. However, this approach needs to
be extended to other requirements such as continuous
variables. In our current version of the approach we
use a discrete representation of time and resources.
We are specially interested in finding tradeoffs be-
tween the scalability, execution errors and discretiza-
tion. The other extension we are developing consists
of dealing with multiple agents using a distributed
MDP. Future work may concern the construction of
the graphs that we consider as given in this paper.
REFERENCES
Blythe, J. (1999). Planning under uncertainty in Dynamic
domains. PhD, Carnegie Mellon University.
Boutilier, C., Dean, T., and Hanks, S. (1999). Decision-
theoretic planning: Structural asumptions and com-
putational leverage. Journal of Articicial Intelligence
Research, 1:1–93.
Bresina, J., Dearden, R., Meuleau, N., Ramkrishnan, S.,
Smith, D., and Washington, R. (2002). Planning un-
der continuous time and resource uncertainty: A chal-
lenge for ai. In Proceedings of the 18th Annual Con-
ference on Uncertainty in Artificial Intelligence (UAI-
02), pages 77–84, San Francisco, CA. Morgan Kauf-
mann Publishers.
Bresina, J. and Washington, R. (2000). Expected utility dis-
tributions for flexible contingent execution. In AAAI
Workshop on Representation issues for Real World
Planning system.
Cardon, S., Mouaddib, A., Zilberstein, S., and Washing-
ton, R. (2001). Adaptive control of acyclic progressive
processing task structures. In IJCAI, pages 701–706.
Estlin, T. A., Gray, A., Mann, T., Rabideau, G., Castano,
R., Chien, S., and Mjolsness, E. (1999). An inte-
grated system for multi-rover scientific exploration. In
AAAI/IAAI, pages 613–620.
Howard, R. A. (1960). Dynamic Programming and Markov
Processes. MIT Press.
Morris, P., Muscettola, N., and Vidal, T. (2001). Dynamic
control of plans with temporal uncertainty. In IJCAI,
pages 494–502.
Mouaddib, A.-I. and Zilberstein, S. (1998). Optimal
scheduling for dynamic progressive processing. In
ECAI-98, pages 499–503.
Sutton, R. S., Precup, D., and Singh, S. P. (1999). Between
MDPs and semi-MDPs: A framework for temporal
abstraction in reinforcement learning. volume 112,
pages 181–211.
Tsamardinos, I., Pollack, M., and Ramakrishnan, S. (2003).
Assessing the probability of legal execution of plans
with temporal uncertainty. In ICAPS Workshop on
Planning under uncertainty and Incomplete informa-
tion.
Vidal, T. and Ghallab, M. (1996). Dealing with uncertain
durations in temporal constraints networks dedicated
to planning. In 12
th
ECAI, pages 48–54.
Zilberstein, S. and Mouaddib, A.-I. (1999). Reactive con-
trol for dynamic progressive processing. In IJCAI-99,
pages 1269–1273.
OPTIMAL PLANNING FOR AUTONOMOUS AGENTS UNDER TIME AND RESOURCE UNCERTAINTY
187