Offline Feature-Based Reinforcement Learning with Preprocessed Image Inputs for Liquid Pouring Control

Stephan Pareigis, Jesus Hermosilla-Diaz, Jeeangh Reyes-Montiel, Fynn Maaß, Helen Haase, Maximilian Mang, Antonio Marin-Hernandez

2023

Abstract

A method for the creation of a liquid pouring controller is proposed, based on experimental data gathered from a small number of experiments. In a laboratory configuration, a UR5 robot arm equipped with a camera near the end effector holds a container. The camera captures the liquid pouring from the container as the robot adjusts its turning angles to achieve a specific pouring target volume. The proposed controller applies image analysis in a preprocessing stage to determine the liquid volume pouring from the container at each frame. This calculated volume, in conjunction with an estimated target volume in the receiving container, serves as input for a policy that computes the necessary turning angles for precise liquid pouring. The data received on the physical system is used as Monte-Carlo episodes for training an artificial neural network using a policy gradient method. Experiments with the proposed method are conducted using a simple simulation. Convergence proves to be fast and the achieved policy is independent of initial and goal volumes.

Paper Citation

in Harvard Style

Pareigis S., Hermosilla-Diaz J., Reyes-Montiel J., Maaß F., Haase H., Mang M. and Marin-Hernandez A. (2023). Offline Feature-Based Reinforcement Learning with Preprocessed Image Inputs for Liquid Pouring Control. In Proceedings of the 20th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO; ISBN 978-989-758-670-5, SciTePress, pages 320-326. DOI: 10.5220/0012235700003543

in Bibtex Style

@conference{icinco23,
author={Stephan Pareigis and Jesus Hermosilla-Diaz and Jeeangh Reyes-Montiel and Fynn Maaß and Helen Haase and Maximilian Mang and Antonio Marin-Hernandez},
title={Offline Feature-Based Reinforcement Learning with Preprocessed Image Inputs for Liquid Pouring Control},
booktitle={Proceedings of the 20th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO},
year={2023},
pages={320-326},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012235700003543},
isbn={978-989-758-670-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO
TI - Offline Feature-Based Reinforcement Learning with Preprocessed Image Inputs for Liquid Pouring Control
SN - 978-989-758-670-5
AU - Pareigis S.
AU - Hermosilla-Diaz J.
AU - Reyes-Montiel J.
AU - Maaß F.
AU - Haase H.
AU - Mang M.
AU - Marin-Hernandez A.
PY - 2023
SP - 320
EP - 326
DO - 10.5220/0012235700003543
PB - SciTePress