![](bg1.png)
Hand-Drawn Diagram Correction Using Machine Learning
Tenga Yoshida
1 a
and Hiroyuki Kobayashi
2 b
1
Graduate School of Robotics and Design, Osaka Institute of Technology, Osaka, Japan
2
Department of System Design, Osaka Institute of Technology, Osaka, Japan
Keywords:
Machine Learning, Hand-Drawn Diagram, Time Series Data.
Abstract:
This paper introduces a real-time correction technique for hand-drawn diagrams on tablets, leveraging machine
learning to mitigate inaccuracies caused by hand tremors. A novel fusion of classification and regression mod-
els is proposed; initially, the classification model discerns the geometric shape being drawn, aiding the regres-
sion model in making precise corrective predictions during the drawing process. Additionally, a unique Mean
Angle of Vector (MAV) loss function is introduced to minimize angle changes in vectors formed by consec-
utive points, thereby reducing hand tremors especially in straight line segments. The MAV function not only
facilitates real-time corrections but also preserves the drawing fluidity, enhancing user satisfaction. Experi-
mental results highlight improved correction accuracy, particularly when employing classification alongside
regression. However, the MAV function may round off sharp corners, indicating areas for further refinement.
This work paves the way for more intuitive and user-friendly digital sketching and diagramming applications.
1 INTRODUCTION
Hand-drawn diagrams on tablet devices are recog-
nized as intuitive and convenient. However, a greater
susceptibility to shakes is observed compared to tradi-
tional paper-based drawing. As a result, various hand-
shake correction methods have been developed. One
notable method that has been employed is the mov-
ing average process. In this method, the output point
is computed from the moving average of the preced-
ing and subsequent n steps for the input point at time
t. Advanced methods, wherein the filter size n is ad-
justed based on the path’s curvature, have also been
introduced (Kawase et al., 2012). Predictions can be
made using past coordinate data by these methods, but
the prediction of specific shapes remains a challenge.
Moreover, in applications designed for the correc-
tion of hand-drawn figures, the shape is typically clas-
sified after being drawn, and corrections are then ap-
plied. This approach ensures that geometric figures
are drawn without imperfections such as shakes or
blurs. However, pauses in prediction are often intro-
duced, leading to a less fluid drawing experience.
In the realm of hand-drawn diagram correction,
most existing methods have been centered around
bitmap images. The novelty of this research lies in
a
https://orcid.org/0009-0002-2766-1493
b
https://orcid.org/0000-0002-4110-3570
leveraging vector data for real-time correction, a facet
that has not been extensively explored before. This
vector-based approach is anticipated to offer better
accuracy and efficiency in correcting hand-drawn di-
agrams on digital mediums.
In response to the challenges aforementioned, it
was recognized that online hand-drawn figures can be
treated as time series data. A proposal has been made
to utilize machine learning for real-time regressive
path predictions, allowing corrections to be made dur-
ing the drawing process. This research further aims to
enhance correction accuracy by introducing concur-
rent classification prediction. This allows the regres-
sion prediction model to adapt based on the type of
figure drawn.
2 RELATED WORK
Up to now, numerous applications of machine learn-
ing to hand-drawn data, often utilizing datasets like
MNIST, have been reported. Distinct from these stud-
ies that rely on bitmap images, the present research
is characterized by its emphasis on vector images for
correction.
When attention is given to the correction of hand-
drawn figures and time series prediction, several per-
tinent studies can be identified.
346
Yoshida, T. and Kobayashi, H.
Hand-Drawn Diagram Correction Using Machine Learning.
DOI: 10.5220/0012239300003543
In Proceedings of the 20th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2023) - Volume 1, pages 346-351
ISBN: 978-989-758-670-5; ISSN: 2184-2809
Copyright © 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)