Design Patterns for Monitoring and Prediction Machine Learning

Systems: Systematic Literature Review and Cluster Analysis

Richard May

, Tobias Niemand

, Paul Scholz

and Thomas Leich

Harz University of Applied Sciences, Wernigerode, Germany

Siemens Mobility GmbH, Brunswick, Germany

Hilti AG, Th

uringen, Austria

Keywords:

Monitoring, Prediction, Machine Learning, Systematic Literature Review, Cluster Analysis.

Abstract:

Although machine learning methods for industrial maintenance systems have already been well described

in recent years, their practical implementation is only slowly taking place. One of the reasons is a lack

of comparable analyses of machine learning systems. To address this gap, we ﬁrst conducted a systematic

literature review (2012–2021) of 104 monitoring and prediction systems. Second, we extracted 5 design

patterns (i.e., high-level construction manuals) based on a k-means cluster analysis. Our results show that

monitoring and prediction systems mainly differ in their choice of operations. However, they usually share

similar learning strategies (i.e., supervised learning) and tasks (i.e., classiﬁcation, regression). With our work,

we aim to help researchers and practitioners to understand common characteristics, contexts, and trends.

1 INTRODUCTION

The manufacturing industry produces an ever increas-

ing data amount due to the digitalization of machines

(e.g., cyber-physical systems) (Tang et al., 2014). The

generated data is particularly valuable for applications

that are able to use it in machine learning systems

(MLS), enabling diverse possibilities, such as moni-

toring and prediction applications (Monostori et al.,

2016). Since the costs of maintaining machines is a

major part of the total costs of manufacturing sys-

tems (Mobley, 2002), the importance of monitoring

and prediction systems, e.g., condition monitoring

(CM) or predictive maintenance (PM) applications,

increased in recent years (Motaghare et al., 2018;

Serin et al., 2020). However, their practical imple-

mentation in actual manufacturing companies is only

slowly taking place, e.g., due to a missing under-

standing of relevant technological layers (Schuh et al.,

2020). Usually, MLS lack comparable investigations

of their technological structure. So, despite the ex-

tensive research regarding maintenance systems, we

are missing a comprehensive systematization of MLS

which is oriented towards a comparable tool to ex-

tract design patterns representing their technological

structure. Precisely, we deﬁne a design pattern as

the systematization of the most frequently occurring

characteristics of a speciﬁc software system, i.e., a

high-level construction manual as a decision aid for

system design and development, consisting of prin-

cipal MLS-related properties. In this context, we ar-

gue that similar problems offer similar solution pat-

terns. Thus, these patterns may provide a high trans-

ferability to facilitate the transfer of theoretical de-

scribed MLS into practice. Moreover, design patterns

can provide a highly valuable basis for the engineer-

ing process of MLS software for both researchers and

practitioners (Schuh et al., 2020).

Overall, our goal is to classify and discuss the

technological structure of monitoring and predic-

tion MLS. To achieve this goals, we ﬁrst conduct

a systematic literature review (SLR) of 104 papers

(2012–2021). Second, the results of the SLR are fur-

ther analyzed by applying a k-means clustering algo-

rithm. In detail, we contribute the following:

• An analysis of the current state-of-the-art of moni-

toring and prediction MLS (SLR).

• An identiﬁcation and discussion of monitoring and

prediction design patterns (clustering).

• An open-access repository to ensure a replicability

of our study.

With our results, we aim to provide an understanding

and uniform decision aid for the design and imple-

mentation of MLS in the manufacturing industry.

https://doi.org/10.5281/zenodo.7540506

May, R., Niemand, T., Scholz, P. and Leich, T.

Design Patterns for Monitoring and Prediction Machine Learning Systems: Systematic Literature Review and Cluster Analysis.

DOI: 10.5220/0012005800003538

In Proceedings of the 18th International Conference on Software Technologies (ICSOFT 2023), pages 209-216

ISBN: 978-989-758-665-1; ISSN: 2184-2833

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

209

2 BACKGROUND

Next, we provide information regarding MLS and

monitoring and prediction systems.

2.1 Machine Learning Systems

Machine learning (ML) refers to the artiﬁcial acqui-

sition of information based on experiences (e.g., sen-

sor data), which can be applied to new data to start

learning processes (Patil et al., 2017) that reduce time,

costs, or improve product and process quality (Wang

et al., 2018). When ML is implemented in a holis-

tic system to address such value added, the resulting

system is called MLS (Schuh et al., 2020). Overall,

MLS are mostly categorized in terms of their learning

strategies, tasks, and operations (Dogan and Birant,

2021). Learning strategies are related to the way in-

formation is generated, including 3 major categories:

supervised learning, unsupervised learning, and rein-

forcement learning (Wuest et al., 2016). Tasks refer to

how systems receive feedback according to the learn-

ing strategies, e.g., classiﬁcation (Schuh and Scholz,

2019). Operations involve models or methods used to

fulﬁll tasks, e.g., support vector machines (SVM) or

artiﬁcial neural networks (ANN) (Schuh et al., 2020).

2.2 Monitoring and Prediction Systems

Monitoring and prediction mainly refer to the term

maintenance, which is considered as part of opera-

tion phases in the product and machine life cycle (IEC

60300-3-3:2017, 2017). Precisely, Monitoring refers

to identifying the current condition state of a ma-

chine or process. So, it is focused on the derivation

of symptoms (e.g., failure) based on input data, e.g.,

generated by sensors (VDI/VDE 2651 Part 1:2017-

05, 2017). Prediction (also called prognosis) deals

with the derivation of future machine or tool condi-

tions based on the previous and current conditions.

Predictive techniques estimate the remaining useful

life (RUL) based on retrieved data to improve oper-

ating time and to schedule maintenance tasks (ISO

13372:2012, E/F). Usually, the selection of suitable

monitoring and prediction systems is made in relation

to the failure and wear behavior of a system. This

selection depends on the use case, such as tool wear

monitoring (TWM), i.e., determining the actual con-

dition state, or tool wear prediction (TWP), i.e., esti-

mating the RUL and (actual) condition states in rela-

tion to following machining cycles. However, MLS

provide diverse operations to address diverse types of

systems or analytics, making the appropriate selection

of strategies highly challenging (Qiao et al., 2020).

3 METHODS

To achieve our research goal, we employed an

SLR (Kitchenham et al., 2015) to collect data and an-

alyzed it by applying a k-means clustering algorithm

to extract common design patterns (cf. Figure 1).

automated search

IEEE

Scopus

ACM

data extraction

I. systematic literature review

II. k-means cluster analysis

cluster creation

design pattern extraction

104 after selection

Figure 1: Methodological overview consisting of 2 steps.

3.1 Systematic Literature Review

In the following sections, the methodological steps of

the SLR are described.

3.1.1 Study Design

Search String. To identify suitable papers, the fol-

lowing search string consisting of relevant terms in

the context of MLS, manufacturing, and maintenance

was applied to SCOPUS, IEEE XPLORE, and the

ACM GUIDE TO COMPUTING LITERATURE.

("machine learning" OR "deep learning"

OR "artificial intelligence" OR "neural

network") AND "manufacturing" AND ("machine

tool" OR "cutting" OR "machining" OR

"turning" OR "milling" OR "drilling" OR

"CNC") AND ("condition monitoring" OR

"predictive maintenance" OR "tool wear

prediction")

Selection Criteria. We selected only papers that

have been published in 2012–2021 at a conference or

in a journal and deal with manufacturing-related MLS

approaches focusing on CM, PM, or TWP. As we did

not perform a detailed quality assessment of all pa-

pers, a certain quality was ensured by deﬁning a min-

imum of 3 pages of only peer-reviewed papers.

Extraction Criteria. To extract data, we relied on

an extended version of the framework by Schuh and

Scholz (2019) as it provides a well-founded basis for

a uniform classiﬁcation of MLS approaches:

• Manufacturing Processes, i.e., actual manufac-

turing operations performed by a machine, e.g.,

milling (Sevic and Keller, 2019).

• Machines, i.e., hardware performing certain manu-

facturing processes, e.g., a CNC machine (Krishna

and Kannadaguli, 2020).

• Use Cases, i.e., technological tasks of the applica-

tion, e.g., monitoring of health state in water pump-

ing (Grifﬁths, 2020).

ICSOFT 2023 - 18th International Conference on Software Technologies

210

• Technology Value Add, i.e., economic incentives

to use ML-driven technology, e.g., improvement of

quality, time, or costs (Wang et al., 2018).

• Implementation Procedures, i.e., tools for devel-

oping the MLS, e.g., programming languages such

as Python (Lutz et al., 2019).

• Working Areas, i.e., ML-related activities or ap-

plication ﬁelds, e.g., CM (Davies, 2012).

• Learning Strategies, i.e., the strategy used by MLS

to learn from data, e.g., supervised learning (Good-

fellow et al., 2016).

• Tasks, i.e., main workﬂows used to recognize cer-

tain patterns in data, e.g., classiﬁcation or cluster-

ing (Schuh et al., 2019).

• Operations, i.e., representation forms and charac-

teristics of the tasks applied for analysis purposes,

e.g., decision trees or ANN (Schuh et al., 2019).

• Data Base, i.e., data used to learn, e.g., vibration

data (Espinoza Sep

ulveda and Sinha, 2021).

3.1.2 Study Conduct

The ﬁrst and second authors conducted an automated

search on December 01, 2022, retrieving 246 publica-

tions (231 after duplication removal). Next, the publi-

cations’ titles and abstracts were analyzed to identify

the approaches relevant for our study (111 papers). To

ﬁnd missing related articles, we performed backward

snowballing with an unrestricted number of iterations

(117 papers). After reading the full-texts, we selected

104 papers. These papers are not explicitly listed in

our paper, but can be accessed via the open access

repository including their analysis.

3.2 Cluster Analysis

An unsupervised k-means algorithm was imple-

mented (scikit-learn, Python 3.9) to recognize pat-

terns in the collected data (Kanungo et al., 2002). The

clustering method consists of the following 5 steps:

1. Text values (i.e., extracted MLS properties) are

converted into numerical values, including the

correction of assignments. We mitigated the

threat of affecting the external validity by exclud-

ing the technology value add and implementation

procedures. Both properties contained too little

information or too much space for interpretation.

2. Cluster seeds were initialized randomly according

to a predeﬁned number of expected clusters. In

this context, the elbow method was used which

suggested a number of 5 clusters as appropriate.

3. The Euclidean distance between each point and

seed was calculated by our method, meaning the

smallest distance became part of the given seed.

4. The seeds were placed in the centroids of the gen-

erated clusters to become their representatives.

5. The numerical values are converted to their text

values, including the correction of assignments.

4 RESULTS

Next, the SLR and clustering results are described (or-

ganized according to monitoring and prediction).

4.1 Literature Review Results

This section presents the results of the SLR. For rea-

sons of simplicity, each publication that is concerned

with CM, PM, and TWP is assigned to its predomi-

nant concern.

4.1.1 Monitoring Systems

We selected 67 monitoring MLS publications. Note

that we do not address working areas since we only

focus on CM.

Manufacturing Processes. The most mentioned

manufacturing process is milling (32). Others include

cutting (10), turning (8), drilling (4), grinding (3), and

sawing (2), which are typically concerned with a spe-

ciﬁc use case, e.g., gun drilling. Besides, we found

processes that were unique and thus could not be clas-

siﬁed adequately, e.g., processes for bearings (2).

Machines. The majority of the papers refers to CNC

machines (49). The remaining publications focus on

diverse (parts of) machines, e.g., lathes (4).

Use Cases. We divide the use cases into 2 categories:

monitoring approaches regarding the condition of ma-

chines, machine parts or their components (62), or

monitoring approaches focusing on the workpiece (5).

Most papers refer to CM of machines, machine parts,

or components, including TWM or tool CM. The sec-

ond category was mentioned by less papers, however,

also including 2 TWM approaches.

Technology Value Add. We identiﬁed 5 major

incentives for monitoring MLS, including improv-

ing manufacturing quality (51), product quality (32),

productivity (38) and life time (12), and reducing

waste/consumption (16) to reduce costs (50).

Learning Strategies. The majority of the monitor-

ing systems is based on supervised learning (54). A

small number of publications refers to unsupervised

learning (8) or a combination of supervised and unsu-

pervised learning methods (5).

Tasks. 38 publications are based on classiﬁcation.

Moreover, there exist 2 approaches using a combi-

nation of dimension reduction with classiﬁcation al-

Design Patterns for Monitoring and Prediction Machine Learning Systems: Systematic Literature Review and Cluster Analysis

211

gorithms. 20 papers mention regression as their task.

Furthermore, there exist combinations of regression

and dimension reduction (2) or clustering algorithms

(1). Only few papers refer solely to clustering (3).

Operations. Most approaches are based on ANN

(23), including traditional ANN (15). In addition,

14 papers are based on convolutional NN (CNN), in-

cluding besides the traditional CNN speciﬁc forms,

Bayesian CNN. Moreover, we found approaches us-

ing SVM (10) or recurrent NN (RNN) (6), e.g., recur-

rent long-short term memory network. The remaining

papers present various ML operations, such as deci-

sion trees (4), autoencoders (3), ensemble algorithms

(2), c-means (2), k-star (1), or deep belief networks

(1). We also identiﬁed several mixed operations oc-

curring only once, e.g., ANN and cluster analysis.

Data Base. The dominant signals used as a single

data base are vibration (36), force (21), audio (19),

images (6), current (9), and power (6). The combina-

tions of signals are commonly based on force or vi-

bration and include other signals, such as audio, tem-

perature, or power. Moreover, we found out that the

most applied combination is force, vibration, and au-

dio signals (9).

Implementation Procedures. The most preferred

programming language is MATLAB (21). Moreover,

in 11 publications the usage of Python is described,

usually in terms of libraries, such as Tensorﬂow. Oth-

ers include G (3), Visual Basic (1), Java (1), Excel (1),

and both Python and MATLAB (2). 27 papers did not

mention a certain implementation procedure.

4.1.2 Prediction Systems

We selected 37 prediction MLS publications. Note

that we intentionally considered both PM and TWP.

Manufacturing Processes. The most common pro-

cess is milling (22). Others include cutting (5), turn-

ing (3), or machining (2). Among the general process

categories, there are several use case-speciﬁc manu-

facturing processes, e.g., laser melting.

Machines. The most common machines are CNC

machines (28). Other machines mentioned are lin-

ear axes (2), lathes (2), steckel mills (1), melting ma-

chines (1), and wafer-handling robotics (1). Two pa-

pers that did not focus on a speciﬁc machine (2).

Use Cases. The use cases are mainly in the context

of TWP (31). Prediction systems are typically com-

bined with monitoring processes, e.g., bearing CM

and TWP of the machine tools. Other papers relied

on more speciﬁc prediction use cases, such as degra-

dation state prediction.

Technology Value Add. The major incentives are im-

proving manufacturing quality (30), product quality

(16), productivity (25) and tool life (10), and reduc-

ing waste (9) to reduce costs (31).

Working Areas. We extracted 2 working areas: tradi-

tional PM (14) and TWP (23). Both areas are primar-

ily concerned with predicting future conditions for

maintaining purposes rather than monitoring.

Learning Strategies. Most papers are based on su-

pervised learning (27). A small number of papers

refer to unsupervised learning (5). Others rely on a

combination of supervised and unsupervised learning

methods (4). 1 approach is based on semi-supervised

learning.

Tasks. The most common task is regression (19). In

addition, combinations of regression with other tasks

were found, e.g., classiﬁcation (1) or dimension re-

duction (2). Classiﬁcation is mentioned in 9 cases.

We also found 2 combinations of dimension reduction

and classiﬁcation. Moreover, there are also clustering

(3) and dimension reduction approaches (1).

Operations. ANN are implemented the most (8), in-

cluding traditional ANN and deep NN. Other opera-

tions mentioned in the papers are CNN (6), ensem-

ble algorithms (6), decision trees (5), RNN (4), clus-

ter analyses (3), SVM (3), and Bayesian approaches

(3). These general categories often include use case-

speciﬁc operations, e.g., multi-scale convolutional

gated recurrent unit networks.

Data Base. The most frequently signals used are vi-

bration (22), force (17), and audio (14). Combina-

tions of signals are more common than single signals,

mostly based on at least force or vibration.

Implementation Procedures. The majority did not

mention an implementation procedure (27). Concrete

programming languages are rarely mentioned, e.g.,

Python (4), MATLAB (3), or G (1).

4.2 Cluster Analysis Results

In this section, we present the clustering results based

on the k-means method applied to the SLR data.

Overall, we identiﬁed 5 clusters, where each cluster

represents a design pattern (cf. Table 1).

4.2.1 Monitoring Design Patterns

We identiﬁed 3 relevant clusters for monitoring MLS.

Monitoring Design Pattern 1. This pattern (22%

of all monitoring approaches) comprises supervised

learning, classiﬁcation, and ANN/CNN, i.e., 2 oper-

ations. Vibration signals (67%) are used the most

as data base for the certain learning processes. This

design pattern refers mainly to cutting processes by

CNC machines. Precisely, it is focused on CM ap-

plications for machines, parts, or individual compo-

ICSOFT 2023 - 18th International Conference on Software Technologies

212

Table 1: Monitoring and prediction design patterns based on the framework of Schuh and Scholz (2019).

Monitoring Prediction

Technological layer Design pattern 1 Design pattern 2 Design pattern 3 Design pattern 1 Design pattern 2

Manufacturing processes cutting milling milling milling milling

Machines CNC CNC CNC CNC CNC

Use cases TWM/TWP TWM/TWP TWM/TWP TWM/TWP TWM/TWP

Working areas CM CM CM TWP PM

Learning strategies supervised learning supervised learning supervised learning supervised learning supervised learning

Tasks classiﬁcation classiﬁcation regression regression classiﬁcation

Operations CNN/ANN ANN/CNN/SVM ANN CNN/ANN/ensemble ANN/clustering/RNN

Data base vibration acoustic/vibration vibration/force vibration/force/acoustic vibration

nents. For example, the pattern is actually imple-

mented by Alegeh et al. (2019).

Monitoring Design Pattern 2. The most applied pat-

tern (40% of all monitoring approaches) is based on

a combination of supervised learning, classiﬁcation,

and ANN/CNN/SVM, i.e., diverse operations. Vibra-

tion and acoustic signal (both 37%) are mentioned the

most. Force is only described in 26% of the papers.

The characteristics regarding machines and process

focus on milling for CNC machines. The use case

is usually TWM or TWP. For instance, this pattern is

followed by Kotsiopoulos et al. (2021).

Monitoring Design Pattern 3. This pattern (37% of

all monitoring systems) relies on supervised learning,

regression, and ANN, e.g., fulﬁlled by Javed et al.

(2012). Data bases are vibration (64%) and force

(56%). Moreover, it refers mainly to CNC-based

milling where the use case is mostly TWM or TWP.

4.2.2 Prediction Design Patterns

We found 2 prediction clusters, i.e., 2 design patterns.

Prediction Design Pattern 1. Regarding the ﬁrst de-

sign pattern (59% of all prediction approaches), the

combination of supervised learning, regression, and

CNN/ANN/ensemble algorithms, i.e., diverse opera-

tion, is most common. Interestingly, design pattern

1 consists exclusively of TWP approaches, which are

a dominant decisive criteria. The data base of these

TWP approaches typically consists of force (59%),

vibration (59%), and acoustic signals (41%). Simi-

lar to 2 monitoring patterns, methods for milling with

CNC machines are often proposed. For instance, this

design pattern is followed by Li et al. (2021).

Prediction Design Pattern 2. This pattern (41% of

all prediction approaches) relies on supervised learn-

ing, classiﬁcation, and ANN/cluster analysis/RNN,

i.e., diverse operations. The data base used is usually

based on vibration signals (64%). Similar to pattern

1, this pattern focuses on milling for CNC machines.

Interestingly, all but 1 paper deals with PM, which in-

dicates that the working area is decisive here. For ex-

ample, this pattern is addressed by Luo et al. (2018).

5 DISCUSSION

Next, the extracted design patterns are compared

to each other, additional ﬁndings are presented and

threats to validity are discussed.

5.1 Design Pattern Comparison

Monitoring Design Patterns. First, the monitoring

design patterns are mainly characterized by their MLS

tasks. While the ﬁrst and second pattern deal with

classiﬁcation, the third pattern focuses on regression

tasks. Second, in all patterns TWM and TWP are the

most mentioned use cases. Here, the paper count in-

creases with cluster number, i.e., design pattern 1 has

the lowest and design pattern 3 the highest propor-

tion, i.e., classiﬁcation MLS are applicable in more

diverse applications. Third, the manufacturing pro-

cesses correspond to the machines. So, in pattern

2 and 3 milling is most common. In contrast, the

ﬁrst pattern consists primarily of cutting and barely

milling processes. Fourth, while in design pattern 1

mainly vibration is used, design pattern 2 is quite bal-

anced based on acoustic, vibration, or other data like

force and current. Design pattern 3 is focused on vi-

bration, force, and acoustic.

Prediction Design Patterns. First, the major dif-

ference are the working areas (i.e., TWP and PM).

We argue that most other differences in the technol-

ogy layers are caused by this fact. Second, although

TWM/TWP is the most applied use case in both pat-

terns, the proportion in the ﬁrst design pattern is sig-

niﬁcantly higher corresponding to the working area.

While TWM/TWP is the only use case in design pat-

tern 1, the second pattern shows more diverse use

cases, e.g., machine health monitoring. Third, a simi-

lar situation exists regarding the machine and the cor-

Design Patterns for Monitoring and Prediction Machine Learning Systems: Systematic Literature Review and Cluster Analysis

213

responding process (i.e., CNC and milling). Fourth,

while the ﬁrst pattern deals in general with regression

tasks, classiﬁcation is dominant in the second pattern.

Thus, supervised learning is the main strategy for

both. However, the second pattern consists of more

unsupervised learning applications, possibly due to

the fact that PM requires much data from various data

sources. These are not only limited to information

from machines or processes but also from the mainte-

nance management system. In consequence, MLS in

PM applications comprise upstream dimension reduc-

tion or clustering methods to handle the complexity

of high dimensional huge amounts of data. Fifth, de-

sign pattern 1 consists mainly of vibration, force, and

acoustic signals related to the working area and use

case TWM/TWP. Design pattern 2 is more balanced,

except the dominance of vibration, i.e., showing many

other information, such as power or temperature.

Monitoring & Prediction Design Patterns. The de-

sign patterns seem quite similar, but there is a signif-

icant number of differences. The ratio between mon-

itoring and prediction is about 2:1, possibly due to

the fact that prediction is located on a higher com-

plexity level than monitoring. This assumption is

emphasized by the prediction approaches which ad-

dress more unsupervised learning tasks. In addition,

prediction MLS mainly use regression in contrast to

monitoring which are usually based on classiﬁcation.

We argue that regression is more suitable to predic-

tion systems because it deals with continuous target

values in contrast to classiﬁcation with discreet tar-

get variables. As a result, monitoring MLS deploy

more ANN, CNN, and SVM while prediction MLS

are signiﬁcantly more balanced with a higher num-

ber of RNN, ensemble methods, and variations of DT

which are very powerful, e.g., on time series data.

5.2 Additional Findings

Data Base. Interestingly, the design patterns are

mostly based on a similar data base: vibration or

force, sometimes also acoustic signals. The main rea-

son for this particularity is the use of open data sets,

e.g., the PHM challenge data set or the NASA milling

data set. We highlight that such data sets are the ba-

sis for uniform and comparable evaluations of MLS

regardless of their working area. By considering the

same evaluation metrics, ideally based on evaluation

criteria or practice-oriented MLS requirements, as-

sessments could increase in value. So, the qualitative

assessment of the operations could be supported by a

benchmark with quantitative evaluation criteria, e.g.,

accuracy or efﬁciency.

Learning Strategies. The extracted learning strate-

gies seem quite similar, i.e., the MLS are usually

based on supervised learning. We assume that super-

vised learning is often implemented because it is cur-

rently one of the most common learning strategies and

is well suited due to its associated tasks, i.e., classiﬁ-

cation in the context of monitoring MLS and regres-

sion in the context of prediction MLS. Furthermore, it

is conceivable that the implementation of more com-

plex forms of learning such as reinforcement learning

is still too challenging or too critical referring to data

security or privacy (e.g., data manipulation).

Machines. The majority of the MLS approaches

is based on CNC machines. This fact indicates

the widespread use of CNC machines in small and

medium-sized manufacturing companies, mainly due

to their ﬂexibility. We argue that this ﬁnding could

also be related to the use of the open data sets. How-

ever, there is a number of machine tools that cannot

be addressed by our design patterns in detail, even

though we support the trend towards CNC machines.

Technology Value Add. The technology value add is

quite similar in most of the cases, i.e., improving qual-

ity and productivity, as well as reducing costs. These

incentives are closely related to the widespread trian-

gle of production (Wang et al., 2018). However, we

note that more than 3 properties would also be feasi-

ble in the manufacturing domain, e.g., by adding ﬂex-

ibility (Erlach, 2013).

5.3 Threats to Validity

There are several threats that could impair the validity

of our study.

Systematic Literature Review. We could not ﬁnd

out whether all authors share the same understanding

of certain terms, e.g., monitoring systems with actual

predictive capabilities. However, this fact exempli-

ﬁes the convergence of these technologies to beneﬁt

from their speciﬁc strengths in different contexts. In

addition, there was a lack of completeness in depth

of content and consistency. Interestingly, more than

half of the authors did not mention a concrete learning

strategy, leading to assignments by ourselves (e.g.,

tasks as indicator for learning strategies). Neverthe-

less, misclassiﬁcations cannot be excluded. Although

we are aware of the fact that an even higher number

of papers might have gone beyond the thematic scope

of the paper, we know that the lower the number of

papers, the higher the impact of misinterpretations.

Clustering. There is no guarantee for an objective va-

lidity of the generated clusters, e.g., k-means is sen-

sitive to outliers that could mitigate our validity. So,

it is important to validate our design patterns based

on the clusters and further investigate the exact nature

ICSOFT 2023 - 18th International Conference on Software Technologies

214

and role of the approaches that do not entirely ﬁt in

one of the design patterns.

Although the described issues could threaten our

ﬁndings, we aimed to mitigate them, e.g., we re-

lied only on peer-reviewed publications of 3 relevant

databases. Moreover, in several previous studies, it

was already demonstrated that the selected framework

is appropriate and valuable for the uniform classiﬁca-

tion of MLS, e.g., Schuh et al. (2020). Note that we

mitigated the threat of possibly affecting the external

validity of the identiﬁed design patterns by excluding

the technological layers that do not provide enough

or reliable information, namely technology value add

and the implementation procedures. Overall, we ar-

gue that our study is highly valuable, reproducible,

and provides detailed insights into monitoring and

prediction MLS research.

6 RELATED WORK

We are aware of related work in our subject areas.

However, none of these publications provides a com-

parable overview of the technological structure from

a technology management perspective. In contrast,

our study is based on a successfully evaluated frame-

work that has been developed by practitioners and re-

searchers to identify design patterns. In the following

we present a selection of the closest related work.

Monitoring Systems. Serin et al. (2020) employed

a study to summarize CM systems using deep learn-

ing and described opportunities of CM in Industry 4.0

environments. Mohanraj et al. (2020) presented a lit-

erature review focusing on monitoring methods for

tool CM in the milling process which have been de-

scribed in manufacturing-related literature. Soother

and Daudpoto (2019) as well as Choudhary et al.

(2019) reviewed CM techniques in an engineering

context (induction motors). However, Choudhary

et al. (2019) focused more on the fault diagnosis pro-

cess and diagnostic schemes.

Prediction Systems. Zonta et al. (2020) presented

a literature review of PM applications in Industry

4.0 environments focusing on existing methods, stan-

dards, and applications. Carvalho et al. (2019) pub-

lished a literature review of ML methods focusing on

their performance, challenges, and opportunities in

the context of PM. Theissler et al. (2021) surveyed

PM approaches in the automotive industry from an

application and ML perspective. Moreover, C¸ ınar

et al. (2020) reviewed smart manufacturing PM ap-

plications by focusing on ML-related criteria, such as

data acquisition and data classiﬁcation.

7 CONCLUSION

We presented a comprehensive overview of monitor-

ing and prediction MLS in the manufacturing indus-

try, i.e., performing a classiﬁcation of 104 papers and

extracting 5 design patterns.

We conclude that research on monitoring and pre-

diction MLS is quite diverse. This trend is highlighted

by the growing capabilities of MLS, their operations,

and tasks in recent year, especially in the ﬁeld of NN.

We emphasize the dependence of MLS operations and

tasks on use cases, the resulting systems’ variability,

as well as the high impact of data sets on the compa-

rability of MLS. We strongly recommend continuing

research, e.g., analyzing further MLS working areas

and manufacturing-related diagnosis applications.

REFERENCES

Alegeh, N., Shagluf, A., Longstaff, A. P., and Fletcher, S.

(2019). Accuracy in detecting failure in ballscrew

assessment towards machine tool servitization. In-

ternational Journal of Mechanical Engineering and

Robotics Research, 8(5):667–673.

Carvalho, T. P., Soares, F., Vita, R., Francisco, R. d. P.,

Basto, J., and Alcal

a, S. G. S. (2019). A systematic

literature review of machine learning methods applied

to predictive maintenance. Computers & Industrial

Engineering, 137:106024.

Choudhary, A. K., Goyal, D., Shimi, S. L., and Akula, A.

(2019). Condition monitoring and fault diagnosis of

induction motors: A review. Archives of Computa-

tional Methods in Engineering, 26(4):1221–1238.

C¸ ınar, Z. M., Abdussalam Nuhu, A., Zeeshan, Q., Korhan,

O., Asmael, M., and Safaei, B. (2020). Machine

learning in predictive maintenance towards sustain-

able smart manufacturing in industry 4.0. Sustainabil-

ity, 12(19):1–42.

Davies, A. (2012). Handbook of Condition Monitoring:

Techniques and Methodology. Springer Science &

Business Media.

Dogan, A. and Birant, D. (2021). Machine learning and

data mining in manufacturing. Expert Systems with

Application, 166:1–45.

Erlach, K. (2013). Value Stream Design. Springer.

Espinoza Sep

ulveda, N. F. and Sinha, J. K. (2021). Blind

application of developed smart vibration-based ma-

chine learning (svml) model for machine faults diag-

nosis to different machine conditions. Journal of Vi-

bration Engineering & Technologies, 9(4):587–596.

Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep

Learning. The MIT Press.

Grifﬁths, I. (2020). Automated condition monitoring us-

ing artiﬁcial intelligence. In International Conference

on Vision, Image and Signal Processing, pages 1–7.

ACM.

IEC 60300-3-3:2017 (2017). Dependability management –

Design Patterns for Monitoring and Prediction Machine Learning Systems: Systematic Literature Review and Cluster Analysis

215

part 3-3: Application guide – life cycle costing. Stan-

dard, IEC.

ISO 13372:2012(E/F) (2012). Condition monitoring and

diagnostics of machines – vocabulary. Standard, ISO.

Javed, K., Gouriveau, R., Zerhouni, N., Zemouri, R., and

Li, X. (2012). Robust, reliable and applicable tool

wear monitoring and prognostic: Approach based on

an improved-extreme learning machine. In Confer-

ence on Prognostics and Health Management, pages

1–9. IEEE.

Kanungo, T., Mount, D. M., Netanyahu, N. S., Piatko,

C. D., Silverman, R., and Wu, A. Y. (2002). An efﬁ-

cient k-means clustering algorithm: Analysis and im-

plementation. Transactions on Pattern Analysis and

Machine Intelligence, 24(7):881–892.

Kitchenham, B. A., Budgen, D., and Brereton, O. P. (2015).

Evidence-Based Software Engineering and Systematic

Reviews. Taylor and Francis.

Kotsiopoulos, T., Leontaris, L., Dimitriou, N., Ioanni-

dis, D., Oliveira, F., Sacramento, J., Amanatiadis,

S., Karagiannis, G., Votis, K., and Tzovaras, D.

(2021). Deep multi-sensorial data analysis for produc-

tion monitoring in hard metal industry. The Interna-

tional Journal of Advanced Manufacturing Technol-

ogy, 115(3):823–836.

Krishna, M. and Kannadaguli, P. (2020). Iot based cnc

machine condition monitoring system using machine

learning techniques. In Conference on Communica-

tion Systems and Network Technologies, pages 61–65.

IEEE.

Li, G., Wang, Y., Wang, J., He, J., and Huo, Y. (2021).

Tool wear prediction based on multidomain feature

fusion by attention-based depth-wise separable con-

volutional neural network in manufacturing. The In-

ternational Journal of Advanced Manufacturing Tech-

nology, pages 1–18.

Luo, B., Wang, H., Liu, H., Li, B., and Peng, F. (2018).

Early fault detection of machine tools based on deep

learning and dynamic identiﬁcation. Transactions on

Industrial Electronics, 66(1):509–518.

Lutz, B., Kisskalt, D., Regulin, D., Reisch, R., Schifﬂer,

A., and Franke, J. (2019). Evaluation of deep learn-

ing for semantic image segmentation in tool condition

monitoring. In International Conference on Machine

Learning and Applications, pages 2008–2013. IEEE.

Mobley, R. K. (2002). An Introduction to Predictive Main-

tenance. Elsevier.

Mohanraj, T., Shankar, S., Rajasekar, R., Sakthivel, N. R.,

and Pramanik, A. (2020). Tool condition monitoring

techniques in milling process – a review. Journal of

Materials Research and Technology, 9(1):1032–1042.

Monostori, L., K

ar, B., Bauernhansl, T., Kondoh, S., Ku-

mara, S., Reinhart, G., Sauer, O., Schuh, G., Sihn, W.,

and Ueda, K. (2016). Cyber-physical systems in man-

ufacturing. CIRP Annals, 65(2):621–641.

Motaghare, O., Pillai, A. S., and Ramachandran, K. (2018).

Predictive maintenance architecture. In International

Conference on Computational Intelligence and Com-

puting Research, pages 1–4. IEEE.

Patil, R. B., Patil, M. A., Ravi, V., and Naik, S. (2017). Pre-

dictive modeling for corrective maintenance of imag-

ing devices from machine logs. In International Con-

ference of the IEEE Engineering in Medicine and Bi-

ology Society, pages 1676–1679. IEEE.

Qiao, H., Wang, T., and Wang, P. (2020). A tool wear moni-

toring and prediction system based on multiscale deep

learning models and fog computing. The Interna-

tional Journal of Advanced Manufacturing Technol-

ogy, 108:2367–2384.

Schuh, G. and Scholz, P. (2019). Development of a frame-

work for the systematic identiﬁcation of ai applica-

tion patterns in the manufacturing industry. In Inter-

national Conference on Management of Engineering

and Technology, pages 1–8. IEEE.

Schuh, G., Scholz, P., Leich, T., and May, R. (2020). Iden-

tifying and analyzing data model requirements and

technology potentials of machine learning systems in

the manufacturing industry of the future. In Interna-

tional Scientiﬁc Conference on Information Technol-

ogy and Management Science, pages 1–10. IEEE.

Schuh, G., Scholz, P., Schorr, S., Harman, D., M

oller, M.,

Heib, J., and B

ahre, D. (2019). Prediction of work-

piece quality: An application of machine learning in

manufacturing industry. In International Conference

on Computer Science, Engineering and Information

Technology, pages 189–202. CS & IT.

Serin, G., Sener, B., Ozbayoglu, A. M., and Unver, H. O.

(2020). Review of tool condition monitoring in ma-

chining and opportunities for deep learning. The In-

ternational Journal of Advanced Manufacturing Tech-

nology, 109(3):953–974.

Sevic, M. and Keller, P. (2019). Design of cnc milling ma-

chine as a base of industry 4.0 enterprise. MM Science

Journal, 12:3555–3560.

Soother, D. K. and Daudpoto, J. (2019). A brief review

of condition monitoring techniques for the induction

motor. Transactions of the Canadian Society for Me-

chanical Engineering, 43(4):499–508.

Tang, L.-A., Han, J., and Jiang, G. (2014). Mining sensor

data in cyber-physical systems. Tsinghua Science and

Technology, 19(3):225–234.

Theissler, A., P

erez-Vel

azquez, J., Kettelgerdes, M., and

Elger, G. (2021). Predictive maintenance enabled by

machine learning: Use cases and challenges in the au-

tomotive industry. Reliability Engineering & System

Safety, 215:107864.

VDI/VDE 2651 Part 1:2017-05 (2017). Plant asset manage-

ment (pam) in the process industry deﬁnition, model,

task, beneﬁt. Standard, VDI/VDE-Gesellschaft Mess-

und Automatisierungstechnik (GMA).

Wang, J., Ma, Y., Zhang, L., Gao, R. X., and Wu, D. (2018).

Deep learning for smart manufacturing: Methods

and applications. Journal of Manufacturing Systems,

48:144–156.

Wuest, T., Weimer, D., Irgens, C., and Thoben, K. D.

(2016). Machine learning in manufacturing: Ad-

vantages, challenges, and applications. International

Journal of Production Research and Manufacturing

Research, 4(1):23–45.

Zonta, T., da Costa, C. A., da Rosa Righi, R., de Lima,

M. J., da Trindade, E. S., and Li, G. P. (2020). Pre-

dictive maintenance in the industry 4.0: A systematic

literature review. Computers & Industrial Engineer-

ing, 150:106889.

ICSOFT 2023 - 18th International Conference on Software Technologies

216