Multi-agent Manufacturing Execution System (MES): Concept,

Architecture & ML Algorithm for a Smart Factory Case

Soujanya Mantravadi

, Chen Li

and Charles Møller

Department of Materials & Production, Aalborg University, Fibigestræde 16, Aalborg, Denmark

Keywords: AI Applications, Industry 4.0, Intelligent Manufacturing, Manufacturing Operations Management (MOM),

Multi-Agent Systems, Enterprise Information Systems, Architectural Solution, Automated Reasoning,

Uncertainty, Work-in-Progress (WIP).

Abstract: Smart factory of the future is expected to support interoperability on the shop floor, where information systems

are pivotal in enabling interconnectivity between its physical assets. In this era of digital transformation,

manufacturing execution system (MES) is emerging as a critical software tool to support production planning

and control while accessing the shop floor data. However, application of MES as an enterprise information

system still lacks the decision support capabilities on the shop floor. As an attempt to design intelligent MES,

this paper demonstrates one of the artificial intelligence (AI) applications in the manufacturing domain by

presenting a decision support mechanism for MES aimed at production coordination. Machine learning (ML)

was used to develop an anomaly detection algorithm for multi-agent based MES to facilitate autonomous

production execution and process optimization (in this paper switching the machine off after anomaly

detection on the production line). Thus, MES executes the ‘turning off’ of the machine without human

intervention. The contribution of the paper includes a concept of next-generation MES that has embedded AI,

i.e., a MES system architecture combined with machine learning (ML) technique for multi-agent MES. Future

research directions are also put forward in this position paper.

1 INTRODUCTION

Context-aware Manufacturing Systems.

Automated operations in a manufacturing enterprise

require both plant control systems as well as

enterprise software. In the era of digital

transformation, smart factories can automate

manufacturing operations by being context-aware.

Given that smart factories are key components of

Industry 4.0 (Kagermann et al., 2013), it is essential

to develop manufacturing information systems that

can assist humans and machines in the execution of

their tasks on the shop floor. Manufacturing

execution system (MES) is an information system and

a real-time compliant software, which is identified to

enable smart factories due to its ability to act as a

digital twin (Mantravadi and Møller, 2019). It

supports shop floor as well as the supply chain level

https://orcid.org/0000-0001-9382-8314

https://orcid.org/0000-0001-6249-8957

https://orcid.org/0000-0003-0251-3419

activities of a manufacturing enterprise (Mantravadi

et al., 2018) (Mantravadi et al., 2018).

The revolutionary wave of computing (Internet of

Things, IoT) which is the phenomenon of connecting

objects over the internet, has enabled us to have

intelligent manufacturing systems. As a core of any

manufacturing system, MES controls production

process that involves the physically connected

production units/physical assets/equipment. Modern

factories generate massive amounts of production

data during the production process, where MES faces

certain challenges such as:

• Make best use of ever-increasing amounts of

logged production data to find meaning,

dependencies, relations and problems in

production which are not apparent upfront

Mantravadi, S., Li, C. and Møller, C.

Multi-agent Manufacturing Execution System (MES): Concept, Architecture ML Algorithm for a Smart Factory Case.

DOI: 10.5220/0007768904770482

In Proceedings of the 21st International Conference on Enterprise Information Systems (ICEIS 2019), pages 477-482

ISBN: 978-989-758-372-8

477

• Possess analytical solutions to support

extracting, storing and analyzing the data to

obtain an optimized decision for MES

This paper is motivated by the need on how to

share and process valuable information in a much

more efficient and flexible way to solve the above two

challenges and fill the gap of lacking decision support

capability of MES (Li, 2012).

Machine Learning for Manufacturing Execution

System. Machine learning (ML) is an important

toolbox which can be used to make sense of the data

generated from the production and ML is already

known to serve the purpose of knowledge synthesis

in engineering automation (Lu, 1990). MES can

perform learning to apply on a wide range of

production processes, including optimization of

individual module behavior, optimization across each

module or one or more production lines (Gröger et al.,

2012). ML can help MES to export valuable

information from the production modules and feed it

to the modern computing power and learning

algorithms. This will consequently result in exploring

new opportunities, business models and solving the

challenges that were not possible before.

Some examples from the literature that apply ML

for manufacturing problems:

1. For optimizing the process to achieve energy

efficiency, Palensky P et.al. suggested that time

periods need to be around 15-30 minutes to

switch off the equipment if the energy

consumption beyond the upper bound according

to the production scheduling of MES (Palensky

and Dietrich, 2011).

2. Vieira et al. proposed an analytical model that

can predict the performance of rescheduling

strategies and quantify the trade-offs between

different performance measures for

manufacturing system (Vieira et al., 2003).

3. In order to build prediction models to increase

sustainability performance in machining

operations, Woo et al. developed a big data

analytics platform for manufacturing system

(Woo et al., 2016).

ML is also able to identify the anomaly behavior

in a production line, which has been a hot topic in

recent years, i.e., anomaly detection. It has become a

manufacturing imperative for high velocity real-time

production to analyze patterns of data streams and

look for anomalies that can reveal something

unexpected on the production line. Ko et al., use ML

based anomaly detection to estimate the products’

quality by integrating manufacturing, inspection and

after-sales service data (Ko et al., 2017). Liu J et al.,

developed a structured neural networks which

efficiently reduces anomaly detection

misclassification for a manufacturing system (Liu et

al., 2018). Van Stein et al., proposed a GLOSS

anomaly detection algorithm which helps to detect

anomalies in high dimensional mixed data sets of

manufacturing process (Van Stein et al., 2017).

Vodencarevic et al. presented anomaly detection

algorithm, AN-ODA, to detect anomalies in the

cyber-physical systems (Asmir Vodencarevic et al.,

2011). Windmann et al. identify the abnormal

behavior, OTALA and QRM were developed for

modeling learning of discrete states and continuous

behavior (Windmann et al., 2015).

There are several other studies that also show how

ML was used for solving/improving a specific task on

the shop floor. All these studies also indicate the fact

that using ML in manufacturing has been studied

extensively and that it is a well-established research

line. However, not many studies address the benefit

utilization aspect of existing manufacturing informa-

tion systems that could use toolboxes such as ML.

Against this backdrop, we argue that larger impact

is created for a manufacturing enterprise when

researchers can maximize the value of existing MES

with systems thinking approach to address a bigger

problem for the enterprise. Such approach can be

realized by deploying MES with collaborating

technologies. A collaboration system, which is a

combination of different elements such as hardware,

software, organizational practices and other tool

boxes like ML could include MES software as a main

actor. Such a collaborating system, designed based on

an information system (MES) can derive significant

benefit for the overall enterprise. Whereas a single

task solving approach might not derive maximum

value from the existing IT assets in a factory. A

similar concept of ‘Work System Theory’ proposed

by Alter also advocates linking people, processes and

IT tools for improving business performance (Alter,

2011).

In this paper, a smart factory scenario of detecting

anamoly using multi-agent MES is outlined as an

example. It supports the enquiry on how to provide

decision making for MES using ML techniques to

detect the abnormal behaviors on the production line;

a question that was not widely researched before.

Section 2 introduces the theoretical framing to the

MES research, section 3 describes the approach

followed by a proposal of a system architecture and

an algorithm to support the concept. Section 4

concludes the position and presents the future work.

ICEIS 2019 - 21st International Conference on Enterprise Information Systems

478

2 CONCEPT OF ‘AI EMBEDDED

MES FOR A SMART FACTORY’

The Intelligent Manufacturing Systems (IMS) were

first outlined in 1978 (Hatvany and Nemes, 1978). AI

research results are highlighted as promising tools for

managing complexity, uncertainty, unforeseen

problems, dynamic changes and disturbances in

manufacturing systems (Hatvany and Lettner, 1983)

(Monostori and Prohaszka, 1993) (Shen et al., 2000).

Holonic manufacturing and multi-agent based

manufacturing control are two popular approaches for

distributed intelligent manufacturing control. Since

multi-agent based control systems are pure software

environments unlike holonic manufacturing systems

(McFarlane et al., 2003), an agent based approach is

more suitable for implementation on a software like

MES to improve process performance.

Requirements of Future Factories. Two of the few

design principles of Industry 4.0 are ‘interconnection’

and ‘decentralized decisions’ (Hermann et al., 2016)

that require smart factories to use MES to implement

production scheduling in real-time via intelligent data

acquisition and analysis (Chen et al., 2017). For this,

OPC UA based interaction in multi-agent systems is

a recommended technology (Chen et al., 2017). With

the changing manufacturing requirements, MES too

needs rethinking and MES research needs to combine

the aspects of AI.

3 APPROACH

The proposed multi-agent MES consists sub-agents

(that run on Raspberry pi platform) and central agent

(a middle-ware running on MES server). The agent is

designed as a virtual digital shell of each physical

asset of the production line. The main idea behind this

agent-based approach is to use a sub-agent as an

assistant to collect the data which is generated by the

asset during the production, and leverage central

agent to identify the abnormal behavior and aid MES

to execute. The extension of this work would be to

test and implement such software system. The

proposed software falls under the category of

centralized multi-agent system (Kamdar et al., 2018).

For this paper the problem is chosen to be the

anomaly behavior on current production line of AAU

Smart lab (Madsen and Møller, 2017) where the two

anomalies are identified as:

(1) Unusual drilling speeds (too high or too low)

and (2) Unusual number of parts finished per minute

The main approach can be described in the

following steps:

 Monitoring - This step is performed by the sub-

agent of each asset. The data is generated during

the production process monitored by the sub-

agent.

 Data Collection - The production data will be

collected and stored by sub-agent during the

production. The selected features of the data will

be used for building the behavior model.

 Modelling - In order to learn the behavior model

of the system, the sub-agents push the data

sample to the central agent that is running on

server. Based on the above identified two

features, the data on drilling speed and the

processing time will be extracted from the

production data. The extracted data will be fed

into the formulae (see the following formula in

section 4) to obtain the mean and variance for

building the normal behavior model. To

distinguish an abnormal behavior, a statistical

model is used to detect the normal behavior of

the system based on the collected data metrics.

 Anomaly Detection - The central agent applies

a statistical test to detect whether it is a normal or

abnormal behavior according to the data point.

 Group Decision Making - If the anomaly is

detected, the central agent needs to quickly

identify the cause of the abnormal behavior, flag

the corresponding sub-agent(s) and generate a

new decision for adjusting the behavior of the

physical assets through the sub-agent(s). The

server also holds a repository for storing the

system topology. If the decision requires

reconstructing the system topology, the topology

repository also needs to be updated.

 Behavior Adjustment - The central agent will

send the commands to the sub-agent which is

involved in causing the anomaly. The current

behavior adjustment considered in this case is

switching the machine off if the abnormal

behavior is detected.

This approach to the situation helps us in presenting

the architecture and an algorithm in section 4.

4 SYSTEM ARCHITECTURE

AND ALGORITHM

The system architecture can be represented as Fig.1.

Multi-agent Manufacturing Execution System (MES): Concept, Architecture ML Algorithm for a Smart Factory Case

479

Figure 1: System architecture of ML based multi-agent manufacturing execution system.

The purpose of this work is to integrate the ML

into multi-agent MES for detecting the anomaly

behaviors in a supervised fashion. In order to achieve

that, we chose the client-server (CS) style

architecture.

Anomaly detection algorithm helps central agent

to distinguish the abnormal behavior from the normal

production activities of the production. We assume

that the features of the production data follow the

Gaussian (Normal) distribution. The main algorithm

is described as:

Feature Selection. Two features are selected for our

example, drilling speed (x1) and number of parts

finished per minute (x2).

Fit Parameters. Given the number m sample data

where µ1 and µ2 represent the mean of the feature x1

and x2 of sample data separately, and σ1 and σ2

stands for the variance of the feature x1 and x2 of

sample data separately.























=









−











Calculating p(x): Calculating the probability p(x) of

the new production data x to see if it is lower than the

predefined lower bound ε, where the anomaly is

detected if p(x) < ε (predefined as a threshold).

(

)

=



;



,











=

√

2





exp(−

(



−



)



2





)





Below is the partial python code to calculate the mean

µ, co-variance σ and probability density p:

mport numpy as np

from scipy.stats import

multivariate_normal

from sklearn.metrics import f1_score

# This function estimates the

parameters: mu and sigma

# input: X - data sample X

# output: mu: the mean of the data

ICEIS 2019 - 21st International Conference on Enterprise Information Systems

480

# set X; sigma: Covariance

def estimate_gaussian(X):

mu = np.mean(X, axis=0)

sigma = np.cov(X.T)

return mu, sigma

# This function is to calculate the

probability density function

def multivariateGaussian(X, mu,

sigma):

pro =

multivariate_normal(X,mean=mu,

cov=sigma)

return pro

5 CONCLUSIONS & FUTURE

WORK

The following contribution of the paper is directed to

improve MES by combining its deployment with AI

techniques:

• A perspective on carrying out high impact

research for a manufacturing enterprise is put

forward. Given that information systems

research concerns socio-technical and

organizational aspects, a position was taken

that future research on benefit utilization of

MES and deploying it with collaborating

systems (such as agent-based systems or

prediction systems etc.) can improve business

performance of a manufacturing enterprise,

which ultimately aligns with strategic and

functional objectives of a smart factory.

• The study inferred that existing MES

literature lacked attention on its benefit

realization using decision support

mechanisms. It established the potential in

applying AI to the factory floor to leverage

the existing manufacturing IT tools of

MOM layer (as per ISA 95 standard).

• Agent based approach is suitable for MES

implementation in the smart factory context.

• A concept was developed to establish the

future research agenda on the topic of

combining AI with MES. To verify the

concept, system architecture and anomaly

detection algorithm that can run on top of

MES to execute decision on the production

line, were proposed. Based on this proposal,

an artifact: a smart factory with multi-agent

MES could be designed in the future.

As far as we know, it is the first time combining

the anomaly detection machine learning algorithm

with agent based MES in manufacturing.

Currently, the authors’ team is working on

implementing this approach at AAU Smart lab’s

production line. The project “AAU Open Source

MES Framework” undertaken by the authors’, aims

to design and develop an open platform (an open

source software stack) for smart factory solutions.

The project contributes to the manufacturing

digitalization using “Odoo” open source enterprise

resource planning (ERP) system to achieve

interoperability through vertical integration of the

factory floor.

The future work intends to develop a toolchain

that builds the communication channel for

exchanging the operational commands and

production data between MES and sub agents that run

on programmable logic controllers and raspberry-pi.

This multi-agent setup works in real-time to improve

the process performance in the factories.

ACKNOWLEDGEMENTS

This research work is partially funded by the

Manufacturing Academy of Denmark.

REFERENCES

Alter, S. (2011) ‘USF Scholarship: a digital repository @

Gleeson Library | Geschke Center The Work System

Method: Systems Thinking for Business Professionals

The Work System Method: Systems Thinking for

Business Professionals’. Available at: http://repository.

usfca.edu/at%5Cnhttp://repository.usfca.edu/at/32.

Asmir Vodencarevic, H. K. B., Niggemann, O. and Maier,

A. (2011) ‘Identifying behavior models for process

plants’, IEEE International Conference on Emerging

Technologies and Factory Automation, ETFA. doi:

10.1109/ETFA.2011.6059080.

Chen, B. et al. (2017) ‘Smart Factory of Industry 4.0: Key

Technologies, Application Case, and Challenges’,

IEEE Access. IEEE, 6, pp. 6505–6519. doi: 10.1109/

ACCESS.2017.2783682.

Gröger, C., Niedermann, F. and Mitschang, B. (2012) ‘Data

Mining-driven Manufacturing Process Optimization’,

Proceedings of the World Congress on Engineering

2012 Vol III , III, p. 7. doi: 10.1016/j.cplett.2005.

06.119.

Hatvany, J. and Lettner, F. J. (1983) ‘The Efficient Use of

Deficient Knowledge’, CIRP Annals - Manufacturing

Technology, 32(1), pp. 423–425. doi: 10.1016/S0007-

8506(07)63433-7.

Multi-agent Manufacturing Execution System (MES): Concept, Architecture ML Algorithm for a Smart Factory Case

481

Hatvany, J. and Nemes, L. (1978) ‘Intelligent

Manufacturing Systems— A Tentative Forecast’, IFAC

Proceedings Volumes. Elsevier, 11(1), pp. 895–899.

doi: 10.1016/S1474-6670(17)66031-2.

Hermann, M., Pentek, T. and Otto, B. (2016) ‘Design

principles for industrie 4.0 scenarios’, Proceedings of

the Annual Hawaii International Conference on System

Sciences, 2016–March, pp. 3928–3937. doi:

10.1109/HICSS.2016.488.

Kagermann, H., Wahlster, W. and Helbig, J. (2013)

‘Recommendations for implementing the strategic

initiative INDUSTRIE 4.0: Final report of the Industrie

4.0 Working Group’, Final report of the Industrie 4.0

WG, (April), p. 82. doi: 10.13140/RG.2.2.14480.20485.

Kamdar, R., Paliwal, P. and Kumar, Y. (2018) ‘A State of

Art Review on Various Aspects of Multi-Agent

System’, Journal of Circuits, Systems and Computers,

27(11), p. 1830006. doi: 10.1142/S02181266183000

64.

Ko, T. et al. (2017) ‘Machine learning-based anomaly

detection via integration of manufacturing, inspection

and after-sales service data’, Industrial Management &

Data Systems, 117(5), pp. 927–945. doi:

10.1108/IMDS-06-2016-0195.

Li, F. (2012) ‘Study of Multi-Agent Based Integratable

Manufacturing Execution System Model’, Advanced

Materials Research. Trans Tech Publications, 366, pp.

268–271. doi: 10.4028/www.scientific.net/AMR.366.

268.

Liu, J.; Guo, J.; Orlik, P.V.; Shibata, M.; Nakahara, D.; Mii,

S.; Takac, M. (2018) Anomaly Detection in

Manufacturing Systems Using Structured Neural

Networks. Available at: https://www.merl.com/

publications/docs/TR2018-097.pdf.

Lu, S. C. Y. (1990) ‘Machine learning approaches to

knowledge synthesis and integration tasks for advanced

engineering automation’, Computers in Industry, 15(1–

2), pp. 105–120. doi: 10.1016/0166-3615(90)90088-7.

Madsen, O. and Møller, C. (2017) ‘The AAU Smart

Production Laboratory for Teaching and Research in

Emerging Digital Manufacturing Technologies’,

Procedia Manufacturing. The Author(s), 9, pp. 106–

112. doi: 10.1016/j.promfg.2017.04.036.

Mantravadi, S., Cheng, Y. and Møller, C. (2018)

‘MES/MOM systems for Manufacturing Networks : An

exploratory study from operations in India’, 22nd

Cambridge International Manufacturing Symposium,

(September), pp. 27–28.

Mantravadi, S. and Møller, C. (2019) ‘An Overview of

Next-generation Manufacturing Execution Systems :

How important is MES for Industry 4.0 ?’, Elsevier

Procedia Manufacturing.

Mantravadi, S., Moller, C. and Christensen, F. M. M.

(2018) ‘Perspectives on Real-Time Information

Sharing through Smart Factories: Visibility via

Enterprise Integration’, in 2018 International

Conference on Smart Systems and Technologies (SST).

IEEE, pp. 133–137. doi: 10.1109/SST.2018.8564617.

McFarlane, D. et al. (2003) ‘Auto ID systems and

intelligent manufacturing control’, Engineering

Applications of Artificial Intelligence, 16(4), pp. 365–

376. doi: 10.1016/S0952-1976(03)00077-0.

Monostori, L. and Prohaszka, J. (1993) ‘A Step towards

Intelligent Manufacturing: Modelling and Monitoring

of Manufacturing Processes through Artificial Neural

Networks’, CIRP Annals - Manufacturing Technology,

42(1), pp. 485–488. doi: 10.1016/S0007-8506(07)62

491-3.

Palensky, P. and Dietrich, D. (2011) ‘Demand side

management: Demand response, intelligent energy

systems, and smart loads’, IEEE Transactions on

Industrial Informatics. IEEE, 7(3), pp. 381–388. doi:

10.1109/TII.2011.2158841.

Shen, W., Maturana, F. and Norrie, D. H. (2000)

‘Enhancing the performance of an agent-based

manufacturing system through learning and

forecasting’, Journal of Intelligent Manufacturing,

11(4), pp. 365–380. doi: 10.1023/A:1008926202597.

Van Stein, B. et al. (2017) ‘Towards data driven process

control in manufacturing car body parts’, Proceedings

- 2016 International Conference on Computational

Science and Computational Intelligence, CSCI 2016,

pp. 459–462. doi: 10.1109/CSCI.2016.0093.

Vieira, G. E., Herrmann, J. W. and Lin, E. (2003)

‘Rescheduling Manufacturing Systems: a Framework

of Strategies, Policies, and Methods’, Journal of

Scheduling, 6, pp. 39–62. Available at: https://link-

springer-com.ezproxy2.utwente.nl/content/pdf/10.102

3%2FA%3A1022235519958.pdf.

Windmann, S., Niggemann, O. and Stichweh, H. (2015)

‘Energy efficiency optimization by automatic

coordination of motor speeds in conveying systems’,

Proceedings of the IEEE International Conference on

Industrial Technology. IEEE, 2015–June(June), pp.

731–737. doi: 10.1109/ICIT.2015.7125185.

Woo, J., Shin, S.-J. and Seo, W. (2016) ‘Developing a Big

Data Analytics Platform for Increasing Sustainability

Performance in Machining Operations’, Flexible

Automation and Intelligent Manufacturing, (June), pp.

1–8. Available at: https://www.researchgate.net/

publication/305222784_Developing_a_Big_Data_Ana

lytics_Platform_for_Increasing_Sustainability_Perfor

mance_in_Machining_Operations.

ICEIS 2019 - 21st International Conference on Enterprise Information Systems

482