Business Intelligence and Data Analytics (BI&DA) to Support the
Operation of Smart Grid
Business Intelligence and Data Analytics (BI&DA) for Smart Grid
G. Escobedo, Norma Jacome and G. Arroyo-Figueroa
Instituto de Investigaciones Eléctricas, Reforma 113, Cuernvaca, Morelos, Mexico
Keywords: Data Mining, Data Analytics, Business Intelligence, Operational BI, Smart Grid, Electric Power Utility,
Information Systems.
Abstract: Smart Grid is the modernization of electrical networks using intelligent systems and information
technologies. The growing interest that the smart grid is attracting and its multidisciplinary nature motivate
the need for solutions coming from different fields of knowledge. Due to the complexity, and heterogeneity
of the smart grid and the high volume of information to be processed, Business Intelligence and Data
Analytics (BI&DA) appear to be some of the enabling technologies for its future development and success.
The aim of this article is proposed a framework for the development of BI&DA techniques applied to the
different issues that arise in the smart grid development. As case study the paper presents the applications of
BI&DA in database of processes security for Distribution System. The goal is to have available and timely
information to make better decisions, to reduce the number of accidents and incidents. This work is
therefore devoted to summarize the most relevant challenges addressed by the smart grid technologies and
how BI&DA systems can contribute to their achievement.
1 INTRODUCTION
In the last decade the Electric Power Utilities
(EPUs) industry has undergone major changes in
terms of liberalization, increased competition, efforts
to improve energy efficiency, in a context of
environmental sustainability.
This situation has led governments and the
scientific community to look for solutions that allow
an efficient, reliable and responsible use of energy,
appealing to an optimized and more flexible
conception of the electrical grid (Mejia 2009).
The modernization of the electrical grid is known
as the Smart Grid. The NIST defines as smart grid
how a modernized grid that enables bidirectional
flows of energy and uses two-way communication
and control capabilities that will lead to an array of
new functionalities and applications (NIST 2010).
Smart Grid is the convergence of information
technologies, sensors and intelligent systems to
monitor and manage power generation, transmission,
and distribution. Despite the highly diverse nature of
the technological challenges, they share a common
set of features that need to be considered as the
starting point to propose solutions based on
Information Tecnology (IT).
The EPUs need a way to collect, correlate and
analyze information from multiple sources, for
different task, such as: processes optimization,
planning, prediction, diagnosis, make decision, and
re-evaluate the situations to determine whether
further actions are required (Khanna et al, 2015).
Due to the complexity and challenges in the
design, optimization, scheduling and management of
smart grids, IT and computational intelligence
techniques (CIT) have proven to be a good
alternative to achieve these goal (Morais et al,
2009). Two of these techniques are Business
Intelligence and Data Analityc (BI&DA). The
opportunities associated with data and analysis in
different organizations have helped generate
significant interest in BI&A, which is often referred
to as the techniques, technologies, systems,
practices, methodologies, and applications
(Chaudhuri et al, 2011). BI&DA transforms the raw,
massive data collected by various sources into useful
information. BI&A includes business-centric
practices and methodologies that can be applied to
various high-impact applications such as electric
market intelligence, planning, make decisions and
Escobedo, G., Jacome, N. and Arroyo-Figueroa, G.
Business Intelligence and Data Analytics (BI&DA) to Support the Operation of Smart Grid - Business Intelligence and Data Analytics (BI&DA) for Smart Grid.
DOI: 10.5220/0005936604890496
In Proceedings of the International Conference on Internet of Things and Big Data (IoTBD 2016), pages 489-496
ISBN: 978-989-758-183-0
Copyright
c
 2016 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
489

cyber security, among others.
The aim of this paper is to present a proposed
framework for the application of BI&DA in smart
grids environment. In section 2 a brief of smart grids
concept and challengers for the information
management is shown. Section 3 presents a resumen
of BI&DA techniques. Section 4 shows the proposed
of a framework for BI&DA applications. Section 5
presents a case study of application of BI&DA in
database of processes security for Distribution area.
Finally, Section 6 summarizes the most relevant
ideas presented in this paper.
2 SMART GRID
An EPU is a complex environment. The structure of
traditional power grid comprises different stages:
generation, transmission, distribution and
commercialization of electricity, see figure 1. The
first stage is the power generation that takes place in
large power plants or renewables power plants; the
second stage is the transmission that transports
energy to the areas where it will be consumed; the
third stage is the distribution that delivered energy to
the end user. To support the electrical grid operation
information systems needed for management and
control the processes of generation, transmission,
distribution and trading.
Figure 1: Main Processes and its information systems of
EPU.
The NIST defines as smart grid how a
modernized grid that enables bidirectional flows of
energy and uses two-way communication and
control capabilities that will lead to an array of new
functionalities and applications. The Smart Grid
integrates electricity and communications in an
electric network that supports the new generation of
interactive energy and communication services and
supplies digital quality electricity for the final
customer. In this sense, the smart grid can be defined
as a system that employs digital information and
control technologies to facilitate the deployment and
integration of distributed and renewable resources,
smart consumer devices, automated systems,
electricity storage and peak-saving technologies.
The most common applications of Smart Grids are
shown in the figure 2 (Gulich 2010).
Figure 2: Applications of Smart Grid.
Advanced Metering Infrastructure (AMI).
Remote Meter Reading, Remote
Disconnect/Connect, Theft Detection, Customer
Prepay, Mobile Workforce Management.
Demand Response. Advanced Demand
Maintenance and Demand Response; Load
Forecasting and Shifting.
Grid Optimization. Self-healing Grid: Fault
Protection, Outage Management, Remote Switching,
Minimal Congestion, Dynamic Control of Voltage,
Weather Data Integration, Centralized Capacitor
Bank Control.
Distributed Generation & Store. Monitoring of
Distributed Assets.
EV Charging and Discharging. Application Data
Flow for PHEVs.
Customer Support. Application Data Flow to/from
End-User Energy Management Systems.
Electricy Market. Real Time Energy Markets.
Smart Grid will integrate all the components of
power system to enhance the performance of the
grid. Much of the integration of components relates
to communication systems, IT systems, and business
RAIBS 2016 - Special Session on Recent Advancement in IoT, Big Data and Security
490

processes. To satisficed this challenges for Smart
Grid, the IT involved include the following:
• Integrated communications across the grid.
• Advanced control methods.
• Intelligent sensing, metering, and measurement.
• Advanced grid components
• Decision support and human interfaces
• Business Intelligence, Data Analytics and Big
Data.
3 BUSINESS INTELLIGENCE
AND DATA ANALITYC
The term intelligence has been used by researchers
in artificial intelligence since the 1950s. Business
intelligence became a popular term in the business
and IT communities only in the 1990s. In the late
2000s, business analytics was introduced to
represent the key analytical component in BI. More
recently big data and big data analytics have been
used to describe the data sets and analytical
techniques in applications that are so large (from
terabytes to exabytes) and complex (from sensor to
social media data) that they require advanced and
unique data storage, management, analysis, and
visualization technologies (Chen, 2012). Business
intelligence and data analytics (BI&DA) is an
unified term and treat big data analytics.
BI&DA is a collection of decision support
technologies for gathering, providing access to, and
analyzing data for the purpose of helping enterprise
users (executives, managers and analysts) make
better and faster business decisions (Obeidat, 2015).
The term implies having a comprehensive
knowledge of all of the factors that affect the
Figure. 3: Typical BI architecture.
business. It is imperative that companies have an in
depth knowledge about factors such as the
customers, competitors, business partners, economic
environment, and internal operations to make
effective and good quality business decisions.
Business intelligence enables firms to make these
kinds of decisions. Enterprise aimed at enabling
knowledge executives, managers and analysts to
make better and faster decisions (Yeoh, 2010).
The typical components of Business Intelligence
architecture for an Enterprise are shown in the figure
3:
3.1 Data Sources
Data sources can be operational databases, historical
data, external data for example, from market
research companies or from the Internet), or
information from the already existing data
warehouse environment. The data sources can be
relational databases or any other data structure that
supports the line of business applications. They also
can reside on many different platforms and can
contain structured information or unstructured
information. Thus the problems of integrating,
cleansing, and standardizing data in preparation for
BI tasks can be rather challenging.
3.2 Data Integration
Extract-Transform-Load (ETL) refers to a collection
of tools that play a crucial role in helping discover
and correct data quality issues and efficiently load
large volumes of data into the warehouse.
3.3 Data Warehouse and Data Marts
The data warehouse is the significant component of
business intelligence. It is subject oriented,
integrated. The data warehouse supports the physical
propagation of data by handling the numerous
enterprise records for integration, cleansing,
aggregation and query tasks. A data mart is a
collection of subject areas organized for decision
support based on the needs of a given department.
The key difference is that the creation of a data mart
is predicated on a specific, predefined need for a
certain grouping and configuration of select data
(Watson, 2007).
3.4 Data Presentation
BI&DA includes several tools for the data analysis.
It refers to the way in which business users can slice
Business Intelligence and Data Analytics (BI&DA) to Support the Operation of Smart Grid - Business Intelligence and Data Analytics
(BI&DA) for Smart Grid
491

and dice their way through data using sophisticated
tools that allow analytical processing and advance
analytics. Online analytic processing (OLAP)
provides multidimensional, summarized views of
business data and is used for reporting, analysis,
modeling and planning for optimizing the business.
OLAP tools provide the common BI operations such
as filtering, aggregation, drill-down and pivoting.
Advanced analytics is referred to as data mining,
text analytics, forecasting or predictive analytics and
artificial intelligence algorithms, this takes
advantage of statistical analysis and artificial
intelligence techniques to predict or provide
certainty measures on facts.
There are several popular frontend applications
through which users perform BI tasks: spreadsheets,
enterprise portals for searching, performance
management applications that enable decision
makers to track key performance indicators of the
business using visual dashboards, tools that allow
users to pose ad hoc queries, viewers for data mining
models, and so on. Rapid, ad hoc visualization of
data can enable dynamic exploration of patterns,
outliers and help uncover relevant facts for BI.
A high-level diagram depicting the
interrelationships among the various BI&DA
technologies and their applications in the different
subsystems of power grid is shown in Figure 3
(Zeyar, 2013).
Figure 4: BI&DA applications for Smart Grid.
In addition to the traditional BI&DA paradigm
on static and centralized data, there are new
paradigms distributed data, data stream, and time-
series data are much relevant to the smart grid
because of its very nature of distributiveness and
having to deal with numerous data streams and time
series data from various data sources: smart meters,
sensors, and power system machinery.
BI&A is an unified term and treat big data
analytics. In the case of smart grid there are some
potential applications of this technology, see figure
4:
• Frequent Pattern Mining: to discover some
sub-patterns or motifs those occur frequently in
a dataset.
• Association Rule Mining: to uncover which
causes usually lead to which effects in a dataset.
The association rules can generally be derived
from the frequent patterns described above.
• Classification: to classify instances in a dataset
into pre-defined groups (called class labels).
Classification is a supervised learning process.
• Clustering: to organize similar instances in a
dataset into groups which are not predefined.
Clustering is an unsupervised learning process
in which we do not know the class labels of all
the instances in the data set in advance.
• Regression: to predict the value of the target
attribute (called dependent variable) of an
instance based on the values of other attributes
(independent variables). Regression is also a
type of supervised learning which works in the
similar way as classification.
• Outlier Detection: to identify anomalous
instances, which might be interesting or indicate
errors and require further investigation. It can be
supervised, unsupervised, or semi-supervised
learning.
4 FRAMEWORK OF BI&DA FOR
SMART GRID
The framework proposed provides the methods and
tools for assisting in the acceptance, production, use
and maintenance of BI&DA implementation. It is
based on TOGAF framework; an iterative process
model supported by best practices and a re-usable
set of existing architecture assets, see figure 5.
For a successful BI implementation is to be
applied formal methods for managing information
and fully understand the operation of the
organization.
• Project Definition: The first is to have clearly
defined the purpose of the application of
BI&DA aligned business vision and mission;
RAIBS 2016 - Special Session on Recent Advancement in IoT, Big Data and Security
492

fully understand the strategic objectives of the
organization.
• Business and IT Architecture: An Enterprise
Architecture (EA) is a scheme whereby
represented through models, the business of the
company (strategy, objectives and processes),
the needs of information (data and applications)
and the technologies that support it. The BA
determines that the government provides to the
business of information technologies (IT) and
how are you must be aligned with strategic
business goals.
• Business Process Mapping: It requires
mapping the processes and technological
applications involved in the definition of the
data to develop BI&DA applications. The data
quality is critical for success of the application
of BI&DA.
• IT Applications: The IT applications must be
the automation of the business process. The IT
applications are the origin of the data.
• Data Quality: For a successful BI&DA
applications, you must ensure that the data are
available and reliable.
• B&DA Techniques: Design of data marts and
tools for extraction, transformation, and load
(ETL) are essential for converting and
integrating enterprise-specific data. Database
query, online analytical processing (OLAP), and
reporting tools based on intuitive, but simple,
graphics are used to explore important data
characteristics.
• Front End & Applications: Front-end-
applications can be the most relevant because it
show the result of BI&DA applications. In
addition to these well-established business
reporting functions, statistical analysis and data
mining techniques are adopted for association
analysis, data segmentation and clustering,
classification and regression analysis, anomaly
detection, and predictive modelling in various
business applications.
• Results and Improvements: Finally the results
have to analyser in order to improve the BI&DA
applications.
BI&DA is a matter not only of technology, it is
necessary that the company deploys a Business and
IT technology strategy that is executed by the
BI&DA Centre and this requires an infrastructure of
software and hardware as well as organization and
models.
Figure 5: Framework for BI&DA.
5 CASE STUDY
The goal of BI&DA is to provide new knowledge to
the company, from the automated exploitation of
historical information for business actions are taken
to be better supported. The results obtained by
applying techniques of BI&DA to the database of
industrial safety of Mexican Electric Utility are
shown in this section.
The benefits it will bring to the implementation
of business intelligence technologies are mainly:
a) Accessing security information in a timely and
reliable, which allow reducing the time in
making and decisions, creating more effective
decisions to have the information available.
b) Display detailed information of the security
process, making further analysis as a result of
having consolidated historical information and
current information.
c) Allow delivery of data in a flexible, dynamic and
in many cases to solve unplanned queries.
d) As result of above: having a decrease in the
number of injured or dead; having a decrease in
the economic impact caused by accidents and
Decrease the number of days lost due to
accidents.
Today there are many tools that offer similar
products to both large and small organizations. BI
vendors propose solutions both horizontal and
vertical and the best choice will depend on the
specific need of each organization. With horizontal
Business Intelligence and Data Analytics (BI&DA) to Support the Operation of Smart Grid - Business Intelligence and Data Analytics
(BI&DA) for Smart Grid
493

solutions from scratch by an application tailored to
the need. Vertical solutions are aimed at an industry
already developed components and only fit specific
needs. In this case the implementation of BI tools for
the industrial security was done by development
proprietary tools under Windows platform and a
solution horizontal. For the design and development
of BI tools take in count the following queries:
Project Definition. Goal alignment queries: the
application of BI tools has the aim to reduce the
number of accidents and incidents.
Business & IT Architecture. The enterprise
architecture meets the information needs related to
the process. The elements of the entreprise
architecture are bussines architecture and IT
architecture. The business architecture includes the
strategic plan definition and the BP mapping. The
technology architecture includes the definition of the
data set and data flow; the information systems
applications management, the communication
definition, the enterprise bus, the drives, the security
system and soon. The figure 6, shows the Enterprise
architecture.
Figure 6: Enterprise Architecture.
IT Applications. Baseline queries: the source of
information is generated by security information
system (in Spanish, Sistema de Integral de
Seguridad y Salud en el Trabajo –SISST). The
system manages three areas of industrial safety for
power system processes: accident, infrastructure
safety and health protection; and safety management
(Jacome et al, 2011).
Customer and stakeholder queries: There three
kinds of users: operative, analytical and executive.
Metric-related queries: the metrics was defined
by the security expert. The metrics includes: security
indicators, and related variables. A success factor for
the development of BI&DA applications is the
definition of metrics.
Data Quality. The operational variables are load
by the security information system. There are
procedures and methodologies defined by the
company for the variables and its load frequency.
BI&DA Techniques. The architecture of
BI&DA techniques considers the construction of a
single enterprise data warehouse and from it will
emerge Data Marts (small units of analysis), with an
overall vision of the company. The figure 7 shows
the BI&DA-SISST architecture.
Figure 7: BI&DA-SISST architecture.
The primary tasks include gathering, preparing
and analyzing data. From operational database of
industrial security system, extract the most relevant
data to the user through an ETL process, which
loads and makes the necessary transformations and
data cleansing. These are the integrated operational
data (ODS-SISST) of the security system. The data
itself must be of high quality.
Once the data is complete and clean, they pass
through another ETL process to data warehouse
(DWH-SISST). Finally, data marts are created for
accidents, safety programs and hazards. From these
data were carried out the data analysis, through
multidimensional analysis and consulted manager
dashboards.
Front-end Applications. The results of the BI
application are the front-end-applications through of
dashboards. The Dashboard should provide the
RAIBS 2016 - Special Session on Recent Advancement in IoT, Big Data and Security
494

executive with a tool for navigating through
company's information. The figure 8 shows the
accident frequency by area with pie contribution and
table of results with goal and indicator state. Also
shows the severity by area.
Figure 8: Accident frequency and severity.
The figure 9 shows the historical, real and
forecast accident bye year and by moth. This
information is obtained to accident module.
The main indicators of the security have
presented in the figure 10. The definition of
indicators is an important task to ensure the success
of business intelligence. This dashboard shows the
security maturity in terms of identification of
hazards, legal requirements, compliance of security
programs, safety forms, incidents and acts of
government.
With the business intelligence is possible to
combine different information. For example it is
possible to relate the accident with the attitudes.
Figure 9: Accident frequency: historical, real and forecast.
Figure 10: Management indicators.
For this case, the figure 11 presents the relations
between the accidents with attitudes. Also, it is
possible to classify the kind of attitudes: learning,
knowledge and responsibility. This means that the
an accident occurs can determine if the accident
occurred due to lack of training, lack of knowledge
or lack of responsibility. For the case of incidents
also it is possible determinate if occurred by lack of
training, knowledge or responsibility.
BI&DA-SISST system is a traditional business
intelligence application with back-end database and
front-end user interface, software that processes the
information and reporting systems.
Figure 11: Relationship between accidents and incidents
with attitudes.
6 CONCLUSIONS
BI&DA as a concept is becoming more common in
everyday business life. BI&DA incorporates people,
Business Intelligence and Data Analytics (BI&DA) to Support the Operation of Smart Grid - Business Intelligence and Data Analytics
(BI&DA) for Smart Grid
495

process, and also knowledge as an end product. The
implementation of BI&DA is a complex undertaking
requiring considerable resources.
An important factor to build BI&DA
applications is the information management.
BI&DA requires reliable and timely information and
generates summary information for the operative
and strategic decision making. In addition, the
implementation of a BI&DA is often associated with
the following challenges: underlying original back-
end systems and processes which were not adapted
for BI&DA applications; poor data quality derived
from source systems that can often go unnoticed
until cross-systems analysis is conducted; and the
maintenance process that tends to be vague and ill-
defined
To attack this problem is necessary to implement
enterprise architecture with its two main
components: business architecture and technological
architecture can help ensure that the data source will
be reliable.
The BI&DA tools developed for the industrial
security have had good results. The information
displayed through dashboards make career choices
have led to the decrease of accidents. In particular
the relationship between accidents and attitudes has
been a great help to generate preventive actions to
avoid accidents. Also indicate if the accident
occurred due to lack of training, knowledge, or lack
of responsibility.
As many research challenges remain in all
aspects of BI&DA, several new open research
challenges appear on horizon for recent
technologies, such as Cloud Computing, Near Real-
Time BI, Enterprise Search, distributed data mining,
data stream mining, time-series data mining,
information security and more.
ACKNOWLEDGEMENTS
The authors wish to thank Israel Paredes Rivera,
Department Head of Technical Services Unit of CFE
for their important work in supporting, organizing
and promoting the project.
REFERENCES
Zeyar Aung, “Database Systems for the Smart Grid”,
Book Smart Grids: Opportunities, Developments, and
Trends, Sringer Verlag, pp 151-168, 2013.
Victor Chang, Yen-Hung Kuo, Muthu Ramachandran,
“Cloud computing adoption framework: A security
framework for business clouds”, Future Generation
Computer Systems, Volume 57, Pages 24–41, 2016.
S. Chaudhuri, U. Dayal and V. Narasayya, “An Overview
of Business Intelligence Technology”,
Communications of the ACM, Vol. 54, No. 8, pp: 88-
98, August 2011.
Oleg Gulich, “Technological and Business challenges of
Smart Grids”, Msc Theses, Lappeenranta University of
Technology, 2010.
Hsinchun Chen, Roger H. L. Chiang and Veda C. Storey,
“Business Intelligence and Analytics from Big Data to
Big Impact”, MIS Quarterly, Vol. 36, No. 4, pp: 1165-
1188, December 2012.
N. Jacome-Grajales, G. Escobedo-Briones and E.
Guadarrama Villa “Inteligencia de Negocios en el area
de seguridad de la CFE” (In Spanish), Congreso
Internacional sobre Innovación y Desarrollo
Tecnológico, pp. 677-685, 2011.
Manju Khanna, N. K. Srinath, and J. K. Mendiratta, “Data
Mining in Smart Grids-A Review”, Vol. 5, No. 3, pp:
709-712, 2015.
I. Martin-Rubio, A. E. Florence-Sandoval, J. Jimenez-
Trillo, and D. Andina, “From Smart Grids to Business
Intelligence, a Challenge for Bioinspired Systems”,
Lecture Notes in Computer Science, Vol. 9108, pp
439-450, 2015.
M. Mejía-Lavalle, G. Arroyo-Figueroa, E. F. Morales,
“Innovative applications of diagnosis, forecasting,
pattern recognition, and knowledge discovery in
power systems”, IEEE Power & Energy Society
General Meeting PES’09, Otawa Canada, pp. 1-9,
2009.
J. Morais,, Y. Pires,, C. Cardoso, A. Klautau “An
overview of data mining techniques applied to power
systems. In: J. Ponce, A. Karahoca (eds.) Data Mining
and Knowledge Discovery in Real Life Applications. I-
Tech Education and Publishing, 2009.
NIST Framework and Roadmap for Smart Grid
Interoperability Standards N. S. P. 1108., Release 1.0,
January 2010.
M. Obeidat, M. North, R. Richardson V. Rattanak and S.
North, “Business Intelligence Technology,
Applications, and Trends”, International Management
Review, Vol. 11, No.2, pp: 47-55, 2015.
Sarvapali D. Ramchurn, Perukrishnen Vytelingum, Alex
Rogers, and Nicholas R. Jennings, “Putting the
‘Smarts’ into the Smart Grid: A Grand Challenge for
Artificial Intelligence”, Communications of the ACM,
Vol. 55, No. 4, pp: 86-97, April 2012.
William Yeoh, Andy Koronios, “Critical Success Factors
for Business Intelligence Systems”, Journal of
Computer Information Systems, pp:23-32, 2010.
H. Wang, and S. Wang, “A Knowledge Management
Approach to Data Mining Process for Business
Intelligence,” Industrial Management & Data Systems,
Vol. 108, No. 5, pp: 622-634, 2008.
H. J. Watson, H. Barbara Wixom, The current state of
business intelligence, Computer, Vol. 40, No. 9, 96-99,
2007.
RAIBS 2016 - Special Session on Recent Advancement in IoT, Big Data and Security
496