data model is present in the loaded HTML document.
In this scenario, we store this entity in the data model,
remove the last element from the stack because we are
no longer interested in the current web page (since we
have found the entity), and go back a step (there’s a
higher probability that the next unvisited clickable el-
ement is situated on the previous page than others).
The algorithm for detecting an entity for the data
model is based on the premise that a label associ-
ated with an input is situated on its North-West side.
Therefore, we firstly detect the inputs (<input>, <se-
lect>, <textarea>) present on the HTML web page
and then try to find for each input the closest textual
label that is positioned in the North-West quarter of a
circle with the centre in the middle of the input. If for
more than one input we have found a pair in the list of
labels, it means that the plugin has found an entity for
the data model of the page, and the associated textual
labels are the properties/fields of the entity of the data
model.
5 VALIDATION
We conducted an experiment in which we tried to de-
termine the data model for a business web application.
The application we used to test our browser plugin is
Microsoft Dynamics 2016 CRM. This business appli-
cation has more than 150 distinct types of web pages
generated (i.e. more than 150 functionalities). We
let the plugin discover entities for 30 minutes, time in
which it found 21 different entities out of 21 possibili-
ties (apparent 100% precision; also the plugin did not
detect any false entity - an entity that does not exist
in the application). This precision of 100% is appar-
ent because although the plugin detected all the avail-
able entities in the time frame of the experiment (30
minutes), occasionally some of the properties/labels
of the detected entities were not properly detected.
We realize that it would have been better to let the
plugin run for a significantly longer period of time, so
that it discovers all entities in the data model of the
Microsoft Dynamics 2016 CRM application, but the
web pages generated by this application are similar
and we consider that running the data model detec-
tion algorithm of our plugin for about 30 minutes is
statistically relevant for the whole Microsoft Dynam-
ics 2016 CRM application. After running the browser
plugin for 30 minutes, we report the entities it has
found. Some of these entities were found multiple
times because of their numerous occurrences in the
web pages of the application. Being a complex ap-
plication, it has a variety of buttons and features, such
as generating reports or downloading files. Therefore,
the plugin had to traverse these steps with the aim of
discovering an entity.
Running the data model discovery algorithm for a
longer time period would, of course, allow the plugin
to discover more entities from the data model of the
target business web application.
In screenshot Fig. 1, we can see how the plugin
detected the clickable elements and bordered them in
red. We compute the XPATH of these HTML ele-
ments and push them into a stack, in order to be later
clicked one by one in a depth-first search manner.
In Fig. 2 and Fig. 3, the plugin firstly detected the
input fields present on this web page and then associ-
ated the closest textual label to each input. Therefore,
the labels which are associated with an input, repre-
sent the properties of an entity, or more exactly, the
column’s names of an entity in a database. In rare
cases, where the wrong labels are closer to an input,
the associations are incorrect.
The list of entities that make up the model is saved
by the browser plugin in JSON format due to the ubiq-
uitous support for this format in web applications.
Each entity is described by its name and a list of at-
tributes/fields (which are equivalent to column names
in a database table). For example, a small snippet of
the entities from the Microsoft Dynamics 2016 CRM
application identified by our browser plugin tool is the
following :
[ "Invoice" :
["Invoice ID", "Name", "Currency",
"Price list", "Prices locked",
"Opportunity", "Order", "Customer",
"Description", "Date delivered",
"Due date", "Shipping method",
"Payment terms", "Bill to address",
"Ship to address"],
"Order" :
["Order ID", "Name", "Currency",
"Price list", "Prices locked",
"Opportunity", "Quote", "Potential
Customer", "Description", "Requested
delivery", "Date fulfilled",
"Shipping method", "Payment terms",
"Freight Terms", "Bill to address",
"Ship to address"]
]
We should also note that the detected model of
the application is just an approximation of the data
model of the database used by the target web appli-
cation. The actual data model of the database used
by the web application is hidden from the human user
(i.e. the human user does not have direct access to the
underlying database of the web application). The de-
tected data model is the one imprinted in the UI of the
web application (i.e. it is comprised from the labels
Conceptual Model Detection in Business Web Applications Within the RPA Context
523