![](bg2.png)
web service to obtain the status of a purchase order.
The innovation with respect to the previous version
is that consider two scenarios for web service
activation: local (C2B) and remote (B2B). An
important benchmarking aspect is that an average
thinking time of 10 seconds is used between the
successive requests of the clients. This new version
is also too simple to be considered as an acceptable
benchmark for web services.
A more complete benchmark for web services is
@Bench (Doculabs, 2003) that exposes 3 services:
GetOrderDetails, GetCustomer, NewCustomer. The
users request the three services with the same
probability and the time between two successive
requests is chosen as a random value between 2 and
8 seconds. This benchmark models the interactive
requests that users send to the application server of
their own company (this represents a C2B scenario).
However this benchmark does not model the
relations of a server with other servers, that is, the
typical B2B scenario integrated in a server based on
web services.
The Spidermark benchmark (Subramanyam,
2003) models a set of users sending interactive
requests to the application server of their own
company (this represents a C2B scenario). The
benchmark also models the transactions carried out
by the application server with servers of external
suppliers of the company to satisfy the requests of
the users (this represents a B2B scenario). All the
interactions are implemented using web services.
Later, Sun Microsystems proposed the
benchmark WSTest (Sun, 2004) to compare the
technologies used to implement web services. This
benchmark only invocates empty methods in the
remote server, that echo the variables received. This
benchmark has been designed to evaluate only the
communication aspects involved in web services.
Microsoft modified this benchmark adding a method
to generate load in the server (Microsoft, 2004).
WSTest does not model any specific e-commerce
scenario and it is too simple to be considered as a
general benchmark for web services platforms.
Finally, the Transaction Processing Performance
Council organization (TPC) launched the
specification 1.0 of the benchmark TPC-App
(TPC 2004) that is an application server and web
services benchmark. The application modelled by
this benchmark is a retail distributor operating
through Internet that support ordering and retrieving
information of products (this represents a typical
B2B scenario).
For our research work on benchmarking of web
services platforms, we have selected the TPC-App
benchmark, because it models very well the
operations and the workload of a typical e-business
application server that interacts with other
e-business servers through web services.
3 AN OVERVIEW OF TPC-APP
The TPC-App benchmark emulates the activities of
a B2B transactional application server system with
the goal of obtaining an indication of the
performance capabilities of the server system.
The benchmarking architecture includes three
main elements: the System Under Test (SUT), the
Remote Business Emulator (RBE) and the external
emulators. Figure 1 gives a general overview of
these three elements of the benchmark, showing also
their main internal components.
3.1 The Server Under Test
The server (SUT) exposes 7 remote methods to the
remote business emulator (RBE). Figure 1 shows
these methods, indicating also the percentage of
invocations of each method and the maximum
admissible value of the 90-percentile response time
for the invocations of each particular method.
The most important method is Create Order,
whose operation is explained in the following
paragraphs. The Create Order method creates an
order on the database and sends a message to the
order fulfilment subsystem using the shipping queue.
An order summary is returned to the RBE.
Later the orders are processed asynchronously by
the Shipping Process. It extracts the messages with
the orders from the shipping queue and process the
order in two different ways, as a function of the
order status:
1) If the status is pending, there are enough items
in stock to complete the order and send the items to
the customers. The Shipping Process sends a request
to the external shipment notification emulator (SNE)
which represents an external packet delivering
company. The SNE returns an image that represents
a shipping label and a tracking number for the
shipment package.
2) If the status is back, there are not enough
items in stock to complete the order, and therefore, a
message is sent to the stock management queue in
order to the stock management process add new
items to the stock. Then, it sends a message to the
shipping queue containing the order with its status
assigned to pending.
WEBIST 2006 - INTERNET TECHNOLOGY
76