![](bg2.png)
pendence and obstruct the ability to obtain repeatable
test results. Thus, destructive E2E tests must be run
serially, increasing the time cost of effectively testing
a system.
While this seems challenging, we can apply the
same frameworks and logic which enabled automated
E2E testing in the first place to make E2E database-
conscious testing feasible. Using a base database con-
tainer image and certain customization SQL scripts
for different tests, testers can effectively assign a
database to each test to create a closed environment
for testing. This would allow tests to be run in par-
allel, increasing utility and feasibility of E2E testing.
To avoid imposing high resource costs on the testing
infrastructure, we can group tests which do not mod-
ify the database or which require the resulting data
from another test and run them serially on the same
database.
In this paper, we propose a framework which
accomplishes this and comment on our work in
progress. Section 2 addresses related work to our field
and comments on its limitations for our topic. Section
3 details our proposed framework. Section 4 details
the work we have completed thus far, and Section 5
addresses our planned future work. Section 6 evalu-
ates our framework’s benefits and limitations. Finally,
Section 7 closes the paper with some final remarks.
2 RELATED WORK
End-to-end testing of complex, interconnected sys-
tems is a known problem which multiple projects
have attempted to solve. One of these is ElasTest
(Bertolino et al., 2018), a distributed architecture
designed explicitly for ”cloud testing.” It offers a
language-independent framework for writing and de-
ploying tests and uses container deployment to allow
itself to stay application-agnostic. Since its inception,
the ElasTest software (Gort
´
azar et al., 2017) has con-
tinued to evolve and now supports features such as
chaos testing, security testing, and external integra-
tions such as Jenkins. However, ElasTest appears to
lack the native ability to change the database between
tests or specify a specific schema for a test. Other
E2E testing frameworks which also lack this feature
include AWS Device Farm (Amazon Web Services,
2021), an Amazon Web Services offering which only
allows client-side application upload, and TestCraft
(Testim, 2021), a codeless UI testing framework.
The high cost of running many E2E tests has led to
research in limiting the resources used. Augusto (Au-
gusto, 2020) addresses these concerns using a similar
framework to our own, the RETORCH framework. It
groups tests by resource usage, including levels of ac-
cess (read, read/write, write-only, or dynamic) and ad-
ditionally schedules them for maximum usage. While
this framework is an excellent E2E testing tool, it
does not address the problem of preparing databases
for different tests or handling destructive E2E tests.
Others propose limiting the number of tests in the re-
gression test set, the set used to continually test the
software (Ammann and Offutt, 2017). Gligoric et al.
(Gligoric et al., 2015) present Ekstazi, a framework
which automatically analyzes software modifications
to select only the tests which would be affected by the
change. Ekstazi has been shown to produce signifi-
cant cost savings for production environments (Vasic
et al., 2017). We believe that Ekstazi and similar ef-
forts are complimentary to our work and that further
investigation could lead to more cost savings for our
framework.
More general efforts to test reactive systems vary.
Schrammel et al. proposes the idea of test chaining,
a way of grouping similar tests together which re-
duces execution time (Schrammel et al., 2013). This
presents a great opportunity to reduce test data re-
quirements by pairing these test chains with a sin-
gle database; however, the authors make no men-
tion of this strategy. Modern efforts focus on con-
tainerization. Garc
´
ıa et al. pairs ElasTest with Se-
lenium WebDriver to construct an advanced user im-
personation testing platform (Garcia et al., 2018). In
a case study on implementing an E2E testing strategy,
Lindell and Johnson create a Selenium testing appa-
ratus using Docker containers and Microsoft Azure
pipelines (Johnson and Lindell, 2020). However, nei-
ther of these address destructive E2E tests.
Research on destructive E2E testing is less com-
mon. Early works (Tsai et al., 2001) and (Xiaoy-
ing Bai et al., 2001) present E2E testing frameworks
which include the ability to specify test database
configurations. The introduction of containerization
since that time makes this configuration much easier.
The primary challenge of this project is ensuring that
it is application-independent, decreasing coupling and
enabling wider acceptance. Frameworks like ElasTest
accomplish this, but at the cost of being able to di-
rectly manipulage the application database. We be-
lieve we can accomplish this by requiring a standard
way of specifying the database address across all ap-
plications which will use the framework. The next
section details our attempt at creating a suite to run
database-swapping E2E tests on an existing reactive
system.
ICEIS 2021 - 23rd International Conference on Enterprise Information Systems
378