EXPERIMENTATION MANAGEMENT KNOWLEDGE SYSTEM
Information Systems Integration for DoD Network-Centric Operations
R. William Maule, Shelley P. Gallup, Gordon Schacher
Information Sciences, Naval Postgraduate School, Monterey, California, USA
Keywords: Systems integration, Distributed systems, Knowledge management, Workflow management, Web services
Abstract: A current major focus in the DoD involves the integration of information across the different military
branches for operations. Network-centric information methods will enable efficiencies through the
integration of best-of-breed software and hardware from each branch of the military, together with the latest
advances from government laboratories and the private sector. Information merging will promote synergy
and expand effective use of the enterprise infrastructure to realize improved operational and organizational
processes. Research to date has focused on core network and infrastructure capabilities but has not fully
addressed strategic organizational objectives in the context of systems integration. A model is advanced that
establishes variables for enterprise analysis to assess strategic technical objectives enabled or hindered
through new network-cen-tric capabilities. Examples are derived from operational experimentation in
network-centric warfare but presented generically to apply to any organization seeking to assess the
effectiveness of organizational strategy as enabled or hindered through network-based communications and
enterprise-level systems integration.
1 INTRODUCTION
DoD systems integration problems are similar to
those encountered in the private sector. However,
the complexity of relationships in military
exchanges and the inherent uncertainties of dynamic
environmental variables make DoD systems
integration problems much more difficult.
Warfare in the past was mainly laying ordnance
on targets. Today we are in a systems-based
environment with network-centric operations and the
management of knowledge as a top priority. Various
techniques are being advanced to help integrate
distributed databases and legacy information
systems into virtual enterprise architecture. Data
types, sources, taxonomy, ontology, and XML
schemas are being mapped to enable information
exchange.
Key to military operations is the sequence of
processes: information development, information
sharing, knowledge development, situation
assessment, shared situation understanding, and
collaborative action—herein achieved as a facet of
enterprise-level integration of large-scale databases
and database driven systems.
It is worth noting that knowledge development is
needed for more than military operations.
Management processes such as budget and
procurement decisions must also be supported.
These decisions are even more complex in the new
environment because they involve more than
hardware design and acquisition. Someone has to
decide how much a “pound” of information
management is worth.
2 INFORMATION INTEGRATION
Networked enterprises are becoming a new
organizational paradigm, creating challenging
opportunities in terms of management (Azevedo and
Sousa, 2000). Technological analysis needs to
incorporate organizational context as well as
application and data sources (Ericsson, 2001).
Enterprise systems integration, in the system
discussed, addresses technological, organizational,
and information [context] variables to improve
management decision-making. The referenced KM
enterprise system was developed to interface with
and assess systems and systems integration
initiatives and make recommendations based on
experimentation results.
185
William Maule R., P. Gallup S. and Schacher G. (2005).
EXPERIMENTATION MANAGEMENT KNOWLEDGE SYSTEM - Information Systems Integration for DoD Network-Centric Operations.
In Proceedings of the First International Conference on Web Information Systems and Technologies, pages 185-189
DOI: 10.5220/0001226601850189
Copyright
c
 SciTePress

2.1 Objective Integration
QoS issues generally involve the effectiveness
and/or efficiency of the systems integration
initiatives. For example, the recent Trident Warrior
2004 experiment considered the effectiveness of
individual networks,
interfaces between information systems,
coherence to emerging standards for enterprise
architecture (i.e., Web Services, Global Information
Grid), the viability of specific components of the
infrastructure and the information they produced,
human-systems integration, and organizational
decision processes supported or hindered by the
systems integration initiative(s).
A sponsor provides experimentation objectives
for the particular systems integration initiative. This
is the top-level definition of the experiment. At the
next level, the experiment’s physical structure is
chosen to meet those objectives, including the
operational forces, the processes to be used by
operational personnel, and the systems that will
support those processes. The next level is concerned
with situations to be run, measures to be produced ,
data to be captured, and analysis techniques. All of
this information is integrated in the KM system with
appropriate relationships for reference, to establish
fitness across components, and to construct and
make available a data capture plan.
A key to experimentation is development of
experiment threads. For each thread, specific data
elements are identified, generally as pass-through
from system to system and increasingly as a web
service or XML-based exchange. Data that is
captured during the experiment are input into the
KM system. The result is development of an
automatic association from top-level objectives
down through data, analysis, and results. The KM
information can be entered at any point of
experimentation and relationships to all associated
information examined.
Results reporting follows a similar structure.
Data is archived with a relationship to experiment
threads. Measures resulting from analyses are filed
in the KM system with the correct relationship to the
data from which they are produced. The final step in
the results production process is interpretation of
meaning by subject-matter-experts (SMEs). A form-
based process in the KM system is used to file both
interpretation results (interpretation is with respect
to the experiment’s original sponsor objectives) and
the context within which the experiment was carried
out. The relationships between results and
objectives are made transparent in the system, as are
references to all levels of planning and analysis.
For example, a recent evaluation of a web
services implementation in a distributed
environment tested the ability of a portal to
dynamically assemble web services under various
network conditions. Of particular interest was that
one of the tested services was itself a compilation of
XML feeds from several different servers, and
another was processing metadata input from
distributed sources (also encapsulated and passed as
a web service). Additional tested systems included
networks, routers, and communication technologies
employed in the process (various configurations of
optical, Ethernet, satellite, and wireless). The thread
used by the KM system to analyze such a process
involved a live event (MSEL) to stimulate an
operational scenario (terrorist attack). The thread
was the means to tie together the systems, the
information output, and the results of the test within
context.
The experimentation and analysis KM system
therein has two primary objectives: the creation of
knowledge through the experimentation process, and
the retrieval of knowledge as results or
recommendations that are forwarded to decision-
makers and/or into subsequent experimentation.
Information and knowledge is drawn from the
distributed systems and integration initiatives, plus
reach-back into supporting systems and archives.
Knowledge retrieval is essentially a reversal of
creation. The objective is not the usual meaning of
information retrieval via a search or a relational SQL
query, although both of these techniques, plus some
additional AI-based means, are used to help sort
experimentation results. Rather, the focus of
information or knowledge output from the KM
system is to answer a question.
At the lowest level, system logs and network data
are assessed to determine the performance of tested
systems against various integration scenarios and
network loading conditions. The advent of web
services and service-oriented architectures have
added increased emphasis to comprehensive
evaluation that includes the context in which the
tested system operated and communicated. Results
are derived at technical and operational levels.
Together it is possible to judge system performance
and interoperability within the tested context.
2.2 Application Integration
Enterprise integration is the study of an
organization, its business processes, and resources,
understanding how they are related to each other so
as to efficiently and effectively execute the
enterprise goals, focusing on organization, process,
WEBIST 2005 - INTERNET COMPUTING
186

application, data, and network (Nunez, Giachetti,
Truex, and Arteta, 2004). Modern knowledge
management must go beyond data mining and search
to provide collaboration as an intrinsic part of any
business process (Hawryszkiewycz, 2001).
Economies of scale are also realized, as data no
longer needs to be moved from independent systems
or data marts into the warehouse. As Belo (1999)
noted, significant effort involving enterprise
functional and operational analysis processes is
necessary in the migration of data into a warehouse.
Figure 1 provides a use-case of technical and
operational integration around high-level system
objectives, focusing on user requirements and
specifications for information retrieval. Generally,
data output from higher-level systems and databases
are summarized or encapsulated for the next level.
At the lower level additional database (systems)
information is integrated with the higher-level
information.
Previous research has noted that information
systems too often require specialized managerial
skills to interpret data and derive useful conclusions
and where the data volume is large a decision
support capability can help structure relationships
(Malhotra, 2001). In addition, cross-domain
enterprise architectures must support information
flows between internal and cross-enterprise
processes with a high level of automation while
remaining flexible and integrated (Martakos,
Kanellis, and Alexopoulou, 2004). Satisfaction
concerns were a driving force for the development
of the KM experimentation management system and,
we believe, effectively addressed.
Similar to the private sector, in the military
different information systems satisfy different
information needs. For example, a field sales person
may access specific product data for a sales call and
this may be represented in a sector of the corporate
enterprise system. Upon completion of the sale, and
similar sales from peer sales persons, a mid-level or
tactical information system [or sector in the
enterprise system] would synthesize and represent
these results for mid-level management. Sales across
a region would similarly be derived for top
management. Middle and top managers are
cognizant of the importance of information
resources to assure decision effectiveness (Carneiro,
1999).
Military systems have this same information flow
but in addition there are specialized systems at each
level. For example, a warfighter in the field would
require situational awareness specific to his or her
immediate surroundings (operational) while a mid-
level manager may require tactical data and situation
assessment at the theater level. At the strategic level
a commander’s situational assessment would require
understanding of issues at the tactical and
operational levels but not necessary the types of
information required at those levels. Thus, a
difference between military and corporate systems is
not only the additional specialized information
technologies at each level that still need to be
integrated but also the filtering mechanisms to refine
the output for appropriate audiences at each level,
and in each environmental context. The addition of
highly dynamic context is an additional variable.
The use-case therein establishes the necessary
systems integration relationships and in an
information-driven network-centric environment the
types of queries and linkages necessary for efficient
information retrieval. Each of the use-cases
represents one or more systems and the output of
those systems the information needed by both the
actor/user and the other systems employed at that
level (operational, tactical, strategic).
Figure 1: Use-case of information and systems integration showing levels of usage, data requirements and pertinent
databases (systems) for each level of usage
EXPERIMENTATION MANAGEMENT KNOWLEDGE SYSTEM
187

2.3 Data Integration and Analysis
Finally is the data processing for integration. Data
maintained in operational systems is not commonly
arranged for analytical needs or management
perspectives (Gonçalves, Lourenço and Belo, 2001).
Figure 2 represents a process through which the
previously discussed efforts achieve fruition.
Experimentation processes are mapped from
initiatives through to end-user or sponsor objectives.
This includes delineation of requirements and
specifications that lead to the initial series of
applications designed to aid in data processing.
These applications help structure the measures and
metrics that govern the systems integration testing
and evaluation. The KM system processes operate in
tandem to help assess processes and collect pertinent
data. This occurs throughout the experiment.
Next represented in the figure is the data scrub
that occurs after an experiment. Increasing much of
the initial processing is occurring parallel to
experimentation phases.The scrub provides structure
and contextualization that maps repository metadata
to the systems, projects or business areas (Ramírez,
Merayo, and Baizán, 2003). A significant number of
applications are employed to help in this process.
These applications are served to experimentation
and sponsor participants through an application
server with a portal interface and secured Internet
connection.
Finally is the evaluation process that leads to a
recommendation on the effectiveness of a particular
system or the efficiency of a systems integration
initiative. If successful the tested system [or system
of systems] is forwarded for possible inclusion in
upcoming acquisition cycles, if unsuccessful
dropped from consideration, if partially successful
returned to the experimentation cycle for follow-on
tests. The experiments generally evaluate for
timeframes 5-8 years into the future and the tested
environments simulate those time periods and the
technologies that will exist at those dates.
3 CONCLUSION
This paper presented a systems integration scenario
and supporting KM-based enterprise-class
experimentation management system operational in
military network-centric technology
experimentation. The discussion addressed both the
environment through which systems integration is
addressed and also the evaluation systems used to
monitor and assess the functioning of the systems
and initiatives. This led to an overview of output
processes for various levels of decision makers, use-
cases for the information requirements, and an
operational diagram of the knowledge management
systems and processes supporting the use-cases.
Comparisons were drawn between military and
corporate systems integration.
REFERENCES
Azevedo, A., and Sousa, J., 2000. Distributed order
planning for networked enterprises. In ICEIS’00, 2nd
International Conference on Enterprise Information
Systems. ICEIS Press.
Belo, O., 1999. Gathering the right information at the right
time: An agent based approach to data warehouses
loading processes. In ICEIS’99, 1st International
Conference on Enterprise Information Systems. ICEIS
Press.
Figure 2: Application server and repository for system and application integration
WEBIST 2005 - INTERNET COMPUTING
188

Carneiro, A., 1999. How decision support systems
influence management decision effectiveness? In
ICEIS’99, 1st International Conference on Enterprise
Information Systems. ICEIS Press.
Ericsson, F., 2001. Information systems integration. In
ICEIS’01, 3rd International Conference on Enterprise
Information Systems. ICEIS Press.
Gonçalves, J., Lourenço, A., and Belo, O., 2001.
Specializing agents on data integration in a data
warehousing system environment. In ICEIS’01, 3rd
International Conference on Enterprise Information
Systems. ICEIS Press.
Hawryszkiewycz, I., 2001. Developing the infrastructure
for knowledge based enterprises. In ICEIS’01, 3rd
International Conference on Enterprise Information
Systems. ICEIS Press.
Malhotra, V., 2001. A decision support system model for
subjective decisions. In ICEIS’01, 3rd International
Conference on Enterprise Information Systems. ICEIS
Press.
Martakos, D., Kanellis, P., and Alexopoulou, N., 2004.
Managing information flow dynamics with agile
enterprise architectures. In ICEIS’04, 6th International
Conference on Enterprise Information Systems. ICEIS
Press.
Nunez, A., Giachetti, R., Truex, D., and Arteta, B., 2004.
A framework for assessment of enterprise integration
approaches and technologies. In ICEIS’04, 6th
International Conference on Enterprise Information
Systems. ICEIS Press.
Ramírez, E., Merayo, F., and Baizán, C., 2003. Data
warehousing: A repository model for metadata storage
and retrieval based on the human information
processing. In ICEIS’03, 5th International Conference
on Enterprise Information Systems. ICEIS Press.
EXPERIMENTATION MANAGEMENT KNOWLEDGE SYSTEM
189