Assessing Technology Readiness for Artificial Intelligence and

Machine Learning based Innovations

Tobias Eljasik-Swoboda

, Christian Rathgeber

and Rainer Hasenauer

ONTEC AG, Ernst-Melchior-Gasse 24/DG, Vienna, Austria

Marketing Management Institute, Wirtschaftsuniversität Wien and Hi-Tech Center, Vienna, Austria

Keywords: Artificial Intelligence Readiness, Technology Readiness, Market Readiness, Innovation, Innovation

Management, Organizational Concepts and Best Practices, Data Privacy and Security, Data Management and

Quality, Data and Information Quality.

Abstract: Every innovation begins with an idea. To make this idea a valuable novelty worth investing in requires

identification, assessment and management of innovation projects under two primary aspects: The Market

Readiness Level (MRL) measures if there is actually a market willing to buy the envisioned product. The

Technology Readiness Level (TRL) measures the capability to produce the product. The

READINESSnavigator is a state of the art software tool that supports innovators and investors in managing

these aspects of innovation projects. The existing technology readiness levels neatly model the production of

physical goods but fall short in assessing data based products such as those based on Artificial Intelligence

(AI) and Machine Learning (ML). In this paper we describe our extension of the READINESSnavigator with

AI and ML relevant readiness levels and evaluate its usefulness in the context of 25 different AI projects.

1 INTRODUCTION

Innovation is an important foundation for

entrepreneurial success and has great economic

importance (Niever et al., 2019). But what do the

terms innovation, success and even invention actually

mean? According to Rogers (2003), “invention is the

process by which a new idea is discovered or created;

the adoption of an innovation is the process of using

an existing idea”. Another definition for invention

and innovation is that an invention is not necessarily

positive and can be purely imagined while an

innovation aims to create value (Merriam-Webster,

2019). According to Schumpeter (1939), an idea and

technical solution leads to an invention, which can

become an innovation by a successful market launch.

In short, an invention can be regarded as an idea while

an innovation strives to be a successful and profitable

invention.

There are multiple measures to define success and

profitability for innovations. The classic approach is

to measure success as maximum monetary return on

investment. Another more modern approach is to

consider the triple bottom line, which is defined as the

https://orcid.org/0000-0003-2464-8461

tradeoffs between economic drivers (the monetary

return on investment), environmental impact and

social impact of the innovation (Hasenauer et al.,

2016). Examples for data based Machine Learning

(ML)- and Aritificial Intelligence (AI) innovation

projects aiming for a triple bottom line include Social

Assistive Robots for Elderly Care (SAR) and Sensor

Enabled Affective Computing for Enhancing Medical

Care (SENSECARE) (Belviso et al., 2018),

(Donovan et al., 2018), (Healy et al., 2018). SAR

aims to develop caregiving robots for the elderly,

SENSECARE aims to monitor dementia patients

using AI so that they can continue living in their home

and help can be alerted if necessary. Wellbeing of

elderly or dementia patients are important aspects in

these innovation projects, not solely the monetary

return on investment. Lepak et al. (2007) aim to

define value creation and have shown, that the

concept is heterogeneously used depending on the

academic field of study. Creators and users of value

can differ and stretch from society, over organizations

to individuals which all have different value creation

and capture processes.

Eljasik-Swoboda, T., Rathgeber, C. and Hasenauer, R.

Assessing Technology Readiness for Artiﬁcial Intelligence and Machine Learning based Innovations.

DOI: 10.5220/0007946802810288

In Proceedings of the 8th International Conference on Data Science, Technology and Applications (DATA 2019), pages 281-288

ISBN: 978-989-758-377-3

281

However one decides to measure success, in order

to achieve it, innovations must be managed and

assessed with regards to their readiness (Hasenauer et

al., 2015). Investors are highly unlikely to provide

capital for innovations that are not ready. To facilitate

informed decisions, two dimensions of readiness need

to be assessed: The Technology Readiness Level

(TRL) expresses the degree of readiness for a

technology while the Market Readiness Level (MRL)

measures the maturity of a given need in the market

considering potential obstacles (Sadin et al.,

1989)(Dent and Pettit, 2011).

The READINESSnavigator is a software product

that addresses the identification, assessment,

management and protection of investments through

analyzing innovations for their triple bottom line by

assessing their TRL and MRL (Ontec, 2019). Its

underlying methodology has been used in the

assessment of 57 startups and 26 high-tech products.

Hasenauer et al. (2016) have shown that startups that

used the READINESSnavigator’s underlying

readiness assessment method had a significantly

higher success rate than startups not following this

approach. More details about the method and tool can

be found in section 2. Even though the underlying

TRL and MRL models are market and technology

versatile, they do not express the specific problems

associated with innovations in the field of Artificial

Intelligence (AI) and Machine Learning (ML). To

overcome this shortcoming, our research goals are to

identify and specify levels of readiness for AI and

ML. We subsequently implement this model as

extension for the READINESSnavigator and use it in

the assessment of 25 AI innovations to

experimentally evaluate its usefulness.

To do so, this paper is structured as follows:

Section two describes the relevant state of the art in

science and technology for our endeavour. Section

three describes our AI readiness model, which we

implemented as extension of the

READINESSnavigator. Section four describes our

observations in using the READINESSnavigator for

AI while section five finishes our contribution by

describing conclusions drawn from our observations.

2 STATE OF THE ART

The idea to model readiness of technologies was

originally conceived by NASA in 1974 and formally

defined in 1989 (Sadin et al., 1989). Dent and Pettit

(2011) adopted the concept to include market

readiness. Hasenauer et al. (2015) built on this to

propose a framework to manage technology push

which was extended to also address the triple bottom

line (Hasenauer et al., 2016). The

READINESSnavigator was developed by Ontec in

colaboration with Hasenauer et al. to aid in

documenting and accessing innovations and their

respective readiness levels.

While NASA uses nine levels of technology

readiness, from basic idea to flight proven on

missions, Hasenauer et al. (2015) define three

dimensions of technology readiness which are each

expressed in nine levels: Intellectual property

readiness (IPR-RL) expresses if the underlying

intellectual property has been protected, integration

readiness (INT-RL) expresses if the technology can

be integrated where needed by the envisioned

customers while manufacturing readiness (MAN-

RL) expresses if the innovation can actually be

produced.

The market readiness is likewise split into four

dimensions. The competitive supply readiness

(COM-RL) expresses if competitors have similar

products and how much the innovator is aware of -

and has evaluated them. The demand readiness

(DEM-RL) assesses if there is a demand for the

product. The customer readiness (CUS-RL)

expresses if a customer is ready to use and adopt the

product while the product readiness (PRO-RL)

expresses if the product itself is ready for widespread

use. Figure 1 illustrates a visualization module within

the READINESSnavigator. In this example, an

Innovation has a very high MAN-RL but poor IPR-

RL and mediocre MRL levels. The

READINESSnavigator highlights fields of action and

shows the necessity to address issues in certain fields

to raise overall readiness, for example by addressing

intellectual property rights issues. Hasenauer et al.

(2016) have shown a success optimizing development

curve in which market readiness always is one or two

levels above technologry readiness during product

development. The intuition for this curve is simple: If

potential customers are willing to purchase an

innovation, further technology development can be

financed by this revenue. The READINESSnavigator

compares an innovation’s current development with

the success optimizing curve to highlight necessary

next steps.

As part of technology readiness, manufacturing

readiness strongly focuses on the capability to

produce physical goods. As the benefits of AI and ML

are much more data and information based, their

readiness comes with an additional set of challenges.

There is some work in assessing AI readiness by

multiple organisations. Intel (2019) published a

model for AI readiness that assesses organisations.

DATA 2019 - 8th International Conference on Data Science, Technology and Applications

282

Figure 1: READINESSnavigator visualization.

They address three dimensions of AI readiness:

Foundational AI readiness expresses if the

appropriate infrastructure (hard-, and software) is

available. Operational AI readiness expresses if the

necessary management mechanisms are in place.

Transformational AI readiness expresses how ready

an organisation is to maximize the value it obtains

from applying AI. According to Intel (2019), there are

three fundamental levels for companies in regard to

AI readiness: New to AI, ready to scale up, and

broadly implementing.

Capgemini Consulting has created an AI

readiness benchmark for countries that measures the

countries competitiveness regarding AI in terms of

institutional readiness, IT maturity and available IT

skills (Tinholt et al., 2018). Neither Intel’s nor

Capgemini’s model focuses on the necessary

readiness dimensions to innovate. Obviously one can

see a start-up as a company which needs to have AI

readiness in Intel’s sense of the term in order to have

any technology readiness. The ability to create

beneficial innovations goes beyond Intel’s three

dimensions as shown in section three of this paper.

Big Data is a related field to AI and ML, which

has many overlaps. In order to better understand Big

Data endeavours, Kaufmann (2016) proposed the Big

Data Management Meta Model (BDMcube). The

BDMcube is based on epistemology and sees Big

Data as continuous cycle in which the results of data

analysis influence the world (effectuation). Physical

signals are gathered (datafication) to be centrally

stored (data integration), analysed and interacted

with resulting in a new innovation or decision support

for any enterprise. Based on this value cycle,

Kaufmann et al. (2017) created and evaluated the Big

Data Management Canvas (BDMC). The BDMC

takes the five cycle stages of the BDMcube and

assigns each of them a technical and a business

dimension. Each of these 10 dimensions represents a

field of action for any big data endeavour. On top of

these 10 fields, there are two meta-fields of data

intelligence which Kaufmann et al. define as the

ability to execute in terms of available skills and

infrastructure. Both data intelligence fields clearly

have connections to Intel’s and Capgemini’s views on

AI readiness: Without the necessary skills or

equipment one cannot carry out any Big Data or AI

Assessing Technology Readiness for Artiﬁcial Intelligence and Machine Learning based Innovations

283

endeavour. The epistemological value cycle is

especially interesting because of nescience. In

information science, nescience is the unawareness of

an information need. One could refer to it as unknown

unknown. Ignorance on the other hand is knowingly

not having information, which in contrast can be

referred to as a known unkown (Weber et al., 2018).

The cycle of creating knowledge that leads to a new

information need is neatly modelled by the BDMC.

3 MODEL

The READINESSnavigator’s technology readiness

currently assesses three fields of technology

readiness. Intellectual property readiness and

integration readiness are equally as important for AI

or ML based innovations as for any other. The

manufacturing readiness however is not directly

applicable, as the challenges of physical production

are often times out of scope for ML or AI endeavours.

Instead of manufacturing readiness, we propose six

AI specific readiness dimensions, split into two main

categories of AI readiness and data readiness. We

base these dimensions on the existing state of the art

by (Sadin et al., 1989), Hasenauer et al. (2016),

Kaufmann et al. (2017), Tinholt et al., (2018) and

Intel (2019) as well as five years of practical

experience in implementing ML and AI based

systems.

It is noteworthy that readiness dimensions are

optional within the READINESSnavigator. This

means that if one field of readiness is superfluous for

a specific innovation, one can always skip assessing

it. The overall readiness level of an innovation is its

lowest readiness level in one dimension (see Figure

1). Levels within one dimension are always strictly

ordered. This means that an innovation cannot reach

a higher level if it has not fulfilled all requirements of

the previous levels. Currently, all readiness fields

have exactly nine levels.

Figure 2 maps AI- and data readiness levels onto

the BDMC’s fields of action. It also illustrates

important links between readiness levels. Different

from the existing technology readiness model, the

individual levels of different fields can have

prerequisites. One can for example not run the

envisioned algorithm on relevant real-world data if

one doesn’t have access to this data.

The first important readiness level for AI is

specification readiness. For now, it has six different

levels that express how clearly the use case for AI is

defined. These range from having a vague idea of

applying AI to a complete specification. An important

intermediate level is level five, which defines success

criteria for the AI innovation. These are important for

many other fields, for example when defining

effectiveness measures for machine learning based

applications. Figure 2 illustrates this with the arrow

from specification readiness to algorithmic

readiness. Having only six specification

readiness levels spotlights, that the computation

of an innovation’s overall readiness level needs

to normalize r eadiness levels in order to generate

Figure 2: Mapping of our proposed readiness levels on Kaufmann et al.’s BDMC.

DATA 2019 - 8th International Conference on Data Science, Technology and Applications

284

meaningful progress graphs similar to those shown in

figure 1. We have abstained from inventing redundant

readiness levels just to get up to nine levels.

Specification readiness is related to the BDMC’s

fields of effectuation and interaction by assessing if

there is a specification of what the invention should

achieve how users interact with it and how its success

can be measured. Also similar to the BDMC method,

the specification influences the analytics field. We

refer to readiness in the BDMC fields of analytics as

algorithmic readiness. It has eight levels which

model stages from knowing no algorithmic approach

to solve the issue at hand to using algorithms for this

specific problem in production. In between the

algorithm family has to be identified. Possible

algorithm families include classification, regression,

clustering, time-series analysis, structural equation

models, fuzzy logic applications or symbolic

knowledge reasoning among many others. This

obviously depends on the goals specified within the

specification readiness. Stage four of algorithmic

readiness expresses the selection of effectiveness

measures. E.g. for the classification task, the

difference between precision and recall can have

massive impacts on the result. Other important levels

of algorithmic readiness are level 5, indicating that

the algorithm is being evaluated using real world data

and level 6, indicating that the hyper-parameters are

tuned. Hyper-parameter tuning does not necessarily

yield good results when the data readiness is poor,

because one potentially overfits a solution to

inaccurate data. Additionally, low quality data can

lack important features resulting in poor system

convergence with extreme computation times. To

reach levels > 4 of algorithmic readiness, real world

quality data must be available.

This creates a link to the main category of data

readiness. This field is related to Intel’s operational

AI readiness and Capgemini’s IT maturity fields in

the sense that it measures how accessible and

understood the necessary data for the envisioned

analytics are. As such, we place it in Kaufmann et

al.’s (2017) fields of data integration and

datafication. Because readiness levels are supposed

to be mono-dimensional, we split up the main

category into four individual readiness dimensions.

The relevance of all four fields depends on the

specification readiness and algorithm readiness.

Data existence readiness expresses if the required

data for the envisioned algorithm actually exists.

While this could be expressed in two levels, we opted

for nine different levels taking the possibility to

gather non-existing data into account. These nine

data existence levels closely mirror Sadin et al.’s

(1989) original NASA technology readiness levels,

substituting the readiness of flight hardware with that

of data gathering technology so that level one implies

that no data exists and one is unaware of a method to

gather it while level nine reflects existing data and a

productive data gathering technology and process.

Data format and quality readiness reflects how

well the existing data format is understood and of

what quality the available data is. Understanding the

data format is of high importance to create any feature

extraction scheme required for ML based algorithms.

Having quality data is equally as important so that the

resulting innovation actually fulfils its specified

goals. Our model expresses these issues using 5 levels

that identify if the format is understood, a method to

measure quality is identified and data actually is of

high quality. Low data quality can manifest itself in

multiple ways, such as pragmatic quality, semantic

quality, syntactic quality and social quality (Shanks

and Corbitt, 1999). While low pragmatic, semantic

and syntactic quality point to irregularities in the data

model and entries, low social quality data can reflect

a high degree of biases If a machine learns to simulate

these biases, it automatically creates biased results.

Biased AI systems based on their underlying data are

problematic as Caliskan et al. (2017), Sweeney

(2013) and Holstein et al. (2019) among many others

point out. Such a bias doesn’t need to be exclusively

social. Tasks such as fraud detection, text

classification and detection of oil spills in satellite

images oftentimes work with 1 positive example out

of 100,000 negative examples (Chawla et al., 2002).

If such imbalance is the case, it must be understood

and addressed in the AI system, which is modelled by

our readiness levels.

Data legal readiness is another important aspect

modelling the legality of data usage. The General

Data Protection Regulation (GDPR) aims at

protecting the personal data of EU citizens (EU

2016). It is exemplary for multiple pieces of

legislature that regulate how and by whom data can

be used. If one wants to base an innovation on

processing data, one needs to be sure that it is legal to

do so. We model this circumstance using eight

different stages. At level one, the legality of data

usage is completely unclear where as at level eight

there is a Supreme Court ruling explicitly allowing

the use of this kind of data. In our model, one does

not need to take a lawsuit through all instances before

launching an innovation. One should however be

aware of potential risks along the way. Important

intermediate steps are the identification if natural

person’s personal data is used because it is much

more protected than other types of data. If this is the

Assessing Technology Readiness for Artiﬁcial Intelligence and Machine Learning based Innovations

285

case, at least within the EU the extra requirement of

being capable to explain the AI’s results manifest as

the GDPR states that every EU citizen has the right of

explanation why a specific result was generated. This

is also expressed within our eight data legal readiness

levels. An important aspect of every product launch

is to perform a Freedom to Operate (FTO) analysis,

which is a patent information process that determines

if an innovation does not infringe on any existing

patents (European Patent Office, 2016). In the case of

critical data being used as resource for an innovation,

a similar analysis must occur to reach high data legal

readiness.

Expert knowledge readiness is our final group of

readiness levels. It is of particular importance if a

symbolic AI is implemented. In contrast to a machine

learning based AI, a symbolic AI explicitly models

rules in human-readable form (Haugeland, 1985). If

one plans to implement a symbolic AI, one needs to

capture the necessary domain knowledge from

relevant domain experts. Some degree of explicitly

modelled domain knowledge might also be required

for ML based AI innovations for example for

labelling training data. Neural-symbolic integration is

the act of constructing hybrid machine learning /

symbolic systems (Bader and Hitzler, 2005). No

matter what kind of ML or AI based innovation is

implemented, checking for access to the required

expert knowledge is important to ascertain the

innovation’s readiness. An expert knowledge

readiness level of one indicates that the knowledge

domain is not yet identified let alone any necessary

knowledge captured in a meaningful way. In contrast

at level 7, high quality (see data format and quality

readiness) expert knowledge is captured in a machine

readable fashion. Important intermediate steps are the

identification of appropriate experts and signing

collaboration contracts with them before capturing

their knowledge.

In its current version, the READINESSnavigator

models readiness levels as entries within a relational

database. We implemented our prototype by

importing our proposed readiness levels into that

database. As of now, the READINESSnavigator

lacks two features our model ultimately requires: The

capability to model interdependencies between

readiness levels and normalization for readiness

categories with less than nine levels.

4 OBSERVATIONS

We used the READINESSnavigator AI extension on

25 ideas to determine their potential for becoming

successful innovations. At this point in time, none of

Figure 3: READINESSnavigator showing all proposed readiness levels.

DATA 2019 - 8th International Conference on Data Science, Technology and Applications

286

these ideas has been fully implemented and marketed.

As we also have no control group capable of

measuring the success of AI innovations not using the

READINESSnavigator, we cannot yet reliably prove

its positive impact on the innovation process.

The following observations were made while

working with the READINESSnavigator for AI:

1. The READINESSnavigator prevents that

important aspects during the innovation process are

overlooked as it demands assessment.

2. The READINESSnavigator helps to steer

innovation projects by highlighting weaknesses

required for a successful market launch.

3. During this evaluation, the

READINESSnavigator for AI was used in a

predominately technical company. This means, that

the more technology dependent readiness levels were

typically higher than those of market readiness,

intellectual property rights readiness and data legal

readiness. These can require legal counsel and market

research, which the company would need to

outsource thus creating additional external cost. This

effect can be considered as a structural bias as

engineering firms usually excel at engineering tasks

while legal or marketing firms excel at their specific

tasks. The aforementioned bias should be taken into

account when planning, staffing and managing

innovation projects.

Figure 3 shows a screenshot of the current version

of the Readiness Navigator. It shows its current lack

of normalizing readiness levels. This especially

impacts the data readiness diagrams, where the

available levels range from 5 (data format and quality

readiness) to 9 (data existence readiness).

From the 25 ideas used to evaluate the

READINESSnavigator for AI, one is closer to market

introduction than the remaining 24. When this

specific innovation was first assessed, its manually

normalized AI readiness lacked one level behind its

technology readiness. The reason for this was, that a

concrete learning target has not been defined reducing

its specification readiness. Similarly, market

readiness was one level below the optimal curve,

requiring the definition of specific product options in

order to raise its product readiness. Both issues were

remedied before the subsequent implementation

began. During technology development, the

READINESSnavigator was used as a scenario-

modelling tool to see where the readiness levels

would be after development if no market readiness

related activities were undertaken. In this scenario,

after successful development, normalized

technology-, data-, and AI readiness are at levels >6.

To be on Hasenauer et al.’s (2016) optimal curve,

market readiness should be >7. This created an

additional list of work packages to be addressed in

parallel to the technology development.

This specific project highlights our third

observation: Technical personnel tends to dismiss the

necessity of marketing and sales related activities.

The READINESSnavigator for AI helped to raise

awareness and lead to the initialization of the required

work packages. Additionally, the

READINESSnavigator’s assessment was used to

convince investors, that the development is on track

and likely successful.

5 CONCLUSIONS

We started work on the READINESSnavigator AI

extension because we are convinced of its positive

impact on ML or AI innovation projects. This

conviction comes from the basic

READINESSnavigator’s significant positive impact

on other high-technology innovation projects and the

solid literature foundation of our proposed ML and AI

readiness levels. Using this tool, we evaluated a

backlog of 25 potential AI based innovations to

determine which have the highest potential for

success. At the point of writing this paper, the most

promising innovation was nearing market

introduction. The READINESSnavigator

externalizes expert knowledge about the innovation

process to help at every phase of it. This way it

functions as automated innovation coach/mentor.

The READINESSnavigator highlights

weaknesses in plans. For instance a system can be at

a highly algorithmic ready level but lacking legal

prerequisites and potential customers if the market

readiness is too low.

In future works we intend to either implement or

stop work on the innovation projects in our backlog.

Stopping work with a too low success probability is

equally as much a success for the

READINESSnavigator for AI as successful projects.

Using a control group of innovation projects not using

the READINESSnavigator for AI can prove its

usefulness in future works.

REFERENCES

Bader, S., Hitzler, P. (2005). Dimensions of neural-

symbolic integration - a structured survey.

arXiv:cs/0511042 [cs.AI]

Belviso, C., Hasenauer, R., Bechthold, U., 2018. Social

Assistive Robots Diffusion in Elderly Care: A Pre-

Assessing Technology Readiness for Artiﬁcial Intelligence and Machine Learning based Innovations

287

Adoption Study through Agent-Based Modeling. In:

Journal of Strategic Innovation and Sustainability, vol.

13(5), ISSN 1718-2077, p.58-75

Caliskan, A., Bryson, J. J., Narayanan, A., 2017. Semantics

derived automatically from language corpora contain

human-like biases. In: Science 14. April 2017, Vol.

356, Issue 6334, pp. 183-186, DOI: 10.1126/

science.aal4230

Chawla, N. V., Bowyer, K. W., Hall, L. O., Kegelmeyer,

W. P., 2002: SMOTE: Synthetic Minority Over-

sampling Technique, In: Journal of Artificial

Intelligence Research 16, pp. 321-357

Dent, D., Pettit, B., 2011. Technology and Market

Readiness Levels, White Paper Dent Associates

Donovan, R., Healy, M., Zheng, H., Engel, F., Vu, B.,

Fuchs, M., Walsh, P., Hemmje, M., Mc Kevitt, P. 2018.

SenseCare: using automatic emotional analysis to

provide effective tools for supporting wellbeing. In:

Proceedings of the 3

International Workshop on

Affective Computing in Biomedicine & Healthcare

(ACBH-2018), IEEE International Conference on

Bioinformatics and Biomedicine (BIBM-2018), pp.

2682-2687

EU 2016. Regulation (EU) 2016/679 of the European

Parliament and of the Council of 27 April 2016 on the

protection of natural persons with regard to the

processing of personal data and on the free movement

of such data, and repealing Directive 95/46/EC

(General Data Protection Regulation) (Text with EEA

relevante); OJL 119, 4.5.2016, pp. 1-88.

European Patent Office 2016. FTO analysis: a king among

patent searches. In: Patent Information News 1/2016

Hasenauer, R., Weber, C. M., Filo, P., Orgonas, J., 2015.

Managing Technology Push through Marketing

Testbeds: The Case of the Hi-Tech Center in Vienna,

Austria. In: MANAGEMENT OF THE TECHNOLOGY

AGE Proceedings of PICMET 2015, IEEE Catalog

Number: CFP15766-USB PICMET ISBN USB: 978-1-

890843-32-8, pp. 99 – 127

Hasenauer R., Gschöpf, A., Weber, C., 2016. Technology

Readiness, Market Readiness and the Triple Bottom

Line: An Empirical Analysis of Innovating Startups in

an Incubator, In: Proceedings of PICMET 2016 pp.

1387-1428

Haugenland, J., 1985. Artificial Intelligence: The Very Idea,

Cambridge, Mass: MIT Press, ISBN 0-262-08153-9

Healy, M., Donovan, R., Walsh, P., Zheng, H.: A machine

learning emotion detection platform to support

affective wellbeing. In: IEEE International Conference

on Bioinformatics and Biomedicine (BIBM-2018), pp.

2694-2700

Holstein, K., Vaughan, J. W., Daumé H., Dudík, M.,

Wallach, H., 2019. Improving fairness in machine

learning systems: What do industry practitioners need?,

arXiv:1812.05239v2 [cs.HC]

Intel, 2019, The AI Readiness Model, Judging an

Organization’s Ability to Generate Business Value

from Artifical Intelligence, Retrieved March 28, 2019

from https://www.intel.com/content/dam/www/public/

us/en/documents/white-papers/ai-readiness-model-whi

tepaper.pdf

Kaufmann, M., 2016. A Reference Model For Big Data

Management, research report, Faculty of Mathematics

and Computer Science, University of Hagen, Germany

Kaufmann, M., Eljasik-Swoboda, T., Nawroth, C.,

Berwind, K., Bornschlegl, M., Hemmje, M., 2017. The

Big Data Management Canvas Method, In:

Proceedings of the 6

international conference on data

science, technology and applications (DATA 2017),

DOI: 10.5220/0006397101490156

Lepak, D. P., Smith, K. G., Taylor, M. S., 2007. Value

Creation and Value Capture: A Multilevel Perspective.

In: Academy of Management Review, vol. 32, no. 1, pp.

180-194

Merriam-Webster.com, 2019, “innovation”. Retrieved

March 27, 2019 from https://www.merriam-

webster.com/dictionary/innovation#note-1

Niever, M., Marthaler, F., Kosejian, M., Hahn, C., Albers,

A., 2019. The Future of Innovation Coaching in Product

Engineering: A Systematic Approach to Deriving the

Future Competence Profile and its Development

through Strategic Potential Identification, In:

Proceedings of the Collaborative European Research

Conference (CERC 2019), pp. 299-310

Ontec AG, The READINESSnavigator. Retrieved April 10,

2019 from https://ontec.at/the-readinessnavigator/

Rogers, E. M., 2003. Diffusion of innovations, Free Press.

New York, 5

edition

Sadin, S. R., Povinelli, F. P., Rosen, R., 1989. The NASA

technology push towards future space mission systems.

In: Acta Astronautica, Volume 20, p. 73-77

Schumpeter, J., 1939. Business Cycles. A Theoretical,

Historical and Statistical Analysis of the Capitalist

Process, New York/London

Shanks, G., Corbitt, B., 1999. Understanding Data Quality:

Social and Cultural Aspects, In: Proceedings of the 10

Australasian Conference on Information Systems, pp.

785 - 797

Sweeney, L, 2013. Discrimination in Online Ad Delivery,

arXiv:1301.6822v1 [cs]

Tinholt, D., van Niel, E., van Kraaij, C., Knödler, M., 2018.

Artificial Intelligence Benchmark. Retrieved March 29,

2019 from https://www.capgemini.com/wp-

content/uploads/2018/07/AI-Readiness-Benchmark-

POV.pdf

Weber, C. M., Hasenauer, R. P., Mayande, N. V., 2018.

Toward a Pragmatic Theory for Managing Nescience.

In: International Journal of Innovation and Technology

Management, Vol. 15, No. 5, DOI: 10.1142/

S0219877018500451

DATA 2019 - 8th International Conference on Data Science, Technology and Applications

288