Implementing Learning Analytic Systems in Educational Institutions:

The Importance of Transparent Information for User Acceptance

Lynn Schmodde

, Marius Wehner

, Paulin Zahn

and Jasmin Georgy

Chair of Business Administration, Esp. Digital Management and Digital Work, Heinrich-Heine-University,

Universitätsstr. 1, Düsseldorf, Germany

Keywords: Education, Learning Analytics, Fairness, User Perceptions.

Abstract: Learning analytics (LA) systems have to meet high standards to ensure effective implementation in

educational institutions, but knowledge about which factors play the most important role for users is limited.

With two studies, we investigate the importance of different attributes of LA systems (Study 1) and the

influence of different information fragments (i.e., benefits, drawbacks, and auditing information of the LA

system) on users’ (i.e., students and teachers) perceived fairness and attractiveness of the institution (Study

2). In Study 1, we conducted a choice-based conjoint analysis to examine the relative importance of fairness,

accuracy, audits, and methods of use. Our results show that both students and teachers consider fairness to be

the most important feature. In Study 2, we conducted an experimental video vignette study to examine how

different fragments of information influence perceived fairness (i.e., informational justice) and attractiveness

of the institution. We show that more information increases students’ and teachers’ acceptance, even when

potential drawbacks are communicated, although the results of the teacher sample are less pronounced overall.

1 INTRODUCTION

In recent years, digitalization and the implementation

of algorithm-based systems increased rapidly

(Fischer et al., 2023; Mai et al., 2022). This

development can also be observed in educational

institutions (Winter et al., 2021). Schools and

universities started to offer online learning, resulting

in teachers having to assess student performance

online. Learning analytics (LA) systems can improve

this process and support teachers and students to

make the learning process more effective (Martin and

Ndoye, 2016). LA involves the process of measuring,

collecting, analyzing and reporting data about

learners and their environment (Siemens and Baker,

2012). The data is used to improve understanding of

learning processes and to optimize both the learning

experience itself and the learning environment in

which it takes place (Ifenthaler and Drachsler, 2020).

LA systems can measure learners’ activity and

consistency in using the learning platform, but also

https://orcid.org/0000-0001-9242-6110

https://orcid.org/0000-0002-1932-3155

https://orcid.org/0009-0003-3797-110X

https://orcid.org/0009-0006-0234-0553

provide information on how well students complete

their exercises and prepare for exams (Mai et al.,

2022). Based on this, predictions are made and LA

systems can help identify at-risk learners and support

learning success (Siemens and Long, 2011).

Despite this, LA systems are not yet widespread,

particularly in Germany, and the process of

implementing a LA system is not well understood

(Ifenthaler et al., 2021). Discussions often include

concerns about the fairness and accuracy of the

analysis (Roberts et al., 2016). Although algorithms

and the use of artificial intelligence (AI) can increase

the objectivity of decisions (Kaibel et al., 2019),

biased training datasets can lead to unfair tendencies

and systematically discriminate against certain

groups (Köchling and Wehner, 2020; Greller and

Drachsler, 2012).

Exploring factors that mitigate perceived

uncertainties is crucial for safe implementation of

these systems. Current literature highlights a gap in

understanding the conditions under which users are

Schmodde, L., Wehner, M., Zahn, P. and Georgy, J.

Implementing Learning Analytic Systems in Educational Institutions: The Importance of Transparent Information for User Acceptance.

DOI: 10.5220/0012555000003693

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 16th International Conference on Computer Supported Education (CSEDU 2024) - Volume 1, pages 297-304

ISBN: 978-989-758-697-2; ISSN: 2184-5026

297

receptive to learning platforms with integrated LA

(Schumacher and Ifenthaler, 2018). Furthermore,

little is known about how educational institutions can

enhance users’ perceptions of fairness in LA systems

(Roberts et al., 2016).

Hence, first, it is important to know what LA

systems should look like from the users’ perspectives

and to assess how to minimize users’ concerns about

fairness and accuracy, thereby giving credit to a user-

centered design and making users feel more

comfortable implementing LA systems in their

educational routines (Lawson et al., 2016). Based on

this knowledge, systems can be designed to be

efficient and supportive. Yet the significance of

different aspects of LA systems, including audits, has

not been thoroughly studied (Schumacher and

Ifenthaler, 2018). Therefore, we investigate the

relative importance of auditing LA systems for users

in comparison to other features, such as fairness,

accuracy, and methods of use, with a choice-based

conjoint analysis (Study 1).

Second, in Study 2, we conducted an experimental

vignette study to examine the role of transparent

communication of a newly introduced LA system for

users’ perceived fairness and organizational

attractiveness. Transparent information is crucial for

introducing new technologies to convey the context

of their usage and functionality (Lawson et al., 2016).

Following the reasoning of justice theory, the

communication of information creates a sense of

being actively involved in decisions (Colquitt, 2001).

This can foster a more favorable assessment of

emerging technologies, thereby promoting increased

acceptance of LA systems within educational settings

(Greenberg, 1994).

2 BACKGROUND AND THEORY

Educational learning platforms gather log data

associated with learners’ behaviors and actions

(Almosallam and Ouertani, 2014), enabling

automated analysis and enhancing information

accessibility about learners (Mai et al., 2022). This

information can be used to promote learner reflection

or to predict learning success (Greller and Drachsler,

2012). LA facilitates information-based interventions

for students, which enables adaptive and personalized

learning. In this study, we only focus on algorithmic

LA systems that can predict learners’ success.

Although LA systems offer various benefits for

enhancing learning processes, they come with

challenges that might decrease users’ perceptions of

fairness and their acceptance of the system.

In education, data about learners is sensitive,

which is the reason why the use of data is a critical

issue (Khalil and Ebner, 2015). Furthermore, LA

systems are trained by existing data. If this training

data is biased LA systems replicate or even reinforce

biases (Mehrabi et al., 2022), resulting in potential

discrimination based on gender, origin, or religion

(Köchling et al., 2021). Hence, it is crucial to avoid

biases from a technical perspective, but users also

need to have the feeling of a fairly treating system.

The acceptance of LA systems can also depend on

the accuracy of the algorithm, as errors can also arise

from inaccurate data used by algorithms and lead to

errors in user evaluation (Mehrabi et al., 2022). These

errors are often unnoticed and therefore cannot be

reported (Kim, 2017).

Due to these concerns regarding the use of LA

systems, it is useful to conduct an audit. Audits aid in

early problem detection and bias prevention (Calders

and Zliobaite, 2013; Riazy et al., 2020; Rzepka et al.,

2022). A regular audit is also recommended in the

European Artificial Intelligence Act (European

Commission, 2021).

To measure the effects of addressing the concerns

on the user perceptions, we followed the reasoning of

justice theory and measured perceived fairness by

using one dimension of justice (Starke et al., 2022).

Justice refers to “perceptions of fairness in decision-

making” (Colquitt and Rodell, 2011) and can be

categorized into four dimensions (Colquitt, 2001). In

our research, the dimension of informational justice

was of particular interest, which describes the extent

to which justification and truthfulness are provided

during procedures (Colquitt and Rodell, 2011). We

assume that communicating information about LA

systems will make users feel more involved in

decisions (Colquitt, 2001). In addition, research by

Shapiro et al. (1994) found that detailed explanations

are perceived as more satisfying. Accordingly,

information about the LA system can lead to the

development of trustworthiness (Colquitt and Rodell,

2011) and, hence, acceptance of the system. Further,

Langer et al. (2018) found that reactions to

technologies are positively influenced by more

information.

Based on these assumptions, we propose that

transparent communication of relevant information

enhances the acceptance of LA systems. Furthermore,

we assume that perceived fairness positively

influences students’ and teachers’ reactions to LA

systems and, in turn, increases institutional

attractiveness.

In this study, institutional attractiveness reflects

individuals’ attitudes towards the educational

CSEDU 2024 - 16th International Conference on Computer Supported Education

298

institution (Chapman et al., 2005), which is a key

precondition for intentions and behavior according to

the theory of planned behavior (Ajzen, 1991).

University students and teachers have freedom in

selecting their educational institution and employer.

If a university lacks characteristics and support

deemed suitable by students and teachers, they may

find this university less attractive and choose another

university. Thus, attractiveness of an institution is

crucial to remain a competitive university (Platz and

Holtbrügge, 2016).

This leads to our two research questions

concerning the implementation of a new LA system:

First, which factors of LA systems do students and

teachers rate to be important? Second, how do these

information elements influence the perceived fairness

and the institutional attractiveness of a university?

3 STUDY 1

3.1 Study Design and Sample

We conducted our Study 1 and 2 between autumn

2022 and spring 2023. In Study 1, we employed a

choice-based conjoint analysis, a popular

experimental research design for evaluating the

importance of certain factors due to its similarity to

real-life situations and ability to elicit spontaneous

decision-making (Balderjahn et al., 2009; Karren and

Barringer, 2002; Shepherd and Zacharakis, 1999). In

our study, participants were asked to imagine their

educational institution implementing a LA system.

They chose between two systems in several rounds,

selecting their preferred option. The systems differed

in randomized attributes, visually represented with

icons and pictures for clarity (see Table 1).

The first attribute methods of use varied across

two levels, therefore differentiating between LA

systems that give a grade recommendation and the

ones that additionally forecast the learning success.

The type of audit differed in the verification of the

system within the educational institution, by third

parties or no verification at all. Fairness of the

analysis included an equal treatment of all learners

regardless of origin, gender or religion or an

accidental unequal treatment. The fourth attribute

described the accuracy of the analysis, which varied

between the correct evaluations of seven, eight or

nine out of ten learners.

The specifications of attributes allow 36 (2 x 2 x

3 x 3) possible combinations of a LA system. As more

than 20 rounds can overwhelm the participants, we

used a fractional factorial design and applied twelve

rounds in our survey (Balderjahn et al., 2009). We

tested the number of rounds with the preliminary

counting test in Sawtooth based on 300 versions.

For the recruitment of our sample, we employed a

European ISO-certified online sampling provider

(ISO 20252:2019). Our sample consisted of 440

participants from Germany, including 212 teachers

age

= 44.82) and 228 students (M

age

= 21.23). 70%

of the sample were female. We made a distinction

between teachers and students because teachers

benefit from the results of the LA process and are

expected to take action based on those results, while

students provide the data that is analyzed (Greller and

Drachsler, 2012). This highlights the need to

understand the preferences of both groups.

3.2 Results

To analyze the results, we used the Sawtooth

Software Lighthouse Studio 9.15.0 and evaluated our

data using Hierarchical Bayes (HB) model and

counting analysis. While the HB model shows the

importance (I) of the attributes in a ranking, the

counting analysis rates the levels within the attributes

(Orme and Sawtooth Software, Inc., 2002).

All groups (i.e., students and teachers) rated

fairness (I = 34.95[SD = 14.47]) as the most important

attribute of a LA system. The type of audit (I =

28.87[SD = 11.61]) was ranked second, followed by

accuracy of the analysis (I = 28.23[SD = 10.61]). The

least important attribute was the method of use (I =

7.95[SD = 8.42]).

The counting analysis revealed the preferred

levels of the different attributes. Table 1 shows the

results of the level being selected of the times it

occurred. All groups preferred a system with equal

treatment of all learners regardless of origin, gender,

or religion (0.70), audits verified by third parties

(0.59), the correct evaluation of nine out of ten

learners (0.62) as well as LA systems that provides

grade recommendation as well as a prediction of the

learning success (0.52).

4 STUDY 2

4.1 Study Design and Sample

While Study 1 highlighted the factors that are

important for students and teachers when introducing

a new LA system, Study 2 focuses on the extent to

which communicating these information about LA

systems may increase fairness perceptions and the

institutional attractiveness of a university.

Implementing Learning Analytic Systems in Educational Institutions: The Importance of Transparent Information for User Acceptance

299

Table 1: Level rank of attributes.

We used a video vignette study, which is

recommended to gain insights into the perceptions

and attitudes of individuals (Shapiro et al., 1994). In

these videos, the principal of a fictitious university,

played by a trained actress, announced the

implementation of a new LA system.

A total of eight vignette videos were included,

resulting in a 2 x 2 x 2 factorial between-subject

design to examine the effects of different

combinations of information on users’ perceptions of

LA. Every participant watched only one of the eight

videos. Each video aimed to manipulate the stimulus

information in the following ways: (1) it highlighted

the benefits of the LA system, including personalized

feedback opportunities, (2) it discussed possible

drawbacks of the LA system, such as the potential

biases in algorithmic predictions, and (3) it provided

supplementary details about an external audit of the

LA system and explained that gender discrimination

or ethnical discrimination against groups of people

can be ruled out. Participants in one vignette (video 1:

used as the reference category in the analysis) were

solely informed about the implementation of a LA

system, without any elaboration on its benefits,

drawbacks, or the auditing

process.

We assessed participants’ perceived fairness

using three items from the informational justice scale

(Colquitt, 2001) (e.g., “Was the principal of the

university open in her communication with you?”;

Cronbach’s alpha = .87). Participants’ perception of

the institutional attractiveness was measured with

three items adapted from Aiman-Smith et al. (2001)

(e.g., “The described university would be a good

university to study at”; Cronbach’s alpha = .94). Both

scales were Likert scales (1 = strongly disagree; 7 =

strongly agree).

The vignette study was conducted with two

samples. We employed a European ISO-certified

online sampling provider (ISO 20252:2019) to recruit

students from German universities (n = 458; M

age

30.33; female = 57%). The second sample included

teachers from educational institutions in Germany (n

= 269; M

age

= 42.85; female = 58%), with participants

recruited through social networks and the sampling

provider.

4.2 Results

We analyzed the impact of the seven videos, each

presenting various manipulation combinations, in

contrast to the reference category (i.e., video 1

without further details on the LA system).

Detailed results are shown in Table 2. Although

we found negative direct effects on institutional

attractiveness, our results show that more information

(i.e., benefits, drawbacks, audit) for students led to

positive indirect effects on the institutional

attractiveness. These relationships were mediated by

perceived informational justice, which highlights the

underlying mechanism governing users’ responses to

the provided information. Those vignettes containing

information about the external audit of the LA system

(i.e., Video 2, 4, 6, and 8) demonstrated the most

substantial increase in informational justice

Icons Attributes and levels I

Fairness

The algorithm treats all learners equally (regardless of gender, origin,

religion).

0.70

The algorithm may inadvertently disadvantage learners (due to gender,

origin, religion).

0.31

Audit

A third

art

review of the s

stem took

lace. 0.59

There was a review of the system by the teaching institution. 0.56

No review of the s

stem took

lace. 0.35

Accurac

The algorithm correctly assesses 9 out of 10 learners. 0.62

The al

orithm correctl

assesses 8 out of 10 learners. 0.50

The algorithm correctly assesses 7 out of 10 learners. 0.38

Methods of use

The algorithm gives a grade recommendation and additionally predicts

learning success (successful completion of the course).

0.52

The al

orithm

ives a

rade recommendation. 0.48

CSEDU 2024 - 16th International Conference on Computer Supported Education

300

Table 2: Results of the student sample (Note. B = unstandardized effect; SE = standard error; b = standardized

effect; N (students) = 458, N (teachers) = 269. Reference category: Information that LA is implemented without further details

(video 1).

Student Sample Teacher Sample

Direct Effects of the Video Vignettes B SE b p B SE b p

Video 2 Audit  Informational Justice .64

(

.21

)

.20 .01 .42

(

.30

)

.12 .16

Video 3 Benefits  Informational Justice .64

(

.20

)

.20 .01 .05

(

.30

)

.01 .88

Video 4 Benefits+Audit  Informational Justice .76

(

.21

)

.24 .01 .10

(

.30

)

.03 .74

Video 5 Benefits+Drawbacks  Informational Justice .66

(

.21

)

.21 .01 .08

(

.29

)

.02 .78

Video 6 Benefits+Drawbacks+Audit  Informational

Justice

.78 (.21) .24 .01 .82 (.31) .22 .01

Video 7 Drawbacks  Informational Justice .64

(

.21

)

.20 .01 .47

(

.31

)

.13 .13

Video 8 Drawbacks+Audit  Informational Justice .64

(

.21

)

.20 .01 .16

(

.29

)

.05 .59

Video 2 Audit  Institutional Attractiveness -.38 (.22) -.09 .08 -.12 (.29) -.03 .68

Video 3 Benefits  Institutional Attractiveness -.42 (.22) -.10 .05 -.17 (.29) -.04 .55

Video 4 Benefits+Audit  Institutional Attractiveness -.38 (.22) -.09 .08 -.36 (.29) -.08 .22

Video 5 Benefits+Drawbacks  Institutional

Attractiveness

-.69 (.22) -.16 .01 -.48 (.28) -.12 .09

Video 6 Benefits+Drawbacks+Audit  Institutional

Attractiveness

-.43 (.22) -.10 .05 -.39 (.30) -.09 .20

Video 7 Drawbacks  Institution Attractiveness -.40 (.22) -.09 .07 -.70 (.30) -.15 .02

Video 8 Drawbacks+Audit  Institution Attractiveness -.82 (.22) -.19 .01 -.67 (.29) -.16 .02

Effect of Mediator on Outcome Variable

Informational Justice  Institution Attractiveness .98

(

.07

)

.71 .01 .76

(

.08

)

.63 .01

Indirect Effects

Video 2 Audit  Institution Attractiveness .62 (.20) .14 .02 .32 (.21) .07 .10

Video 3 Benefits  Institution Attractiveness .63 (.21) .15 .01 .03 (.23) .01 .88

Video 4 Benefits+Audit  Institution Attractiveness .74 (.22) .17 .01 .08 (.23) .02 .74

Video 5 Benefits+Drawbacks  Institution

Attractiveness

.65 (.22) .15 .01 .06 (.22) .01 .68

Video 6 Benefits+Drawbacks+Audit  Institution

Attractiveness

.76 (.21) .17 .01 .63 (.25) .14 .02

Video 7 Drawbacks  Institution Attractiveness .63 (.22) .14 .01 .12 (.23) .03 .56

Video 8 Drawbacks+Audit  Institution Attractiveness .63 (.23) .14 .01 .36 (.21) .08 .06

perceptions, subsequently contributing to higher

institutional attractiveness ratings.

To compare the two groups of teachers and

students, we conducted a multi-group comparison in

which we compared three models, an unconstrained

model, a measurement invariance model, and a

structural weights model (Steinmetz, 2013). The

results show that the measurement invariance model

(χ²(10) = 82.069, p = .86) does not differ significantly

from the unconstrained model (χ²(72) = 87.48, p =

.20), which in turn shows that both teachers and

students understood the constructs in the same way.

The structural weights model (χ²(25) = 125.58, p =

.01), on the other hand, differs significantly from the

measurement invariance model, which means that the

model is different overall, that is, the groups differ

significantly in their responses. This means that a

group comparison makes sense

The results show differences in the effects

between the two groups. For the direct effects on

informational justice, the results of the students show

clear positive directions, while only video 6 (i.e., the

provision of information on all available information)

led to an increase in the perception of fairness for

teachers. In both groups, however, informational

justice was positively related to institutional

attractiveness. Concerning the direct effects of the

videos on institutional attractiveness, we observed

negative effects for the students, when potential

drawbacks were mentioned (videos 7 and 8). With

regard to indirect effects, the results show that video

6 with all information had a positive indirect effect on

institutional attractiveness in both groups.

Implementing Learning Analytic Systems in Educational Institutions: The Importance of Transparent Information for User Acceptance

301

5 DISCUSSION

By conducting two empirical studies, we investigated

the relevance of different characteristics of LA

systems as well as the role of transparent information

for fairness perceptions at educational institutions.

The results show that especially the fairness aspect is

a sensitive and relevant characteristic, and great

importance should be attached to ensuring a fair

evaluation by the systems. Furthermore, we found

that transparent communication can increase fairness

perceptions and thus also the perceived attractiveness

of an institution.

In particular, the results of our conjoint analysis

underline the fairness aspects and show the

importance of external audits for both teachers and

students. In contrast, the methods of use are not

highly relevant, suggesting this does not need to be

prioritized during system implementation. However,

ensuring proper verification and technical bias

prevention is still crucial.

The results of our vignette study shed light on the

role of transparent information about the benefits and

drawbacks of LA for students. We also show that

information about an existing system audit by an

external institution has an additional positive

influence on perceptions. This shows that users

perceive LA systems more positively the more

information they receive—even if they are made

aware of possible drawbacks. Although it may seem

counterintuitive to point out drawbacks of the LA

system, informing users of potential drawbacks is

paramount, especially from an ethics perspective.

At the same time, our results show that students

perceive a system to be fairer when they receive

information that the LA system has been audited,

which raises the need for universities to seek audits of

their LA systems. Not only will this ensure that LA

systems are functioning well from a technical and

fairness perspective, but it will also help students

assess whether they accept this new technology.

The results imply that teachers are apparently less

easily influenced by the information provided to

them, or that they themselves already have clearer

opinions on the topic of LA. Another explanation may

be that teachers see their profession as a job that they

have to do anyway, even if new LA systems are

introduced. Students, on the other hand, might change

their educational institution if they do not like the LA

system or if the provided information about the LA

system is insufficient. In sum, our results suggest that

it is important to provide students and teachers with

all available information, as they are both affected by

new LA systems.

Our findings have practical implications for

higher education institutions, planning to implement

LA and aiming to maximize users’ acceptance. Given

that literature is still in its infancy with regard to

reactions to LA systems, our study adds to this

literature by showing that revealing different

information fragments to users has distinct

implications for their perceptions and assessments of

the institution. This has practical implications for the

design and auditing of LA systems prior to their

implementation and underscores the need to

transparently communicate benefits, drawbacks, and

the audit that has been performed by institutions.

In particular, both studies have highlighted the

crucial role of audits, preferably by an external body,

which in turn can ensure a fair assessment of students.

The recommendation by the European Commission

(2021) to audit systems based on artificial intelligence

can therefore be supported and underlined as relevant

by our findings.

6 LIMITATIONS

Our research is not without limitations. First, we

conducted our studies with German-speaking

participants. An exploration of our research questions

in other countries with their own peculiarities (i.e., the

state of digitization in educational instructions) would

be an interesting and important avenue for future

research.

Second, the novelty of LA systems presents a

challenge for users, particularly concerning their

usage scenarios. Despite employing visual aids in our

studies to improve understanding, many users may

lack prior experience with LA systems, hindering

their full comprehension and empathetic engagement

(Köchling and Wehner, 2020; Simbeck, 2023).

Third, in Study 1, to avoid overwhelming the

participants (Balderjahn et al., 2009), we focused on

just four attributes derived from existing literature

and discussion surrounding LA. However, it is crucial

to acknowledge the potential existence of other

significant attributes and factors, which might emerge

through additional research or during actual LA

system implementation.

7 CONCLUSION

In order to ensure a meaningful use of LA systems in

educational institutions, LA systems have to be

accepted by all users. In Study 1, we investigated the

CSEDU 2024 - 16th International Conference on Computer Supported Education

302

importance of different attributes of a LA system and

found that the fairness of the LA system and an

external audit are most important for students and

teachers. In Study 2, we investigated the influence of

different information fragments on users’ perceived

fairness and institutional attractiveness of a

university. Our results show that all users value more

information about the LA system, even though

possible drawbacks were communicated. However,

the results for students and teachers differ

significantly, indicating that students who are

affected by the predictions of LA systems are more

sensitive to the provided information in comparison

to teachers. Future research is needed to investigate

successful ways of implementing LA systems and to

highlight the positive aspects for all user groups.

REFERENCES

Aiman-Smith, L., Bauer, T. N., and Cable, D. M. (2001).

Are You Attracted? Do You Intend to Pursue? A

Recruiting Policy-Capturing Study. Journal of Business

and Psychology, 16(2), 219–237. https://doi.org/10.

1177/1094428114547952

Ajzen, I. (1991). The Theory of Planned Behavior.

Organizational Behavior And Human Decision

Processes, 50(2), 179–211. https://doi.org/10.

1016/0749-5978(91)90020-T

Almosallam, E. A. and Ouertani, H. C. (2014). Learning

Analytics: definitions, applications and related fields.

In T. Herawan, M. M. Deris, and J. Abawajy (Eds.),

Proceedings of the First International Conference on

Advanced Data and Information Engineering (DaEng-

2013) (pp. 721–730). Springer. https://doi.org/10.1

007/978-981-4585-18-7_81

Balderjahn, I., Hedergott, D., and Peyer, M. (2009).

Choice-Based Conjointanalyse. In D. Baier and M.

Brusch (Eds.), Conjointanalyse (pp. 129–146).

https://doi.org/10.1007/978-3-642-00754-5_9

Calders, T. and Zliobaite, I. (2013). Why Unbiased

Computational Processes Can Lead to Discriminative

Decision Procedures. In B. Custers, T. Calders, B.

Schermer, and T. Zarsky (Eds.), Discrimination and

Privacy in the Information Society. Data Mining and

Profiling in Large Databases (pp. 43–57).

https://doi.org/10.1007/978-3-642-30487-3_3

Chapman, D. S., Uggerslev, K. L., Carroll, S. A., Piasentin,

K. A., and Jones, D. A. (2005). Applicant Attraction to

Organizations and Job Choice: A Meta-Analytic

Review of the Correlates of Recruiting Outcomes.

Journal of Applied Psychology, 90(5), 928–944.

https://doi.org/10.1037/0021-9010.90.5.928

Colquitt, J. A. (2001). On the dimensionality of

organizational justice: A construct validation of a

measure. Journal of Applied Psychology, 86(3), 386–

400. https://doi.org/10.1037/0021-9010.86.3.386

Colquitt, J. A. and Rodell, J. B. (2011). Justice, Trust, and

Trustworthiness: A Longitudinal Analysis Integrating

Three Theoretical Perspectives. The Academy of

Management Journal, 54(6), 1183–1206.

https://doi.org/10.5465/amj.2007.0572

European Commission (2021). Proposal for a Regulation of

the European Parliament and of the Council laying

down harmonised rules on artificial intelligence

(Artificial Intelligence Act) and amending certain

Union legislative acts, COM(2021) 206 final, 1–107.

Fischer, C., Siegel, J., Proeller, I., and Drathschmidt, N.

(2023). Resilience through digitalisation: How

individual and organisational resources affect public

employees working from home during the COVID-19

pandemic. Public Management Review, 25(4), 808–

835. https://doi.org/10.1080/14719037.2022.2037014

Greenberg, J. (1994). Using socially fair treatment to

promote acceptance of a work site smoking ban.

Journal of Applied Psychology, 79(2), 288–297.

https://doi.org/10.1037/0021-9010.79.2.288

Greller, W. and Drachsler, H. (2012). Translating Learning

into Numbers: A Generic Framework for Learning

Analytics. Educational Technology & Society, 15(3),

42–57.

Ifenthaler, D. and Drachsler, H. (2020). Learning Analytics.

In Handbuch Bildungstechnologie (pp. 515–534).

Springer.

Ifenthaler, D., Gibson, D., Prasse, D., Shimada, A., and

Yamada, M. (2021). Putting learning back into learning

analytics: Actions for policy makers, researchers, and

practitioners. Educational Technology Research and

Development, 69(4), 2131–2150. https://doi.org/10.

1007/s11423-020-09909-8

Kaibel, C., Koch-Bayram, I., Biemann, T., and

Mühlenbock, M. (2019). Applicant Perceptions of

Hiring Algorithms—Uniqueness and Discrimination

Experiences as Moderators. Academy of Management

Proceedings, 2019(1), 18172. https://doi.

org/10.5465/AMBPP.2019.210

Karren, R. J. and Barringer, M. W. (2002). A Review and

Analysis of the Policy-Capturing Methodology in

Organizational Research: Guidelines for Research and

Practice. Organizational Research Methods, 5(4), 337–

361. https://doi.org/10.1177/109442802237115

Khalil, M. and Ebner, M. (2015). Learning Analytics:

Principles and Constraints. Proceedings of World

Conference on Educational Multimedia, Hypermedia

and Telecommunications 2015, 1326–1336. http://

rgdoi.net/10.13140/RG.2.1.1733.2083

Kim, P. T. (2017). Data-Driven Discrimination at Work.

William and Mary Law Review, 58(3), 857–936.

Köchling, A., Riazy, S., Wehner, M. C., and Simbeck, K.

(2021). Highly Accurate, But Still Discriminatory: A

Fairness Evaluation of Algorithmic Video Analysis in

the Recruitment Context. Business & Information

Systems Engineering, 63(1), 39–54. https://doi.org/10.

1007/s12599-020-00673-w

Köchling, A. and Wehner, M. C. (2020). Discriminated by

an algorithm: A systematic review of discrimination

and fairness by algorithmic decision-making in the

Implementing Learning Analytic Systems in Educational Institutions: The Importance of Transparent Information for User Acceptance

303

context of HR recruitment and HR development.

Business Research, 13(3), 795–848. https://doi.org/10.

1007/s40685-020-00134-w

Langer, M., König, C. J., and Fitili, A. (2018). Information

as a double-edged sword: The role of computer

experience and information on applicant reactions

towards novel technologies for personnel selection.

Computers in Human Behavior, 81, 19–30.

https://doi.org/10.1016/j.chb.2017.11.036

Lawson, C., Beer, C., Rossi, D., Moore, T., and Fleming, J.

(2016). Identification of ‘at risk’ students using

learning analytics: The ethical dilemmas of intervention

strategies in a higher education institution. Educational

Technology Research and Development, 64(5), 957–

968. https://doi.org/10.1007/s11423-016-9459-0

Mai, L., Köchling, A., and Wehner, M. C. (2022). “This

Student Needs to Stay Back”: To What Degree Would

Instructors Rely on the Recommendation of Learning

Analytics? SN Computer Science, 3(4), 259.

https://doi.org/10.1007/s42979-022-01137-6

Martin, F. and Ndoye, A. (2016). Using Learning Analytics

to Assess Student Learning in Online Courses. Journal

of University Teaching and Learning Practice, 13(3),

110–130. https://doi.org/10.53761/1.13.3.7

Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., and

Galstyan, A. (2022). A Survey on Bias and Fairness in

Machine Learning. ACM Computing Surveys, 54(6), 1–

35. https://doi.org/10.1145/3457607

Orme, B. and Sawtooth Software, Inc. (2002). Interpreting

Conjoint Analysis Data.

Platz, S. and Holtbrügge, D. (2016). Student Expectations

and Experiences in Higher Education: A Comparison

of State and Private Universities in Germany. In T. Wu

and V. Naidoo (Eds.), International Marketing of

Higher Education (pp. 171–190). Palgrave Macmillan.

https://doi.org/10.1057/978-1-137-54291-5

Riazy, S., Simbeck, K., and Schreck, V. (2020). Fairness in

Learning Analytics: Student At-risk Prediction in

Virtual Learning Environments: Proceedings of the

12th International Conference on Computer Supported

Education, 1, 15–25. https://doi.org/10.5220/

0009324100150025

Roberts, L. D., Howell, J. A., Seaman, K., and Gibson, D.

C. (2016). Student Attitudes toward Learning Analytics

in Higher Education: “The Fitbit Version of the

Learning World”. Frontiers in Psychology, 7, 1–11.

https://doi.org/10.3389/fpsyg.2016.01959

Rzepka, N., Simbeck, K., Müller, H.-G., and Pinkwart, N.

(2022). Fairness of In-session Dropout Prediction:

Proceedings of the 14th International Conference on

Computer Supported Education, 2, 316–326.

https://doi.org/10.5220/0010962100003182

Schumacher, C., and Ifenthaler, D. (2018). Features

students really expect from learning analytics.

Computers in Human Behavior, 78, 397–407.

https://doi.org/10.1016/j.chb.2017.06.030

Shapiro, D. L., Buttner, E. H., and Barry, B. (1994).

Explanations: What Factors Enhance Their Perceived

Adequacy? Organizational Behavior and Human

Decision Processes, 58(3), 346–368. https://doi.

org/10.1006/obhd.1994.1041

Shepherd, D. A., and Zacharakis, A. (1999). Conjoint

analysis: A new methodological approach for

researching the decision policies of venture capitalists.

Venture Capital, 1(3), 197–217. https://doi.org/

10.1080/136910699295866

Siemens, G., and Baker, R. S. J. d. (2012). Learning

analytics and educational data mining: Towards

communication and collaboration. LAK ’12:

Proceedings of the 2nd International Conference on

Learning Analytics and Knowledge, 252–254.

https://doi.org/10.1145/2330601.2330661

Siemens, G., and Long, P. (2011). Penetrating the Fog:

Analytics in Learning and Education. EDUCAUSE

Review, 32–40.

Simbeck, K. (2023). They shall be fair, transparent, and

robust: Auditing learning analytics systems. AI and

Ethics, 1–17. https://doi.org/10.1007/s43681-023-

00292-7

Starke, C., Baleis, J., Keller, B., and Marcinkowski, F.

(2022). Fairness perceptions of algorithmic decision-

making: A systematic review of the empirical literature.

Big Data & Society, 9(2), 1–16. https://doi.org/

10.1177/20539517221115189

Steinmetz, H. (2013). Analyzing Observed Composite

Differences Across Groups: Is Partial Measurement

Invariance Enough? Methodology, 9(1), 1–12.

https://doi.org/10.1027/1614-2241/a000049

Winter, E., Costello, A., O’Brien, M., and Hickey, G.

(2021). Teachers’ use of technology and the impact of

Covid-19. Irish Educational Studies, 40(2), 235–246.

https://doi.org/10.1080/03323315.2021.1916559

CSEDU 2024 - 16th International Conference on Computer Supported Education

304