Embedding a Data-Driven Decision-Making Work Culture in a Social

Housing Environment

Srinidhi Karthikeyan

, Takao Maruyama

and Sankar Sivarajah

Faculty of Management, Law & Social Sciences, University of Bradford, Richmond Rd, Bradford BD7 1DP, U.K.

Keywords:

Social Housing, Artiﬁcial Intelligence, Machine Learning, Rent Arrears, Explainable AI.

Abstract:

This paper explores the issue of delayed rent payments in social housing in the United Kingdom and its impact

on tenants and housing providers. Our approach is to use machine learning algorithms to analyse payment

patterns and identify tenants who may be at risk of falling behind on rent payments. By doing this, we aim to

equip housing providers with the necessary tools to intervene early and maintain consistent tenancies. We have

conducted research using machine learning models such as decision trees and random forests to address this

issue. The paper emphasises the potential beneﬁts of Explainable AI, which can help build trust in data-driven

decision-making and AI among employees unfamiliar with AI and machine learning.

1 INTRODUCTION

Social housing is vital in the UK, providing accom-

modation to over 5 million households (Stone, 2003).

Timely rent payments are crucial for the sustainabil-

ity of social housing, as tenants’ early exits can hinder

social providers’ objectives. The transition to Uni-

versal Credit (UC) has shifted the responsibility of

rent payments to tenants, and only 8% of those in

Direct Payment Demonstration Projects consistently

paid their rent on time and in full (Hickman et al.,

2017). By 2023, over 7 million households will re-

ceive Universal Credit (Hickman, 2021). Addressing

this issue is essential to alleviate the ﬁnancial pres-

sure on social housing providers, who face an average

cost of £10,000 per eviction (The Guinness Partner-

ship and Tickell, 2015). Evictions can be traumatic

for tenants and jeopardise their well-being (Bond

et al., 2018). Effective rent collection strategies en-

able providers to function efﬁciently, increase hous-

ing supply, and fulﬁl their responsibilities to tenants.

The development of artiﬁcial intelligence (AI) and

machine learning (ML) has brought signiﬁcant ad-

vancements in various ﬁelds, allowing for data anal-

ysis, pattern recognition, and autonomous decision-

making. This has made AI and ML increasingly valu-

able in the current digital age. These complex algo-

https://orcid.org/0000-0001-6863-0760

https://orcid.org/0000-0002-4830-7322

https://orcid.org/0000-0002-6401-540X

rithms enable machines to learn from data, recognise

patterns, and make decisions without human interven-

tion (Holzinger et al., 2017). Due to their immense

potential, people from various application domains,

including healthcare (Shaheen, 2021), ﬁnance (Cao,

2020), and marketing (Mariani et al., 2022), are in-

creasingly interested in utilising these algorithms.

As a result, AI and ML are now employed in dif-

ferent application domains, such as speech recogni-

tion systems (Amberkar et al., 2018) and self-driving

cars (Rao and Frtunikj, 2018), to provide new solu-

tions to previously unsolvable problems. However, AI

in social housing is an area that needs more research.

The paper proposes using machine learning algo-

rithms to predict rent arrears before they occur by

analysing tenants’ payment history. By identifying

patterns, social housing landlords can take timely ac-

tion to prevent the situation from escalating. The pa-

per explains how a machine learning model was de-

veloped to accurately predict tenants’ payment be-

haviour to alert Income ofﬁcers of potential issues.

Our team aims to prevent the situation from worsen-

ing, leading to further arrears or court cases and will

offer ample support to help tenants manage their ﬁ-

nances.

Karthikeyan, S., Maruyama, T. and Sivarajah, S.

Embedding a Data-Driven Decision-Making Work Culture in a Social Housing Environment.

DOI: 10.5220/0012700700003690

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 26th International Conference on Enterprise Information Systems (ICEIS 2024) - Volume 1, pages 807-811

ISBN: 978-989-758-692-7; ISSN: 2184-4992

807

2 LITERATURE REVIEW

Despite its widespread adoption in various indus-

tries, the potential of Artiﬁcial Intelligence (AI) in

social housing still needs to be explored. This re-

view delves into the applications of AI in other do-

mains, the causes of rent arrears in social housing,

and potential areas for future AI research. Several

successful applications of AI can be seen in ﬁelds

such as loan default prediction (Lagasio et al., 2022;

Neisen and Geraskin, 2022; Shaheen and Elfakha-

rany, 2018; Turiel and Aste, 2019) and healthcare risk

prediction (Ehlers et al., 2017; Shinde and Rajeswari,

2018; Ferdousi et al., 2021; Karthick et al., 2022; Wu

et al., 2021; SK and P, 2017; Sawhney et al., 2023;

Dutta et al., 2022). These examples illustrate the

capability of AI to analyse data, recognise patterns,

and make informed decisions. Explainable AI (XAI)

has been used to minimise loan default risk (Egan,

2021) and interpret default forecasting models (Cas-

carino et al., 2022). By explaining AI’s decisions,

XAI can enhance user conﬁdence and trust in the AI

systems (Weitz et al., 2019; Druce et al., 2021).

Rent underpayment is a signiﬁcant problem. A

study by (Irvine et al., 2007) found a strong con-

nection between ’capability’ and rent arrears accrual

and identiﬁed three types of money managers: ’or-

dered,’ ’ﬂexible,’ and ’chaotic.’ The study indicated

that ’ﬂexible’ and ’chaotic’ money managers were

more likely to experience challenges than their ’or-

dered’ counterparts.

Rent underpayments are affected by opportu-

nity (Johnson and O’Halloran, 2017) and mental

health (Bond et al., 2018). Claimants’ arrears are

caused by their ﬁnancial situation, including irregu-

lar or insufﬁcient income, the ﬁve-week wait for their

initial UC payment, and administrative delays in re-

ceiving the beneﬁt (Johnson and O’Halloran, 2017).

It is essential to acknowledge that many reasons ten-

ants fall into arrears are beyond their control. If land-

lords allow tenants to continue accruing arrears, it can

lead to a loss of proﬁt and additional expenses such

as legal processes. Despite an extensive review of the

literature, the authors have not been able to ﬁnd any

previous research studies that focus on the application

of AI in social housing, particularly in assisting social

landlords in reducing their tenants’ rent arrears. This

indicates a signiﬁcant gap in the current research.

3 METHODOLOGY

3.1 Dataset

The original dataset contained around 15 million

records, which consisted of tenants’ everyday pay-

ment records. It included transaction details such as

the transaction date, year, amount, account type, and

other relevant parameters for 20,867 out of 23,033 ac-

tive tenants in the weekly payment group. The dataset

was then transformed into a more usable format to

provide insights into tenant behaviour. As a result of

the transformation, the new dataset now includes:

• Transaction sum of WK1-WK4 (weeks 1-4),

WK5-WK8 (weeks 5-8), WK9-WK12 (weeks 9-

12), WK13-WK16 (weeks 13-16), WK17-WK20

(weeks 17-20), WK21-WK24 (weeks 21-24), and

WK25-WK28 (weeks 25-28).

• Yearly balance of 2021-2022,2022-2023 and

2023-2024.

• Arrear score of 2021-2022, 2022-2023 and 2023-

2024.

• Transaction score of 2021-2022,2022-2023 and

2023-2024.

• Transaction score of all three years.

• Arrear score of all three years.

With the weekly transaction sums, tracking ten-

ants’ payment patterns and identifying inconsisten-

cies is easy. The past three years’ yearly balance

helps us paint a clearer ﬁnancial picture of the ten-

ant and their payment history. The transaction and

arrear scores provide valuable insights into tenant be-

haviour, allowing us to identify potential risks and

take necessary actions to mitigate them. Overall,

this transformed dataset provides a more detailed and

comprehensive understanding of tenant behaviour,

which can help us make informed decisions and im-

prove our services.

During the exploratory data analysis, some interest-

ing ﬁndings were discovered. Figure 1 shows a graph

illustrating the average number of days tenants take to

fall into their ﬁrst arrears since the beginning of their

tenancy. It appears that the period has decreased and

is now the lowest in 2022 compared to 2019. One

possible reason for this is inﬂation in the UK.

Figure 2 displays the average number of days tenants

receive their ﬁrst Notice of Seeking Possession (NSP)

notice since the beginning of their tenancy. As with

The ﬁnancial year is deﬁned from April of one year to

March of the next, so years are mentioned as ranges. For

the future, this will include the current ﬁnancial year and

the past two ﬁnancial years.

ICEIS 2024 - 26th International Conference on Enterprise Information Systems

808

the previous graph, this ﬁgure has decreased, indicat-

ing that tenants are falling behind on their rent pay-

ments more quickly than in last years. As a result,

Income ofﬁcers are responding more promptly by is-

suing NSP notices earlier to prevent the accumulation

of more signiﬁcant debts.

Figure 1: The Average number of days tenants take to get

into their ﬁrst arrears since the start of the tenancy.

Rent is collected on a weekly basis, and a cycle

lasts for a maximum of 52 weeks. Each cycle has 13

periods, with each period consisting of four weeks.

The payment patterns of tenants are represented

using two critical features. The ﬁrst feature is the

transaction pattern score (ranges from 0 to 1), which

indicates changes in the payment pattern of tenants

over 13 periods. A higher score for the transaction

pattern indicates frequent changes in payment pat-

terns, which may be caused by ﬁnancial instability

or other factors. In contrast, a lower score suggests

a consistent payment pattern, indicating that the

tenant is ﬁnancially stable and can meet payment

obligations.

Figure 2: The Average number of days taken for tenants to

get their ﬁrst NSP notice since the start of tenancy.

The second feature used to represent payment pat-

terns is the 13-period arrear scores (ranges from - ∞

to + ∞). The arrear score is calculated by summing

the balances or arrears at the end of each 13-period

period. A positive score indicates the tenant is in ar-

rears, meaning they have failed to pay rent on time. In

contrast, a negative score implies that the tenant has

credit and is not in arrears, indicating that they have

paid their rent on time or in advance. This information

is crucial in determining the risk of tenants defaulting

on rent payments. The target variable is whether the

person will be in arrears for the next four weeks (25

to 28).

4 PRELIMINARY RESULTS

The problem at hand is a binary classiﬁcation task,

and two well-known machine learning algorithms,

Decision Tree (Wu et al., 2008) and Random For-

est (Breiman, 2001), were used to construct models.

Our evaluation criteria included accuracy, precision,

recall, and F1 scores. As shown in Table 1, the Ran-

dom Forest model had greater accuracy than the De-

cision Tree model, making it the baseline model for

the task.

Table 1: Evaluation results of the two models.

Metrics DecisionTree RandomForest

Accuracy 0.807646 0.856696

Precision 0.755488 0.839635

Recall 0.756410 0.786325

F1-Score 0.755949 0.812106

Upon further analysis of the Random Forest

model’s feature importance (as shown in Table 2), the

top ﬁve features signiﬁcantly impacting the classiﬁ-

cation were analysed. These features were Yearly

balance of 2023-2024, Arrear score of 2023-2024,

Transaction sum of WK17-WK20, Transaction sum

of WK21-WK24, and Transaction sum of WK25-

WK28. Among these ﬁve features, the Yearly balance

of 2023-2024, the Arrear score of 2023-2024, and the

Transaction sum of WK25-WK28 supported the ﬁnd-

ings provided by our domain experts.

However, we noticed that the Transaction sum of

WK17-WK20, which corresponds to April, was a

non-debit raise week due to UK holidays.

5 DISCUSSION AND FUTURE

WORKS

Throughout the course of our project, we encoun-

tered a signiﬁcant issue with data quality. We dis-

covered that some of the data we were working with

was less accurate, incomplete, and inconsistent, mak-

Embedding a Data-Driven Decision-Making Work Culture in a Social Housing Environment

809

Table 2: The importance of features according to Random

Forest model.

Features Importance

score

Transaction sum of WK1-WK4 0.0668

Transaction sum of WK5-WK8 0.0556

Transaction sum of WK9-WK12 0.0721

Transaction sum of WK13-WK16 0.0734

Transaction sum of WK17-WK20 0.0904

Transaction sum of WK21-WK24 0.079

Transaction sum of WK25-WK28 0.0763

Yearly balance of 2021-2022 0.03

Yearly balance of 2022-2023 0.0416

Yearly balance of 2023-2024 0.1229

Arrear score of 2021-2022 0.0291

Arrear score of 2022-2023 0.0376

Arrear score of 2023-2024 0.1181

Transaction score of 2021-2022 0.0162

Transaction score of 2022-2023 0.0177

Transaction score of 2023-2024 0.0176

Transaction score of all three years 0.0242

Arrear score of all three years 0.0313

ing it challenging to consider other important factors,

such as tenant arrangements and the number of house-

hold members. High-quality data is essential for any

business to thrive, while low-quality data can lead to

missed opportunities and ﬁnancial losses. A study by

Duvier et al. (Duvier et al., 2018) revealed that data

quality issues can arise due to various organisational,

cultural, or computational challenges. Therefore, we

understood the importance of addressing these chal-

lenges to establish an effective data quality program.

We also faced another challenge of building trust

among employees new to working with AI. We un-

derstood that trust is crucial when working with AI,

and we wanted our employees to feel conﬁdent about

AI’s decision-making process. Therefore, we decided

to use explainable AI to help employees understand

and trust AI’s decisions.

Based on the preliminary ﬁndings, the authors aim

to broaden the dataset by adding more transaction

data from the past three years. Additionally, the plan

is to incorporate data from previous tenancy transac-

tions, as the current study only utilises current ten-

ancy data. Furthermore, the authors intend to intro-

duce more relevant features, such as tenancy duration,

property demographics, and payment methods, as rec-

ommended by domain experts, to ensure a compre-

hensive and robust dataset. We are preparing a follow-

up paper that will present a comprehensive analysis

of our ﬁndings, detailing strategies employed to over-

come the identiﬁed challenges and the impact of our

work on the business.

6 CONCLUSIONS

Late rent payments in social housing in the UK have

been a persistent problem for many years. This pa-

per delves into the issue and explores potential solu-

tions and challenges when implementing them. One

possible solution that the paper proposes is using Ex-

plainable AI to build trust in data-driven decision-

making processes, which can improve the accuracy

of rent payment predictions and reduce the number

of late payments. However, more research is needed

to understand the full potential of AI in addressing

the problem of late payments in social housing effec-

tively. Despite the challenges of implementing AI in

social housing, there is a growing belief that it could

help tackle the issue more efﬁciently and fairly. It is

crucial to ﬁx the issue of late payments in social hous-

ing because it affects not only the housing providers

but also the tenants, who may face eviction and ﬁ-

nancial difﬁculties. By addressing the problem, we

can ensure that tenants can maintain stable and secure

housing while also achieving positive outcomes for

housing providers.

ACKNOWLEDGEMENTS

The authors thank Innovate UK and Incommunities

for part funding this research.

REFERENCES

Amberkar, A., Awasarmol, P., Deshmukh, G., and Dave,

P. (2018). Speech recognition using recurrent neu-

ral networks. In 2018 international conference on

current trends towards converging technologies (IC-

CTCT), pages 1–4. IEEE.

Bond, N., Evans, K., and Holkar, M. (2018). Where the

heart is-social housing, rent arrears and mental health.

Breiman, L. (2001). Random forests. Machine Learning,

45(1):5–32.

Cao, L. (2020). Ai in ﬁnance: A review. Available at SSRN

3647625.

Cascarino, G., Moscatelli, M., and Parlapiano, F. (2022).

Explainable artiﬁcial intelligence: interpreting default

forecasting models based on machine learning. Bank

of Italy Occasional Paper, (674).

Druce, J., Harradon, M., and Tittle, J. (2021). Explainable

artiﬁcial intelligence (xai) for increasing user trust in

deep reinforcement learning driven autonomous sys-

tems. arXiv preprint arXiv:2106.03775.

Dutta, A., Hasan, M. K., Ahmad, M., Awal, M. A., Islam,

M. A., Masud, M., and Meshref, H. (2022). Early

prediction of diabetes using an ensemble of machine

ICEIS 2024 - 26th International Conference on Enterprise Information Systems

810

learning models. International Journal of Environ-

mental Research and Public Health, 19(19):12378.

Duvier, C., Neagu, D., Oltean-Dumbrava, C., and Dickens,

D. (2018). Data quality challenges in the uk social

housing sector. International Journal of information

management, 38(1):196–200.

Egan, C. (2021). Improving Credit Default Prediction Us-

ing Explainable AI. PhD thesis, Dublin, National Col-

lege of Ireland.

Ehlers, A. P., Roy, S. B., Khor, S., Mandagani, P., Maria,

M., Alfonso-Cristancho, R., and Flum, D. R. (2017).

Improved risk prediction following surgery using ma-

chine learning algorithms. eGEMs, 5(2).

Ferdousi, R., Hossain, M. A., and El Saddik, A. (2021).

Early-stage risk prediction of non-communicable dis-

ease using machine learning in health cps. IEEE Ac-

cess, 9:96823–96837.

Hickman, P. (2021). Understanding social housing tenants’

rent payment behaviour: evidence from great britain.

Housing Studies, 36(2):235–257.

Hickman, P., Kemp, P. A., Reeve, K., and Wilson, I. (2017).

The impact of the direct payment of housing bene-

ﬁt: evidence from great britain. Housing Studies,

32(8):1105–1126.

Holzinger, A., Biemann, C., Pattichis, C. S., and Kell,

D. B. (2017). What do we need to build explainable

ai systems for the medical domain? arXiv preprint

arXiv:1712.09923.

Irvine, A., Kemp, P. A., and Nice, K. (2007). Direct Pay-

ment of Housing Beneﬁt: What Do Claimants Think?

Chartered Institute of Housing.

Johnson, S. and O’Halloran, A. (2017). Nudging your way

to reduced rent arrears.

Karthick, K., Aruna, S., Samikannu, R., Kuppusamy, R.,

Teekaraman, Y., Thelkar, A. R., et al. (2022). Im-

plementation of a heart disease risk prediction model

using machine learning. Computational and Mathe-

matical Methods in Medicine, 2022.

Lagasio, V., Pampurini, F., Pezzola, A., and Quaranta, A. G.

(2022). Assessing bank default determinants via ma-

chine learning. Information Sciences, 618:87–97.

Mariani, M. M., Perez-Vega, R., and Wirtz, J. (2022). Ai

in marketing, consumer research and psychology: A

systematic literature review and research agenda. Psy-

chology & Marketing, 39(4):755–776.

Neisen, M. and Geraskin, P. (2022). Improved credit default

prediction using machine learning and its impact on

risk-weighted assets of banks. Journal of AI, Robotics

& Workplace Automation, 2(1):49–62.

Rao, Q. and Frtunikj, J. (2018). Deep learning for self-

driving cars: Chances and challenges. In Proceedings

of the 1st international workshop on software engi-

neering for AI in autonomous systems, pages 35–38.

Sawhney, R., Malik, A., Sharma, S., and Narayan, V.

(2023). A comparative assessment of artiﬁcial intelli-

gence models used for early prediction and evaluation

of chronic kidney disease. Decision Analytics Jour-

nal, 6:100169.

Shaheen, M. Y. (2021). Applications of artiﬁcial intel-

ligence (ai) in healthcare: A review. ScienceOpen

Preprints.

Shaheen, S. K. and Elfakharany, E. (2018). Predictive ana-

lytics for loan default in banking sector using machine

learning techniques. In 2018 28th International Con-

ference on Computer Theory and Applications (IC-

CTA), pages 66–71. IEEE.

Shinde, S. A. and Rajeswari, P. R. (2018). Intelligent health

risk prediction systems using machine learning: a re-

view. International Journal of Engineering & Tech-

nology, 7(3):1019–1023.

SK, S. and P, A. (2017). A machine learning ensemble clas-

siﬁer for early prediction of diabetic retinopathy. Jour-

nal of Medical Systems, 41:1–12.

Stone, M. E. (2003). Social housing in the UK and US:

Evolution, issues and prospects. Goldsmiths College,

Centre for Urban Community Research London.

The Guinness Partnership, C. o. A. i. t. S. E. and Tickell, C.

(2015). Tenancy sustainment: Summary report may

2015.

Turiel, J. D. and Aste, T. (2019). P2p loan acceptance and

default prediction with artiﬁcial intelligence. arXiv

preprint arXiv:1907.01800.

Weitz, K., Schiller, D., Schlagowski, R., Huber, T., and

Andr

e, E. (2019). ” do you trust me?” increasing user-

trust by integrating virtual agents in explainable ai in-

teraction design. In Proceedings of the 19th ACM In-

ternational Conference on Intelligent Virtual Agents,

pages 7–9.

Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q.,

Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu,

P. S., et al. (2008). Top 10 algorithms in data mining.

Knowledge and information systems, 14:1–37.

Wu, Y.-T., Zhang, C.-J., Mol, B. W., Kawai, A., Li, C.,

Chen, L., Wang, Y., Sheng, J.-Z., Fan, J.-X., Shi, Y.,

et al. (2021). Early prediction of gestational diabetes

mellitus in the chinese population via advanced ma-

chine learning. The Journal of Clinical Endocrinology

& Metabolism, 106(3):e1191–e1205.

Embedding a Data-Driven Decision-Making Work Culture in a Social Housing Environment

811