Decoding AI’s Evolution Using Big Data: A Methodological Approach

Sophie Gvasalia

, Mauro Pelucchi

, Simone Perego

and Rita Porcelli

Global Data Science, Lightcast, Italy

INAPP – Istituto per l’Analisi delle Politiche Pubbliche, Italy

{sophie.gvasalia, mauro.pelucchi, simone.perego}@lightcast.io, r.porcelli@inapp.gov.it

Keywords:

Big Data Methodology, Artiﬁcial Intelligence Impact, Job Market Analysis.

Abstract:

This study presents a novel approach to measuring the impact of Artiﬁcial Intelligence on occupations through

an analysis of the Atlante del Lavoro dataset and web job postings. By focusing on data preparation and

model selection, we provide real-time insights into how AI is reshaping job roles and required skills. Our

methodological framework enables a detailed examination of speciﬁc labour market segments, emphasizing

the dynamic nature of occupational demands. Through a rigorous mixed-method approach, the study high-

lights the AI impact on sectors such as ICT, telecommunications, and mechatronic, revealing distinct skill

clusters and their signiﬁcance. This innovative analysis not only delineates the convergence of digital, soft,

and hard skills but also offers a multidimensional view of future workforce competencies. The ﬁndings serve

as a valuable resource for educators, policymakers, and industry stakeholders, guiding workforce development

in line with emerging AI-driven demands.

1 INTRODUCTION

The integration of artiﬁcial intelligence into work pro-

cesses is set to signiﬁcantly inﬂuence various types of

workers, leading to changes in wage structures and

skill requirements. Public policies will, therefore,

play a critical role in promoting training and ensur-

ing that workers are adequately prepared for the trans-

formations in the labour market. This study aims to

provide an innovative analysis of the labour market,

focusing on the measure of the impact of AI-related

skills across different economic sectors. By utilizing

data from the Atlante del Lavoro

and online job post-

ings (provided by Lightcast

), this study explores the

competencies demanded in job postings and charac-

terizes professional proﬁles, offering a detailed per-

spective on the emerging labour dynamics in sectors

signiﬁcantly impacted by AI.

To explore this relationship, we apply big data

principles, focusing on ﬁve key dimensions: volume,

by leveraging a large dataset of online job postings for

comprehensive labour market analysis; velocity, with

near real-time processing to capture dynamic changes

in AI skill demands; variety, by combining structured

data from Atlante del Lavoro with unstructured job

postings for a multifaceted view; veracity, using NLP

https://atlantelavoro.inapp.org/

https://lightcast.io

techniques to ensure data accuracy and reliability; and

value, providing actionable insights for policymak-

ers, training institutions, and industry stakeholders on

AI-related skills and their economic impact. Through

rigorous quantitative analysis and qualitative interpre-

tation, we aim to characterize emerging professional

proﬁles and elucidate the nuanced interplay between

AI adoption and skill demand. This approach allows

for a granular examination of labour market trends,

with particular emphasis on sectors experiencing sig-

niﬁcant AI-driven transformations.

AI’s labour market impact includes job displace-

ment, creation, and transformation. Studies indicate

AI automates routine and non-routine tasks, altering

work and skill demands. (Lane et al., 2023) high-

light how AI transforms roles by automating repeti-

tive tasks, increasing the need for cognitive and so-

cioemotional skills. Generative Pre-trained Trans-

formers (GPT) further revolutionize labour by au-

tomating tasks requiring natural language understand-

ing. GPT is widely used in customer service, content

creation, and data analysis, enhancing productivity

(Eloundou et al., 2023). As GPT reshapes job roles,

new training programs are essential for effective AI

collaboration. (Squicciarini and Nachtigall, 2021) in-

vestigations conﬁrm that the majority of workers de-

veloping and maintaining AI possess these special-

ized skills, although not all workers involved in AI

424

Gvasalia, S., Pelucchi, M., Perego, S. and Porcelli, R.

Decoding AI’s Evolution Using Big Data: A Methodological Approach.

DOI: 10.5220/0013016600003838

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2024) - Volume 1: KDIR, pages 424-432

ISBN: 978-989-758-716-0; ISSN: 2184-3228

have these skills to the same extent. The demand for

these specialized AI skills has grown signiﬁcantly in

recent years, particularly in the United States, with

similar trends observed in Canada, Singapore, and

the United Kingdom. Job advertisements increasingly

demand AI skills alongside transversal competencies

such as social skills and management abilities, indi-

cating their complementary nature.

Our approach innovatively analyzes the effects of

AI on job competencies by integrating large-scale

job posting data with the Atlante del Lavoro frame-

work. This method provides a detailed view of evolv-

ing skill demands and AI-related skills across sectors.

Advanced text mining techniques capture emerging

trends in real-time, offering more immediate insights

than traditional surveys. The dynamic mapping of AI

adoption and skill demand reveals subtle shifts in job

roles, often missed by conventional methods.

The study begins with a review of the state of the

art in research on AI’s impact on the labour market.

This section will introduce the current understand-

ing and highlight gaps that this research aims to ﬁll.

Following this, section 3 presents the data used in

our analysis. The data section also covers the pre-

processing steps undertaken to ensure the accuracy

and relevance of the data for our study. In the section

4, we detail our innovative approach that integrates

both qualitative and quantitative data to evaluate the

impact of AI on work activities. Advanced Machine

Learning and NLP techniques are employed to esti-

mate AI’s impact efﬁciently. This section provides a

comprehensive description of the data preparation and

model selection processes, highlighting the nuances

of our methodological framework. The results section

presents our ﬁndings on the impact of AI in speciﬁc

sectors such as ICT, telecommunications, and mecha-

tronic. The conclusion discusses the implications of

our ﬁndings, the limitations of the study, and suggests

directions for future research.

2 RELATED WORK

Recent studies indicate a signiﬁcant acceleration in

the adoption of Artiﬁcial Intelligence (AI) across var-

ious sectors of the economy. The PwC AI Barom-

eter

reports that 52% of companies have expedited

their AI adoption plans, with 86% anticipating AI to

become a mainstream technology within their organ-

isations by 2024. This trend is corroborated by the

OECD AI surveys (Lane et al., 2023), which found

that 24% of businesses across OECD countries are

https://www.pwc.com/AIJobsBarometer

currently utilising AI technologies. Notably, there ex-

ists a substantial disparity in adoption rates based on

ﬁrm size, with large ﬁrms being ten times more likely

to adopt AI than their smaller counterparts.

(Brynjolfsson and McAfee, 2014) highlight AI’s

dual effect on the workforce: automation of rou-

tine tasks and creation of new roles requiring ad-

vanced skills. This underscores the complex interplay

between technological innovation and labour market

shifts, with both job displacement and new opportu-

nities emerging. (Autor, 2015) delved into the po-

larization of the labour market caused by technolog-

ical advancements. The research indicates that AI

and automation technologies tend to replace middle-

skill jobs that involve routine tasks, while simultane-

ously increasing demand for both high-skill jobs that

require creative and cognitive abilities and low-skill

jobs that involve non-routine manual tasks. This po-

larization highlights the need for targeted educational

and training programs to equip workers with the skills

necessary to thrive in an AI-driven economy. (Frey

and Osborne, 2017) utilize a Gaussian process classi-

ﬁer to estimate automation probabilities for 702 occu-

pations, based on O*NET data. The study highlights

sectoral automation risks, though it focuses on current

technology and technical feasibility, neglecting future

advancements and workforce adaptability.

Recent empirical investigations have delved into

the speciﬁc competencies. (Alekseeva et al., 2021)

conducted a comprehensive analysis of job postings,

revealing a marked upsurge in demand for AI-centric

skills, encompassing proﬁciency in programming lan-

guages such as python, expertise in big data manage-

ment, and capabilities in model development. Com-

plementing this work, (Acemoglu et al., 2022) ex-

plored the intricate relationship between AI technol-

ogy adoption and evolving workforce skill demands.

Their ﬁndings underscore the symbiotic nature of

technical proﬁciencies and soft skills in the contem-

porary labour landscape, highlighting the complex in-

terplay between technological advancement and hu-

man capital development in shaping employment dy-

namics.

The methodological approaches for analyzing the

impact of AI on job postings have also evolved.

(Manca, 2023) utilized advanced NLP techniques to

parse and analyze large datasets of job advertise-

ments, providing real-time insights into emerging

skill demands. This approach enables a more dy-

namic understanding of labour market trends, as op-

posed to traditional static analyses.

(Eloundou et al., 2023) integrate expert judgments

with datasets from O*NET, ILO, and the World Bank,

assessing generative AI’s impact on 923 occupations

Decoding AI’s Evolution Using Big Data: A Methodological Approach

425

across 199 countries. The broad scope and AI expo-

sure scoring system are strengths, though expert re-

liance and rapid AI evolution pose limitations. De-

spite extrapolation challenges, the study offers valu-

able insights for global labour markets. (Weichsel-

braun et al., 2024) employs a deep learning-based ap-

proach to anticipate future job market demands by as-

sessing the automatability and offshorability of skills.

The authors use a combination of Support Vector Ma-

chines (SVMs), Transformers, and Large Language

Models (LLMs) to classify skills and estimate their

future relevance. Their ﬁndings highlight the increas-

ing demand for skills related to automation and off-

shoring, driven by trends like the Gig economy and

technological advancements.

3 DATA

The Atlante del Lavoro is the Italian classiﬁcatory and

informative device for work and qualiﬁcations, cre-

ated based on the descriptive sequences of the Classi-

ﬁcation of 24 Economic Professional Sectors (SEPs).

The Atlante del Lavoro was developed as part of the

construction of the National Repository of Education

and Training Titles and Professional Qualiﬁcations,

as stipulated by Legislative Decree No. 13 of January

16, 2013

. It aims to systematize and correlate the

competencies of qualiﬁcations from the public life-

long learning offerings with work activities. The sec-

tors were generated by intersecting two independent

ISTAT classiﬁcations, both in terms of the object rep-

resented and the constructive criteria used: the clas-

siﬁcation of economic activities (ATECO 2007

) and

the classiﬁcation of professions (CP 2011, updated in

2023 with CP 2021

All the codes constituting the aforementioned sta-

tistical classiﬁcations, at their maximum extension,

have been aggregated in the Atlante del Lavoro Eco-

nomic Professional Sectors (SEPs) to meet the empir-

ical need to identify a ”perimeter” where sets of work

processes and activities with relative internal homo-

geneity (intra-sectoral) and sufﬁcient external distinc-

tion (inter-sectoral) can be placed and ordered in their

information ﬁeld. The Economic Professional Sec-

tors (SEPs) are articulated into work processes, pro-

cess sequences, activity areas, and individual activi-

ties described following the typical logic of the value

https://www.gazzettaufﬁciale.it/eli/id/2013/02/15/

13G00043/sg

https://www.istat.it/en/classiﬁcation/ateco-

classiﬁcation-of-economic-activity-2007/

https://www.istat.it/en/classiﬁcation/classiﬁcation-of-

occupations/

chain model (Mazzarella et al., 2017). These descrip-

tors are constantly updated to meet the need to track

the evolution of constantly changing work activities.

The INAPP

study utilized online job advertise-

ments to calculate skill rates and assess the rele-

vance of these skills for the descriptors within the

Atlante del Lavoro, which detail each segment of

work. Three indicators were deﬁned to measure the

evolution of work dynamics in processes, sequences,

and area of activities (ADAs), quantifying the skill

rate for each system component (Mezzanzanica et al.,

2018). These indicators measure the incidence of dig-

ital, soft, and hard non-digital skills on the Atlante del

Lavoro’s descriptive elements:

Skill

Rate

+ f

Where t denotes the skill type (digital, soft, or

hard non-digital), with f

representing the frequency

of type t, f

the frequency of soft skills, f

the fre-

quency of digital skills, and f

the frequency of hard

non-digital skills in the dataset. Three indicators were

identiﬁed, relating to classes of macro-competencies

based on ESCO skills

, extracted from the job post-

ings database. These indicators measure and monitor

over time the degree of digitalization, the demand for

soft skills, and the demand for technical/hard skills

within Economic Professional Sectors (SEPs), Pro-

cesses, Sequences, and Area of Activities (ADAs).

The working method can be summarized in the

following steps:

(i) Job advertisements are used as measurement tools

to elaborate all indices on macro-competencies.

Each job advertisement is classiﬁed according to

the CP 2011 standard, at the 5-digit level.

(ii) Ads are linked to Area of Activities (ADAs)

through the associated profession. It should be

noted that only advertisements classiﬁed accord-

ing to the occupations belonging to the area of ac-

tivity contribute to the calculation of indicators.

During the association between area of activity

and job advertisements, the correspondence be-

tween a single area of activity and the occupations

may not be one-to-one: each profession can be

associated with multiple activities. In this case,

if the same profession is associated with multiple

areas belonging to the same sequence, the job ad-

vertisements for this sequence are considered only

once.

(iii) Job advertisements associated with each area can

be further ﬁltered based on the industry codes

https://www.inapp.gov.it/en/homepage

https://esco.ec.europa.eu/en/classiﬁcation

KDIR 2024 - 16th International Conference on Knowledge Discovery and Information Retrieval

426

associated with the process Sequence to which

the area belongs. The industry sectors ﬁlter, in

some cases, reduces the expressive capacity of the

database, but when present, it reﬁnes the match.

(iv) Job advertisements associated with each area re-

port the required skills, categorized according to

the ESCO classiﬁcation and grouped by macro-

competency classes.

The skills rate, broken down into macro-

competence areas, monitors the results within the area

of activity, sequence, or process through successive

aggregations and it tracks the evolution of jobs over

time.

3.1 Job Postings Dataset

The Lightcast database currently consists of over 21

million online job postings for Italy. After thorough

data cleaning, it contains more than 8 million vali-

dated job postings. These postings, often in semi-

structured or unstructured text, require rigorous sci-

entiﬁc, methodological, and technical work to extract

useful information. Covering the entire national ter-

ritory, they provide a rich data source for analyzing

various dimensions (occupations, industry sectors, re-

gions, and skills) (Vrolijk et al., 2022).

The processing phases are: (i) Data Collection:

Extracting job postings via API, bulk extraction, and

scraping. (ii) Data Treatment: Structuring data to

meet shared standards. (iii) Text Processing: Prepar-

ing unstructured texts for classiﬁcation. (iv) Classi-

ﬁcation: Extracting professions and skills from job

postings.

Occupations are extracted from texts using a com-

bination of machine learning algorithms that train the

classiﬁer based on previously classiﬁed and expert-

validated occurrences. Skills are extracted using fea-

ture extraction techniques and mapped to the ESCO

standard. Each skill is then associated with a macro-

competence class: digital, soft, or hard-no-digital, de-

ﬁned by the working group using ESCO and O*NET

classiﬁcation pillars. Hard skills are speciﬁc job abil-

ities, while soft skills are interpersonal and environ-

mental interaction abilities. Digital skills within hard

skills include ICT tool usage to complex system de-

sign. Soft skills include thinking, social interaction,

knowledge application, and attitudes and values. For

each occupation, the required competencies are ana-

lyzed, and the frequency of soft, hard non-digital, and

digital skills is calculated.

(Lovaglio, 2022) introduces a methodology for

analyzing labour market trends using web-scraped job

vacancies, revealing the growing importance of digi-

tal skills across sectors. Despite potential biases in

online recruitment, the approach provides real-time

insights. Similarly, (Vermeulen and Amaros, 2024)

and (Enrique and Matteo, 2024) assess the validity of

Lightcast job posting data compared to national statis-

tics across Europe (2019–2022). Their benchmarking

highlights discrepancies but emphasizes the comple-

mentary value of online postings for tracking labour

demand trends.

4 METHODOLOGY

The methodology involves selecting a representative

sample from the Atlante del Lavoro using stratiﬁed

sampling by Economic Professional Sector (SEP).

Sector experts then evaluated the AI impact on sam-

pled work activities. Next, NLP techniques were ap-

plied to efﬁciently analyze the data corpus and esti-

mate AI’s impact across all activities. Finally, work

activities were classiﬁed based on AI impact esti-

mates, enabling a clear assessment of sectoral trans-

formations. This methodology partially draws from

(Frey and Osborne, 2017) study. A 5% random sam-

ple of areas of activity was extracted using stratiﬁed

sampling to ensure fair sector representation. The

sample comprises 48 Area of Activities (ADAs), with

each of the 24 Economic Professional Sectors (SEPs)

represented by 2 ADAs (see Table 1).

To ensure the reliability and consistency of the

labeling effort, we assessed inter-rater agreement

among the ﬁve industry experts who evaluated the AI

impact on various work activities. Each expert in-

dependently assigned scores ranging from 1 to 5, re-

ﬂecting the perceived impact of AI on speciﬁc activ-

ities. To gauge the consistency of these assessments,

we calculated inter-rater agreement using the Fleiss’

Kappa method. The calculated Fleiss’ Kappa value

of 0.2128 suggests fair agreement among the raters.

This indicates some level of consistency in their eval-

uations, but the variability implies differences in their

assessment criteria (see Table 2).

To efﬁciently analyze the impact of AI on all

work activities, a methodology based on advanced

machine ,earning techniques was adopted. Initially,

Sentence BERT (Bidirectional Encoder Representa-

tions from Transformers, (Reimers and Gurevych,

2019)), a natural language representation model, was

used to create a data corpus containing descriptions

of work activities and their related embeddings. Sub-

sequently, the dataset was divided into training and

test sets to train and evaluate the ML models. Vari-

ous models were explored, including XGBoost (Chen

and Guestrin, 2016), linear and polynomial regres-

sion, neural networks, and support vector machines

Decoding AI’s Evolution Using Big Data: A Methodological Approach

427

Table 1: Sample of the area of activities evaluated by the experts.

Sector Area of Activity description

Chemistry Operation and control of plants/machines in the production of

sterile and non-sterile drugs

Chemistry Processing of plastics and rubber

Construction Execution of foundations and tunnels

Construction Painting works

Extraction of gas, oil, coal, minerals, and stone processing Environmental recovery of disused extraction areas

Extraction of gas, oil, coal, minerals, and stone processing Preparation and squaring of blocks

Wood and furniture Selection and storage of lots

Wood and furniture Packaging of curtains and drapes

Mechanics, production, and maintenance of machinery, plant

engineering

Maintenance and repair of mechanical and structural compo-

nents of aircraft

Mechanics, production, and maintenance of machinery, plant

engineering

Maintenance and repair of household appliances and electri-

cal devices

Table 2: Sample of AI Impact Evaluation on work activities from the experts.

Sector Description Expert evaluation

Social and Health Services Implementation of clown therapy interventions 1

Tourism Services Operational management of bathing services 2

Tourism Services Operational management of ski slopes and implementation of

rescue interventions

Printing and Publishing Handcrafted production of prints using lithographic processes 3

Printing and Publishing Digital archiving of the publishing house’s documentary her-

itage

(SVM), in order to estimate the impact of AI on work

activities. Among the various models tested (see Fig-

ure 1), those that showed the best performance were

selected for subsequent analysis. The performances,

computed on the test set and reported in table 3 indi-

cate that the Gradient Boosting model performs best,

with an R² of 0.417, meaning it explains 41.7% of the

variance in the data. The Gradient Boosting model

utilizes XGBoost. Key parameters include the num-

ber of trees (100), which dictates the ensemble’s size,

and the learning rate (0.1), controlling how much

each tree contributes to the model. The regularization

(Lambda: 10) adds penalties to prevent overﬁtting,

and the depth of trees (10) controls tree complexity,

balancing model expressiveness and overﬁtting risk.

Subsampling parameters are set to 1.0, meaning the

entire dataset and all features are used at each step.

Table 3: Comparison of model performance metrics evalu-

ated on the test set.

Model MSE RMSE MAE R²

Gradient Boosting 0.400 0.633 0.490 0.417

Linear Regression 0.494 0.703 0.641 0.282

Neural Network 0.703 0.838 0.635 -0.022

SVM 0.503 0.709 0.587 0.268

Based on the estimates obtained through models,

work activities were classiﬁed according to their rel-

ative AI impact. Activities were divided into cat-

egories, including areas with high impact (score >

3.5), medium impact (score > 2 and ≤ 3.5), and low

impact (score > 1 and < 2). This categorization pro-

vides a clear overview of AI’s inﬂuence on different

work activities and sectors.

5 RESULTS

The results of the analysis provided a detailed

overview of the impact of AI on work activities across

various economic sectors.

5.0.1 AI Impact

Initially, we examined the number of activities classi-

ﬁed based on AI impact, distinguishing between high

impact (category A), medium impact (category B),

and low impact (category C). The results indicate that

the most signiﬁcant number of activities fall into cat-

egory B (564 activities), followed by category C (254

activities), while category A includes 84 activities.

Analyzing the industry sectors and their respec-

tive areas of activity with signiﬁcant AI impact, data

shows considerable variation across sectors (Table

4). For instance, in the ”Digital Services” sec-

tor, most ADA (63.64%) are classiﬁed as high im-

pact; in ”Printing and Publishing” and ”Construction”

45.45% and 4.17%, respectively. The strong AI pres-

ence in ICT related services drives demand for spe-

ciﬁc skills such as programming, data analysis, and

AI itself. Sectors with medium AI impact, like print-

ing and manufacturing, require professionals to adapt

KDIR 2024 - 16th International Conference on Knowledge Discovery and Information Retrieval

428

Figure 1: The text mining process to evaluate the AI impact on all the area of activities. The process begins with document

embedding using SBERT

, followed by a data split for training and testing. Four models (Neural Network, Gradient Boost-

ing, Support Vector Machine, and Linear Regression) are trained and evaluated using test data. The best-performing model

generates predictions, which are output for further analysis.

their skills to optimize work processes using AI tech-

nologies. Even in sectors with limited AI impact, such

as education, professional development must address

digitalization demands, preparing the workforce for

future innovations.

The analysis highlights the need for continuous

adaptation of the education and skill development

system in response to technological evolution. Train-

ing should focus on transversal digital skills, such as

critical thinking, problem-solving, and communica-

tion, in addition to AI-speciﬁc skills and their appli-

cations in various sectors (Pedone A. 2024, Conforti

D. 2024).

To complete the study, we explored the link be-

tween digital skills (using the digital skills rate at the

Area of Activity level) and the AI impact on ADA.

A polynomial model was used to calculate the co-

efﬁcient of determination (R²), which measures how

much variation in AI impact is explained by digital

skills. The R² of 0.2095 indicates that 20% of the

variability in AI impact is explained by digital skills.

The adjusted R² was 0.2078, suggesting the model is

appropriate and does not overﬁt. While a correlation

exists, the relatively low R² suggests that other factors

contribute to explaining AI impact variability, indicat-

ing the need for further research (Figure 2).

5.0.2 Digital Services

The analysis of the ICT sector highlights several key

ﬁndings. The sector shows a high susceptibility to AI-

driven changes, with a signiﬁcant portion of activities

impacted by AI. Skill demands are shifting towards

AI-related competencies, with a notable emphasis on

machine learning, NLP, and data analytics. Digital

Figure 2: Digital Skills Rate and AI Impact. The digital

skills rate is represented on the horizontal axis of this scatter

plot, while the vertical axis presents the AI impact.

skills dominate the sector, and soft skills are increas-

ingly valued, especially for roles managing AI. These

insights stress the need for an adaptable ICT work-

force, capable of continuous upskilling to keep pace

with AI advancements (Table 5).

Job postings in 2023 already highlight AI-related

skills such as Apache Spark, Machine Learning, NLP,

Computer Vision, PyTorch, Deep Learning, Keras,

Generative AI, Cognitive Computing, and Large Lan-

guage Modeling. To attract talent in the ICT sector,

it is important to introduce policies focused on up-

skilling and reskilling the workforce. These initiatives

enhance existing employee skills, making the region

more competitive and appealing to potential talent.

Continuous learning ensures that the workforce stays

adaptable, ﬁlling skill gaps and positioning the region

as a hub for innovation and professional growth (Gatti

et al., 2022).

Decoding AI’s Evolution Using Big Data: A Methodological Approach

429

Table 4: AI Impact on the economic sectors.

Industry A (High Impact) B (Medium Im-

pact)

C (Low Impact)

Agriculture, forestry and ﬁshing 2.00% 58.00% 40.00%

Food production 4.76% 69.05% 26.19%

Wood and furniture 0.00% 21.74% 78.26%

Paper and papermaking 0.00% 75.00% 25.00%

Textiles, clothing, footwear and fashion system 0.00% 27.50% 72.50%

Chemistry 0.00% 76.00% 24.00%

Extraction of gas, oil, coal, minerals and stone processing 0.00% 43.33% 56.67%

Glass, ceramics and building materials 0.00% 28.57% 71.43%

Construction 4.17% 41.67% 54.17%

Mechanics, machine production and maintenance, plant engineering 11.32% 63.21% 25.47%

Transport and logistics 7.35% 89.71% 2.94%

Commercial distribution services 0.00% 90.00% 10.00%

Financial and insurance services 2.08% 81.25% 16.67%

Digital Services 63.64% 36.36% 0.00%

Telecommunication and postal services 38.46% 61.54% 0.00%

Public utilities services 18.18% 68.18% 13.64%

Printing and publishing 45.45% 45.45% 9.09%

Education, training and employment services 9.38% 90.63% 0.00%

Social and health services 4.17% 79.17% 16.67%

Personal services 0.00% 29.41% 70.59%

Recreational and sports services 0.00% 62.50% 37.50%

Cultural and entertainment services 11.11% 64.81% 24.07%

Tourism services 9.68% 83.87% 6.45%

Common area 20.55% 73.97% 5.48%

Table 5: ADA SEP – High AI Impact Digital Services and corresponding digital skills rate.

Area of activity AI Impact Digital Skills Rate

Engineering ICT Systems 4.57 64%

Improving ICT Processes 4.46 20%

Innovation in ICT 4.4 61%

Data Science and Analytics 4.38 34%

Sustainability Management in ICT 4.29 59%

Monitoring Technological Trends 4.23 64%

Information and Knowledge Management 4.23 64%

Problem Management in ICT 4.18 67%

Deﬁning IT Strategy and Aligning with Business 4.16 60%

User Experience Design 3.95 63%

Application Development 3.8 63%

Developing Cybersecurity Strategy 3.62 47%

Providing ICT Services 3.57 66%

Supporting System Changes and Evolutions 3.55 72%

5.0.3 Telecommunications

In the telecommunications sector, AI signiﬁcantly im-

pacts work activities, as shown by the digital skill rate

of area of activity analyzed for this SEP (Table 6).

The highest AI impact is seen in Network Architec-

ture Design and Planning (5.00), requiring a 26% dig-

ital skill rate. Installation, Conﬁguration, and Testing

of TLC Systems have a high AI impact (4.64) with an

18% digital skill rate. Network Management and Su-

pervision also show signiﬁcant AI impact (4.18) with

a 26% digital skill rate. Conversely, Online Shipping

Service Programming has a lower AI impact (3.88)

with a 19% digital skill rate, while TLC System Main-

tenance Assistance has a moderate AI impact (3.87)

with an 8% digital skill rate.

Job postings in the telecommunications sector

highlight the demand for AI-related skills, such as

Machine Learning, K-Means Clustering, Deep Learn-

ing, and Natural Language Processing. Technologies

like Apache Spark, TensorFlow, PyTorch, and Keras

are widely adopted, emphasizing the need for skills in

data analysis, ICT system management, and project

management methodologies.

KDIR 2024 - 16th International Conference on Knowledge Discovery and Information Retrieval

430

Table 6: ADA SEP – Telecommunications and Postal Services with High AI Impact and Corresponding Digital Skills Rate.

Area of activity AI Impact Digital Skills Rate

Network Architecture Design and Planning 5.00 26%

Installation, Conﬁguration, and Testing of TLC Systems 4.64 18%

Management, Supervision, and Control of TLC System Components and Networks 4.18 26%

Programming and Control of Online Shipping Services 3.88 19%

Assistance/Maintenance of TLC Systems 3.87 8%

5.0.4 Mechatronic

In the mechatronic, AI integration is revolutionizing

several operational activities (Table 7). Key areas

include programming and automating electronic sys-

tems, utilizing AI platforms to optimize production

processes, and improving assembly line efﬁciency. AI

solutions in electrical/electronic installation on boats

integrate smart sensors and control algorithms, en-

hancing system safety and reliability. Aerospace sec-

tor AI optimizes production of components and ve-

hicles through advanced modeling and virtual sim-

ulations. Building automation systems use machine

learning algorithms to optimize energy consumption

and occupant comfort.

Job postings in this sector frequently mention

roles such as electromechanics, industrial engineers,

telecommunication technicians, and aerospace engi-

neers. Skills in Machine Learning, Apache Spark,

Computer Vision, and Natural Language Processing

are highly sought after, indicating a growing need for

AI competencies to enhance automation, safety, and

efﬁciency in industrial and electronic systems.

6 CONCLUSIONS

The analysis investigated AI’s impact across various

labour segments in the Atlante del Lavoro using on-

line job vacancies. Results revealed differing AI im-

pacts, categorized as high, medium, and low, with a

correlation between digital skills and AI impact on

work activities. Analysis of selected sectors focused

on high-impact area of activities, identifying key pro-

fessions and AI-related skills. A clear distinction

was found between AI application roles, which re-

quire digital literacy and domain-speciﬁc expertise,

and AI development roles, which demand specialized

skills like Python, SQL, and machine learning. These

ﬁndings underscore the varied skillsets needed across

AI’s inﬂuence on the labour market.

In Digital Services, AI automates repetitive tasks,

requiring new skills for designing intelligent appli-

cations like Robotic Process Automation (RPA) and

data analysis tools. In telecommunications, AI auto-

mates network management, enhances customer in-

teractions through NLP, and improves predictive ana-

lytics for network maintenance. In the mechatronics

sector, AI-driven robots boost efﬁciency, and machine

learning predicts equipment failures, while simulators

optimize production processes and predictive mainte-

nance reduces downtime. In addressing the feasibil-

ity of satisfying the required shift in skills, it is im-

portant to acknowledge the gap between the demand

for advanced AI skills, such as deep learning, and the

realistic capacity for most workers to acquire them.

While reskilling efforts can help non-technical work-

ers adopt AI-related competencies, expecting a sig-

niﬁcant portion of the workforce to master complex

areas like deep learning is unrealistic. Recent studies

suggest that non-technical workers are better suited to

focus on skills such as AI collaboration, data literacy,

and problem-solving, which are more attainable and

still highly relevant in an AI-driven environment (see

(Whelan and Redmond, 2024)).

The study has limitations, such as potential bi-

ases from relying on online job vacancies and over-

looking future AI advancements. The analysis is not

exhaustive across sectors, and digital skills measure-

ment may miss emerging trends. Expert evaluations

introduce subjectivity, and the study’s focus on a spe-

ciﬁc timeframe and region limits broader applicabil-

ity. Economic shifts and sectoral AI adoption rates

are not fully considered. However, the study empha-

sizes the need for investment in training programs,

identifying two key skill areas: technical competen-

cies for AI system development and general skills for

AI adoption and interaction.

ACKNOWLEDGEMENTS

Simone Perego handled data preprocessing and

preparation for online job posting datasets. Sophie

Gvasalia performed data extraction and linked the At-

lante del Lavoro dataset with job postings. Mauro

Pelucchi implemented data mining and model selec-

tion. Rita Porcelli curated the manuscript and synthe-

sized policy-relevant insights.

Decoding AI’s Evolution Using Big Data: A Methodological Approach

431

Table 7: ADA SEP – Mechatronic with High AI Impact and Corresponding Digital Skills Rate.

Area of activity AI Impact Digital Skills Rate

Programming Electronic Systems for Automation Control 4.53 17%

Installation of Electrical/Electronic Systems on Boats 3.82 7%

Manual and Automated Machine Forming 3.75 0%

Installation and Repair of TV Reception and Signal Systems 3.75 0%

Designing Renewable Energy Source (RES) Systems 3.73 27%

System Integration for Optimizing Aerospace Components and Vehicles Production 3.73 25%

Customer Installation, Commissioning, and Testing 3.71 12%

Design of Thermohydraulic Systems (e.g., civil, industrial, HVAC) 3.69 27%

Installation/Maintenance of Industrial Electrical Systems 3.61 14%

Management and Improvement of Aerospace Production Processes and Logistics 3.61 21%

Building Automation Systems Setup and Management 3.59 26%

Installation/Maintenance of Civil and Commercial Electrical Systems 3.57 11%

REFERENCES

Acemoglu, D., Autor, D., Hazell, J., and Restrepo, P.

(2022). Artiﬁcial intelligence and jobs: Evidence

from online vacancies. Journal of Labor Economics,

40(S1):S293–S340.

Alekseeva, L., Azar, J., Gin

e, M., Samila, S., and Taska, B.

(2021). The demand for ai skills in the labor market.

Labour economics, 71:102002.

Autor, D. H. (2015). Why are there still so many jobs? the

history and future of workplace automation. Journal

of economic perspectives, 29(3):3–30.

Brynjolfsson, E. and McAfee, A. (2014). The second ma-

chine age: Work, progress, and prosperity in a time of

brilliant technologies. WW Norton & Company.

Chen, T. and Guestrin, C. (2016). Xgboost: A scalable

tree boosting system. In Proceedings of the 22nd acm

sigkdd international conference on knowledge discov-

ery and data mining, pages 785–794.

Eloundou, T., Manning, S., Mishkin, P., and Rock, D.

(2023). Gpts are gpts: An early look at the labor mar-

ket impact potential of large language models. arXiv

preprint arXiv:2303.10130.

Enrique, F. M. and Matteo, S. (2024). Skewed signals?

confronting biases in online job ads data. Technical

report, Joint Research Centre.

Frey, C. B. and Osborne, M. A. (2017). The future of em-

ployment: How susceptible are jobs to computerisa-

tion? Technological forecasting and social change,

114:254–280.

Gatti, A. C., Colombo, E., Magrini, E., Perego, S., and

Pelucchi, M. (2022). Understanding talent attraction

using online job ads: the impact of artiﬁcial intelli-

gence and green jobs. In The Relevance of Artiﬁ-

cial Intelligence in the Digital and Green Transfor-

mation of Regional and Local Labour Markets Across

Europe, pages 129–164. Nomos Verlagsgesellschaft

mbH & Co. KG.

Lane, M., Williams, M., and Broecke, S. (2023). The im-

pact of ai on the workplace: Main ﬁndings from the

oecd ai surveys of employers and workers.

Lovaglio, P. G. (2022). Do job vacancies variations antici-

pate employment variations by sector? some prelimi-

nary evidence from italy. Labour, 36(1):71–93.

Manca, F. (2023). Six questions about the demand for arti-

ﬁcial intelligence skills in labour markets.

Mazzarella, R., Mallardi, F., and Porcelli, R. (2017). At-

lante lavoro: un modello a supporto delle politiche

dell’occupazione e dell’apprendimento permanente.

Sinappsi, 7:2–3.

Mezzanzanica, M., Mercorio, F., and Colombo, E. (2018).

Digitalisation and automation: Insights from the on-

line labour market. In Developing Skills in a Changing

World of Work, pages 259–282. Rainer Hampp Verlag.

Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sen-

tence embeddings using siamese bert-networks. arXiv

preprint arXiv:1908.10084.

Squicciarini, M. and Nachtigall, H. (2021). Demand for ai

skills in jobs: Evidence from online job postings.

Vermeulen, W. and Amaros, F. G. (2024). How well do on-

line job postings match national sources in european

countries?: Benchmarking lightcast data against sta-

tistical and labour agency sources across regions, sec-

tors and occupation.

Vrolijk, J., Mol, S. T., Weber, C., Tavakoli, M., Kismih

ok,

G., and Pelucchi, M. (2022). Ontojob: Automated on-

tology learning from labor market data. In 2022 IEEE

16th International Conference on Semantic Comput-

ing (ICSC), pages 195–200. IEEE.

Weichselbraun, A., S

usstrunk, N., Waldvogel, R., Glatzl,

A., Bras¸oveanu, A. M., and Scharl, A. (2024). An-

ticipating job market demands—a deep learning ap-

proach to determining the future readiness of profes-

sional skills. Future Internet, 16(5):144.

Whelan, A., M. S. S. E. and Redmond, P. (2024). Skill re-

quirements for emerging technologies in ireland. ESRI

Research Series 191.

KDIR 2024 - 16th International Conference on Knowledge Discovery and Information Retrieval

432