Machine Learning for All: A More Robust Federated Learning

Framework

Chamatidis Ilias

and Spathoulas Georgios

1,2

Department of Computer Science and Biomedical Informatics, University of Thessaly, Greece

Center for Cyber and Information Security, Norwegian University of Science and Technology, Gjovik, Norway

Keywords:

Deep Learning, Federated Learning, Blockchain, Security, Privacy, Integrity, Incentives.

Abstract:

Machine learning and especially deep learning are appropriate for solving multiple problems in various do-

mains. Training such models though, demands signiﬁcant processing power and requires large data-sets.

Federated learning is an approach that merely solves these problems, as multiple users constitute a distributed

network and each one of them trains a model locally with his data. This network can cumulatively sum up

signiﬁcant processing power to conduct training efﬁciently, while it is easier to preserve privacy, as data does

not leave its owner. Nevertheless, it has been proven that federated learning also faces privacy and integrity

issues. In this paper a general enhanced federated learning framework is presented. Users may provide data

or the required processing power or participate just in order to train their models. Homomorphic encryption

algorithms are employed to enable model training on encrypted data. Blockchain technology is used as smart

contracts coordinate the work-ﬂow and the commitments made between all participating nodes, while at the

same time, tokens exchanges between nodes provide the required incentives for users to participate in the

scheme and to act legitimately.

1 INTRODUCTION

Machine learning ﬁeld has recently attracted a lot of

interest. Advancements in hardware and algorithmic

breakthroughs have made it easier and faster to pro-

cess large volumes of data. In particular deep learn-

ing scheme, trains neural networks with a large num-

ber of nodes and multiple hidden layers. Taking ad-

vantage of the parallel processing capabilities of mod-

ern graphic cards, deep learning became quickly the

main option for training large machine learning mod-

els upon big data data-sets.

Another relevant advancement, federated learning

refers to multiple nodes which train models locally

and then fuse these partial models into a single one.

The resulting distributed network has a lot more pro-

cessing power than a single machine, so it can per-

form faster and cope with larger volumes of data. An-

other critical issue is the collection of data to train

the model. Traditionally data is gathered at a single

host and training is carried out locally, but in feder-

ated learning the training happens at users’ devices,

so data does not need to be sent to a central server and

thus privacy of the data holder is preserved. Although

federated learning seems very interesting, it still has

problems such as coordination of the whole process,

privacy of the users data and performance issues.

Personal data are being gathered and used for

training machine learning models. This happens with

or without users’ consent and usually gives them no

control over the resulting models. For example, data

such as biometrics, text input and location coordinates

are private personal data, but are required in order

to train models for biometric authentication, text pre-

diction or navigation services respectively. Federated

learning offers a solution to the problem mentioned

above, because no central server gathers the users’

data. In this scheme, models are trained locally at

the users devices, without any third parties accessing

their data and users only share the resulting trained

models.

In this paper we present an approach for enhanc-

ing federated learning model in terms of privacy, man-

agement and integrity. Speciﬁcally we discuss the

use of homomorphic encryption, blockchain technol-

ogy and integrity mechanisms, in order to construct a

more robust scheme for federated learning. Speciﬁ-

cally Section 2 discusses deep learning and federated

learning in more detail, Section 3 presents some of

the most serious threats for the current model of fed-

544

Ilias, C. and Georgios, S.

Machine Learning for All: A More Robust Federated Learning Framework.

DOI: 10.5220/0007571705440551

In Proceedings of the 5th International Conference on Information Systems Security and Privacy (ICISSP 2019), pages 544-551

ISBN: 978-989-758-359-9

erated learning, Section 4 discusses the main points

of the proposed methodology and Section 5 discusses

related research efforts through recent years. In Sec-

tion 6 our future plans are presented and Section 7

discusses our conclusions.

2 MACHINE LEARNING

In this Section both deep learning and federated learn-

ing paradigms are presented.

2.1 Deep Learning

Deep learning is a rapidly growing ﬁeld of machine

learning. Because of the breakthrough in algorithms

and also of the fact that hardware has recently become

more efﬁcient and less expensive to build, deep learn-

ing models have recently been massively employed in

various applications. Deep learning is essentially the

paradigm of training very large neural networks, with

multiple hidden layers consisting of numerous nodes,

by the use of very large data-sets (Deng et al., 2014).

The main advantage of deep neural networks is

the large number of neurons allowing them to learn a

great depth of detail of the data, used as input. Be-

cause of their ability to efﬁciently adapt to any data-

set, deep learning networks are called universal ap-

proximators, for their ability to efﬁciently solve di-

verse problems. Thus deep learning is used in differ-

ent domains, such as computer vision (Zeiler, 2013),

speech recognition (Deng et al., 2013), natural lan-

guage processing (Collobert and Weston, 2008), au-

dio recognition (Noda et al., 2015), social network

ﬁltering (Liu et al., 2013), machine translation (Na,

2015), bioinformatics (Min et al., 2017), drug de-

sign (Jing et al., 2018) and biometric authentication

(Chamatidis et al., 2017). Learning in deep networks

can be supervised or unsupervised. Also there is an

hierarchical relation between the nodes of the hidden

layers, so each layer learns to a different abstraction

of the data outputted by the previous layer, thus re-

sulting into higher accuracy.

2.2 Federated Learning

As deep learning became more popular, researchers

started experimenting on different problems, an im-

portant conclusion was quickly evident; the accuracy

of the results of deep learning networks is highly cou-

pled with the size of the training data-set. Larger net-

works are characterized by higher degree of freedom

and thus more data is required for their training. The

Figure 1: Architecture of a federated deep learning network.

need for larger data-sets and consequently more com-

puting power is an important issue. Also the form of

the computing power required (GPU clusters) made

it harder for individual users, to utilize proposed al-

gorithms. The previous factors practically made deep

learning available only to large corporations and orga-

nizations with the required resources for researching

and developing systems under the aforementioned re-

strictions.

Additionally several issues have recently emerged

regarding the collection of data required for the train-

ing of deep networks. Training of such networks re-

quires real-world data, which in most of the cases is

personal data. Such data are usually captured by ser-

vices that interact with multiple users, with or without

the consent of the latter. Most data that is created by

people is considered as personal information and it is

protected by law, so a difﬁcult legislative process is

needed to collect and use this data.

Even if the data used is anonymous, it has

been proved that signiﬁcant privacy leaks may oc-

cur through various linking attacks (Backes et al.,

2016). Recently Netﬁx released a data-set (Netﬂix

prize (Bennett et al., 2007; Zhou et al., 2008)) con-

sisting of 500,000 anonymous movie ratings, in an

attempt to optimize their predictions algorithms. Au-

thors in (Narayanan and Shmatikov, 2008) success-

fully demonstrated that it is possible to ﬁnd a Netﬂix

subscribers records in this data-set and furthermore

uncover their political preferences along with other

sensitive information about them.

As private information leaks become common,

users become more concerned about the personal data

they share. Also these collected data-sets may be too

large to be processed or saved locally, while the train-

ing of the network is a resources demanding task, and

requires large training times even for simple tasks.

A proposed architecture that aims to solve the prob-

lem of data collection and concurrently make required

computing resources available to more users is feder-

ated learning, also referred to as collaborative learn-

ing or distributed learning.

Machine Learning for All: A More Robust Federated Learning Framework

545

Federated learning architecture is usually based on

the assumption that a node needs to train a machine

learning model and that this training is partially com-

mitted in multiple other nodes. The main node, also

identiﬁed as coordinator collects the trained models

and combines those into a single model. The rest of

the nodes train the partial models upon data that reside

locally and send the trained models to the coordinator

to be fused into a single ﬁnal model.

Federated learning architecture is characterized by

two signiﬁcant changes with respect to the traditional

model, where a single node has to collect all data

and then conduct the required training of the model.

Firstly there is no need for a single node to collect

and store all the data locally. Secondly the process-

ing power required is split among different parties,

so it is easier to concentrate the required resources.

This characteristic enables any node with no signif-

icant computing power available to participate into

the scheme. In an example Google applied federated

learning, using a mobile light version of Tensorﬂow

(Abadi et al., 2015) and conducted text prediction

tasks across its mobile platforms. Deep learning mod-

els were trained locally in mobile devices of users and

then the results were fused in the Google cloud AI.

During training of the model ﬁrst the coordina-

tor node notiﬁes the other nodes of a speciﬁc initial

model to be used. Then the all other nodes train the

same initial mode locally with data-sets they own.

Finally, when they ﬁnish training, only the trained

model is shared to the coordinator so that the ﬁnal

model can be calculated by aggregating all partial

models.

3 SECURITY AND PRIVACY

THREATS

Personal users’ data are of great interest to compa-

nies, as they can use those to train models for targeted

advertisement or product suggestion. At the same

time, users tend to share personal information online

in increasing rates with out thinking that their data

might be used by someone else with out their con-

sent. As technology and the internet progressed per-

sonal users’ data have become a commodity for com-

panies that is sold between them. Federated learning

seems to be a reasonable solution to that issue. To be-

gin with, there is no central entity that collects all the

data. Instead, data is processed locally from its origi-

nal owner and it is not required to be handled by any

third party.

Federated learning seems to be the next big ad-

vancement in the deep learning research ﬁeld, al-

though there are many issues that need to be ad-

dressed, in order to be used in a real world use cases.

Privacy and security problems still exist, despite the

fact that the users share the minimum required data

for the training process. In the following sections,

these threats are analyzed.

3.1 Data Leak with Adversarial Attack

In federated learning environment, users train their

own deep learning models locally. After training their

model they sent the local gradients to the coordina-

tor, in order to fuse those into a global model. As

it is demonstrated in (Melis et al., 2018), through an

adversarial attack it is possible to extract information

about users’ data set just by having access to the local

gradients they sent to the coordinator.

3.2 General Adversarial Network

Attack

This kind of attack is also executed by a malicious

adversary node that pretends to be honest. This node

tries to extract information about data classes he does

not own by inﬂuencing the learning process to deceive

the victim into releasing further information about the

targeted data.

The adversary node has a General Adversarial

Network(GAN) (Hitaj et al., 2017) generating data

samples that look similar to the class of the victim.

Then these fake classes are injected into the federated

learning network, which then needs to work harder to

distinguish between the real and the fake classes and

thus reveals more information about the real data set

than initially designed. The adversary node by forc-

ing the victim to train between the real data and the

generated fake data, can learn the distribution of the

data, without accessing to the actual data points.

3.3 User Linkability Attacks

Another type of attack, as demonstrated in (Orekondy

et al., 2018), is user linkability attack. In a federated

learning environment, users share only the gradient to

the central node, but it is possible to link these param-

eters to the users they belong to and extract informa-

tion about them.

In this attack the malicious adversary node knows

a set of gradient updates and the corresponding train-

ing data of the user. The adversary can learn to iden-

tify the user based on the produced gradient updates.

In this kind of attack even adding random noise to the

data doesn’t seem to mitigate it. This type of attack

proves that every communication during the training

ICISSP 2019 - 5th International Conference on Information Systems Security and Privacy

546

process has to be protected from privacy related at-

tacks.

3.4 Data Poisoning

Another technique, as shown in (Bagdasaryan et al.,

2018), aims to make a federated learning system com-

posed of Convolutional Neural Networks(CNN) clas-

sifying images, to under-achieve in terms of classiﬁ-

cation accuracy. The attacker forces the CNN to miss-

classify certain car images by feeding ”poisoned” im-

ages to the CNN.

Another experiment is also described, in a fed-

erated learning word predictor, where multiple cell-

phones share text predictions that were trained locally

in their devices, to update the text predictor of the ser-

vice in a server. Authors showed that the attacker can

force the text predictor to show a desired word as it’s

prediction. In fact with less than 1% of the total nodes

being malicious the attacker can achieve 100% accu-

racy in the task. This attack can be mitigated by ap-

plying differential privacy to the individual nodes.

4 PROPOSED METHODOLOGY

We propose a general framework that will enable

users to construct machine learning work-ﬂows by

sharing data, processing power and problems’ deﬁ-

nitions.

Each user of the system may have :

• A problem to solve by using machine learning

methodology

• Data that can be used to solve such problems

• Processing power to conduct the required training

In practice each one of the users may posses one of

the aforementioned assets/issues or even two of them.

If a user has a problem, data to solve it and the re-

quired processing power then he can produce the so-

lution on his own. In every other case he may take

advantage of the proposed system. There are mainly

three different scenarios to take into account :

• A user has a problem to solve, without data or re-

quired processing power

• A user has a problem to solve, has the required

data but lacks processing power

• A user has a problem to solve has the required

processing power but lacks required data

To tackle the problems of federated learning, we

propose a general framework that will protect privacy

of personal data, orchestrate the procedure and offer

incentives for users to participate and also provide in-

tegrity and quality checking mechanisms for the re-

sulting trained model. Data exchanged between nodes

have to be encrypted, nodes that participate by pro-

viding data or processing power have to be rewarded,

while in order for the produced models to be useful

the integrity of the procedure has to be crosschecked.

The design of the system is going to be based on

three main pillars, which are homomorphic encryp-

tion, blockchain technology and integrity checking

mechanisms.

4.1 Privacy

4.1.1 Homomorphic Encryption

Homomorphic encryption algorithms allow for oper-

ations on the ciphertext, that correspond to actual op-

erations on the original plain text. There are multiple

algorithms that differ with regards to the level of pro-

cessing allowed and to their efﬁciency. In general,

data is decrypted with the private key and it is pos-

sible for a node that does not posses this private key

to conduct speciﬁc processing on encrypted data and

produce an encrypted result. This result is then re-

turned to the initial data owner and he can decrypt

that to get the result of the processing conducted in

plain text form.

Homomorphic encryption has been studied as a

mechanism for achieving privacy preserving machine

learning. The model trainer can use sensitive pri-

vate data in encrypted form, to train an encrypted ma-

chine learning model. Then he can return the trained

model to the data owner to decrypt that. There is a

lot of ongoing research in this ﬁeld, as it could even-

tually enable outsourcing the training procedure of

machine learning models to cloud services. Recently

homomorphic encryption has been successfully used

in deep learning and machine learning tasks (Takabi

et al., 2016; Li et al., 2017; Rouhani et al., 2018;

Aono et al., 2017).

4.1.2 Proposed Encryption Scheme

A proposed encryption scheme is depicted in Fig-

ure 2. In this example there are ﬁve different nodes

A,B,C,D,E.

• Node A has a problem to solve

• Nodes B,C have the appropriate data

• Nodes D,E have processing power

Hypothesizing that node A needs to train a ma-

chine learning model, to solve his problem, he will

utilize the resources offered by other nodes to do so.

Machine Learning for All: A More Robust Federated Learning Framework

547

Figure 2: The system architecture with 3 kinds of nodes.

The procedure that the nodes need to go through is as

follows:

1. Node A creates a pair of public and private keys

and distributes the public one to other nodes

2. Nodes B and C encrypt their data with the com-

mon public encryption key

3. Nodes B and C distribute their encrypted data to

nodes D and E

4. Node A deﬁnes a training model and encrypts its

parameters

5. Node A sends the model to nodes D and E

6. Nodes D and E train the model with the data they

hold and then return the encrypted model param-

eters to node A

7. Node A checks if the model has been successfully

trained

8. If not, node A updates model parameters and steps

5 to 7 are repeated

9. If yes, the model is decrypted and used by node

Through the aforementioned steps and given the

fact that no nodes are maliciously colluding, no node

gets to know the personal data of nodes B and C. At

the end of the procedure, node A has received the re-

quired trained machine learning model, with conduct-

ing minimal processing on its side.

Adding an encryption layer to federated learning

approach in order to ensure privacy of participating

nodes will of course create a processing overhead. On

the other the proposed scheme will enable the par-

ticipation of numerous additional, privacy conscious,

processing nodes that would not take part in a sim-

ple federated learning collaboration. The processing

overhead will be shared among multiple nodes and

thus it will be more feasible to conduct large scale

machine learning projects. Homomorphic encryption

will have a minimal effect on the classiﬁcation ratio

of the produced models. The encrypted models pro-

duced by homomorphic approaches are characterized

by a bearable reduction of classiﬁcation ratio with re-

spect to that of normally trained models.

4.2 Management

4.2.1 Blockchain Technology

Blockchain is an emerging technology that was made

widely known to the research community through Bit-

coin in 2008 (Nakamoto, 2008). Blockchain is a

immutable, distributed and transparent shared ledger.

This ledger is stored in every node of a peer to peer

network of similar nodes. The action to append infor-

mation to the ledger is called a transaction, and such

transactions are stored into structures called blocks.

The latter contain the transactions and a reference

to the previous block (i.e, the hash of the previous

block), thus forming a chain of blocks that cannot be

altered under normal circumstances. For a transaction

to be stored in the blockchain, it needs to be ﬁrst val-

idated by miners who compute a complex math func-

tion which is called proof-of-work and then add the

new block to the chain. The miners gain a reward

when they calculate the block and respectively the

users who commit transactions pay a small transac-

tion fee.

Blockchain technology has been used in many in-

dustrial and research applications, mainly through the

use of smart contracts. Smart contracts are computer

programs that can be executed on a virtual machine

based on blockchain. The integrity of both the state

and the execution of such programs is guaranteed by

blockchain technology. In this sense, smart contracts

are ideal for orchestrating the collaboration between

parties that do not trust each other.

4.2.2 Procedure Orchestration

A user announces that he needs to solve a speciﬁc

problem by training a model. He has to collaborate

with a number of data owners and data processors to

do so.

He instantiates a smart contract, through which he

sets all the required details. He deﬁnes the network

initial parameters and the data structure for each data

point. He also sets a speciﬁc number of data points to

be used as input and also the maximum time that each

training epoch shall last and the maximum number

of epochs allowed. Additionally he sets a maximum

budget (in terms of tokens) he is willing to spent on

both data owners and data processors. During the cre-

ation of the contract, he is obliged to send to the con-

ICISSP 2019 - 5th International Conference on Information Systems Security and Privacy

548

tract the sum of the two maximum budget amounts of

tokens he has set. Finally he sets the expiration time

for the bidding procedure.

Nodes that hold compatible data points may bid

with the number of data points they can provide. They

also set the number of tokens per data point they re-

quire for providing their data. Nodes that are able to

provide the required processing power bid with the

number of data points they are able to process in the

speciﬁed epoch maximum time duration. Addition-

ally they set the number of tokens per data point pro-

cessing they require.

When the biding ends the contract automatically

checks all submitted offers and resolves the bidding.

In practice it selects the most proﬁtable set of data

owners and data processors for the user and publishes

the results.

After the successful end of the procedure the con-

tract pays the proper amount of tokens to the data

owners and data processors and returns the rest of the

tokens to the user. If a data processor fails to submit

training results in the pre-set maximum time, then a

penalty is activated and the ﬁnal amount of tokens that

is ﬁnally sent to that node gets reduced.

User is given the opportunity to back off from the

procedure at each training epoch. Speciﬁcally at the

end of each epoch the user decides to continue or not

with the training procedure. This is done through the

smart contract and in practice continuing to the next

iteration means that every party agrees that the proce-

dure is acceptable as legitimate by this point. If the

user decides to back off before the procedure ends or

because one of the other nodes seems to misbehave,

then the payments go through according to the pro-

cedure up until this speciﬁc iteration and the user re-

ceives the partially trained model.

4.3 Integrity

An important factor for the successful training of the

model is data integrity. Any change in data transferred

between data owners and processing nodes and model

parameters transferred between the user and the pro-

cessing nodes may signiﬁcantly alter the ﬁnal result.

Additionally, processing nodes shall be monitored in

terms of the results they produce in order to mitigate

abnormal behaviors such as trying to avoid processing

by returning random parameters or even intentionally

training the model with artiﬁcial data in order to ma-

liciously alter the ﬁnal result.

In order to establish an integrity mechanism for

the behavior of the processing nodes, a modiﬁcation

has to be made to the procedure orchestration, ana-

lyzed in Subsection 4.2.2. Speciﬁcally the process-

ing nodes are required to transfer to the contract an

amount of tokens, as a stake, equal to their possible

maximum earnings, during the bid process. Eventu-

ally, if the procedure goes on normally, then at the end

they are getting paid for their service, according to

the commitments made at the bidding phase and they

also have their stake returned. If they are detected

to act maliciously or abnormally during the process,

then they do not get paid, while their stake is paid out

evenly to all other participating nodes.

A mechanism to validate if processor nodes are

malicious and trying to corrode the process by pro-

viding false results or detect they not doing any work

at all is to compare the gradients that they return at

each training epoch. Individuals processor nodes train

their version of the model locally, upon their share of

training data, and return the resulting error parame-

ters to the user. Given that the training data points

are evenly split among processing nodes, the parame-

ters returned from processing nodes should converge

to similar values for all nodes, after some training it-

erations. Additionally the values returned for these

parameters should gradually stabilize for each node.

Nodes that return values with high variance, in con-

sequent training epochs, probably misbehave. The

user can monitor the parameters returned from each

processing node, for each training iteration, and de-

cide upon the criteria mentioned for any misbehaving

node.

An alternative mechanism could be applied in

cases where the update of the model is conducted

upon the average of errors for all data points. Specif-

ically the integrity of the procedure carried out in the

data processors side can be tested upon the error re-

sults of distinct data points. If the user is able to do

some processing on his side, then he can test the ini-

tial epoch model with random data points out of the

ones distributed to processors and then calculate the

hash of the data point along with the produced error.

Finally he has to require from the processors to submit

which are these random data points. There is no other

way for the processor than to do the actual training

with all data points until he picks the ones the user has

selected. In the case where the user cannot conduct

any processing then an alternative approach would be

to have partial overlapping of data points distributed

to users and utilize the results received from other

nodes to test the integrity of each processing node.

Data processors are obliged to transfer to the con-

tract a stake equal to their maximum earnings accord-

ing to their bid. If they do not act according to the

protocol then the stake is lost, so economic loss is a

strong incentive for nodes, to perform a valid training

procedure.

Machine Learning for All: A More Robust Federated Learning Framework

549

5 RELATED WORK

There are some recent research efforts in the lit-

erature that combine federated learning with either

blockchain or privacy preserving methods.

In (Weng et al., 2018) a privacy preserving

deep learning distributed network is presented, where

blockchain is used for the orchestration of the pro-

cess. The participating parties do computations lo-

cally without exposing their data and then upload the

local gradients to a central node to in order to combine

those. Blockchain is utilized to offer a direct incentive

to users to be behave trustfully and non-maliciously.

The system is tested in a local Corda network and in-

teresting results are obtained.

Also in (Mendis et al., 2018) a theoretical decen-

tralized model is presented, in which there are three

kinds of nodes. First one is the initiator who pro-

poses the deep learning model and the proﬁt that each

node will earn. Apart from that there is the contrib-

utor which does the computations and gets tokens as

a reward and the veriﬁcator who veriﬁes the correct-

ness of the results. No practical implementation is

presented.

In (Kurtulmus and Daniel, 2018) a trust less ma-

chine learning distributed network is presented where

the orchestration of the whole process is done exclu-

sively through Ethereum smart contracts. Nodes com-

pute the models and upload the results to smart con-

tracts for veriﬁcation. Smart contracts automatically

verify the results and pay the required rewards. That

creates a machine learning market where users utilize

available GPU power to solve machine learning tasks

and get proﬁt instead of mining cryptocurrencies.

Our approach, while still in early stages, combines

privacy preserving techniques, blockchain technology

and integrity checking mechanisms, in order to pro-

vide a ﬂexible infrastructure that will enable all users

to engage with modern machine learning. We envi-

sion an open system where end users can either train

machine learning models or offer processing power

and data to others. Such a system is not described in

any of the previous research efforts.

6 FUTURE WORK

Regarding future work, a detailed privacy and security

threat model has to be deﬁned for federated learning

schemes. The main actors that may interfere with per-

sonal data privacy and the procedure’s security are go-

ing to identiﬁed along with the probable attacks that

may be executed by each one of them.

According to this threat model the general archi-

tecture of the proposed system will be designed. This

will include the deﬁnition of nodes taxonomies the

collaboration protocol between such nodes along with

the main building blocks of the proposed system.

Homomorphic encryption techniques perfor-

mance will be tested with regards to machine

learning models training requirements. Also a

large scale federated learning network incorporating

blockchain will be developed to test if any scalability

issues may arise.

Blockchain technology will also be tested in the

context of the proposed system’s requirements. It

must be seamlessly integrated to the system in order

not to hinder the adoption of required users due to

usability issues or limited familiarity with blockchain

technology.

Lastly integrity violations is a great threat to such

an ecosystem. A detailed analysis of the attack vec-

tors is going to be applied, in order to research on all

probable issues and design speciﬁc counter-measures

that can be enforced.

7 CONCLUSIONS

In conclusion, federated learning is a powerful tool

for deploying machine learning in distributed net-

works. Meanwhile, combining blockchain in this ar-

chitecture makes it even more robust and creates an

ecosystem where users can share their data or pro-

cessing power for monetary reward, which is a great

incentive for engaging more users. Also security and

privacy measures need to be applied because during

the training sensitive and personal data are shared.

In general this is a promising approach that could

make recent machine learning technology advance-

ments available to more users. In order for it to be

effective, a lot of effort is required for designing and

implementing all the required building blocks.

REFERENCES

Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z.,

Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin,

M., Ghemawat, S., Goodfellow, I., Harp, A., Irving,

G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kud-

lur, M., Levenberg, J., Man

e, D., Monga, R., Moore,

S., Murray, D., Olah, C., Schuster, M., Shlens, J.,

Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Van-

houcke, V., Vasudevan, V., Vi

egas, F., Vinyals, O.,

Warden, P., Wattenberg, M., Wicke, M., Yu, Y., and

Zheng, X. (2015). TensorFlow: Large-scale machine

ICISSP 2019 - 5th International Conference on Information Systems Security and Privacy

550

learning on heterogeneous systems. Software avail-

able from tensorﬂow.org.

Aono, Y., Hayashi, T., Wang, L., Moriai, S., et al. (2017).

Privacy-preserving deep learning: Revisited and en-

hanced. In International Conference on Applications

and Techniques in Information Security, pages 100–

110. Springer.

Backes, M., Berrang, P., Hecksteden, A., Humbert, M.,

Keller, A., and Meyer, T. (2016). Privacy in epigenet-

ics: Temporal linkability of microrna expression pro-

ﬁles. In USENIX Security Symposium, pages 1223–

1240.

Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., and

Shmatikov, V. (2018). How to backdoor federated

learning. arXiv preprint arXiv:1807.00459.

Bennett, J., Lanning, S., et al. (2007). The netﬂix prize.

In Proceedings of KDD cup and workshop, volume

2007, page 35. New York, NY, USA.

Chamatidis, I., Katsika, A., and Spathoulas, G. (2017). Us-

ing deep learning neural networks for ecg based au-

thentication. In Security Technology (ICCST), 2017

International Carnahan Conference on, pages 1–6.

IEEE.

Collobert, R. and Weston, J. (2008). A uniﬁed architec-

ture for natural language processing: Deep neural net-

works with multitask learning. In Proceedings of the

25th international conference on Machine learning,

pages 160–167. ACM.

Deng, L., Hinton, G., and Kingsbury, B. (2013). New types

of deep neural network learning for speech recogni-

tion and related applications: An overview. In Acous-

tics, Speech and Signal Processing (ICASSP), 2013

IEEE International Conference on, pages 8599–8603.

IEEE.

Deng, L., Yu, D., et al. (2014). Deep learning: methods

and applications. Foundations and Trends

 in Signal

Processing, 7(3–4):197–387.

Hitaj, B., Ateniese, G., and Perez-Cruz, F. (2017). Deep

models under the gan: information leakage from col-

laborative deep learning. In Proceedings of the 2017

ACM SIGSAC Conference on Computer and Commu-

nications Security, pages 603–618. ACM.

Jing, Y., Bian, Y., Hu, Z., Wang, L., and Xie, X.-Q. S.

(2018). Deep learning for drug design: An artiﬁcial

intelligence paradigm for drug discovery in the big

data era. The AAPS journal, 20(3):58.

Kurtulmus, A. B. and Daniel, K. (2018). Trustless ma-

chine learning contracts; evaluating and exchanging

machine learning models on the ethereum blockchain.

arXiv preprint arXiv:1802.10185.

Li, P., Li, J., Huang, Z., Li, T., Gao, C.-Z., Yiu, S.-M., and

Chen, K. (2017). Multi-key privacy-preserving deep

learning in cloud computing. Future Generation Com-

puter Systems, 74:76–85.

Liu, F., Liu, B., Sun, C., Liu, M., and Wang, X. (2013).

Deep learning approaches for link prediction in so-

cial network services. In International Conference

on Neural Information Processing, pages 425–432.

Springer.

Melis, L., Song, C., De Cristofaro, E., and Shmatikov, V.

(2018). Inference attacks against collaborative learn-

ing. arXiv preprint arXiv:1805.04049.

Mendis, G. J., Sabounchi, M., Wei, J., and Roche, R.

(2018). Blockchain as a service: An autonomous,

privacy preserving, decentralized architecture for deep

learning. arXiv preprint arXiv:1807.02515.

Min, S., Lee, B., and Yoon, S. (2017). Deep learn-

ing in bioinformatics. Brieﬁngs in bioinformatics,

18(5):851–869.

Na, S.-H. (2015). Deep learning for natural language pro-

cessing and machine translation.

Nakamoto, S. (2008). Bitcoin: A peer-to-peer electronic

cash system.

Narayanan, A. and Shmatikov, V. (2008). Robust de-

anonymization of large sparse datasets. In Security

and Privacy, 2008. SP 2008. IEEE Symposium on,

pages 111–125. IEEE.

Noda, K., Yamaguchi, Y., Nakadai, K., Okuno, H. G., and

Ogata, T. (2015). Audio-visual speech recognition us-

ing deep learning. Applied Intelligence, 42(4):722–

737.

Orekondy, T., Oh, S. J., Schiele, B., and Fritz, M.

(2018). Understanding and controlling user link-

ability in decentralized learning. arXiv preprint

arXiv:1805.05838.

Rouhani, B. D., Riazi, M. S., and Koushanfar, F. (2018).

Deepsecure: Scalable provably-secure deep learning.

In 2018 55th ACM/ESDA/IEEE Design Automation

Conference (DAC), pages 1–6. IEEE.

Takabi, H., Hesamifard, E., and Ghasemi, M. (2016). Pri-

vacy preserving multi-party machine learning with ho-

momorphic encryption. In 29th Annual Conference on

Neural Information Processing Systems (NIPS).

Weng, J., Weng, J., Li, M., Zhang, Y., and Luo, W. (2018).

Deepchain: Auditable and privacy-preserving deep

learning with blockchain-based incentive. Technical

report, Cryptology ePrint Archive, Report 2018/679.

2018. Available online: https . . . .

Zeiler, M. D. (2013). Hierarchical convolutional deep

learning in computer vision. PhD thesis, New York

University.

Zhou, Y., Wilkinson, D., Schreiber, R., and Pan, R. (2008).

Large-scale parallel collaborative ﬁltering for the net-

ﬂix prize. In International Conference on Algorith-

mic Applications in Management, pages 337–348.

Springer.

Machine Learning for All: A More Robust Federated Learning Framework

551