loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Pu Chen and Hung-Hsuan Chen

Affiliation: Computer Science and Information Engineering, National Central University, Taoyuan, Taiwan

Keyword(s): Matrix Factorization, Collaborative Filtering, SVD, Recommender Systems, Overparameterization.

Abstract: This paper studies overparameterization on the matrix factorization (MF) model. We confirm that overparameterization can significantly accelerate the optimization of MF with no change in the expressiveness of the learning model. Consequently, modern applications on recommendations based on MF or its variants can largely benefit from our discovery. Specifically, we theoretically derive that applying the vanilla stochastic gradient descent (SGD) on the overparameterized MF model is equivalent to employing gradient descent with momentum and adaptive learning rate on the standard MF model. We empirically compare the overparameterized MF model with the standard MF model based on various optimizers, including vanilla SGD, AdaGrad, Adadelta, RMSprop, and Adam, using several public datasets. The experimental results comply with our analysis – overparameterization converges faster. The overparameterization technique can be applied to various learning-based recommendation models, including dee p learning-based recommendation models, e.g., SVD++, nonnegative matrix factorization (NMF), factorization machine (FM), NeuralCF, Wide&Deep, and DeepFM. Therefore, we suggest utilizing the overparameterization technique to accelerate the training speed for the learning-based recommendation models whenever possible, especially when the size of the training dataset is large. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 52.91.0.68

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Chen, P. and Chen, H. (2020). Accelerating Matrix Factorization by Overparameterization. In Proceedings of the 1st International Conference on Deep Learning Theory and Applications - DeLTA; ISBN 978-989-758-441-1, SciTePress, pages 89-97. DOI: 10.5220/0009885600890097

@conference{delta20,
author={Pu Chen. and Hung{-}Hsuan Chen.},
title={Accelerating Matrix Factorization by Overparameterization},
booktitle={Proceedings of the 1st International Conference on Deep Learning Theory and Applications - DeLTA},
year={2020},
pages={89-97},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009885600890097},
isbn={978-989-758-441-1},
}

TY - CONF

JO - Proceedings of the 1st International Conference on Deep Learning Theory and Applications - DeLTA
TI - Accelerating Matrix Factorization by Overparameterization
SN - 978-989-758-441-1
AU - Chen, P.
AU - Chen, H.
PY - 2020
SP - 89
EP - 97
DO - 10.5220/0009885600890097
PB - SciTePress