Kernel Completion for Learning Consensus Support Vector Machines in Bandwidth-limited Sensor Networks

Sangkyun Lee, Christian Pölitz

2014

Abstract

Recent developments in sensor technology allows for capturing dynamic patterns in vehicle movements, temperature changes, and sea-level fluctuations, just to name a few. A usual way for decision making on sensor networks, such as detecting exceptional surface level changes across the Pacific ocean, involves collecting measurement data from all sensors to build a predictor in a central processing station. However, data collection becomes challenging when communication bandwidth is limited, due to communication distance or low-energy requirements. Also, such settings will introduce unfavorable latency for making predictions on unseen events. In this paper, we propose an alternative strategy for such scenarios, aiming to build a consensus support vector machine (SVM) in each sensor station by exchanging a small amount of sampled information from local kernel matrices amongst peers. Our method is based on decomposing a “global” kernel defined with all features into “local” kernels defined only with attributes stored in each sensor station, sampling few entries of the decomposed kernel matrices that belong to other stations, and filling in unsampled entries in kernel matrices by matrix completion. Experiments on benchmark data sets illustrate that a consensus SVM can be built in each station using limited communication, which is competent in prediction performance to an SVM built with accessing all features.

References

  1. Bache, K. and Lichman, M. (2013). UCI machine learning repository.
  2. Bertsekas, D. P. and Tsitsiklis, J. N. (1997). Parallel and Distributed Computation: Numerical Methods. Athena Scientific.
  3. Boser, B. E., Guyon, I. M., and Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual workshop on Computational learning theory, pages 144-152.
  4. Boyd, S., Parikh, N., Chu, E., Peleato, B., and Eckstein, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1):1- 122.
  5. Candès, E. J. and Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6):717-772.
  6. Candès, E. J. and Recht, B. (2012). Exact matrix completion via convex optimization. Communications of the ACM, 55(6):111-119.
  7. Crammer, K., Dredze, M., and Pereira, F. (2012). Confidence-weighted linear classification for natural language processing. Journal of Machine Learning Research, 13:1891-1926.
  8. Forero, P. A., Cano, A., and Giannakis, G. B. (2010). Consensus-based distributed support vector machines. Journal of Machine Learning Research, 11:1663- 1707.
  9. Ji, Y. and Sun, S. (2013). Multitask multiclass support vector machines: Model and experiments. Pattern Recognition, 46(3):914-924.
  10. Joachims, T. (1999). Making large-scale support vector machine learning practical. In Schölkopf, B., Burges, C., and Smola, A., editors, Advances in Kernel Methods - Support Vector Learning, chapter 11, pages 169-184. MIT Press, Cambridge, MA.
  11. Lanckriet, G., Cristianini, N., Bartlett, P., E. G., and L., Jordan, M. (2002). Learning the kernel matrix with semidefinite programming. In Proceedings of the 19th International Conference on Machine Learning.
  12. Lanckriet, G. R. G., De Bie, T., Cristianini, N., Jordan, M. I., and Noble, W. S. (2004). A statistical framework for genomic data fusion. Bioinformatics, 20(16):2626-2635.
  13. Lee, S. and Bockermann, C. (2011). Scalable stochastic gradient descent with improved confidence. In Big Learning - Algorithms, Systems, and Tools for Learning at Scale, NIPS Workshop.
  14. Lee, S., Stolpe, M., and Morik, K. (2012). Separable approximate optimization of support vector machines for distributed sensing. In Flach, P., Bie, T., and Cristianini, N., editors, Machine Learning and Knowledge Discovery in Databases, volume 7524 of Lecture Notes in Computer Science, pages 387-402. Springer.
  15. Lippi, M., Bertini, M., and Frasconi, P. (2010). Collective traffic forecasting. In Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part II, pages 259-273.
  16. Morik, K., Bhaduri, K., and Kargupta, H. (2012). Introduction to data mining for sustainability. Data Mining and Knowledge Discovery, 24(2):311-324.
  17. Rakotomamonjy, A., Bach, F., Canu, S., and Grandvalet., Y. (2007). More efficiency in multiple kernel learning. In Proceedings of the 24th International Conference on Machine Learning.
  18. Recht, B., Fazel, M., and Parrilo, P. A. (2010). Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review, 52(3):471-501.
  19. Recht, B. and Ré, C. (2011). Parallel stochastic gradient algorithms for large-scale matrix completion. Technical report, University of Wisconsin-Madison.
  20. Schoenberg, I. J. (1938). Metric spaces and positive definite functions. Transactions of the American Mathematical Society, 44(3):522-536.
  21. Scholkopf, B. and Smola, A. J. (2001). Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, USA.
  22. Shawe-Taylor, J. and Sun, S. (2011). A review of optimization methodologies in support vector machines. Neurocomputing, 74(17):3609-3618.
  23. Stolpe, M., Bhaduri, K., Das, K., and Morik, K. (2013). Anomaly detection in vertically partitioned data by distributed core vector machines. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2013.
  24. Whittaker, J., Garside, S., and Lindveld, K. (1997). Tracking and predicting a network traffic process. International Journal of Forecasting, 13(1):51-61.
Download


Paper Citation


in Harvard Style

Lee S. and Pölitz C. (2014). Kernel Completion for Learning Consensus Support Vector Machines in Bandwidth-limited Sensor Networks . In Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-018-5, pages 113-124. DOI: 10.5220/0004829401130124


in Bibtex Style

@conference{icpram14,
author={Sangkyun Lee and Christian Pölitz},
title={Kernel Completion for Learning Consensus Support Vector Machines in Bandwidth-limited Sensor Networks},
booktitle={Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2014},
pages={113-124},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004829401130124},
isbn={978-989-758-018-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Kernel Completion for Learning Consensus Support Vector Machines in Bandwidth-limited Sensor Networks
SN - 978-989-758-018-5
AU - Lee S.
AU - Pölitz C.
PY - 2014
SP - 113
EP - 124
DO - 10.5220/0004829401130124