Q-Routing in Cognitive Packet Network Routing Protocol for MANETs

Amal Alharbi, Abdullah Al-Dhalaan, Miznah Al-Rodhaan

2014

Abstract

Mobile Ad hoc Networks (MANET) are self-organized networks which are characterized by dynamic topologies in time and space. This creates an instable environment, where classical routing approaches cannot achieve high performance. Thus, adaptive routing is necessary to handle the random changing network topology. This research uses Reinforcement Learning approach with Q-Routing to introduce our MANET routing algorithm: Stability-Aware Cognitive Packet Network (CPN). This new algorithm extends the work on CPN to adapt it to the MANET environment with focus on path stability metric. CPN is a distributed adaptive routing protocol that uses three types of packets: Smart Packets for route discovery, Data Packets for carrying data payload, and Acknowledgments to bring back feedback information for the Reinforcement Learning reward function. The research defines a reward function as a combination of high stability and low delay path criteria to discover long-lived routes without disrupting the overall delay. The algorithm uses Acknowledgment-based Q-routing to make routing decisions which adapt on line to network changes allowing nodes to learn efficient routing policies.

References

  1. Rahul D., Patil B. (2013) 'MANET with Q-Routing Protocol', Medwell Journal of Information Technology, vol. 12. Issue 1, pp.20-26.
  2. Sachi, M., Prakash, C.(2013) 'QoS Improvement for MANET Using AODV Algorithm by Implementing Q-Learning Approach', International Journal of Computer Science and Technology IJCST vol.4 , issue 1, pp. 407-409.
  3. Wang K. (2012) 'A MANET Routing Protocol Using QLearning Method Integrated with Bayesian Network 78, Proceedings of International Conference on Communication Systems (ICCS), Singapore, 2012, IEEE, pp. 270-274.
  4. Santhi, G., Nachiappan, A. Ibrahime, M.,Raghunadhane, R. (2011) 'Q-Learning Based Adaptive QoS Routing Protocol for MANNETS' , Proceedings of the International Conference on Recent Trends in Information Technology. Chennai: IEEE. Pp. 1233- 1238.
  5. Kulkani, S., Rao, R. (2010) 'Performance Optimization of Reinfrocement Learning Based Routing Algorithm Applied to Ad hoc Networks', International Journal of Computer Networks and Communications, vol. 2, pp. 46-60.
  6. Chetret, D. L., (2009) 'Adaptive Routing in MANETS using RL and CMAC'. Master Thesis submitted to Department of Electrical and Computer Engineering, National University of Singapore.
  7. Skellari, G. (2009), 'Maintaining Quality of Service and Reliability in Self-Aware Networks”, PhD Thesis for Imperial College of London.
  8. Gelenbe, E., Liu P., and Lain, J. (2008) 'Genetic Algorithms for Autonomic Route Discovery', in Proceedings of IEEE Conference on Distributed Intelligent Systems, Washington, USA.
  9. Skellari, G., Hey L., and Gelenbe, E. (2008), 'Adaptability and Failure Resilience of the Cognitive Packet Network', 27th IEEE Conference on Computer Communications, Phoenix, Arizona, USA.
  10. Mehandthan, N. (2008) “Exploring the Stability-Energy Consumption-Delay Network Lifetime Trade-off of MANET Routing Protocol”, Journal of Networks , vol.3, No.2.
  11. Targen, J., Chuang, B. (2007) 'A Novel Stability-Based Routing for Mobile Ad hoc Networks”, IEICE Transactions on Communications, B(4) : 876-884.
  12. Murad, A., Al-Mahadeen, B. (2007) 'Adding Quality of Service Extensions to Enhanced ABR Routing protocol', American Journal of Applied Science.
  13. Forster, A., Murphy, A. (2007) 'Feedback Routing for Optimized Multiple Sinks in WSN with Reinforcement Learning”, Proc. 3rd International Conference of Intelligent Sensors, Sensor Networks, Information Process (ISSNIP).
  14. Han, Y., (2006) 'Maximizing Path Duration in MANETS', Maryland, USA.
  15. Trivino-Cabrera A., Nieves-Perez, I. (2006) 'Ad hoc Routing based on Stability of Routes', Proceedings of the 4th ACM International Workshop on Mobility Management and Wireless Access, ACM, New York, USA. Pp. 100-103.
  16. Gellman, M., Liu, P. (2006), 'Random Neural for Adaptive Control of Packet Networks', Proceedings of 16th International on Artificial Neural Networks, Greece, pp.313-320.
  17. Lent, R. (2006) 'Smart Packet-Based Selection of Reliable Paths in Ad hoc Networks', IEEE.
  18. Toa, T. Tagashira, S. Fujita, S. (2005) 'LQ-Routing Protocol for MANETs', In proc. Of Fourth Annual ACIS International Conference on Computer and Information Science.
  19. Liu, C., Kaiser J. (2005), 'A Survey of Mobile Ad hoc Networks Routing Protocols'. University of Ulm, Germany.
  20. Lent, R., Zanoozi, R. (2005) 'Power Control in Ad hoc Cognitive Packet Network', Texas Wireless Symposium.
  21. Chang, Y. , Ho, T. (2004) 'Mobilized Ad hoc Networks: A Reinforcement Learning Approach.78 Proc. Of First International Conference on Autonomic Computing IEEE Computer Society USA, pp. 240-247.
  22. Chung, W. (2004) 'Probabilistic Analysis of Routing on Mobile Ad hoc Networks', IEEE Communication Letters 8(8): 506508.
  23. Toh, C. K. , Lee , E. C. , and Ramos N. (2004), 'Next Generation Tactical Ad hoc Mobile Wireless Network' ,Technology Review Journal.
  24. Gelenbe E., Lent R., and A. Nunez, (2004), 'Self-Aware Networks and Quality of Service', Proceedings of IEEE vol. 92(9): 1478-1489.
  25. Gelenbe, E., Lent, R. (2004), 'Power-Aware Ad hoc Cognitive Packet Network', Elsevier.
  26. Punde, J., Pissinou, N. (2003) 'On Quality of Service Routing in Ad hoc Networks”, Proceedings of the 28th IEEE International Conference on Local Computer Networks, IEEE Computer Society, Washington, USA.pp.276-278.
  27. Tseng, Y., Chang, C. (2003) 'On Route Lifetime in Multihop Mobile Ad hoc Networks', IEEE Transactions on Mobile Computing 2(4): 366-376.
  28. Camp, T., Davies, J. (2002) 'Survey of Mobility Models for Ad hoc Networks Research”, Wireless Communication & Mobile Computing (WCMC): Special issue on MANET: Research Trend & Application 2 (5): 483-502.
  29. Peshkin, L., Savova, V. (2002) 'Reinforcement Learning for Adaptive Routing', IEEE.
  30. Gelenbe, E., Lent R., and Xu Z.,(2001), 'Design and Performance of Cognitive Packet Network', Performance Evaluation, 46,pp.155-176.
  31. Halici, U. (2000), 'Reinforcement Learning with Internal Expectation for The Random Neural Network', Eur.J.Opns.Res. 126(2)2, pp.288-307.
  32. Perkins, C. E. (2001), Ad Hoc Networking, AddisonWesley.
  33. Gelenbe, E., Lent, R. and Xu Z.,(2001) 'Measurement and Performance of Cognitive Packet Network', Journal of Computer Networks, 46,pp.155-176.
  34. Toh, C. K., Vassiliou, V. (2000) 'The effects of Beaconing on the Battery life of Ad hoc Mobile Computers'.
  35. Manikandan, S., Naveen, L. and Padmanban, R., (2000) 'Optmized Associativity-Based Threshold Routing for Mobile Ad hoc Netwroks', 2000.
  36. Boyan, J., Littman, B. (1999) 'Packet Routing in Dynamic Changing Networks: A Reinforcement Learning Approach', In Advances in Neural Information Processing Systems, vol.12, pp. 893-899.
  37. Brown, T. X, (1999) 'Low Power Wireless Communication via Reinforcement Learning', In Advances in Neural Information Processing Systems.
  38. Sutton, R., Barto, S. (1998) Reinforcement Learning: an Introduction, MIT Press: Cambridge.
  39. Toh, C.K. (1997) 'Associativity Based Routing for Mobile Ad hoc Networks', IEEE Conference on Computer & Communications ( IPCCC), Phoenix, USA.
  40. Dube, R., Rais, C. R. and Wang, K.Y. (1997) 'Signal Stability Based Adaptive Routing SSA for Ad hoc Mobile Networks', Personal Communication, IEEE, 4(1): 36-45.
  41. Gelenbe, E. (1993), 'Learning in the Recurrent Random Neural Network'. Neural Computing, 5(1), pp. 154- 164. 1993.
Download


Paper Citation


in Harvard Style

Alharbi A., Al-Dhalaan A. and Al-Rodhaan M. (2014). Q-Routing in Cognitive Packet Network Routing Protocol for MANETs . In Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014) ISBN 978-989-758-054-3, pages 234-243. DOI: 10.5220/0005082902340243


in Bibtex Style

@conference{ncta14,
author={Amal Alharbi and Abdullah Al-Dhalaan and Miznah Al-Rodhaan},
title={Q-Routing in Cognitive Packet Network Routing Protocol for MANETs},
booktitle={Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014)},
year={2014},
pages={234-243},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005082902340243},
isbn={978-989-758-054-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2014)
TI - Q-Routing in Cognitive Packet Network Routing Protocol for MANETs
SN - 978-989-758-054-3
AU - Alharbi A.
AU - Al-Dhalaan A.
AU - Al-Rodhaan M.
PY - 2014
SP - 234
EP - 243
DO - 10.5220/0005082902340243