A SECURE JAILING SYSTEM FOR CONFINING UNTRUSTED APPLICATIONS

Guido van ’t Noordende, Ádám Balogh, Rutger Hofman, Frances M. T. Brazier, Andrew S. Tanenbaum

2007

Abstract

System call interception based jailing is a well-known method for confining (sandboxing) untrusted binary applications. Existing systems that are implemented using standard UNIX debugging mechanisms are rendered insecure by several race conditions. This paper gives an overview of the most important threats to jailing systems, and presents novel mechanisms for implementing jailing securely on standard UNIX systems. We implemented these solutions on Linux, and achieve competitive performance compared to existing jailing systems. Performance results are provided for this implementation, and for an implementation that uses a special-purpose extension to the Linux kernel designed to improve performance of the jailing system.

References

  1. Alexandrov, A., Kmiec, P., and Schauser, K. (1999). Consh: Confined execution environment for internet computations. http://www.cs.ucsb.edu/˜berto/papers/99- usenix-consh.ps.
  2. Back, G. and Hsieh, W. (1999). Drawing the red line in java. Workshop on Hot Topics in Operating Systems (HotOS VII). pp. 116-121.
  3. Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Pratt, I., Warfield, A., Barham, P., and Neugebauer., R. (2003). Xen and the art of virtualization. Proc. ACM Symposium on Operating Systems Principles (SOSP).
  4. Efstathopoulos, P., Krohn, M., VanDeBogart, S., Frey, C., Ziegler, D., Kohler, E., Mazières, D., Kaashoek, F., and Morris, R. (2005). Labels and event processes in the asbestos operating system. Proc. 20th Symposium on Operating Systems Principles (SOSP), Brighton, United Kingdom.
  5. Engler, D., Kaashoek, M., and O'Toole Jr., J. (1995). Exokernel: an operating system architecture for application-specific resource management. Proc. Fifteenth ACM Symposium on Operating Systems Principles (SOSP). pp. 251-266.
  6. Garfinkel, T. (2003). Traps and pitfalls: Practical problems in system call interception based security tools. Proc. Symposium on Network and Distributed System Security (NDSS). pp. 163-176.
  7. Garfinkel, T., Pfaff, B., and Rosenblum, M. (2004). Ostia: A delegating architecture for secure system call interposition. Proc. ISOC Network and Distributed System Security Symposium (NDSS).
  8. Ghormley, D., Rodrigues, S., Petrou, D., and Anderson., T. (1998). Slic: An extensibility system for commodity operating systems. USENIX 1998 Annual Technical Conference.
  9. Goldberg, I., Wagner, D., Thomas, R., and Brewer, E. (1996). A secure environment for untrusted helper applications - confining the wily hacker. Proc. 6th Usenix Security Symposium. San Jose, CA, USA.
  10. Jain, K. and Sekar, R. (2000). User-level infrastructure for system call interposition: A platform for intrusion detection and confinement. ISOC Network and Distributed System Security Symposium (NDSS). pp. 19- 34.
  11. Kamp, P. and Watson, R. (2000). Jails: Confining the omnipotent root. Proc. 2nd Intl. SANE Conference.
  12. Liang, Z., Venkatakrishnan, V., and Sekar, R. (2003). Isolated program execution: An application transparent approach for executing untrusted programs. 19th Annual Computer Security Applications Conference (ACSAC), Las Vegas, Nevada.
  13. Mazières, D. and Kaashoek, M. (1997). Secure applications need flexible operating systems. Workshop on Hot Topics in Operating Systems (HotOS).
  14. Ousterhout, J., Levy, J., and Welch., B. (1997). The safetcl security model. Sun Microsystems Laboratories Technical Report TR-97-60.
  15. Peterson, D., Bishop, M., and Pandey, R. (2002). A flexible containment mechanism for executing untrusted code. Usenix Security Symposium.
  16. Provos, N. (2003). Improving host security with system call policies. Proc. 12th USENIX Security Symposium. pp. 257-272.
  17. van 't Noordende, G., Balogh, A., Hofman, R., Brazier, F., and Tanenbaum, A. (2006). A secure and portable jailing system. Technical report IR-CS-025, Vrije Universiteit.
  18. Wallach, D., Balfanz, D., Dean, D., and Felten, E. (1997). Extensible security architectures for java. 16th ACM Symposium on Operating Systems Principles. pp. 116- 128.
Download


Paper Citation


in Harvard Style

van ’t Noordende G., Balogh Á., Hofman R., M. T. Brazier F. and S. Tanenbaum A. (2007). A SECURE JAILING SYSTEM FOR CONFINING UNTRUSTED APPLICATIONS . In Proceedings of the Second International Conference on Security and Cryptography - Volume 1: SECRYPT, (ICETE 2007) ISBN 978-989-8111-12-8, pages 414-423. DOI: 10.5220/0002129404140423


in Bibtex Style

@conference{secrypt07,
author={Guido van ’t Noordende and Ádám Balogh and Rutger Hofman and Frances M. T. Brazier and Andrew S. Tanenbaum},
title={A SECURE JAILING SYSTEM FOR CONFINING UNTRUSTED APPLICATIONS},
booktitle={Proceedings of the Second International Conference on Security and Cryptography - Volume 1: SECRYPT, (ICETE 2007)},
year={2007},
pages={414-423},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002129404140423},
isbn={978-989-8111-12-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Second International Conference on Security and Cryptography - Volume 1: SECRYPT, (ICETE 2007)
TI - A SECURE JAILING SYSTEM FOR CONFINING UNTRUSTED APPLICATIONS
SN - 978-989-8111-12-8
AU - van ’t Noordende G.
AU - Balogh Á.
AU - Hofman R.
AU - M. T. Brazier F.
AU - S. Tanenbaum A.
PY - 2007
SP - 414
EP - 423
DO - 10.5220/0002129404140423