Comparing Computerized Physician Order Entry Usability between Expert and Novice Primary Care Physicians

Martina Clarke, Jeffery L. Belden, Min Soon Kim

2015

Abstract

Objectives: To examine usability gaps between expert and novice primary care physicians when using computerized provider order entry (CPOE). Methods: To analyze usability gaps between ten novice and seven expert physicians, using the triangular method approach, usability tests involving video analysis were conducted. Results: While most novice physicians completed tasks less proficiently, and provided a lower System Usability Scale (SUS) score than expert physicians, the result of ‘percent task success rate’ (t(8) = 2.31, p=0.98) was not significant for both physician groups on all five tasks. Seven common and four unique usability issues were identified between the two physician groups. Three themes emerged during analysis: user interface issues, ambiguous terminologies, and training and education issues. Discussion and Conclusion: This study identified varying usability issues for users of CPOE with different expertise. Two additional iterations of the usability data collections are undergoing to uncover comprehensive usability issues and measure the learnability.

References

  1. Ash, J. S., Stavri, P. Z. & Kuperman, G. J. 2003. A consensus statement on considerations for a successful CPOE implementation. Journal of the American Medical Informatics Association, 10, 229- 34.
  2. Bangor, A., Kortum, P. & Miller, J. 2009. Determining what individual SUS scores mean: adding an adjective rating scale. Journal of Usability Studies, 4, 114-123.
  3. Barnum, C. 2003. The magic number 5: Is it enough for web testing? Information Design Journal, 11, 160- 170.
  4. Battaglia, M. 2008. Convenience sampling. In P. Lavrakas (Ed.), Encyclopedia of survey research methods., Thousand Oaks, CA, Sage Publications, Inc.
  5. Berger, R. G. & Kichak, J. P. 2004. Computerized physician order entry: helpful or harmful? Journal of the American Medical Informatics Association, 11, 100-103.
  6. Braun, V. & Clarke, V. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77-101.
  7. Brooke, J. 1996. SUS-A quick and dirty usability scale. Usability Evaluation in Industry, 189, 194.
  8. Chan, J., Shojania, K. G., Easty, A. C. & Etchells, E. E. 2011. Usability evaluation of order sets in a computerised provider order entry system. BMJ Qual Saf, 20, 932-40.
  9. Cordes, R. E. 1993. The effects of running fewer subjects on time-on-task measures. International Journal of Human-Computer Interaction, 5, 393 - 403.
  10. Devine, E. B., Hansen, R. N., Wilson-Norton, J. L., Lawless, N. M., Fisk, A. W., Blough, D. K., Martin, D. P. & Sullivan, S. D. 2010. The impact of computerized provider order entry on medication errors in a multispecialty group practice. Journal of the American Medical Informatics Association, 17, 78-84.
  11. Ghahramani, N., Lendel, I., Haque, R. & Sawruk, K. 2009. User satisfaction with computerized order entry system and its effect on workplace level of stress. Journal of Medical Systems, 33, 199-205.
  12. Higgins, J. 2005. The Correlation Coefficient. The Radical Statistician, 1-26.
  13. Horsky, J., Kaufman, D. R. & Patel, V. L. 2004. Computer-based drug ordering: evaluation of interaction with a decision-support system. Studies in Health Technology and Informatics, 107, 1063-7.
  14. Khajouei, R. & Jaspers, M. W. 2010. The impact of CPOE medication systems' design aspects on usability, workflow and medication orders: a systematic review. Methods of Information in Medicine, 49, 3-19.
  15. Kim, M., Shapiro, J., GENES, N., Aguilar, M., Mohrer, D., Baumlin, K. & Belden, J. 2012. A pilot study on usability analysis of emergency department information system by nurses. Applied Clinical Informatics, 135-153.
  16. Kjeldskov, J., Skov, M. B. & Stage, J. 2010. A longitudinal study of usability in health care: does time heal? Int J Med Inform, 79, e135-43.
  17. Koppel, R., Metlay, J. P., Cohen, A., Abaluck, B., Localio, A. R., Kimmel, S. E. & Strom, B. L. 2005. Role of computerized physician order entry systems in facilitating medication errors. JAMA, 293, 1197- 203.
  18. Medlineplus. 2013. Basic metabolic panel (Online). National Library of Science, National Institues of Health. Available: http://www.nlm.nih.gov/ medlineplus/ency/article/003462.htm (Accessed 2/25 2014).
  19. Neinstein, A. & Cucina, R. 2011. An analysis of the usability of inpatient insulin ordering in three computerized provider order entry systems. J Diabetes Sci Technol, 5, 1427-36.
  20. Peute, L. W. & Jaspers, M. W. 2007. The significance of a usability evaluation of an emerging laboratory order entry system. International Journal of Medical Informatics, 76, 157-68.
  21. Schoen, C., Osborn, R., Huynh, P. T., Doty, M., Peugh, J. & Zapert, K. 2006. On the front lines of care: primary care doctors office systems, experiences, and views in seven countries. Health Affairs, 25, w555-w571.
  22. Centers for Medicare & Medicaid Services. 2010. Eligible professional meaningful use core measures, measure 14 of 15. Stage.
  23. Walji, M. F., Kalenderian, E., Tran, D., Kookal, K. K., Nguyen, V., Tokede, O., White, J. M., Vaderhobli, R., Ramoni, R., Stark, P. C., Kimmes, N. S., Schoonheim-Klein, M. E. & Patel, V. L. 2013. Detection and characterization of usability problems in structured data entry interfaces in dentistry. International Journal of Medical Informatics, 82, 128-138.
  24. Yui, B. H., Jim, W. T., Chen, M., Hsu, J. M., Liu, C. Y. & Lee, T. T. 2012. Evaluation of computerized physician order entry system-a satisfaction survey in Taiwan. Journal of Medical Systems, 36, 3817-24.
  25. Zhan, C. L., Hicks, R. W., Blanchette, C. M., Keyes, M. A. & Cousins, D. D. 2006. Potential benefits and problems with computerized prescriber order entry: Analysis of a voluntary medication error-reporting database. American Journal of Health-System Pharmacy, 63, 353-358.
Download


Paper Citation


in Harvard Style

Clarke M., Belden J. and Kim M. (2015). Comparing Computerized Physician Order Entry Usability between Expert and Novice Primary Care Physicians . In Proceedings of the International Conference on Health Informatics - Volume 1: HEALTHINF, (BIOSTEC 2015) ISBN 978-989-758-068-0, pages 304-311. DOI: 10.5220/0005188503040311


in Bibtex Style

@conference{healthinf15,
author={Martina Clarke and Jeffery L. Belden and Min Soon Kim},
title={Comparing Computerized Physician Order Entry Usability between Expert and Novice Primary Care Physicians},
booktitle={Proceedings of the International Conference on Health Informatics - Volume 1: HEALTHINF, (BIOSTEC 2015)},
year={2015},
pages={304-311},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005188503040311},
isbn={978-989-758-068-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Health Informatics - Volume 1: HEALTHINF, (BIOSTEC 2015)
TI - Comparing Computerized Physician Order Entry Usability between Expert and Novice Primary Care Physicians
SN - 978-989-758-068-0
AU - Clarke M.
AU - Belden J.
AU - Kim M.
PY - 2015
SP - 304
EP - 311
DO - 10.5220/0005188503040311