Presentation Type
Poster Presentation
Abstract
Interpreting free-text medication orders in Electronic Health Record (EHR) systems is challenging due to variability, increasing the risk of errors and non-compliance with Joint Commission (TJC) standards. This research develops Natural Language Processing (NLP) tools to enhance medication order accuracy, particularly for Duplicate PRN orders. Our open-source, vendor-neutral approach ensures broad applicability across healthcare institutions while maintaining data security.
Using Python and the Natural Language Toolkit (NLTK), we implemented a four-phase methodology. First, the NLP framework was developed on Google Colab for collaborative accessibility. Next, models were trained on 10,000 anonymized synthetic free-text medication orders to detect non-compliant entries. Model validation was conducted using precision, recall, and F1 score metrics, achieving high accuracy. The precision score of 1.000 demonstrated no false positives, while a recall of 0.989 indicated minimal missed duplicates, resulting in an F1 score of 0.995. These findings highlight the model’s strong reliability in identifying non-compliant PRN orders.
Our NLP tools flagged 12.7% of PRN orders as potential duplicates, underscoring the prevalence of therapeutic duplication in free-text entries. These results validate the potential of NLP-driven automation for improving TJC compliance and medication safety. Future work will focus on real-world validation and integration into educational resources to enhance healthcare professionals’ understanding of compliance standards.
Faculty Mentor
Dr. Jay Dorris- Lipscomb University College of Pharmacy
Recommended Citation
Breton, Preslie, "Developing NLP-Based Open-Source Tools to Improve Medication Order Interpretation in EHR Systems" (2025). Student Scholar Symposium. 97.
https://digitalcollections.lipscomb.edu/student_scholars_symposium/2025/Full_schedule/97
Developing NLP-Based Open-Source Tools to Improve Medication Order Interpretation in EHR Systems
Interpreting free-text medication orders in Electronic Health Record (EHR) systems is challenging due to variability, increasing the risk of errors and non-compliance with Joint Commission (TJC) standards. This research develops Natural Language Processing (NLP) tools to enhance medication order accuracy, particularly for Duplicate PRN orders. Our open-source, vendor-neutral approach ensures broad applicability across healthcare institutions while maintaining data security.
Using Python and the Natural Language Toolkit (NLTK), we implemented a four-phase methodology. First, the NLP framework was developed on Google Colab for collaborative accessibility. Next, models were trained on 10,000 anonymized synthetic free-text medication orders to detect non-compliant entries. Model validation was conducted using precision, recall, and F1 score metrics, achieving high accuracy. The precision score of 1.000 demonstrated no false positives, while a recall of 0.989 indicated minimal missed duplicates, resulting in an F1 score of 0.995. These findings highlight the model’s strong reliability in identifying non-compliant PRN orders.
Our NLP tools flagged 12.7% of PRN orders as potential duplicates, underscoring the prevalence of therapeutic duplication in free-text entries. These results validate the potential of NLP-driven automation for improving TJC compliance and medication safety. Future work will focus on real-world validation and integration into educational resources to enhance healthcare professionals’ understanding of compliance standards.