The Potential of Automation and AI/ML in Causality Assessment for Safety Vigilance

Causality assessment is a crucial process in safety vigilance that involves determining the relationship between a drug and an adverse event or reaction. The identification of new signals, evaluation of benefit-risk profile, and measurement of evidence strength for pharmaceutical medicinal products heavily rely on the factor of causality. Traditionally, this process has been performed manually by experts, but using technologies such as Machine Learning (ML) and Natural Language Processing (NLP), there is an opportunity to automate this process.

Automated causality assessment is an emerging approach in safety case management that uses ML algorithms and other automated techniques to assess the causes of incidents and hazards. This approach has the potential to improve the accuracy and efficiency of causality assessments, leading to better decision-making and risk management. It overcomes some of the limitations of manual assessments by analysing large volumes of data and identify patterns and correlations. These algorithms can be trained on different types of data including historical data, real-world data, data from spontaneous reporting systems, social media, biomedical literature and knowledge bases with an aim to learn about the causes of incidents and hazards and to develop models that can be used to predict the likelihood of future incidents.

This article will explore various methods and AI/ML models that can be utilized in implementing automated causality assessment in safety vigilance along with the challenges and opportunities.

The World Health Organization's (WHO) Uppsala Monitoring Centre (UMC) and the Naranjo algorithm are currently the two widely used methods for determining the causality of adverse drug reactions (ADRs).

WHO-UMC Causality Assessment Method

The WHO-UMC causality assessment method is a widely used and standardized method for determining the causality of ADRs. It was developed by the UMC, which is a collaborating center of the WHO. The method involves the temporal relationship and some other important criteria for assessing causality such as lab tests results, any sudden abnormalities in the reports, dechallenge and rechallenge values etc. between the drug and the ADR, the known pharmacological properties of the drug, and the presence of alternative explanations for the ADR.

The WHO-UMC method classifies the causality of ADRs into five categories: Certain, Probable/Likely, Possible, Unlikely, and Unclassified. The method provides a standardized framework for evaluating ADRs and is widely used by regulatory agencies and pharmaceutical companies.

Naranjo Scale

The Naranjo Adverse Drug Reaction (ADR) Probability Scale was developed to provide a standardized and systematic approach to assess the likelihood that an ADR was caused by a particular drug.

The Naranjo Scale consists of ten questions that evaluate the relationship between the drug and the ADR. The questions are designed to assess the temporality, the nature and severity of the ADR, the possibility of other causes, the drug's known pharmacology, and the response to rechallenge.

Each question is scored based on the answer, with a score of +1 indicating a positive answer, 0 indicating a neutral answer, and -1 indicating a negative answer. The scores are then totalled to provide an overall score ranging from -4 to +13. The higher the score, the more likely the drug is to have caused the ADR.

While the Naranjo Scale can be a useful tool in causality assessment, it has some limitations. It relies on the accuracy and completeness of the information available, and its results can be subjective, as different assessors may interpret the questions differently. Additionally, the scale was developed for use in assessing ADRs, and may not be appropriate for assessing other types of adverse events.

COLIPA Causality Assessment Method

The guidelines for the assessment of causality of adverse drug reactions (ADRs) related to the use of personal care products and cosmetics have been established by the European Cosmetic, Toiletry and Perfumery Association (COLIPA). These guidelines were published in 1997 and are still used today.

The COLIPA guidelines suggest a structured and systematic approach to assess the causality of ADRs. The guidelines state that the assessment of causality should be based on the following criteria:

• Temporal relationship: The time interval between the use of the cosmetic product and the onset of the adverse reaction is an important factor to consider. The shorter the time interval, the more likely the cosmetic product is to be the cause of the adverse reaction.
• Previous history: If the patient has a previous history of allergic reactions to similar or related products, it may suggest a causal relationship with the current cosmetic product.
• Dechallenge and rechallenge: The effect of stopping and restarting the use of the cosmetic product can help determine causality. If the adverse reaction disappears after stopping the use of the product and reappears after restarting, it suggests a causal relationship.
• Alternative explanations: Other possible causes of the adverse reaction should be considered, such as concurrent medical conditions, concomitant medication use, or exposure to environmental factors.
• Pharmacological properties: Knowledge of the pharmacological properties of the ingredients in the cosmetic product can help determine the plausibility of a causal relationship.
• Dose-response: The severity of the adverse reaction and its relationship to the dose of the cosmetic product used can also help determine causality.

The COLIPA guidelines also suggest that the assessment of causality should be performed by a healthcare professional with experience in the field of dermatology or toxicology. They also recommend that the assessment should be documented and communicated to the patient, the manufacturer of the cosmetic product, and regulatory authorities if necessary.

Overall, the COLIPA guidelines provide a structured and systematic approach to assess the causality of adverse reactions associated with cosmetics and personal care products, which can aid in the evaluation of product safety and inform regulatory decisions.
        
AI/ML Models and Algorithms to Determine Causality

There are various AI/ML models and algorithms that can be used for causality assessment in safety vigilance, including causal inference methods, counterfactual inference, and machine learning techniques. These models can help identify potential causal relationships between risk factors and adverse events, enabling healthcare professionals and safety vigilance teams to take appropriate action to prevent similar events from occurring in the future.

The use of AI/ML models for causality assessment is particularly relevant given the large amount of data generated in healthcare and other industries. ML techniques facilitates the processing and analysis of vast datasets, enabling the identification of patterns and relationships that may not be readily discernible to human analysts. This can help improve the efficiency and effectiveness of safety vigilance programs, reducing the risk of adverse events and improving patient or user safety.

Below are the AI/ML models that can be leveraged to determine Causality in safety vigilance:

1. Decision Trees: Decision trees are one of the most commonly used ML models in safety vigilance. Decision trees are used to identify the causal relationship between a drug and an adverse event. They are used to classify adverse events into different categories based on their cause. Decision trees are useful in identifying the most likely cause of the adverse event and in developing strategies to prevent the recurrence of the event. Rule-based ML Models may be built using Decision Trees. Decision trees are also used in the WHO-UMC and COLIPA Scales. The decision tree is converted into rules that could be expressed as normal ML expressions. This method is excellent when we don't have enough data, or the data quality is undetermined.  However, the predictions are only as good as the underlying rules. Hence, utmost care is needed to define the rules and validate the results over time to build trust in the predictions.

2. Random Forest: Random Forest is a popular ML algorithm used for classification, that is used in safety vigilance to determine causality. Random forest is a type of decision tree algorithm that is used to improve the accuracy of the model. It works by creating a large number of decision trees and then combining their results to produce a more accurate prediction. Random forest is best utilised to address classification challenges, particularly in identifying rare events or events that are difficult to classify. It can be used to identify which features are most strongly associated with a particular outcome.
    
However, for the model to function effectively well, data set availability and data quality are of significance. The FDA supports the use of Random Forest for Causality Prediction, with a human in the loop to validate the findings in a research study in which a random forest model demonstrated the best results in report ranking and accuracy in establishing causal relationships to suspect drugs.

3. Bayesian Networks: Bayesian networks are another type of ML model that can be used to determine causality in safety vigilance. Bayesian networks are probabilistic models that are used to represent the causal relationships between different variables. Bayesian networks are particularly useful in identifying the probability of a particular event occurring given a set of conditions. They can be used to predict the likelihood of an adverse event occurring given certain conditions, such as the patient's age, gender, and medical history.

4. Logistic Regression: Logistic regression is a statistical method used to predict the probability of an event occurring. It is a binary classification algorithm, which means that it predicts the occurrence or non-occurrence of an event. Logistic regression works by fitting a model to a dataset that contains one or more independent variables and a dependent variable that is binary (0 or 1). Logistic regression can be applied in causality assessment to determine the relationship between a drug and an adverse event. The independent variables can be demographic data, medical history, concomitant medications, and the drug in question. The dependent variable is the occurrence or non-occurrence of the adverse event. It can be used to determine the probability of an adverse event occurring as a result of a drug.

5. Neural Networks: Neural networks can be used for causality assessment in pharmacovigilance by training the network on large datasets that contain information about patient demographics, medical history, concomitant medications, and the drug in question. The neural network can then learn the patterns and relationships in the data and use this information to predict the probability of an adverse event occurring because of the drug. One approach to using neural networks for causality assessment is to feed the network with structured data, such as the drug, the patient's age, gender, medical history, and other factors related to the adverse event. The network can learn to identify patterns in the data and use them to make predictions about causality. Another approach is to use unstructured data, such as free-text reports from healthcare providers or social media posts. Natural Language Processing (NLP) techniques can be used to extract relevant information from the text and feed it into the network. The neural network can then learn to identify patterns in the text and use them to make predictions about causality. Neural networks can also be used to improve the accuracy of existing causality assessment methods, such as the Naranjo algorithm or the WHO-Uppsala Monitoring Centre (UMC) system. The network can learn to identify features in the data that are predictive of causality and use them to enhance the performance of the existing method.
    
6. Support Vector Machines (SVMs): Support vector machines are a type of AI/ML model that can be used to determine causality in safety vigilance. SVMs are used to identify the causal relationship between a drug and an adverse event by classifying events into different categories. SVMs work by finding the hyperplane that best separates the data into different classes. SVMs are particularly useful in identifying complex relationships between different variables of events. In a study published in Clinical Pharmacology & Therapeutics, SVM was used to assess the causality between drugs and adverse events (AEs) reported in the US Food and Drug Administration's (FDA) Adverse Event Reporting System (AERS). The authors developed a system that used SVM to classify each drug-AE pair into one of three categories: "positive," "negative," or "unclassified" for causality. The system was trained on a subset of AERS data and then tested on a separate set of data. The results showed that SVM had a high accuracy (over 90%) in classifying drug-AE pairs for causality.

Challenges of Automated Causality Assessment

Automated causality assessment using AI/ML models has the potential to revolutionize the way in which causal relationships between risk factors and adverse events are identified and analyzed. However, there are several challenges associated with the use of AI/ML models for causality assessment that must be considered.

• The first challenge in implementing automated causality assessment in safety vigilance is the need for high-quality data. Causality assessment relies on accurate and detailed information about the drug, the patient, and the adverse event or reaction. This data may be scattered across different sources and formats and may require significant pre-processing and cleaning before it can be used for analysis. Without high-quality data, automated causality assessment may produce inaccurate or unreliable results, which could have serious implications for safety vigilance.
• Second challenge is the need to incorporate expert knowledge and judgment into the analysis. While ML and NLP can be used to process large amounts of data quickly and efficiently, they may not be able to capture the nuances and complexities of safety vigilance. Expert knowledge and judgment are necessary to interpret the results of automated causality assessment, and to ensure that the analysis is consistent with accepted safety vigilance standards and practices.
• Another challenge is the potential for bias and error. ML algorithms and NLP models may produce biased or inaccurate results if they are trained on incomplete or biased data. Additionally, these systems may not be able to capture all the relevant factors that contribute to causality in safety vigilance. As a result, automated causality assessment may produce false positives or false negatives, which could lead to incorrect conclusions about the safety and efficacy of a drug.

Opportunities of Automated Causality Assessment

Despite these challenges, there are advantages for implementing automated causality assessment in safety vigilance:

• One of the biggest advantages of automated causality assessment is its ability to process large amounts of data quickly and efficiently. This can help identify potential safety issues and adverse reactions more quickly, which can lead to faster and more effective interventions to ensure patient safety.
• Another advantage of automated causality assessment is its ability to identify patterns and trends in data that may not be immediately apparent to human experts. ML algorithms can analyse large datasets and identify correlations between drugs and adverse events that may not have been previously recognized. This can lead to new insights into the safety and efficacy of drugs and can help identify potential safety issues before they become widespread.
• A third advantage of automated causality assessment is its potential to reduce the workload of human experts in safety vigilance. With the increasing volume of data being generated in the pharmaceutical industry, it can be challenging for human experts to keep up with the demand for causality assessment. Automated systems can help alleviate this burden by processing data more quickly and efficiently, allowing human experts to focus on more complex or nuanced cases.

In conclusion, automated causality assessment has the potential to revolutionize safety vigilance by improving the efficiency and accuracy of the causality assessment process. However, there are several challenges that must be addressed to ensure the accuracy and reliability of automated systems. High-quality data, expert knowledge and judgment, and careful attention to bias and error are all critical factors in implementing successful automated causality assessment in safety vigilance. With these challenges in mind, the pharmaceutical industry has the opportunity to leverage advanced technologies and safety platforms like DF mSafety AI to improve patient safety and advance the field of safety vigilance.

References

1. Pande S. Causality or Relatedness Assessment in Adverse Drug Reaction and Its Relevance in Dermatology. Indian J Dermatol. 2018 Jan-Feb;63(1):18-21. doi: 10.4103/ijd.IJD_579_17. PMID: 29527021; PMCID: PMC5838749.
2. https://www.who.int/publications/m/item/WHO-causality-assessment
3. Behera, S.K., Das, S., Xavier, A.S. et al. Comparison of different methods for causality assessment of adverse drug reactions. Int J Clin Pharm 40, 903–910 (2018). https://doi.org/10.1007/s11096-018-0694-9
4. Gawai PP. Overview of important methods used for causality assessment of adverse drug events in pharmacovigilance. jpadr [Internet]. 2020Dec.1 ;1(2):6-12
5. https://lifesciencescare.hcltech.com/blog/guide-to-causality-assessment/
6. https://who-umc.org/media/164840/umc-conference-report_2012.pdf
7. Han L, Ball R, Pamer CA, Altman RB, Proestel S. Development of an automated assessment tool for MedWatch reports in the FDA adverse event reporting system. J Am Med Inform Assoc. 2017 Sep 1;24(5):913-920. doi: 10.1093/jamia/ocx022. PMID: 28371826; PMCID: PMC7651970.
8. Zhao Y, Yu Y, Wang H, Li Y, Deng Y, Jiang G, Luo Y. Machine Learning in Causal Inference: Application in Pharmacovigilance. Drug Saf. 2022 May;45(5):459-476. doi: 10.1007/s40264-022-01155-6. Epub 2022 May 17. Erratum in: Drug Saf. 2022 Aug;45(8):927. PMID: 35579811; PMCID: PMC9114053.
9. Huang Y, Zheng S, Liu X, et al. Exploring the FDA Adverse Event Reporting System to Generate Hypotheses for Monitoring of Disease Characteristics. Clin Pharmacol Ther. 2014;95(5):496-498. doi:10.1038/clpt.2013.234
10. Han L, Ball R, Pamer CA, Altman RB, Proestel S. Development of an automated assessment tool for MedWatch reports in the FDA adverse event reporting system. J Am Med Inform Assoc. 2017 Sep 1;24(5):913-920. doi: 10.1093/jamia/ocx022. PMID: 28371826; PMCID: PMC7651970.
11. Sharma, Sushil & Gupta, Ajay & Reddy, G.. (2017). Inter-rater and intra-rater agreement in causality assessment of adverse drug reactions: a comparative study of WHO-UMC versus Naranjo scale. International Journal of Research in Medical Sciences. 5. 4389. 10.18203/2320-6012.ijrms20174564.

--Issue 02--