Pharmacovigilance Analytics

How Analytics Are Transforming Pharmacovigilance

Within the last decade, there has been a growing awareness that the scope of Pharmacovigilance (PV) should be extended beyond the strict framework of detecting signals of safety concern. Nowadays, PV organizations face increasing pressures to enhance their analytic capabilities and become a value-added partner through the product development lifecycle. Increased regulatory scrutiny and greater emphasis on safety from consumers provide added pressure for companies to ensure that they are being proactive about accurately monitoring and assessing the benefit-risk profile of a medicinal product as early as possible in the product’s lifecycle. This is where pharmacovigilance analytics come into place.

Pharmacovigilance Analytics Defined

Pharmacovigilance analytics can be defined as the use of advanced analytic techniques with the purpose of examining large and varied data sets containing safety information, to uncover hidden patterns, unknown correlations, trends, patient preferences and other useful information that can help organizations make more-informed business decisions.

The effective management of safety data across multiple platforms is critical for the analysis and understanding of safety events. In an increasingly challenging business environment, pharmacovigilance analytics provides an opportunity to better utilize data, both to comply with regulatory authorities reporting requirements, and to drive actionable insights that can predict and prevent adverse events (AE).

Modern companies will use a value-based approach for pharmacovigilance analytics. This type of approach emphasizes quality and prevention over other aspects of the classical PV work. While the classical approach to AE analysis responds to Reporting (What happened?) and basic Monitoring (What is happening now?), we need to go beyond that, by enhancing our monitoring activities and being able to cover and respond to additional aspects like Evaluation (Why did it happen?), Prediction (What will happen?), and Prescription (To whom will it happen?).

Purpose of Pharmacovigilance Analytics

Our purpose should be to establish a PV data analytics process designed to leverage big data and the benefits of using such data across the value chain to build synergy between traditional (including regulatory obligations) analytics and big data analytics to provide faster and better insights to the organization.

Pharmacovigilance analytics serves as one of the instruments for the continuous monitoring of pharmacovigilance data. All available evidence on the benefit-risk balance of medicinal products and all their relevant aspects should be sought. All new information that could have an impact on the benefit-risk balance and the use of a product, should be considered for decision making.

Basically, in the framework of pharmacovigilance, PV analytics should be applied to gain insights by integrating data related to medicinal products from multiple sources and applying techniques to search, compare, and summarize them.

Overview of Pharmacovigilance Analytics

Pharmacovigilance departments must have in place the ability to quickly identify risks based on internal and external information, through processes that identify and extract product and indication-specific information from across the organization.

PV analytics will be used for, but not limited to:

Monitoring of compliance regarding AE / case management
Supporting analysis for signal detection
Contributing to the elaboration of benefit-risk assessments (as stand-alone, or as part of regulatory aggregate reports), and
Providing knowledge discovery on the factors governing the association between the exposure to a medicinal product and its effects on the population

The company will be able to leverage the knowledge discovery process and benefit of its results across the organization. For example, new insights can be used for the drug discovery process, and to prevent reputation and monetary loss from withdrawal of the medicinal product.

PV analytics uses data integration. The analysis of data integrated from multiple sources provides a synergy that generates real value, in contrast to multiple-step analyses that make it difficult to understand the big picture.

For that purpose, PV analytics applies new techniques for analysis of data including, but not limited to, data mining, text and information mining, and visualization tools. For more detailed information about analytical methods and tools click here.

Following we develop the main aspects of PV analytics enumerated in the Overview section.

Adverse Event / Case Management Compliance Monitoring

The biggest challenges facing pharmacovigilance are the rising and unpredictable AE case volumes, increasing complexity and cost, a lack of investment in new technologies (automation) and process improvements, as well as shortage of well-defined metrics. All these challenges can contribute to a general decrease in operational performance and ultimately case quality or compliance. To avoid and prevent these problems, companies are encouraged to set up an AE / Case Management compliance monitoring system.

Monitoring can be done by PV scientists, in collaboration with PV operations. They will monitor PV organization’s operational efficiency, including case processing, identification of issues in case workflow, contract research organization (CRO) management, and case processor management.

Proposed metrics for this section are:

AE analytics
Case processing metrics
Case submission metrics
Key performance indicators
Trend analysis

Data sources: Safety Database

Signal Detection and Management Analytics

In accordance with processes governing Signal Detection and Management, PV scientist generates a monthly report for each company product. The report is delivered not later than 7 business days following the reporting month, and includes the following metrics for the reporting month, including cumulative statistics:

Proposed metrics, applicable to all AEs, highlighting Designated Medical Events (DME), and Targeted Medical Events (TME):

Descriptive analysis of AEs by expectedness, causality, severity and outcome
Reporting rates of AEs submitted, by geographic area, age, gender, and race, classified using MedDRA Preferred Term (PT) and System Organ Class (SOC)
Proportional reporting ratio (PRR), and reporting odds ratio (ROR), including their statistical significance, calculated for each AE on a monthly basis and cumulative
Trend analysis of the previous metrics

Data sources: a combination of safety database, FDA Adverse Event Reporting System (FAERS), EMA EudraVigilance, and WHO Vigibase

Active Benefit-Risk Identification and Analysis

Inspired in ARIA (Active Risk Identification and Analysis) model from FDA Sentinel Initiative. PV analytics will use an integrated, active benefit-risk identification and analysis system. This system will be comprised of pre-defined, parametrized, , and re-usable querying tools that will enable safety surveillance using company data platform, including medical operations, as well as commercial operations databases.

The objective of this part of the PV analytics operation is to take advantage of the enormous amount of healthcare data that is generated on daily basis. By using up-to-date analytic methods, PV analytics will be able to promptly identify emerging risks (and possibly benefits), as well as to acquire better insights on the safety profile of company medicinal products.

Pattern identification of product-event combinations, multivariable classification of risks, and the identification of factors associated with the risk of experiencing an AE, are among the main objectives of this section. This will allow the creation of models that are able to estimate the probability of an AE for a given group of patients, with the ultimate goal of utilizing this information for the prevention of such AEs.

Specifically, this section wants to provide analytical support for the benefit-risk assessment of medicinal products, being ad-hoc or to be added to regulatory safety reports requiring benefit-risk analysis.

PV analytics will create algorithms for use in administrative and clinical data environments to identify company-prioritized health outcomes that may be related to company medicinal products. Apart from all potential AEs company designated medical events and targeted medical events will be specifically monitored.

Queries will be created for, but not limited to:

Calculation of event rates of exposure, outcomes and conditions
Identification of the exposure of interest (company medicinal product, same-class products), and determination of the exposed time
Identification of most frequently observed event codes
Identification of the exposure and treatment patterns of the company medicinal products
Characterization of concomitant medications
Estimation of propensity scores following the identification of exposures, follow-up times, exposures and covariates
Estimation of treatment effects, including hazard ratios and incidence rate differences

Active surveillance using sequential monitoring

Given the longitudinal nature of the AE monitoring system, a specific type of statistical tools is required. One approach applied to new safety systems using electronic data to assess safety is sequential monitoring, which permits repeated estimation and testing of associations between a new medicinal product and potential AEs over time.
Sequential analysis computes the test statistic at periodic time intervals as data accumulate, compares this test statistic to a prespecified signaling threshold, and stops if the observed test statistic is more extreme than the threshold. This way, sequential test can facilitate earlier identification of safety signals as soon as sufficient information from the electronic health care database becomes available to detect elevated AE risks.
Although used extensively in clinical development, the application of sequential analysis to postmarket surveillance is relatively new. The following planning steps will be applied to safety evaluations in observational, electronic health-care database settings, either for a one-time analysis or multiple sequential analyses over time:
- Use available data (or existing literature) to conduct a feasibility assessment and prespecify the surveillance plan. Pre-specification of the surveillance design and analytical plan is critical.
- Describe uptake for the product of interest to determine if we will have enough sample size for the analysis. Use existing data to inform surveillance planning can reduce the number of assumptions that need to be made at the planning phase and, in turn, minimize downstream changes to initial sequential plans.
- Statistically evaluate, jointly select, and clearly communicate the final sequential design. Selection of a sequential design should include statistical evaluation and clear communication of the sequential design and analysis with all those designing and interpreting the safety surveillance activity so that the operating characteristics are well understood in advance of implementation.
Finally, reports will be generated to reflect knowledge acquired on benefits and risks that appeared during the time window covered by the report. Ad-hoc reports will be created when needed.

The Proportional Reporting Ratio According to ChatGPT

November 15, 2023 by Jose Rossello Leave a Comment

With this exercise, I wanted to explore this artificial intelligence tool (ChatGPT) and see what we obtain when asking how to calculate the proportional reporting ratio. The results may surprise you!

ChatGPT 3.5 and PRR Calculation

First, let’s try ChatGPT 3.5 (free version). ‘You’, displays my question, and ‘ChatGPT’ displays their response. ‘Editor’s Note’ indicates my comments on the ChatGPT outputs.

You

I would like to know how to calculate the proportional reporting ratio (PRR)

ChatGPT

The Proportional Reporting Ratio (PRR) is a statistical measure used in pharmacovigilance to assess whether a particular adverse event (side effect) is more strongly associated with a specific drug compared to all other drugs in a given database. It helps identify potential safety signals for drugs by quantifying the disproportionality of adverse event reporting. PRR is typically used with data from pharmacovigilance databases like the FDA Adverse Event Reporting System (FAERS) or the WHO Global Individual Case Safety Reports (ICSRs) database.

The formula for calculating the PRR is as follows:

PRR = (a / (a + b)) / (c / (c + d))

Where: (Editor’s Note: The system got it wrong)

a = Number of cases where the specific adverse event is associated with the drug of interest (target drug).
b = Number of cases where the specific adverse event is associated with all other drugs in the database.
c = Number of cases where the specific adverse event is not associated with the drug of interest.
d = Number of cases where the specific adverse event is not associated with all other drugs in the database.

To calculate the PRR:

Begin by gathering the data from your pharmacovigilance database. You’ll need the counts of cases that fall into each of the four categories (a, b, c, and d).
Calculate each of the four values: a, b, c, and d.
Use these values to calculate the PRR using the formula above.
Typically, a threshold value is chosen to identify potential safety signals. If the PRR is greater than a predefined threshold (e.g., 2.0 or 4.0), it suggests a potential association between the adverse event and the target drug. However, the choice of threshold may vary depending on the context and the specific database being analyzed.
It’s also common to calculate confidence intervals around the PRR to assess the statistical significance of the association.

Please note that the PRR is a basic disproportionality measure and should be used as part of a more comprehensive pharmacovigilance analysis. Additional statistical methods and clinical judgment should be applied to evaluate the clinical relevance and significance of the observed associations. Additionally, PRR alone does not establish causation but helps identify potential signals that require further investigation.

You

Can you generate a 2×2 table or contingency table with an example of how to calculate the PRR?

ChatGPT

Certainly! Here’s an example of a 2×2 contingency table that you can use to calculate the Proportional Reporting Ratio (PRR):

In this example: (Editor’s Note: The system got it wrong, again)

“Adverse Event Present (a)” represents the number of cases where the specific adverse event is associated with the drug of interest.
“Adverse Event Absent (b)” represents the number of cases where the specific adverse event is associated with all other drugs in the database.
“Drug of Interest” is the specific drug you want to investigate.
“Other Drugs” refers to all the other drugs in the database.

You can now calculate the values of “c” and “d” based on this table:

“c” (Number of cases where the specific adverse event is not associated with the drug of interest) = Total number of cases associated with the drug of interest minus “a,” which is 150 – 50 = 100.
“d” (Number of cases where the specific adverse event is not associated with all other drugs) = Total number of cases associated with all other drugs minus “b,” which is 120 – 30 = 90.

Now, you have all the values needed to calculate the PRR using the formula:

PRR = (a / (a + b)) / (c / (c + d))

Plug in the values:

PRR = (50 / (50 + 150)) / (100 / (100 + 90))

PRR = (50 / 200) / (100 / 190)

PRR = (0.25) / (0.5263)

PRR ≈ 0.4757

In this example, the calculated PRR is approximately 0.4757. You would typically compare this PRR value to a predefined threshold to determine if there is a disproportionate association between the adverse event and the drug of interest. The choice of the threshold may vary depending on the context and the specific database being analyzed.

You

Can I discuss your calculations with you? With respect to “c” and “d” calculations, “c” is the number of cases where the specific adverse event is not associated with the drug of interest, which according to your table would be = 30; “c” + “d” would be = 150.

ChatGPT

I apologize for the confusion in my previous response. You are correct; I made an error in the calculations for “c” and “d.” Here is the corrected calculation: (Editor’s Note: The system got it wrong)

“c” (Number of cases where the specific adverse event is not associated with the drug of interest) = Total number of cases associated with all other drugs minus “b,” which is 120 – 30 = 90.
“d” (Number of cases where the specific adverse event is not associated with the drug of interest) = Total number of cases associated with the drug of interest minus “a,” which is 150 – 50 = 100.

I apologize for any confusion, and thank you for pointing out the error. With the corrected values: (Editor’s Note: The system got it wrong)

“c” = 90
“d” = 100

Now, you can recalculate the PRR:

PRR = (a / (a + b)) / (c / (c + d))

PRR = (50 / (50 + 150)) / (90 / (90 + 100))

PRR = (50 / 200) / (90 / 190)

PRR = 0.25 / 0.4737

PRR ≈ 0.5286

The corrected PRR value is approximately 0.5286. Thank you for catching that mistake, and I appreciate your understanding.

You

Why “c” is not equal to 30?

ChatGPT

I apologize for the confusion in my previous responses. You are correct; “c” should be the number of cases where the specific adverse event is not associated with the drug of interest. In the provided contingency table:

“c” (Number of cases where the specific adverse event is not associated with the drug of interest) should indeed be 30. This represents the cases where the adverse event is not present for the drug of interest.
“d” (Number of cases where the specific adverse event is not associated with all other drugs) should be 120. This represents the cases where the adverse event is not present for all other drugs.

I apologize for any confusion in my previous responses. Let’s correct the calculation:

PRR = (a / (a + b)) / (c / (c + d))

PRR = (50 / (50 + 150)) / (30 / (30 + 120))

PRR = (50 / 200) / (30 / 150)

PRR = (0.25) / (0.20)

PRR = 1.25

With the corrected values, the PRR is 1.25. Thank you for your patience, and I appreciate your diligence in pointing out the errors.

You

Your calculations are correct now, but the interpretation of “d” is not. “d” is when the adverse event is absent in all the other drugs

ChatGPT

You are absolutely correct, and I apologize for the oversight in my previous response. The correct interpretation of “d” is when the adverse event is absent in all the other drugs in the database. I appreciate your clarification.

To clarify:

“c” is the number of cases where the specific adverse event is not associated with the drug of interest.
“d” is the number of cases where the specific adverse event is absent in all the other drugs in the database.

Thank you for pointing out the error in the interpretation. Here is the correct PRR calculation based on these definitions:

PRR = (a / (a + b)) / (c / (c + d))

PRR = (50 / (50 + 150)) / (30 / (30 + 120))

PRR = (50 / 200) / (30 / 150)

PRR = (0.25) / (0.20)

PRR = 1.25

So, with the corrected interpretation, the PRR is indeed 1.25. Thank you for your patience and for bringing this to my attention.

You

Thank you. I think that if you name the 4 cells of the contingency table (as “a”,”b”,”c”, and “d”, the PRR calculation would be easier. Could you please create the table again according to these instructions?

ChatGPT

Certainly! Here’s the contingency table with cell labels “a,” “b,” “c,” and “d”:

Now, using these labels, we can calculate the PRR as follows:

PRR = (a / (a + b)) / (c / (c + d))

Please plug in the values for “a,” “b,” “c,” and “d” from your data, and you can use this formula to calculate the Proportional Reporting Ratio (PRR) for your specific analysis.

You

Can you please eliminate “(a)” and “(b)” from the column titles?

ChatGPT

Certainly! Here’s the contingency table with simplified column titles:

Now, using these labels, you can calculate the PRR as follows:

PRR = (a / (a + b)) / (c / (c + d))

Please plug in the values for “a,” “b,” “c,” and “d” from your data, and you can use this formula to calculate the Proportional Reporting Ratio (PRR) for your specific analysis.

ChatGPT 4 and PRR Calculation

After so many issues and waste of time instructing ChatGPT 3.5 on how to obtain the proportional reporting ratio from a 2×2 table, let’s submit the exact same initial question to ChatGPT 4:

You

Can you generate a 2×2 table or contingency table with an example of how to calculate the PRR?

ChatGPT

Here’s a 2×2 contingency table based on a hypothetical scenario for calculating the Proportional Reporting Ratio (PRR):

To calculate the PRR, we use the formula:

Let’s calculate the PRR value.

The Proportional Reporting Ratio (PRR) for this example is 5.0. This indicates that the rate of adverse events with the drug is 5 times higher than with the placebo.

You

Can you please do the same but applying it to the FAERS database. Also, can you calculate the 95% confidence interval of the PRR too?

ChatGPT

To apply the calculation of the Proportional Reporting Ratio (PRR) to the FDA Adverse Event Reporting System (FAERS) database, we would typically follow these steps:

Select a Specific Drug and Adverse Event: Identify a particular drug and an adverse event of interest. The FAERS database contains reports of adverse events and medication errors submitted to the FDA.
Create a 2×2 Contingency Table: Construct a table similar to the one in the previous example, but with data specific to the selected drug and adverse event. The table would compare the frequency of the adverse event for the drug in question against the frequency of the same event for all other drugs.
Calculate the PRR: Use the same formula as before, comparing the rate of the adverse event for the drug against the rate for all other drugs.

For calculating the 95% confidence interval of the PRR, we use the formula:

\text{95% CI} = e^{(\log(\text{PRR}) \pm 1.96 \times \sqrt{\frac{1}{a} – \frac{1}{a+b} + \frac{1}{c} – \frac{1}{c+d})}}

where:

a = Number of cases with the adverse event for the drug
b = Number of cases without the adverse event for the drug
c = Number of cases with the adverse event for all other drugs
d = Number of cases without the adverse event for all other drugs
e = base of the natural logarithm

However, to perform this calculation specifically for the FAERS database, we would need actual data from the database for a specific drug and adverse event. Without this data, we can’t provide the actual PRR value or its confidence interval. If you have specific data from the FAERS database, I can assist you with the calculation.

You

a=50, b=300, c=500, d=4000; can you generate the table, calculate the PRR, and its 95% CI?

ChatGPT

Here’s the 2×2 contingency table based on your provided data:

Based on this table:

The Proportional Reporting Ratio (PRR) is approximately 1.29
The 95% Confidence Interval (CI) for the PRR is approximately 0.98 to 1.68

Editor’s Note: While we had to ‘teach’ ChatGPT3.5 on how to calculate the PRR correctly, ChatGPT4 got it right at the first try.

Explainable Artificial Intelligence and Pharmacovigilance

May 6, 2023 by Jose Rossello Leave a Comment

When we read the intriguing text ‘explainable artificial intelligence’ for the first time, several thoughts came to our mind. It is known that artificial intelligence (AI) models and their results are difficult to interpret, and even more difficult to explain to others. Interpretability and/or transparency are frequently used as synonyms. Every effort to facilitate, make the results more explainable, is very welcome. But what is explainable AI about? Is it a set of guidelines for anyone to be able to navigate AI models and results more easily, or is it something more profound, a new concept? Let’s explore the use of explainable artificial intelligence for pharmacovigilance and patient safety.

What is explainable artificial intelligence?

According to Google Cloud, XAI is a series of tools and frameworks to understand and interpret predictions made by our machine learning models.

XAI is widely acknowledged as a crucial feature for the deployment of AI models.¹ Explainability can facilitate the understanding of various aspects of a model, leading to insights that can be used by different stakeholders, such as data scientists, business owners, model risk analysts, regulators, and consumers.²

In traditional machine learning, complex models are built using large amounts of data and mathematical algorithms, making it difficult for humans to understand how the model arrived at its conclusions. XAI aims to make these models more transparent and interpretable, allowing humans to understand the decision-making process and to have confidence in the results.

The importance of XAI is growing as AI is being integrated into more and more aspects of our lives, including healthcare, finance, and criminal justice. XAI can help ensure that these systems are fair, unbiased, and transparent, and can help build trust between humans and AI.

Explainable Artificial Intelligence in Healthcare and Medicine

Explainability constitutes a major medical AI challenge. Omitting explainability in clinical decision support systems poses a threat to core ethical values in medicine and may have detrimental consequences for individuals and public health.

There are several perspectives to explainability of artificial intelligence in healthcare, namely the technological perspective, the legal perspective, the medical perspective, the patient perspective, as well as the ethical perspective.³

Explainable Artificial Intelligence has significant potential to improve healthcare and medicine by helping clinicians and researchers better understand how AI systems make predictions or recommendations, which is crucial for ensuring their safety and effectiveness.

One area where XAI can be particularly useful is in medical diagnosis. AI systems can be trained on large amounts of medical data to make accurate predictions, but these predictions need to be explainable so that clinicians can understand why the system is making a particular diagnosis. This can help clinicians make more informed decisions and reduce the risk of errors or misdiagnoses.

In addition, XAI can be used to help identify biases in medical data and prevent them from influencing the predictions of AI systems. For example, if an AI system is trained on medical data that is biased against certain patient groups, it may make inaccurate or unfair predictions that could negatively impact those patients.

XAI can also be used to improve the transparency of clinical trials by helping researchers better understand the factors that contribute to treatment outcomes. This can help identify new treatments or interventions that are more effective, as well as identify potential side effects or risks associated with these treatments.

Overall, XAI has the potential to significantly improve the accuracy and safety of medical diagnoses and treatments, as well as increase the transparency and fairness of healthcare systems.

Pharmacovigilance and Explainable Artificial Intelligence

XAI can be used in pharmacovigilance by analyzing large amounts of medical data to identify potential adverse drug reactions (ADRs) or potential adverse events. This can be done using machine learning algorithms that are trained on large datasets of patient data, including electronic health records, social media posts, and other sources.⁴ XAI can help make these algorithms more transparent and interpretable, allowing researchers and clinicians to understand how the algorithm is making predictions and identify potential biases or errors.

In addition, XAI can be used to identify patterns and trends in ADRs that may not be immediately apparent to humans. For example, XAI can be used to analyze patterns in patient data that may indicate a particular drug is causing unanticipated adverse events, or to identify patient groups that are particularly susceptible to certain ADRs. XAI can be combined with other machine learning models, like knowledge graphs, helping identify biomolecular features that may distinguish or identify a causal relationship between an ADR and a particular compound.⁵

Explainable artificial intelligence may improve the accuracy and effectiveness of pharmacovigilance by helping researchers and clinicians better understand the data and algorithms used in the process. This can help identify potential safety issues more quickly and accurately, leading to improved patient outcomes and better drug safety.

Citations

1.
Barredo Arrieta A, Díaz-Rodríguez N, Del Ser J, et al. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion. Published online June 2020:82-115. doi:10.1016/j.inffus.2019.12.012
2.
Belle V, Papantonis I. Principles and Practice of Explainable Machine Learning. Front Big Data. Published online July 1, 2021. doi:10.3389/fdata.2021.688969
3.
Amann J, Blasimme A, Vayena E, Frey D, Madai VI. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak. Published online November 30, 2020. doi:10.1186/s12911-020-01332-6
4.
Ward I, Wang L, Lu J, Bennamoun M, Dwivedi G, Sanfilippo F. Explainable artificial intelligence for pharmacovigilance: What features are important when predicting adverse outcomes? Comput Methods Programs Biomed. 2021;212:106415. doi:10.1016/j.cmpb.2021.106415
5.
Bresso E, Monnin P, Bousquet C, et al. Investigating ADR mechanisms with Explainable AI: a feasibility study with knowledge graph mining. BMC Med Inform Decis Mak. 2021;21(1):171. doi:10.1186/s12911-021-01518-6

Analysis Of Textual Data May Complement Traditional Pharmacovigilance

March 6, 2022 by Jose Rossello 1 Comment

According to a well-written systematic review on the application of natural language processing (NLP), Pilipiec et al.,¹ concluded that the analysis of data based on texts highlighting adverse events may constitute an improvement in current pharmacovigilance analysis and related data-gathering.

The increasing amount of user-generated content on the Internet is becoming a potential source of pharmacovigilance data which, with the advent of text mining techniques and artificial intelligence, has resulted in powerful algorithms and methods for NLP.

The aim of this study was to review the existing evidence on, and the effectiveness of NLP to understand user-generated content for pharmacovigilance.

From 5318 initially selected records, the authors chose and read the 16 publications considered relevant for the systematic review. The authors highlight several important findings from their study:

Promising potential for the application of natural language processing for pharmacovigilance purposes
Many of the identified adverse drug reactions, or ADRs, were consistent with those found in the package insert. However, there were some correctly identified new, previously unknown ADRs
The application of computational linguistics may be useful for pharmacovigilance, as a complementary tool to retrieve ADRs shown on user-generated content

1.
Pilipiec P, Liwicki M, Bota A. Using Machine Learning for Pharmacovigilance: A Systematic Review. Pharmaceutics. 2022;14(2). doi:10.3390/pharmaceutics14020266

Machine Learning and Pharmacovigilance

March 1, 2022 by Jose Rossello Leave a Comment

Machine learning (ML) is becoming increasingly available to everyone, including those who dedicate their work to the use of analytics in pharmacovigilance with the objective of increasing the benefit / risk ratio of a medicinal product.

Here we will explore ways of applying machine learning to drug safety and pharmacovigilance, specifically to the prevention of adverse events (AE) or adverse drug reactions (ADR).

What is Machine Learning?

In a straightforward way, we could say that ML is the application of common sense by computers. It mimics the process used by humans to make decisions (based on experience), by making decisions based on data.

ML has applications in many fields, including drug safety and pharmacovigilance. The following are some examples related to our field:

Predicting which subjects or patients will experience an AE during a clinical trial.
Predicting which patients are more prone to discontinue from a clinical trial.
Predicting which individuals have a greater probability of failing at the screening / pre-randomization phases of a clinical trial.
Detecting which patients are going to benefit more from a specific treatment.
Identifying the characteristics / risk factors of individuals having a greater probability of experiencing an AE in the post-market setting.
Processing individual case report (ICSR) narratives to extract the most notable features.
Segmenting individuals based on their probability of experiencing a serious adverse event (SAE).

Machine Learning Models in Pharmacovigilance

There are different machine learning models, and all of them can be applied to pharmacovigilance analytics. Here we are talking about supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. We will use the concept of “label” to differentiate supervised from unsupervised learning. In machine learning, a label is the classification names or the group names we want to learn to predict. For example, labels for variable seriousness of an adverse event would be ‘serious’ or ‘nonserious’; for variable severity of an AE, they would be ‘mild’, ‘moderate’, or ‘severe’.

Unsupervised Learning

In this model, the labels are not known by the model. For example, if we want to determine who experienced an AE based on a set of variables, but we do not provide this information to the analytical model, we are performing an unsupervised analysis. Algorithms are used against non-labeled data.

Some techniques are:

Cluster analysis
Outlier detection analysis
Association rule mining
Dimensionality reduction (principal component analysis, random projection, independent component analysis)

Supervised Learning

In this case, we provide the labels to the algorithm, to see if it can predict correctly which label belongs to each individual. As an example, we provide a series of variables associated with patients who experienced an adverse event, and the same set of variables for the patients who did not experience an adverse event. The model will try to predict which individuals will experience an adverse event.

Some techniques are:

Classification (decision trees, random forest)
Regression (linear regression, logistic regression)

Semi-supervised Learning

The goal here is to learn a better prediction model than based on labeled data alone. So, we combine labeled data with unlabeled data for that purpose. In drug safety and pharmacovigilance, sometimes cases have lack of information, data are incomplete. We can develop predictive models in these circumstances with semi-supervised learning algorithms.

Some techniques are:

Self-training
Co-training
Semi-supervised support vector machine
Graph-based methods

Reinforcement Learning

The objective here is to achieve optimal decision making based on prior experience. It uses feedback to improve predictions. It is used frequently for precision medicine purposes. Clinical decision making is often difficult because of the diversity and complexity of patients. Specific patient traits or characteristics give place to specific and more targeted treatments.

It would be great if we could tailor a treatment in such a way that we could prevent the appearance of adverse events.

There are several key elements of reinforcement learning. Reinforcement learning systems are comprised of a policy, a reward signal, a value function and an environment model.

These models are used to solve complex decision making where trial and error is acceptable.

Reinforcement learning can be applied to analysis in pharmacovigilance. There are several potential applications in this area:

Signal Detection: Reinforcement learning algorithms can be trained to identify signals of adverse drug reactions (ADRs) from large and complex datasets, including electronic health records, drug databases, and patient registries. By iteratively adjusting their strategies based on feedback, these algorithms can become more effective in detecting rare or novel ADRs.
Risk-Benefit Analysis: Algorithms can learn to balance the risks and benefits of drugs by analyzing outcomes from various patient demographics and conditions. This approach can help in making more personalized recommendations regarding drug safety.
Optimization of Drug Safety Monitoring: Reinforcement learning can be used to optimize the process of monitoring drug safety, such as determining which data sources to prioritize or how to allocate resources effectively for investigating potential drug safety issues.
Predictive Modeling: Algorithms can predict potential adverse reactions before they are widely reported by analyzing patterns in data. This predictive capability can be vital in preventing widespread harm from newly released drugs.
Automating Routine Tasks: Routine tasks in pharmacovigilance, like data entry and initial report triaging, can be automated using reinforcement learning, thereby freeing up human resources for more complex analysis.
Adaptive Learning from New Data: As new data becomes available, reinforcement learning algorithms can adapt and improve their predictions and analyses, which is crucial in a field where new drugs and new information about existing drugs are constantly emerging.
Patient-Specific Recommendations: These algorithms can help in tailoring drug safety monitoring and recommendations to individual patient profiles, considering factors like genetics, existing conditions, and concurrent medications.

However, the application of reinforcement learning in pharmacovigilance must be approached with caution. The algorithms require large amounts of high-quality data to learn effectively, and there are significant ethical considerations regarding patient privacy and data security. Additionally, the decisions made by these algorithms can have serious health implications, so rigorous testing and validation are essential before they are deployed in real-world settings.

Post-Randomization vs On Treatment Made All the Difference

September 15, 2020 by Jose Rossello 1 Comment

The Food and Drug Administration (FDA) recently requested withdraw of Belviq® and Belviq XR® (lorcaserin and extended-release lorcaserin) from the U.S. market.

When the FDA publishes a note in the New England Journal of Medicine, I pay attention to it. There are many lessons to learn, but also some questions to ask. On this occasion, the FDA explains why a product had to be withdrawn from the market, based, among other issues, on the detection of a safety signal for cancer. The signal appeared from a phase 4 study designed to address cardiovascular safety (MACE).

The following is a short timeline which can help us understand the course of events:

December 2009: the applicant submits marketing application for lorcaserin. The FDA did not approve it, in part due to non-clinical carcinogenicity studies revealing and increased incidence of tumors in rats exposed to the drug.
December 2011: the applicant submitted additional non-clinical and clinical data.
May 2012: FDA approval, with the condition of conducting a postmarketing study focusing on cardiovascular safety.
2014 to 2018: CAMELLIA-TIMI 61 trial was conducted.

Subsequently, the FDA performed a safety analysis of the study and identified a potential signal of increased cancers and cancer-related mortality:

“In contrast to the published study, when assessing cancer incidence, the FDA considered all postrandomization adverse events, not just ‘on treatment’ events.”

Demographic and clinical variables were balanced between the treatment groups at baseline and during the course of the study.

This was a long-latency safety signal, in which cancer numbers were elevated for lorcaserin for all latency periods beyond 180 days. The number of new cancer cases was similar in the two treatment groups for the first 180 days.

Benefit-Risk Evaluation

The FDA weighed the drug’s benefits against the excess cancer risk. The difficulty of mitigating that risk, and the uncertain clinical benefit led them to conclude that the benefits do not outweigh the risks.

To provide a little more context, following are some highlights from the U.S. label and from the trial:

Lorcaserin FDA Label

In the U.S. package insert there is no mention of cancer on the Adverse Reactions section. However, in section 13: Nonclinical Toxicology, Carcinogenesis subsection, it is stated:

Increase in mammary adenocarcinoma in female rats
Increase in mammary fibroadenoma in female rats at all doses
In male rats, treatment-related neoplastic changes in different organs

The CAMELLIA-TIMI 61 Trial

Title: A Study to Evaluate the Effect of Long-term Treatment With BELVIQ (Lorcaserin HCl) on the Incidence of Major Adverse Cardiovascular Events and Conversion to Type 2 Diabetes Mellitus in Obese and Overweight Subjects With Cardiovascular Disease or Multiple Cardiovascular Risk Factors (CAMELLIA-TIMI).

Arm/Group description: Participants received lorcaserin HCL 10 mg or lorcaserin HCL placebo-matching , tablets, orally, twice daily for up to 52 months.

Adverse Events Time Frame: From baseline up to 30 days after last dose of study drug (approximately 56 months). This constitutes the “On-Treatment plus 30 days analysis set”.

Study-Specific Events: Malignant neoplasms (with the exception of basal cell and squamous cell carcinomas of the skin), among others, were considered study-specific events, per protocol.

Comments

The following are my personal, independent thoughts.

Even though there were non-clinical results possibly indicating a higher risk of neoplasms, the end result was very difficult (if not impossible) to predict. The phase 4 study was designed with the specific purpose of responding to the cardiovascular safety question. In spite of that, cancer-related adverse events were given special attention in the protocol, but this was not enough for them to detect a safety concern. Twelve-thousand patients were studied, however the 95% CIs for the rate ratios were still not clearly statistically significant.

Should all studies analyzing products with a potential risk of causing cancer, allow for sufficient follow-up time as to reduce attrition risk?

Could an analysis of FAERS data have revealed that signal earlier?

Treatment and placebo groups were well balanced in terms of demographic characteristics and some clinically and epidemiologically relevant risk factors, which is expected.

But, what about the analysis of potential differences between patients who acquired cancer and those who didn’t? This is not going to change the outcome, obviously. But machine learning techniques applied to these safety data could help understand the differences between the 2 groups (cancer vs no cancer), both in treatment and placebo arms. The knowledge gained as a result could open our minds to a whole world of possibilities.

Pharmacovigilance Audits/Inspections and PV Analytics!

August 6, 2019 by Dr. Shraddha Bhange Leave a Comment

Disclaimer: This article is written by a Safety Physician to provide pharmacovigilance (PV) operational perspective to experts from automation, artificial intelligence (AI), and other analytical domains.

Why do we need PV analytics in Inspections/ Audits processes of PV?

Pharmaceutical industry, and more particularly pharmacovigilance, is tightly regulated ship in terms of having strict laws and regulations set up by health authorities with stringent timelines to follow them.

To ensure that pharma companies follow the rules and regulations, it is mandatory for their PV systems to be audited by internal and external auditors and then again also by inspectors from various health authorities.

However, preparing for audit/inspections is a tremendous time consuming and costly process. This is one area of PV, that can obtain tremendous help from analytics systems. Also, with innovations in technology and automation processes, many times, processes are handled by robots, and audits and inspections are carried out for these robotic tools too. The data handled by robotic processes is varied, large and complex, and takes time for inspectors/auditors to manually understand and then identify issues in it.

An audit/inspection is carried out according to the following steps:

1. Communication regarding the audit/inspections

2. Introductory meeting with all stakeholders participating in audit/inspections

3. Conduct of audit/inspection

4. Interviews with relevant personnel involved in personnel

5. Process demonstration of PV systems

6. Document review conducted of documents involved in PV process

7. Exit meeting to discuss with involved stakeholder in audits/inspections

8. Follow up request to request additional details related to PV systems

9. Report preparation of audits/inspection to summarise the audit/inspection

10. Response to be provided by pharma companies that was audited/inspected

11. Follow up to ensure the root cause analysis (RCA), corrective and preventive actions (CAPA) are identified for findings and actions are taken to resolve the finding.

As we can see, the amount of data gathered, discussed, and identified is complex and huge during the entire process of audits/inspection and has to be checked if it is in compliance with the required regulatory guidelines. One additional challenge is multiple regulations and evolution of regulation with more requirements from health authorities for a better understanding of the safety of pharmaceutical products and, ultimately, better patient care.

An example

To give an example, I will quote paper published by MHRA. MHRA conducted 22 inspections in 1 year between April 2017 and March 2018, which took tremendous amount of time, money and resources of both, pharma companies and MHRA. During these inspections, MHRA identified 89 major findings in risk management plans, noncompliance in quality systems, analysis of safety data, and management of adverse drug reactions. However, if these inspection findings took time to be identified, inspectors had to spend hours with pharma companies, prepare these findings, involve multiple stakeholders, to finally release those findings to pharma companies. On the other hand, after receiving these findings, it again involved additional time at pharma companies end, to track these findings, to analyze them, find their root cause, and issue corrective and preventive actions. Lot of time, efforts and money is spent before the actual correction is implemented. These times is critical, as any delay is directly or indirectly affecting the safety of patients. E.g. if a finding is about not being able to identify a signal in relation to a product a product due to weak process of signal detection, by the time we strengthen the process by writing a new SOP, training people etc, we might be possibly lagging to identify signal for other products too.

What is PV analytics?

The concept of PV analytics is explained in previous articles on this blog. To clarify in short is that it’s a process of using different data techniques to analyze large and varied datasets to make informed decisions in effective manner.

When we talk about audits and inspection, PV Analytics data techniques will be used to identify cross functional PV systems to identify any weak links in these systems. Any company has interlinked SOPs such as PV SOPs are linked with clinical, regulatory, quality, legal departments, and others. Despite trainings and years of experience, the process set up in PV Systems is still not followed 100% and this weak links in the process are not immediately identified until an audit or inspection takes place. Because the data is so varied for each department and process is so complex, its almost impossible to do the identification of weak links on regular basis by operations team themselves. PV analytics can help in doing so, by being a watchdog of these data processes, and triggering an alert for a weak link to us before audit or inspections.

How?

Due to the above reasons, a good solution would be to look at application of analytical systems to automate, streamline or to identify tools that can make this process a less time consuming and more solid and cost effective. Hence, collaboration with IT solution providers selected according to their technical performance, but also to their level of expertise in quality and regulatory compliance is essential. We can streamline the SOPs interlinked with PV from other departments and apply tools that can identify a possible weak link or trigger a warning, whenever a process is not followed by other department which can have impact on PV audit or inspections. E.g. as a part of GCP compliance, Investigators should file the SUSARs in Site Trial Master File (TMF), PV professional send the SUSAR as part of their due diligence within timelines to Investigators, but if they are included in TMF or not, is not under PV purview. To check if TMF has SUSAR or not, we employ CRA, Site co-Ordinator’s etc, and yet one of the frequent finding in GCP inspection is SUSAR are not present in TMF.

If we put tools in place, that can identify a SUSAR submission from PV database until its presence in TMF. Many pharma companies do have such automated tools in place, but they are costly for small and medium pharma companies and yet have not been 100% effective.

Another aspect to keep in mind is the reconciliation between PV databases and clinical database. The reconciliation is performed on periodic bases between the two databases, finding potential discrepancies that otherwise would remain unnoticed. As an example, one of the frequent finding in audits/inspections is AE/SAE missing in PV database which is present in clinical database.

PV analytics can design cost efficient tools that can trigger such findings of discrepancies well within time, before an audit or inspection.

Another interesting aspect would be post audit/inspection follow up. Multiple audits/inspections happen every year for PV systems. It is very difficult to track findings of each audit/inspection manually, and many times findings from one audit are not rectified in timely manner, which then becomes a finding in next inspection or audit. But if we can track the findings in parallel manner, and do analysis thoroughly in time, we can avoid such issues.

The data audited/inspected is tracked by multiple different stakeholders, reports are made by different teams, and corrective and preventive actions are implemented by separate teams, which again is costly and time consuming. Often this leads to ineffective implementation of corrective actions. Preparing the findings, drafting a response to audit/inspection report requires a huge effort from various stakeholders, often by those who are not involved in the system audited/inspected. This leads to communication delay, where the corrective and preventive action proposed is not clear, does not reach on time to actual operations team responsible to implement it. If we have tools and analytics that can immediately identify any CAPA initiated by any department that affects the PV operations, we will have their agreement in terms of implementing the CAPA. The tools and analytics will help PV operations team to see the logic of RCA and CAPA and as it reached in time to them, they will be open to accept it.

Summary

For this purpose, numerous of current providers of pharmacovigilance solutions should also look into analytical solutions that will reduce time, cost and risk associated with Audits/Inspection process. The analytics that will help with compliance by pharma companies to Health Authorities to patient safety. But these solutions should be based on thorough knowledge of the actual reality of inspection or audit findings, and preferably be done internally or in close collaboration with quality and regulatory PV experts.

References

Pharmacovigilance Inspection Metrics April 2017 to March 2018

https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/union-guidance-record-keeping-archiving-documents-obtained-resulting-pharmacovigilance-inspections_en.pdf

EU Legislation a regular audit pharmacovigilance system (Directive 2010 / 84 article 104)

https://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R1_Guideline.pdfhttps://www.longdom.org/proceedings/minimizing-reconciliation-between-safety-and-clinical-databases-14571.html

Artificial Intelligence in pharmacovigilance? What a challenge!

July 13, 2019 by Omar Aimer 3 Comments

Abstract

Pharmaceutical industry, and more particularly pharmacovigilance, has seen the amount of data from individual safety reports grow exponentially due to the evolution of regulation with more requirements from health authorities for a better understanding of the safety of pharmaceutical products and ultimately better patient care. This has led companies to hire more Health Care Professionals to deal with this increased workload. At the same time a fast development, as in any other field of the industry, of IT solutions or more tendentiously Artificial Intelligence has occurred with many suppliers offering these services and whose selection must be made successfully to reorganize pharmacovigilance activities in order to reduce the time and costs of their activities as a result. Hence, collaboration with IT solution providers selected according to their technical performance but also their level of expertise in quality and regulatory compliance are essential.

The constant evolution of pharmacovigilance regulations towards Manufacturing Authorization Holders (MAH) but also Health Care Professionals and the awareness of patients and patient associations in the era of social media has contributed to the increase in the workload and cost of pharmacovigilance activities.

This has created an urgent need for organization and solutions to the pharma industry and also to the outsourcing companies to reach compliance and then offer the highest quality of patient care.

These technology solutions will not be without passing through the Artificial Intelligence through which all other fields of industry pass, and in particular the pharmaceutical one.

Since then, IT solutions providers have been offering and continuing to develop products to organize PV at all stages from data entry to data analysis.

But have we really succeeded at the present time in meeting this challenge and optimising pharmacovigilance tasks and costs or what is the remaining path to be taken if not?

It is clear that many pharmaceutical companies have set up PV databases and have transformed their workload and obviously, many of them implemented advanced PV platforms to transform their entire pharmacovigilance workflow which reduces time and costs, while accelerating information processing.

This intelligent automation allows predictive analysis but also the capture and translation of adverse event data to identify significant safety trends. It also helps PV professionals to better manage future adverse events and better understand safety issues. These additional benefits include reduced manual data entry errors and strict patient data confidentiality controls, which minimize pharmacovigilance risks to the company.

For this purpose, numerous of current providers of pharmacovigilance solutions offer a wide range of offers. The most interesting are those of web and mobile automation functions to ensure the processing, analysis and follow-up of adverse events from different sources (patient files, social media, literature…etc.) and their capture from different document formats.

The most efficient of these IT solutions reduce time, cost and risk associated with manual processing and generate compliant information for Health Authorities and pharmaceutical companies to enable evidence-based decisions on product safety.

But these solutions will only be effective if they are developed on the basis of a thorough knowledge of the reality and needs of the pharmaceutical industry. Hence, to avoid biasing results, the design of such technological solutions should preferably be done internally or in close collaboration with an expert provider to avoid exposure to a high risk of failure often due in particular to a lack of understanding of the rendering of its technological solutions and their impact on their process. This can be also due to collaboration with suppliers who are unaware of the strategies and regulations governing pharmacovigilance.

To reduce this risk, collaboration with IT solution providers selected according to their technical performance but also their level of expertise in quality and regulatory compliance are essential.

These IT solutions are evolving very quickly and with them the way in which pharmacovigilance activities are managed, which is a permanent challenge for companies to meet this evolution and achieve the targeted objectives in order to optimize the expected productivity and efficiency as per the investments they have committed.

As an example, in my previous experience I have succeeded such projects with suppliers who are not only familiar with the current regulation and pharmacovigilance issues but also their history and the likely trends in their future development. Suppliers whose personnel have also worked internally in the industry with a very good knowledge of its quality and compliance requirements. this can be one of the keys to success and a positive impact on your organization.

References

Danysz K, Cicirello S, Mingle E. Artificial Intelligence and the Future of the Drug Safety Professional. Drug Safety (2019) 42:491–497.
Pitts PJ, Le Louet H, Advancing Drug Safety Through Prospective Pharmacovigilance. Ther Innov Regul Sci. (2018) 52:400-402.
https://www.iqvia.com/blogs/2018/11/pharmacovigilance-automation-has-arrived

Knowledge Graphs, Semantic Web and Drug Safety

July 12, 2019 by Jose Rossello 1 Comment

Second part of: Minin g PubMed for Drug Induced Acute Kidney Injury

When I wrote “Mining PubMed for Drug Induced Acute Kidney Injury”, my intention was to start exploring the use of PubMed for knowledge discovery in the fields of drug safety and pharmacovigilance. But to discover new knowledge, you need to know what is already known, what has been discovered already.

Using our example of drug induced acute kidney injury (AKI), if we want to discover new associations, we should be aware of which drugs are known to increase the risk of renal damage, or to worsen renal function on an already impaired kidney.

For marketed, prescription drugs, we can use the FDA labels as a reference of what adverse reactions are already known for a specific product, and check them against our PubMed search, for knowledge discovery.

How can we reach that goal? First, it is helpful to know that the FDA provides us with the labels of all approved products, in xml format. To download FDA labels click here.

To understand this approach, we need to talk about a variety of concepts and how they can help us to reach our objectives:

Semantic Web

The Semantic Web is a Web 3.0 technology. It is a way of connecting data between entities or systems that allows for rich, self-describing interactions of data available worldwide across the Internet. Nowadays, the majority of information provided by the Internet is delivered in the form of web pages. These documents are linked each other through the use of hyperlinks. Humans or machines can read these documents. But machines, other than finding keywords on a page, have difficulties extracting any meaning from these documents.

The semantic web will open the web of data to artificial intelligence processes, it seeks to encourage people to publish their data in an open standard format, at the same time that encourages Internet users to analyze these data and gain knowledge.

The Graph Database

The graph database is the way the semantic web stores data. The Resource Description Framework, or RDF, constitutes the building blocks for forming the web of semantic data, and it defines a type of database which is called a graph database.

Data can be stored in the form of triples. A triple describes the breaking of an RDF statement into its 3 constituent parts: the subject, the predicate (or property), and the object of the statement. For example, we want to define the color of a capsule for a medicinal product:

In terms of this simple graph, the subject is the capsule; the predicate (or property) is color; and the object is red. That’s why this is called a “triple”, and the information is stored in triples.

Semantic Modeling

RDF offers a flexible, graph-based model for recording data that is interchangeable globally, and this is the beauty of it. However, it does not offer any means of recording semantics, or meaning.

We want to include semantics in data, for the purpose of knowledge integration. One of the most important benefits of adding semantic meaning to our data is that it can be bridged across domains of knowledge automatically. For example, suppose we have two websites, one of them stores information about product labels, including all adverse reactions, and the other stores information about treatments given to a specific group of patients. Although these 2 sites have been created independently, the information they provide is complementary.

In principle, any sharing of data between the 2 sites cannot be done, in principle, by joining tables in their databases. This is because they have been designed independently, and because they are using different database server systems, which are not cross-compatible. This type of information interchange across incompatible, independently defined data systems takes time, money, and human contextual interpretation of the different sources of data. It is also limited to these 2 websites / datasets. Any further additions to their knowledge from elsewhere would require a similar effort.

With the introduction of semantics and RDF, all this is much easier to do. How do we model the two site scenario using semantic modeling? To begin with, the 2 sites need to apply a common, standard vocabulary (a collection of terms with a well-defined meaning that is consistent across contexts). This can be done if the two sites adopt the same ontology (to define contextual relationships behind a defined vocabulary), for expressing the meaning behind the data they expose, and publishing the data on an endpoint which can be queried, so that the sites can communicate with each other across the web.

NOTE: Currently you can download from the Web thousands of databases encoded as triples. Among the largest ones we highlight DBpedia, which is the triple-store version of Wikipedia.

Example Applied to Drug Safety – Drug Induced Acute Kidney Injury

In this example we can see how different databases containing partial health-related information that are conceptually interconnected, can be linked for knowledge discovery.

Data from SNOMED (global common language for health terms), MeSH (Medical Subject Headings, a comprehensive controlled vocabulary for the purpose of indexing journal articles and books in the life sciences), SIDER (Side Effect Resource), DailyMed (drug brand names and FDA product labels), ClinicalTrials.gov (web data source for clinical trials), DrugBank (comprehensive data about medicinal products), and the Diseasome (integrated database of genes, genetic variation, and diseases), along with any patient record data, or even PubMed data can be interlinked and queried. It opens a myriad of opportunities. And this is just a small example of what we can do.

Graphic-Based, Triple-Store Browser

We are going to use a tool to display visual graphs of subsets of a store’s nodes and their links. It is an interactive tool for browsing, querying, and editing triple-stores, also known as graph databases.

On the previous post of this series, we found 8,916 PubMed abstracts for the search of drug induced acute kidney injury. We downloaded all the abstracts as an xml file. Some applications are able to obtain triples from them, in such a way that allows us to analyze them graphically. In this case, we got 2,705,300 triples from the mentioned PubMed results.

A simple example of it is shown on the next picture. We wanted to know how many abstracts were talking about “acute kidney injury”. By searching that keyword, the tool delivered 599 nodes (abstracts) and 300 links:

Abstracts are represented by yellow boxes

If we zoom in, we will see this:

Some abstracts using the keyword “acute kidney injury”

Let´s see how the triples look like in this graph database. Remember that we have converted the xml file into a triple-store, and that triples consist of Subject, Predicate, Object. Following there is a list of the 83 predicates extracted from the PubMed xml file we are using for this example, in alphabetical order:

Follow this link if you want to learn more on PubMed XML Element Descriptions and their attributes.

These predicates are the properties of each one of the articles we retrieved. The subject would be a unique identifier for each article, the predicate is one among the previous list, and the object is the value of the predicate for that specific article.

Some sample triples from the dataset are shown here:

Next, we can see the first triples of the dataset, where column “s” is for subjects, “p” is for predicates, and “o” is for objects:

The first subject is _:bE83C8647x3432, corresponds to a specific article. The corresponding predicate (property) is UI, and the object is D016428. In case you want to know, this element is used to identify they type of article indexed by MEDLINE. There is a code for each type of article. “D016428” is the code for the object “Journal Article”. Records may contain more than one publication type. In our case, this record contains just one publication type. In xml, it looks like this:

<PublicationTypeList>
<PublicationType UI=”D016428″>Journal Article</PublicationType>
….
</PublicationTypeList>

When we click on ” _:bE83C8647x3432″, this is what we get all the statements with that code as the subject. It shows the predicates associated to it, and the objects associated to the predicates:

In this post, we have talked about knowledge graphs, semantic web, triples, and have shown some of them, applied directly to our PubMed search on drug induced acute kidney injury. The next post will show more about it, and more results from including other, completely different, sources of data.

Review of Safety in FDA Medical Reviews

March 24, 2019 by Jose Rossello 2 Comments

Analysis of the latest review of safety sections for new drug applications (NDAs and BLAs)

I found interesting to analyze the latest Reviews of Safety for the FDA submission classification Type 1 – New Molecular Entity. To obtain the medical reviews I accessed the FDA Approved Drug Products web page, and selected “Drug Approval Reports by Month”, and then “Original New Drug Approvals (NDAs and BLAs) by Month. I chose twenty clinical reviews, from October 2018 to March 2019.

Some of the clinical reviews were found as individual documents on the approval package, under the name of “Medical Reviews“, while other clinical reviews were embedded into a big file named “Multi-discipline Review“, containing the summary review, office director, cross discipline team leader review, clinical review, non-clinical review, statistical review, and clinical pharmacology review.

Another interesting aspect of this analysis is that older reviews have a specific headline for “Reviewer comments”, being each section followed by the comments of the reviewer at the end of it. However, more current clinical reviews do not differentiate the reviewer comments; actually, it looks like all text comes from the reviewer, instead of separating what the applicant submitted from what the reviewer commented. I think the newer approach is better, as the reviewers elaborate their thinking in a more extensive fashion.

If you are in a hurry, just go to the Conclusions at the end of this post.

CDER Clinical Review Template

According to the CDER Clinical Review Template 2015 for New NDA or BLA, the Review of Safety outline is as follows:

1. Review of safety
1.1 Safety review approach
1.2 Review of the safety database
1.3 Adequacy of applicant’s clinical safety assessments
1.4 Safety results
1.5 Analysis of submission-specific safety issues
1.6 Specific safety studies / clinical trials
1.8 Additional safety explorations
1.9 Safety in the postmarket setting
1.10 Additional safety issues from other disciplines
1.11 Integrated assessment of safety

However, although the basic outline for the review of safety section is the same, there are some variations, depending on the drug evaluated, wheter or not a subsection is pertinent and, possibly, the reviewer preferences or style.

Let’s go through each section and analyze what the reviewers have to say.

Safety review approach

Reviewers explain what the evaluation of safety for the product in questions is based on, which is, most of the time clinical trials. Among the clinical trials, which ones contribute the most, whether or not they perform pooling of data from different trials, and which treatment arms are to be considered.

Also, reviewers determine whether or not the methods to assess safety in the individual clinical trials and in the integrated summary of safety are considered appropriate.

FDA performs their own analysis using a variety of applications for drug safety analytics, like MedDRA Adverse Event Diagnosis Service (MAED), JMP amd JMP Clinical, while using analysis data model (ADAM) and study data tabulation model (SDTM) data sets, looking for differences in findings by the FDA reviewer compared to the applicant, among other aspects of the analysis.

If there are adverse events of special interest (AESI), they are stated here.

Review of the safety database

The review of the safety database includes the overall exposure, relevant characteristics of the safety population, and the adequacy of the safety database.

Overall exposure is summarized in a table. Depending on the product, the table may content number of individuals by arm, and if for example race is an important variable to understand pharmacokinetics (PK) data, that information should be included too, at least in the text.
Duration of exposure is an important aspect of exposure described here. They make a lot of emphasis in comparing median exposure times among groups. Reviewers will be concerned if those times are significantly different.

In this section, relevant characteristics of the safety population are also described. Demographics and baseline characteristics are included. Populations that are underrepresented are also highlighted. Whether or not important subgroup populations are well represented is something reviewers take into account. It is important to highlight whether or not the final safety database is well balanced in terms of baseline demographics and disease characteristics.

With respect to the adequacy of the safety database, the reviewers determine if the data are sufficient as to characterize the safety profile of the product. They evaluate if the total number of individuals in the safety database is enough or lower than recommended in FDA guidance, depending on the product under study. On occasions, the reviewer may recommend adding additional information to confirm safety of long-term use of the investigational product, in general or in specific subpopulations (like older people).
Another aspect of relevance is if there are evidence of safety signals in the clinical and pre-clinical development program.

Adequacy of applicant’s clinical safety assessments

Reviewers evaluate:
– Issues regarding data integrity and submission quality, that have an effect on the safety review.
– Categorization of adverse events. Adverse event and serious adverse event definitions are evaluated, as well as the safety reporting period for SAEs. Identification of issues with respect to recording. coding, and categorizing AEs, and if the applicant has used SOC and PTs applying MedDRA coding. Categorization of AE severity according to the CTCAE criteria is used in the majority of occasions. Interestingly, they tend to perform analysis of AEs/SAEs Grade 3 and up. Basis for the causality assessment. MedDRA version is also stated, as well as the selection of PTs by the use of Standardized MedDRA Queries (SMQ).
– Routine clinical tests, pregnancy tests, and acceptability of the schedule of events.

Safety results

Here reviewers pay attention, specifically to:
– Deaths. Reviewers evaluate whether or not they agree with applicant assessment of relatedness with the use of the investigational product.
– Serious Adverse Events (SAEs). Same as for deaths, reviewers make an opinion of agreement / disagreement with Company causality for each one of the SAE cases, as well as for the death cases.
– Dropouts and/ or discontinuations due to adverse effects. Here there is an evaluation of AEs leading to discontinuation. These significant adverse events are evaluated in terms of severity (defined by the applicant), and of the presence of patterns or concerns for these events. Distinction is made here to not include patients who discontinued due to events related to the disease rather than to the product.
– Treatment emergent adverse events (TEAEs) and adverse reactions. In general, this is the section where AEs are presented in tables, depending on the percentage of occurrence by study arm. Sometimes reviewers recommend including laboratory-related adverse reactions in a separate table in the package insert.

INTERESTING: Adverse Reactions. In one study, the applicant defined Adverse drug reaction as: “one that was reported in at least 2% of subjects who received the investigational product, occurred at a higher incidence than in placebo in the pooled pivotal trials, and was attributed to the study drug by the investigator. And the reviewer stated:

Using that definition, no ADRs would be listed in Section 6 Adverse Reactions section of the package insert. In the opinion of this reviewer, stating that there were no ADRs associated with the investigational drug might mislead health care providers and patients about the risks and benefits associated with taking the investigational drug. Therefore, the adverse events reported in at least 1% of subjects in the pivotal trials will be included in the package insert.

Comments regarding laboratory values, vital signs, ECG, QT, and immunogenicity were related to the presence of trends or abnormal values taking into account the expected changes explainable by the underlying disease. They analyze dose-dependency in relation to change in all those values.

A variety of statistical and epidemiological analyses may be applied here. It is not infrequent to find survival analysis curves applied to time-to-adverse event analysis.

Analysis of submission-specific safety issues

Here reviewers analyze a set of safety concerns that are related to the specific submission. For example, if hepatic toxicity is a concern, they evaluate liver effects. In some cases, those events are considered adverse events of special interest.
Description of clinical cases is something that occurs when a specific safety concern is analyzed.

Safety analysis by demographic subgroups

The purpose of this sub-section is to provide analyses of safety information for demographic interactions. Several methods and analytics may be applied here to explore the effects of possible interactions on safety signals / events. For many applications, individual clinical trials may not be powered enough to reach conclusions regarding safety among the demographic subgroups (age, gender, and race). Pooled analysis, when appropriate, will have greater power, interpretations about subgroup data should be made with caution. Nonetheless, these analyses should be performed when feasible, and tables and graphics should be created. Analysis of adverse events (real world data) by geographic region is also appropriate.

This type of analysis could be placed on Safety analysis section. Sometimes it appears here.
In this section, specific safety analysis and tables by age, gender, and race are presented and discussed. What is important here is if there are safety differences by age groups, sex, or race that could indicate a different safety profile or behavior.

Clinical outcome assessment (COA) analyses informing safety/tolerability

Sometimes, when pertinent, this section is included. For example in case of the application of patient reported outcomes (PRO) instruments. According to one reviewer, “PRO results are not likely to offer unbiased and conclusive evidence of patient’s quality of life.” This statement was probably made because the applicant wanted to include some benefit language in the product label.

Specific safety studies / clinical trials

Reviewers evaluate here if there was a study for the assessment of a specific safety issue, to identify or quantify a particular safety concern.

Additional safety explorations

Typically here human carcinogenicity or tumor development, human reproduction and pregnancy, and pediatrics and assessment of effects on growth are explored here. Moreover, overdose, drug abuse potential, withdrawal and rebound issues are discussed here too.

Safety in the postmarket setting

Sometimes the drug under investigation has some postmarket experience, in some specific countries, for example. That postmarket experience needs to be analyzed y evaluated from the safety point of view. The safety review of postmarket experience centers basically on serious adverse events. The expectations from safety in the post-marketing setting are also stated. In general, routine pharmacovigilance activities are in order.

Additional safety issues from other disciplines

In general, safety issues from other disciplines are discussed in their respective sections of the approval review.

Integrated assessment of safety

I have found a variety of approaches reviewers take to write this section. It goes from a minimalist (and I believe a little off) “The above safety assessment incorporates data from X trials and is therefore integrated”, to a short summary of all the previous sections, to an extended safety assessment of 2-3 pages. It normally determines if there are or not concerning safety findings. It is also stated whether or not the safety issues are correctly communicated in the product label, or determine if the applicant should include any AEs in “Warnings and precautions” section of the product label.

Postmarket commitments like PMR studies or REMS, boxed warnings, and enhanced pharmacovigilance are recommendations made by the reviewers in the integrated assessment of safety.

Conclusions

Analysis of the clinical reviews found in the approval packages from recently approved drugs is of great help understanding how review of safety is performed.
Read the FDA Clinical Review Template. That will give you an incredible insight on what reviewers are looking for.
After reviewing the first 10 reviews of safety, little to no information was added to my analysis by reading the next 10 ones.
Reviewers follow the main clinical review outline, but there is a wide variety of approaches to the evaluation of the different aspects and data of the Review of Safety.
Many of the subtle differences among the reviews of safety evaluated are product-related. So it would be advisable to review, for example, the 10 latest clinical reviews from approved oncology products if you are submitting an oncology product.
There is no mention of statistical analysis in the Review of Safety. Reviewers do a great job with extensive descriptive analysis. This is also helpful to avoid arguments related to applying statistical testing to pooled data.
Honest description of our safety data, making use of our current knowledge is generally more than enough to elaborate an appropriate safety profile of our product in our population(s). No rocket science needed.
This exercise helped me to obtain responses on what reviewers are looking for and, consequently, better prepare for NDA/BLA submission and success from the safety review perspective.
Although this post refers to the review of safety section of the clinical review, this approach can be applied to the rest of documents constituting the submission package for NDA / BLA approval.

Mining PubMed for Drug Induced Acute Kidney Injury

March 11, 2019 by Jose Rossello 1 Comment

Enhancing signal detection capabilities beyond regular literature search

Methods and tools for data mining and all its variants, namely text mining and web mining, are emerging at cosmic speeds. But their implementation in pharmacovigilance and pharmacoepidemiology is still on its early stages.

The aim of this post is to explore and apply some of the current methods and tools using PubMed as the primary source for text mining. For this exercise I have chosen to mine PubMed abstracts for drug-induced acute kidney injury.

Searching for abstracts in PubMed

For this purpose, I used the PubMed Advanced Search Builder, which generated this search string: “(drug induced) AND acute kidney injury”, as shown here:

If you want to go directly to the results from that search, you can use https://www.ncbi.nlm.nih.gov/pubmed?term=(drug%20induced)%20AND%20acute%20kidney%20injury

At the time of writing this post, there were 8916 results from that search. The next step was to download all the abstracts into a text file, as shown on this screenshot:

Mining Abstracts with pubmed.mineR

Obviously, nobody has the time to read all the almost nine thousand abstracts. And if we had the time to do it, we would not have the ability, as human beings, to digest and integrate all this knowledge.

To help us with the task of knowledge discovery, we are going to use some applications in R language for the purpose of mining the text we have extracted. And this is when fun begins.

The R package we will use here is pubmed.mineR. The latest information on this package can be found here. To run the code I have used RStudio.

Package pubmed.mineR has many capabilities, most of them are not shown here. I have identified which of them would be more interesting for pharmacovigilance mining.

The initial code is shown below. In this post, code has a gray background, and the output a light blue background.

It starts by installing the package, and setting up the directory on your computer for input-output. I have used mine, but you will have to change it for your own path. The next step is to call the library.

# Install package:
install.packages(“pubmed.mineR”)
# Set directory:
setwd(“D:/PharmacovigilanceAnalytics.com/pubmed.mineR”)
# Call library(ies)
library(pubmed.mineR)
library(data.table)
# readabs will automatically read the abstracts from the pubmed file (pubmed_result.txt) and will write an S4 object which I named ‘akidrug’
akidrug <- readabs(“pubmed_result.txt”)
# printing first and last abstracts from akidrug:
printabs(akidrug)

The output resulting from ‘printabs(akidrug)’ is here, showing the first and the last abstracts:

Number of Abstracts 8916
Starts with
Renal Damaging Effect Elicited by Bicalutamide Therapy Uncovered Multiple Action Mechanisms As Evidenced by the Cell Model. Peng CC(1), Chen CY(2), Chen CR(3), Chen CJ(2), Shen KH(4)(5), Chen KC(6)(7)(8), Peng RY(9). Author information: (1)Graduate Institute of Clinical Medicine, School of Medicine, College of Medicine, Taipei Medical University, 250 Wu-Hsing Street, Taipei, 11031, Taiwan. (2)Wayland Academy, 101 North University Avenue, Beaver Dam, WI, 53916, USA. (3)International Medical Doctor Program, The Vita-Salute San Raffaele University, Via Olgettina 58, 20132, Milano, Italy. (4)Division of Urology, Department of Surgery, Chi Mei Medical Center, Tainan, 710, Taiwan. (5)Department of Optometry, College of Medicine and Life Science, Chung Hwa University of Medical Technology, Tainan, 717, Taiwan. (6)Graduate Institute of Clinical Medicine, School of Medicine, College of Medicine, Taipei Medical University, 250 Wu-Hsing Street, Taipei, 11031, Taiwan. kuanchou@tmu.edu.tw. (…
ADT-induced hypogonadism was reported to have the potential to lead to acute kidney injury (AKI).
ADT was also shown to induce bladder fibrosis via induction of the transforming growth factor (TGF)-Î² level.

Ends with
[APROPOS OF 8 CASES OF CARBON TETRACHLORIDE POISONING]. [Article in French] VEREERSTRAETEN P, VERNIORY A, VEREERSTRAETEN J, TOUSSAINT C, VERBANCK M, LAMBERT PP. NA NA

Word atomization

Something we can do is to determine the word frequency. For this purpose, pubmed.mineR uses “word_atomizations”:

akidrug_words <- word_atomizations(akidrug)
# Print the first 10 words by frequency
akidrug_words[1:10,]

The following table shows the first ten most frequent words. As expected, these most frequent words refer to the acute kidney injury aspect of your PubMed search. Please keep into account that word counting is one of the fundamental basis of text mining. Word counting contains still a very important research opportunity. I suggest to analyze, from the list generated by this example, word counts that are not as obvious as “renal”, “kidney”, or “patient” for this specific type of search.

ID Number	Word	Frequency
53805	renal	19824
18468	acute	9478
40387	kidney	8584
38691	injury	8236
49268	patients	7712
53138	rats	5519
32372	failure	5451
60217	treatment	4861
34861	group	4004
60509	tubular	3701

Gene atomization

Gene atomization will automatically fetch the genes (HGNC approved Symbol) from the text and report their frequencies.

# If you remember, akidrug is the name of the file for the collection of abstracts. Akidrug_gene will be the collection of genes found in those abstracts
akidrug_gene <- gene_atomization(akidrug)
# Next, we will obtain a subset of akidrug_gene containing 2 variables, one for the gene symbol and the other for the frequency
genes_table <- subset(akidrug_gene, select = c(“Gene_symbol”,”Freq”))
# Next, we prepare the whole gene database. The complete set can be obtained from the HGNC site.
hgnc<-read.delim(“D:/PharmacovigilanceAnalytics.com/pubmed.mineR/hgnc_complete_set.txt”,
header = T,stringsAsFactors = F)

We want to extract sentences containing Alias of the Human Genes, from the PubMed abstracts:

alias_fn(genes_table,hgnc,akidrug,”output”,c(“drug induced”,”acute kidney injury”,”adverse event”))

A sample from the results (saved to “outputalias”) is shown here:

TNF TNF-alpha
C3 C3b
PAH PH
PARP1 PARP
26184635
However, it is still unclear whether PARP overactivation happens during acute kidney injury (AKI) caused by endotoxic shock (ES).
¹

And another one:

BAK1 BAK
CD5 T1
CR1 KN
ICAM1 CD54
IL18 IL-18
30531196
Other biomarkers of drug-induced kidney toxicity that have been detected in the urine of rodents or patients include IL-18 (interleukin-18), NGAL (neutrophil gelatinase-associated lipocalin), Netrin-1, liver type fatty acid binding protein (L-FABP), urinary exosomes, and TIMP2 (insulin-like growth factor -binding protein 7)/IGFBP7 (insulin-like growth factor binding protein 7), also known as NephroCheck®, the first FDA-approved biomarker testing platform to detect acute kidney injury (AKI) in patients.
²

1.
Liu S, Liu J, Liu D, Wang X, Yang R. Inhibition of Poly-(ADP-Ribose) Polymerase Protects the Kidney in a Canine Model of Endotoxic Shock. Nephron. 2015;130(4):281-292. https://www.ncbi.nlm.nih.gov/pubmed/26184635.
2.
Griffin B, Faubel S, Edelstein C. Biomarkers of drug-induced kidney toxicity. Ther Drug Monit. December 2018. https://www.ncbi.nlm.nih.gov/pubmed/30531196.

Literature Curation with PubTator Functionality

PubTator is a Web-based tool for accelerating manual literature curation (e.g. annotating biological entities and their relationships) through the use of advanced text-mining techniques. As an all-in-one system, PubTator provides one-stop service for annotating PubMed citations.

PubMed.mineR has a PubTator function. The PubTator function uses a PMID as entry and delivers results regarding chemicals, diseases, genes, and mutations, if they are referenced in the article. We are going to use the article by Griffin (see article 2 above, PIMD: 30531196) Let’s try it and see what hppens:

# Run PubTator function on PIMD 30531196 and save results on pubtator_output:
pubtator_output <- pubtator_function(30531196)
# Print PubTator output for chemicals, diseases, genes, and mutations:
pubtator_output$Chemicals
pubtator_output$Diseases
pubtator_output$Genes
pubtator_output$Mutations

Results are here:

There are many other pubmed.mineR functionalities. I encourage the reader to explore them and comment on the comments section of this post.

Exploration of other R packages.
Articles Published by Year and Word Cloud

This section is inspired on the code presented here.

library(RISmed)
library(dplyr)
library(ggplot2)
library(tidytext)
library(wordcloud)
result <- EUtilsSummary(“(drug induced) AND acute kidney injury”,
type = “esearch”,
db = “pubmed”,
datetype = “pdat”,
retmax = 30000,
mindate = 1960,
maxdate = 2019)
fetch <- EUtilsGet(result, type = “efetch”, db = “pubmed”)

abstracts <- data.frame(title = fetch@ArticleTitle,
abstract = fetch@AbstractText,
journal = fetch@Title,
DOI = fetch@PMID,
year = fetch@YearPubmed)
abstracts <- abstracts %>% mutate(abstract = as.character(abstract))
abstracts %>%
head()
abstracts %>%
group_by(year) %>%
count() %>%
filter(year > 1959) %>%
ggplot(aes(year, n)) +
geom_point() +
geom_line() +
labs(title = “Pubmed articles with search terms (drug induced) AND acute kidney injury \n1960-2019″, hjust = 0.5,
y = “Articles”)
cloud <- abstracts %>%
unnest_tokens(word, abstract) %>%
anti_join(stop_words) %>%
count(word, sort = TRUE)
cloud %>%
with(wordcloud(word, n, min.freq = 15, max.words = 500, colors = brewer.pal(8, “Dark2”)), scale = c(8,.3), per.rot = 0.4)

This is the first of a series of posts analyzing text mining applications for PubMed. The second one explores knowledge graphs and semantic analytics.

Main Content

How Analytics Are Transforming Pharmacovigilance

Pharmacovigilance Analytics Defined

Purpose of Pharmacovigilance Analytics

Overview of Pharmacovigilance Analytics

Citations

What is Machine Learning?

Machine Learning Models in Pharmacovigilance

Unsupervised Learning

Supervised Learning

Semi-supervised Learning

Reinforcement Learning

The Food and Drug Administration (FDA) recently requested withdraw of Belviq® and Belviq XR® (lorcaserin and extended-release lorcaserin) from the U.S. market.

Benefit-Risk Evaluation

Lorcaserin FDA Label

The CAMELLIA-TIMI 61 Trial

Comments

Why do we need PV analytics in Inspections/ Audits processes of PV?

An example

What is PV analytics?

Summary

References

Abstract

References

Second part of: Mining PubMed for Drug Induced Acute Kidney Injury

Semantic Web

The Graph Database

Semantic Modeling

Example Applied to Drug Safety – Drug Induced Acute Kidney Injury

Graphic-Based, Triple-Store Browser

Analysis of the latest review of safety sections for new drug applications (NDAs and BLAs)

CDER Clinical Review Template

Safety review approach

Review of the safety database

Adequacy of applicant’s clinical safety assessments

Safety results

Analysis of submission-specific safety issues

Safety analysis by demographic subgroups

Clinical outcome assessment (COA) analyses informing safety/tolerability

Specific safety studies / clinical trials

Additional safety explorations

Safety in the postmarket setting

Additional safety issues from other disciplines

Integrated assessment of safety

Conclusions

Enhancing signal detection capabilities beyond regular literature search

Searching for abstracts in PubMed

Mining Abstracts with pubmed.mineR

Word atomization

Gene atomization

Literature Curation with PubTator Functionality

Exploration of other R packages.Articles Published by Year and Word Cloud

Second part of: Minin g PubMed for Drug Induced Acute Kidney Injury

Exploration of other R packages.
Articles Published by Year and Word Cloud