Pharmacovigilance Analytics

Your best resource for PV analytics news, content and innovation!

Jose Rossello

Explainable Artificial Intelligence and Pharmacovigilance

by Leave a Comment

When we read the intriguing text ‘explainable artificial intelligence’ for the first time, several thoughts came to our mind. It is known that artificial intelligence (AI) models and their results are difficult to interpret, and even more difficult to explain to others. Interpretability and/or transparency are frequently used as synonyms. Every effort to facilitate, make the results more explainable, is very welcome. But what is explainable AI about? Is it a set of guidelines for anyone to be able to navigate AI models and results more easily, or is it something more profound, a new concept? Let’s explore the use of explainable artificial intelligence for pharmacovigilance and patient safety.

What is explainable artificial intelligence?

According to Google Cloud, XAI is a series of tools and frameworks to understand and interpret predictions made by our machine learning models.

XAI is widely acknowledged as a crucial feature for the deployment of AI models.​1​ Explainability can facilitate the understanding of various aspects of a model, leading to insights that can be used by different stakeholders, such as data scientists, business owners, model risk analysts, regulators, and consumers.​2​

In traditional machine learning, complex models are built using large amounts of data and mathematical algorithms, making it difficult for humans to understand how the model arrived at its conclusions. XAI aims to make these models more transparent and interpretable, allowing humans to understand the decision-making process and to have confidence in the results.

The importance of XAI is growing as AI is being integrated into more and more aspects of our lives, including healthcare, finance, and criminal justice. XAI can help ensure that these systems are fair, unbiased, and transparent, and can help build trust between humans and AI.

Explainable Artificial Intelligence in Healthcare and Medicine

Explainability constitutes a major medical AI challenge. Omitting explainability in clinical decision support systems poses a threat to core ethical values in medicine and may have detrimental consequences for individuals and public health.

There are several perspectives to explainability of artificial intelligence in healthcare, namely the technological perspective, the legal perspective, the medical perspective, the patient perspective, as well as the ethical perspective.​3​

Explainable Artificial Intelligence has significant potential to improve healthcare and medicine by helping clinicians and researchers better understand how AI systems make predictions or recommendations, which is crucial for ensuring their safety and effectiveness.

One area where XAI can be particularly useful is in medical diagnosis. AI systems can be trained on large amounts of medical data to make accurate predictions, but these predictions need to be explainable so that clinicians can understand why the system is making a particular diagnosis. This can help clinicians make more informed decisions and reduce the risk of errors or misdiagnoses.

In addition, XAI can be used to help identify biases in medical data and prevent them from influencing the predictions of AI systems. For example, if an AI system is trained on medical data that is biased against certain patient groups, it may make inaccurate or unfair predictions that could negatively impact those patients.

XAI can also be used to improve the transparency of clinical trials by helping researchers better understand the factors that contribute to treatment outcomes. This can help identify new treatments or interventions that are more effective, as well as identify potential side effects or risks associated with these treatments.

Overall, XAI has the potential to significantly improve the accuracy and safety of medical diagnoses and treatments, as well as increase the transparency and fairness of healthcare systems.

Pharmacovigilance and Explainable Artificial Intelligence

XAI can be used in pharmacovigilance by analyzing large amounts of medical data to identify potential adverse drug reactions (ADRs) or potential adverse events. This can be done using machine learning algorithms that are trained on large datasets of patient data, including electronic health records, social media posts, and other sources.​4​ XAI can help make these algorithms more transparent and interpretable, allowing researchers and clinicians to understand how the algorithm is making predictions and identify potential biases or errors.

In addition, XAI can be used to identify patterns and trends in ADRs that may not be immediately apparent to humans. For example, XAI can be used to analyze patterns in patient data that may indicate a particular drug is causing unanticipated adverse events, or to identify patient groups that are particularly susceptible to certain ADRs. XAI can be combined with other machine learning models, like knowledge graphs, helping identify biomolecular features that may distinguish or identify a causal relationship between an ADR and a particular compound.​5​

Explainable artificial intelligence may improve the accuracy and effectiveness of pharmacovigilance by helping researchers and clinicians better understand the data and algorithms used in the process. This can help identify potential safety issues more quickly and accurately, leading to improved patient outcomes and better drug safety.

Citations

  1. 1.
    Barredo Arrieta A, Díaz-Rodríguez N, Del Ser J, et al. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion. Published online June 2020:82-115. doi:10.1016/j.inffus.2019.12.012
  2. 2.
    Belle V, Papantonis I. Principles and Practice of Explainable Machine Learning. Front Big Data. Published online July 1, 2021. doi:10.3389/fdata.2021.688969
  3. 3.
    Amann J, Blasimme A, Vayena E, Frey D, Madai VI. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak. Published online November 30, 2020. doi:10.1186/s12911-020-01332-6
  4. 4.
    Ward I, Wang L, Lu J, Bennamoun M, Dwivedi G, Sanfilippo F. Explainable artificial intelligence for pharmacovigilance: What features are important when predicting adverse outcomes? Comput Methods Programs Biomed. 2021;212:106415. doi:10.1016/j.cmpb.2021.106415
  5. 5.
    Bresso E, Monnin P, Bousquet C, et al. Investigating ADR mechanisms with Explainable AI: a feasibility study with knowledge graph mining. BMC Med Inform Decis Mak. 2021;21(1):171. doi:10.1186/s12911-021-01518-6

Analysis Of Textual Data May Complement Traditional Pharmacovigilance

by Leave a Comment

According to a well-written systematic review on the application of natural language processing (NLP), Pilipiec et al.,​1​ concluded that the analysis of data based on texts highlighting adverse events may constitute an improvement in current pharmacovigilance analysis and related data-gathering.

The increasing amount of user-generated content on the Internet is becoming a potential source of pharmacovigilance data which, with the advent of text mining techniques and artificial intelligence, has resulted in powerful algorithms and methods for NLP.

The aim of this study was to review the existing evidence on, and the effectiveness of NLP to understand user-generated content for pharmacovigilance.

From 5318 initially selected records, the authors chose and read the 16 publications considered relevant for the systematic review. The authors highlight several important findings from their study:

  • Promising potential for the application of natural language processing for pharmacovigilance purposes
  • Many of the identified adverse drug reactions, or ADRs, were consistent with those found in the package insert. However, there were some correctly identified new, previously unknown ADRs
  • The application of computational linguistics may be useful for pharmacovigilance, as a complementary tool to retrieve ADRs shown on user-generated content
  1. 1.
    Pilipiec P, Liwicki M, Bota A. Using Machine Learning for Pharmacovigilance: A Systematic Review. Pharmaceutics. 2022;14(2). doi:10.3390/pharmaceutics14020266

Machine Learning and Pharmacovigilance

by Leave a Comment

Machine learning (ML) is becoming increasingly available to everyone, including those who dedicate their work to the use of analytics in pharmacovigilance with the objective of increasing the benefit / risk ratio of a medicinal product.

Here we will explore ways of applying machine learning to drug safety and pharmacovigilance, specifically to the prevention of adverse events (AE) or adverse drug reactions (ADR).

What is Machine Learning?

In a straightforward way, we could say that ML is the application of common sense by computers. It mimics the process used by humans to make decisions (based on experience), by making decisions based on data.

ML has applications in many fields, including drug safety and pharmacovigilance. The following are some examples related to our field:

  • Predicting which subjects or patients will experience an AE during a clinical trial.
  • Predicting which patients are more prone to discontinue from a clinical trial.
  • Predicting which individuals have a greater probability of failing at the screening / pre-randomization phases of a clinical trial.
  • Detecting which patients are going to benefit more from a specific treatment.
  • Identifying the characteristics / risk factors of individuals having a greater probability of experiencing an AE in the post-market setting.
  • Processing individual case report (ICSR) narratives to extract the most notable features.
  • Segmenting individuals based on their probability of experiencing a serious adverse event (SAE).

Machine Learning Models in Pharmacovigilance

There are different machine learning models, and all of them can be applied to pharmacovigilance analytics. Here we are talking about supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. We will use the concept of “label” to differentiate supervised from unsupervised learning. In machine learning, a label is the classification names or the group names we want to learn to predict. For example, labels for variable seriousness of an adverse event would be ‘serious’ or ‘nonserious’; for variable severity of an AE, they would be ‘mild’, ‘moderate’, or ‘severe’.

Unsupervised Learning

In this model, the labels are not known by the model. For example, if we want to determine who experienced an AE based on a set of variables, but we do not provide this information to the analytical model, we are performing an unsupervised analysis. Algorithms are used against non-labeled data.

Some techniques are:

  • Cluster analysis
  • Outlier detection analysis
  • Association rule mining
  • Dimensionality reduction (principal component analysis, random projection, independent component analysis)

Supervised Learning

In this case, we provide the labels to the algorithm, to see if it can predict correctly which label belongs to each individual. As an example, we provide a series of variables associated with patients who experienced an adverse event, and the same set of variables for the patients who did not experience an adverse event. The model will try to predict which individuals will experience an adverse event.

Some techniques are:

  • Classification (decision trees, random forest)
  • Regression (linear regression, logistic regression)

Semi-supervised Learning

The goal here is to learn a better prediction model than based on labeled data alone. So, we combine labeled data with unlabeled data for that purpose. In drug safety and pharmacovigilance, sometimes cases have lack of information, data are incomplete. We can develop predictive models in these circumstances with semi-supervised learning algorithms.

Some techniques are:

  • Self-training
  • Co-training
  • Semi-supervised support vector machine
  • Graph-based methods

Reinforcement Learning

The objective here is to achieve optimal decision making based on prior experience. It uses feedback to improve predictions. It is used frequently for precision medicine purposes. Clinical decision making is often difficult because of the diversity and complexity of patients. Specific patient traits or characteristics give place to specific and more targeted treatments.

It would be great if we could tailor a treatment in such a way that we could prevent the appearance of adverse events.

There are several key elements of reinforcement learning. Reinforcement learning systems are comprised of a policy, a reward signal, a value function and an environment model.

These models are used to solve complex decision making where trial and error is acceptable.

Post-Randomization vs On Treatment Made All the Difference

by 1 Comment

The Food and Drug Administration (FDA) recently requested withdraw of Belviq® and Belviq XR® (lorcaserin and extended-release lorcaserin) from the U.S. market.

When the FDA publishes a note in the New England Journal of Medicine, I pay attention to it. There are many lessons to learn, but also some questions to ask. On this occasion, the FDA explains why a product had to be withdrawn from the market, based, among other issues, on the detection of a safety signal for cancer. The signal appeared from a phase 4 study designed to address cardiovascular safety (MACE).

The following is a short timeline which can help us understand the course of events:

  • December 2009: the applicant submits marketing application for lorcaserin. The FDA did not approve it, in part due to non-clinical carcinogenicity studies revealing and increased incidence of tumors in rats exposed to the drug.
  • December 2011: the applicant submitted additional non-clinical and clinical data.
  • May 2012: FDA approval, with the condition of conducting a postmarketing study focusing on cardiovascular safety.
  • 2014 to 2018: CAMELLIA-TIMI 61 trial was conducted.

Subsequently, the FDA performed a safety analysis of the study and identified a potential signal of increased cancers and cancer-related mortality:

“In contrast to the published study, when assessing cancer incidence, the FDA considered all postrandomization adverse events, not just ‘on treatment’ events.”

Demographic and clinical variables were balanced between the treatment groups at baseline and during the course of the study.

This was a long-latency safety signal, in which cancer numbers were elevated for lorcaserin for all latency periods beyond 180 days. The number of new cancer cases was similar in the two treatment groups for the first 180 days.

Benefit-Risk Evaluation

The FDA weighed the drug’s benefits against the excess cancer risk. The difficulty of mitigating that risk, and the uncertain clinical benefit led them to conclude that the benefits do not outweigh the risks.

To provide a little more context, following are some highlights from the U.S. label and from the trial:

Lorcaserin FDA Label

In the U.S. package insert there is no mention of cancer on the Adverse Reactions section. However, in section 13: Nonclinical Toxicology, Carcinogenesis subsection, it is stated:

  • Increase in mammary adenocarcinoma in female rats
  • Increase in mammary fibroadenoma in female rats at all doses
  • In male rats, treatment-related neoplastic changes in different organs

The CAMELLIA-TIMI 61 Trial

Title: A Study to Evaluate the Effect of Long-term Treatment With BELVIQ (Lorcaserin HCl) on the Incidence of Major Adverse Cardiovascular Events and Conversion to Type 2 Diabetes Mellitus in Obese and Overweight Subjects With Cardiovascular Disease or Multiple Cardiovascular Risk Factors (CAMELLIA-TIMI).

Arm/Group description: Participants received lorcaserin HCL 10 mg or lorcaserin HCL placebo-matching , tablets, orally, twice daily for up to 52 months.

Adverse Events Time Frame: From baseline up to 30 days after last dose of study drug (approximately 56 months). This constitutes the “On-Treatment plus 30 days analysis set”.

Study-Specific Events: Malignant neoplasms (with the exception of basal cell and squamous cell carcinomas of the skin), among others, were considered study-specific events, per protocol.

Comments

The following are my personal, independent thoughts.

Even though there were non-clinical results possibly indicating a higher risk of neoplasms, the end result was very difficult (if not impossible) to predict. The phase 4 study was designed with the specific purpose of responding to the cardiovascular safety question. In spite of that, cancer-related adverse events were given special attention in the protocol, but this was not enough for them to detect a safety concern. Twelve-thousand patients were studied, however the 95% CIs for the rate ratios were still not clearly statistically significant.

Should all studies analyzing products with a potential risk of causing cancer, allow for sufficient follow-up time as to reduce attrition risk?

Could an analysis of FAERS data have revealed that signal earlier?

Treatment and placebo groups were well balanced in terms of demographic characteristics and some clinically and epidemiologically relevant risk factors, which is expected.

But, what about the analysis of potential differences between patients who acquired cancer and those who didn’t? This is not going to change the outcome, obviously. But machine learning techniques applied to these safety data could help understand the differences between the 2 groups (cancer vs no cancer), both in treatment and placebo arms. The knowledge gained as a result could open our minds to a whole world of possibilities.

Knowledge Graphs, Semantic Web and Drug Safety

by 1 Comment

Second part of: Mining PubMed for Drug Induced Acute Kidney Injury

When I wrote “Mining PubMed for Drug Induced Acute Kidney Injury”, my intention was to start exploring the use of PubMed for knowledge discovery in the fields of drug safety and pharmacovigilance. But to discover new knowledge, you need to know what is already known, what has been discovered already.

Using our example of drug induced acute kidney injury (AKI), if we want to discover new associations, we should be aware of which drugs are known to increase the risk of renal damage, or to worsen renal function on an already impaired kidney.

For marketed, prescription drugs, we can use the FDA labels as a reference of what adverse reactions are already known for a specific product, and check them against our PubMed search, for knowledge discovery.

How can we reach that goal? First, it is helpful to know that the FDA provides us with the labels of all approved products, in xml format. To download FDA labels click here.

To understand this approach, we need to talk about a variety of concepts and how they can help us to reach our objectives:

Semantic Web

The Semantic Web is a Web 3.0 technology. It is a way of connecting data between entities or systems that allows for rich, self-describing interactions of data available worldwide across the Internet. Nowadays, the majority of information provided by the Internet is delivered in the form of web pages. These documents are linked each other through the use of hyperlinks. Humans or machines can read these documents. But machines, other than finding keywords on a page, have difficulties extracting any meaning from these documents.

The semantic web will open the web of data to artificial intelligence processes, it seeks to encourage people to publish their data in an open standard format, at the same time that encourages Internet users to analyze these data and gain knowledge.

The Graph Database

The graph database is the way the semantic web stores data. The Resource Description Framework, or RDF, constitutes the building blocks for forming the web of semantic data, and it defines a type of database which is called a graph database.

Data can be stored in the form of triples. A triple describes the breaking of an RDF statement into its 3 constituent parts: the subject, the predicate (or property), and the object of the statement. For example, we want to define the color of a capsule for a medicinal product:

In terms of this simple graph, the subject is the capsule; the predicate (or property) is color; and the object is red. That’s why this is called a “triple”, and the information is stored in triples.

Semantic Modeling

RDF offers a flexible, graph-based model for recording data that is interchangeable globally, and this is the beauty of it. However, it does not offer any means of recording semantics, or meaning.

We want to include semantics in data, for the purpose of knowledge integration. One of the most important benefits of adding semantic meaning to our data is that it can be bridged across domains of knowledge automatically. For example, suppose we have two websites, one of them stores information about product labels, including all adverse reactions, and the other stores information about treatments given to a specific group of patients. Although these 2 sites have been created independently, the information they provide is complementary.

In principle, any sharing of data between the 2 sites cannot be done, in principle, by joining tables in their databases. This is because they have been designed independently, and because they are using different database server systems, which are not cross-compatible. This type of information interchange across incompatible, independently defined data systems takes time, money, and human contextual interpretation of the different sources of data. It is also limited to these 2 websites / datasets. Any further additions to their knowledge from elsewhere would require a similar effort.

With the introduction of semantics and RDF, all this is much easier to do. How do we model the two site scenario using semantic modeling? To begin with, the 2 sites need to apply a common, standard vocabulary (a collection of terms with a well-defined meaning that is consistent across contexts). This can be done if the two sites adopt the same ontology (to define contextual relationships behind a defined vocabulary), for expressing the meaning behind the data they expose, and publishing the data on an endpoint which can be queried, so that the sites can communicate with each other across the web.

NOTE: Currently you can download from the Web thousands of databases encoded as triples. Among the largest ones we highlight DBpedia, which is the triple-store version of Wikipedia.

Example Applied to Drug Safety – Drug Induced Acute Kidney Injury

In this example we can see how different databases containing partial health-related information that are conceptually interconnected, can be linked for knowledge discovery.

Data from SNOMED (global common language for health terms), MeSH (Medical Subject Headings, a comprehensive controlled vocabulary for the purpose of indexing journal articles and books in the life sciences), SIDER (Side Effect Resource), DailyMed (drug brand names and FDA product labels), ClinicalTrials.gov (web data source for clinical trials), DrugBank (comprehensive data about medicinal products), and the Diseasome (integrated database of genes, genetic variation, and diseases), along with any patient record data, or even PubMed data can be interlinked and queried. It opens a myriad of opportunities. And this is just a small example of what we can do.

Graphic-Based, Triple-Store Browser

We are going to use a tool to display visual graphs of subsets of a store’s nodes and their links. It is an interactive tool for browsing, querying, and editing triple-stores, also known as graph databases.

On the previous post of this series, we found 8,916 PubMed abstracts for the search of drug induced acute kidney injury. We downloaded all the abstracts as an xml file. Some applications are able to obtain triples from them, in such a way that allows us to analyze them graphically. In this case, we got 2,705,300 triples from the mentioned PubMed results.

A simple example of it is shown on the next picture. We wanted to know how many abstracts were talking about “acute kidney injury”. By searching that keyword, the tool delivered 599 nodes (abstracts) and 300 links:

Abstracts are represented by yellow boxes

If we zoom in, we will see this:

Some abstracts using the keyword “acute kidney injury”

Let´s see how the triples look like in this graph database. Remember that we have converted the xml file into a triple-store, and that triples consist of Subject, Predicate, Object. Following there is a list of the 83 predicates extracted from the PubMed xml file we are using for this example, in alphabetical order:

Follow this link if you want to learn more on PubMed XML Element Descriptions and their attributes.

These predicates are the properties of each one of the articles we retrieved. The subject would be a unique identifier for each article, the predicate is one among the previous list, and the object is the value of the predicate for that specific article.

Some sample triples from the dataset are shown here:

Next, we can see the first triples of the dataset, where column “s” is for subjects, “p” is for predicates, and “o” is for objects:

The first subject is _:bE83C8647x3432, corresponds to a specific article. The corresponding predicate (property) is UI, and the object is D016428. In case you want to know, this element is used to identify they type of article indexed by MEDLINE. There is a code for each type of article. “D016428” is the code for the object “Journal Article”. Records may contain more than one publication type. In our case, this record contains just one publication type. In xml, it looks like this:

<PublicationTypeList>
<PublicationType UI=”D016428″>Journal Article</PublicationType>
….
</PublicationTypeList>

When we click on ” _:bE83C8647x3432″, this is what we get all the statements with that code as the subject. It shows the predicates associated to it, and the objects associated to the predicates:

In this post, we have talked about knowledge graphs, semantic web, triples, and have shown some of them, applied directly to our PubMed search on drug induced acute kidney injury. The next post will show more about it, and more results from including other, completely different, sources of data.

Review of Safety in FDA Medical Reviews

by 2 Comments

Analysis of the latest review of safety sections for new drug applications (NDAs and BLAs)

I found interesting to analyze the latest Reviews of Safety for the FDA submission classification Type 1 – New Molecular Entity. To obtain the medical reviews I accessed the FDA Approved Drug Products web page, and selected “Drug Approval Reports by Month”, and then “Original New Drug Approvals (NDAs and BLAs) by Month. I chose twenty clinical reviews, from October 2018 to March 2019.

Some of the clinical reviews were found as individual documents on the approval package, under the name of “Medical Reviews“, while other clinical reviews were embedded into a big file named “Multi-discipline Review“, containing the summary review, office director, cross discipline team leader review, clinical review, non-clinical review, statistical review, and clinical pharmacology review.

Another interesting aspect of this analysis is that older reviews have a specific headline for “Reviewer comments”, being each section followed by the comments of the reviewer at the end of it. However, more current clinical reviews do not differentiate the reviewer comments; actually, it looks like all text comes from the reviewer, instead of separating what the applicant submitted from what the reviewer commented. I think the newer approach is better, as the reviewers elaborate their thinking in a more extensive fashion.

If you are in a hurry, just go to the Conclusions at the end of this post.

CDER Clinical Review Template

According to the CDER Clinical Review Template 2015 for New NDA or BLA, the Review of Safety outline is as follows:

1. Review of safety
1.1 Safety review approach
1.2 Review of the safety database
1.3 Adequacy of applicant’s clinical safety assessments
1.4 Safety results
1.5 Analysis of submission-specific safety issues
1.6 Specific safety studies / clinical trials
1.8 Additional safety explorations
1.9 Safety in the postmarket setting
1.10 Additional safety issues from other disciplines
1.11 Integrated assessment of safety

However, although the basic outline for the review of safety section is the same, there are some variations, depending on the drug evaluated, wheter or not a subsection is pertinent and, possibly, the reviewer preferences or style.

Let’s go through each section and analyze what the reviewers have to say.

Safety review approach

Reviewers explain what the evaluation of safety for the product in questions is based on, which is, most of the time clinical trials. Among the clinical trials, which ones contribute the most, whether or not they perform pooling of data from different trials, and which treatment arms are to be considered.

Also, reviewers determine whether or not the methods to assess safety in the individual clinical trials and in the integrated summary of safety are considered appropriate.

FDA performs their own analysis using a variety of applications for drug safety analytics, like MedDRA Adverse Event Diagnosis Service (MAED), JMP amd JMP Clinical, while using analysis data model (ADAM) and study data tabulation model (SDTM) data sets, looking for differences in findings by the FDA reviewer compared to the applicant, among other aspects of the analysis.

If there are adverse events of special interest (AESI), they are stated here.

Review of the safety database

The review of the safety database includes the overall exposure, relevant characteristics of the safety population, and the adequacy of the safety database.

Overall exposure is summarized in a table. Depending on the product, the table may content number of individuals by arm, and if for example race is an important variable to understand pharmacokinetics (PK) data, that information should be included too, at least in the text.
Duration of exposure is an important aspect of exposure described here. They make a lot of emphasis in comparing median exposure times among groups. Reviewers will be concerned if those times are significantly different.

In this section, relevant characteristics of the safety population are also described. Demographics and baseline characteristics are included. Populations that are underrepresented are also highlighted. Whether or not important subgroup populations are well represented is something reviewers take into account. It is important to highlight whether or not the final safety database is well balanced in terms of baseline demographics and disease characteristics.

With respect to the adequacy of the safety database, the reviewers determine if the data are sufficient as to characterize the safety profile of the product. They evaluate if the total number of individuals in the safety database is enough or lower than recommended in FDA guidance, depending on the product under study. On occasions, the reviewer may recommend adding additional information to confirm safety of long-term use of the investigational product, in general or in specific subpopulations (like older people).
Another aspect of relevance is if there are evidence of safety signals in the clinical and pre-clinical development program.

Adequacy of applicant’s clinical safety assessments

Reviewers evaluate:
– Issues regarding data integrity and submission quality, that have an effect on the safety review.
– Categorization of adverse events. Adverse event and serious adverse event definitions are evaluated, as well as the safety reporting period for SAEs. Identification of issues with respect to recording. coding, and categorizing AEs, and if the applicant has used SOC and PTs applying MedDRA coding. Categorization of AE severity according to the CTCAE criteria is used in the majority of occasions. Interestingly, they tend to perform analysis of AEs/SAEs Grade 3 and up. Basis for the causality assessment. MedDRA version is also stated, as well as the selection of PTs by the use of Standardized MedDRA Queries (SMQ).
– Routine clinical tests, pregnancy tests, and acceptability of the schedule of events.

Safety results

Here reviewers pay attention, specifically to:
– Deaths. Reviewers evaluate whether or not they agree with applicant assessment of relatedness with the use of the investigational product.
– Serious Adverse Events (SAEs). Same as for deaths, reviewers make an opinion of agreement / disagreement with Company causality for each one of the SAE cases, as well as for the death cases.
– Dropouts and/ or discontinuations due to adverse effects. Here there is an evaluation of AEs leading to discontinuation. These significant adverse events are evaluated in terms of severity (defined by the applicant), and of the presence of patterns or concerns for these events. Distinction is made here to not include patients who discontinued due to events related to the disease rather than to the product.
– Treatment emergent adverse events (TEAEs) and adverse reactions. In general, this is the section where AEs are presented in tables, depending on the percentage of occurrence by study arm. Sometimes reviewers recommend including laboratory-related adverse reactions in a separate table in the package insert.

INTERESTING: Adverse Reactions. In one study, the applicant defined Adverse drug reaction as: “one that was reported in at least 2% of subjects who received the investigational product, occurred at a higher incidence than in placebo in the pooled pivotal trials, and was attributed to the study drug by the investigator. And the reviewer stated:

Using that definition, no ADRs would be listed in Section 6 Adverse Reactions section of the package insert. In the opinion of this reviewer, stating that there were no ADRs associated with the investigational drug might mislead health care providers and patients about the risks and benefits associated with taking the investigational drug. Therefore, the adverse events reported in at least 1% of subjects in the pivotal trials will be included in the package insert.

Comments regarding laboratory values, vital signs, ECG, QT, and immunogenicity were related to the presence of trends or abnormal values taking into account the expected changes explainable by the underlying disease. They analyze dose-dependency in relation to change in all those values.

A variety of statistical and epidemiological analyses may be applied here. It is not infrequent to find survival analysis curves applied to time-to-adverse event analysis.

Analysis of submission-specific safety issues

Here reviewers analyze a set of safety concerns that are related to the specific submission. For example, if hepatic toxicity is a concern, they evaluate liver effects. In some cases, those events are considered adverse events of special interest.
Description of clinical cases is something that occurs when a specific safety concern is analyzed.

Safety analysis by demographic subgroups

The purpose of this sub-section is to provide analyses of safety information for demographic interactions. Several methods and analytics may be applied here to explore the effects of possible interactions on safety signals / events. For many applications, individual clinical trials may not be powered enough to reach conclusions regarding safety among the demographic subgroups (age, gender, and race). Pooled analysis, when appropriate, will have greater power, interpretations about subgroup data should be made with caution. Nonetheless, these analyses should be performed when feasible, and tables and graphics should be created. Analysis of adverse events (real world data) by geographic region is also appropriate.

This type of analysis could be placed on Safety analysis section. Sometimes it appears here.
In this section, specific safety analysis and tables by age, gender, and race are presented and discussed. What is important here is if there are safety differences by age groups, sex, or race that could indicate a different safety profile or behavior.

Clinical outcome assessment (COA) analyses informing safety/tolerability

Sometimes, when pertinent, this section is included. For example in case of the application of patient reported outcomes (PRO) instruments. According to one reviewer, “PRO results are not likely to offer unbiased and conclusive evidence of patient’s quality of life.” This statement was probably made because the applicant wanted to include some benefit language in the product label.

Specific safety studies / clinical trials

Reviewers evaluate here if there was a study for the assessment of a specific safety issue, to identify or quantify a particular safety concern.

Additional safety explorations

Typically here human carcinogenicity or tumor development, human reproduction and pregnancy, and pediatrics and assessment of effects on growth are explored here. Moreover, overdose, drug abuse potential, withdrawal and rebound issues are discussed here too.

Safety in the postmarket setting

Sometimes the drug under investigation has some postmarket experience, in some specific countries, for example. That postmarket experience needs to be analyzed y evaluated from the safety point of view. The safety review of postmarket experience centers basically on serious adverse events. The expectations from safety in the post-marketing setting are also stated. In general, routine pharmacovigilance activities are in order.

Additional safety issues from other disciplines

In general, safety issues from other disciplines are discussed in their respective sections of the approval review.

Integrated assessment of safety

I have found a variety of approaches reviewers take to write this section. It goes from a minimalist (and I believe a little off) “The above safety assessment incorporates data from X trials and is therefore integrated”, to a short summary of all the previous sections, to an extended safety assessment of 2-3 pages. It normally determines if there are or not concerning safety findings. It is also stated whether or not the safety issues are correctly communicated in the product label, or determine if the applicant should include any AEs in “Warnings and precautions” section of the product label.

Postmarket commitments like PMR studies or REMS, boxed warnings, and enhanced pharmacovigilance are recommendations made by the reviewers in the integrated assessment of safety.

Conclusions

  • Analysis of the clinical reviews found in the approval packages from recently approved drugs is of great help understanding how review of safety is performed.
  • Read the FDA Clinical Review Template. That will give you an incredible insight on what reviewers are looking for.
  • After reviewing the first 10 reviews of safety, little to no information was added to my analysis by reading the next 10 ones.
  • Reviewers follow the main clinical review outline, but there is a wide variety of approaches to the evaluation of the different aspects and data of the Review of Safety.
  • Many of the subtle differences among the reviews of safety evaluated are product-related. So it would be advisable to review, for example, the 10 latest clinical reviews from approved oncology products if you are submitting an oncology product.
  • There is no mention of statistical analysis in the Review of Safety. Reviewers do a great job with extensive descriptive analysis. This is also helpful to avoid arguments related to applying statistical testing to pooled data.
  • Honest description of our safety data, making use of our current knowledge is generally more than enough to elaborate an appropriate safety profile of our product in our population(s). No rocket science needed.
  • This exercise helped me to obtain responses on what reviewers are looking for and, consequently, better prepare for NDA/BLA submission and success from the safety review perspective.
  • Although this post refers to the review of safety section of the clinical review, this approach can be applied to the rest of documents constituting the submission package for NDA / BLA approval.

Mining PubMed for Drug Induced Acute Kidney Injury

by 1 Comment

Enhancing signal detection capabilities beyond regular literature search

Methods and tools for data mining and all its variants, namely text mining and web mining, are emerging at cosmic speeds. But their implementation in pharmacovigilance and pharmacoepidemiology is still on its early stages.

The aim of this post is to explore and apply some of the current methods and tools using PubMed as the primary source for text mining. For this exercise I have chosen to mine PubMed abstracts for drug-induced acute kidney injury.

Searching for abstracts in PubMed

For this purpose, I used the PubMed Advanced Search Builder, which generated this search string: “(drug induced) AND acute kidney injury”, as shown here:

If you want to go directly to the results from that search, you can use https://www.ncbi.nlm.nih.gov/pubmed?term=(drug%20induced)%20AND%20acute%20kidney%20injury

At the time of writing this post, there were 8916 results from that search. The next step was to download all the abstracts into a text file, as shown on this screenshot:

Mining Abstracts with pubmed.mineR

Obviously, nobody has the time to read all the almost nine thousand abstracts. And if we had the time to do it, we would not have the ability, as human beings, to digest and integrate all this knowledge.

To help us with the task of knowledge discovery, we are going to use some applications in R language for the purpose of mining the text we have extracted. And this is when fun begins.

The R package we will use here is pubmed.mineR. The latest information on this package can be found here. To run the code I have used RStudio.

Package pubmed.mineR has many capabilities, most of them are not shown here. I have identified which of them would be more interesting for pharmacovigilance mining.

The initial code is shown below. In this post, code has a gray background, and the output a light blue background.

It starts by installing the package, and setting up the directory on your computer for input-output. I have used mine, but you will have to change it for your own path. The next step is to call the library.

# Install package:
install.packages(“pubmed.mineR”)
# Set directory:
setwd(“D:/PharmacovigilanceAnalytics.com/pubmed.mineR”)
# Call library(ies)
library(pubmed.mineR)
library(data.table)
# readabs will automatically read the abstracts from the pubmed file (pubmed_result.txt) and will write an S4 object which I named ‘akidrug’
akidrug <- readabs(“pubmed_result.txt”)
# printing first and last abstracts from akidrug:
printabs(akidrug)

The output resulting from ‘printabs(akidrug)’ is here, showing the first and the last abstracts:

Number of Abstracts 8916
Starts with
Renal Damaging Effect Elicited by Bicalutamide Therapy Uncovered Multiple Action Mechanisms As Evidenced by the Cell Model. Peng CC(1), Chen CY(2), Chen CR(3), Chen CJ(2), Shen KH(4)(5), Chen KC(6)(7)(8), Peng RY(9). Author information: (1)Graduate Institute of Clinical Medicine, School of Medicine, College of Medicine, Taipei Medical University, 250 Wu-Hsing Street, Taipei, 11031, Taiwan. (2)Wayland Academy, 101 North University Avenue, Beaver Dam, WI, 53916, USA. (3)International Medical Doctor Program, The Vita-Salute San Raffaele University, Via Olgettina 58, 20132, Milano, Italy. (4)Division of Urology, Department of Surgery, Chi Mei Medical Center, Tainan, 710, Taiwan. (5)Department of Optometry, College of Medicine and Life Science, Chung Hwa University of Medical Technology, Tainan, 717, Taiwan. (6)Graduate Institute of Clinical Medicine, School of Medicine, College of Medicine, Taipei Medical University, 250 Wu-Hsing Street, Taipei, 11031, Taiwan. kuanchou@tmu.edu.tw. (…
ADT-induced hypogonadism was reported to have the potential to lead to acute kidney injury (AKI).
ADT was also shown to induce bladder fibrosis via induction of the transforming growth factor (TGF)-β level.

Ends with
[APROPOS OF 8 CASES OF CARBON TETRACHLORIDE POISONING]. [Article in French] VEREERSTRAETEN P, VERNIORY A, VEREERSTRAETEN J, TOUSSAINT C, VERBANCK M, LAMBERT PP. NA NA

Word atomization

Something we can do is to determine the word frequency. For this purpose, pubmed.mineR uses “word_atomizations”:

akidrug_words <- word_atomizations(akidrug)
# Print the first 10 words by frequency
akidrug_words[1:10,]

The following table shows the first ten most frequent words. As expected, these most frequent words refer to the acute kidney injury aspect of your PubMed search. Please keep into account that word counting is one of the fundamental basis of text mining. Word counting contains still a very important research opportunity. I suggest to analyze, from the list generated by this example, word counts that are not as obvious as “renal”, “kidney”, or “patient” for this specific type of search.

ID NumberWordFrequency
53805renal19824
18468acute9478
40387kidney8584
38691 injury8236
49268 patients7712
53138 rats5519
32372 failure 5451
60217 treatment4861
34861 group4004
60509 tubular3701

Gene atomization

Gene atomization will automatically fetch the genes (HGNC approved Symbol) from the text and report their frequencies.

# If you remember, akidrug is the name of the file for the collection of abstracts. Akidrug_gene will be the collection of genes found in those abstracts
akidrug_gene <- gene_atomization(akidrug)
# Next, we will obtain a subset of akidrug_gene containing 2 variables, one for the gene symbol and the other for the frequency
genes_table <- subset(akidrug_gene, select = c(“Gene_symbol”,”Freq”))
# Next, we prepare the whole gene database. The complete set can be obtained from the HGNC site.
hgnc<-read.delim(“D:/PharmacovigilanceAnalytics.com/pubmed.mineR/hgnc_complete_set.txt”,
header = T,stringsAsFactors = F)

We want to extract sentences containing Alias of the Human Genes, from the PubMed abstracts:

alias_fn(genes_table,hgnc,akidrug,”output”,c(“drug induced”,”acute kidney injury”,”adverse event”))

A sample from the results (saved to “outputalias”) is shown here:

TNF TNF-alpha
C3 C3b
PAH PH
PARP1 PARP
26184635
However, it is still unclear whether PARP overactivation happens during acute kidney injury (AKI) caused by endotoxic shock (ES).

1

And another one:

BAK1 BAK
CD5 T1
CR1 KN
ICAM1 CD54
IL18 IL-18
30531196
Other biomarkers of drug-induced kidney toxicity that have been detected in the urine of rodents or patients include IL-18 (interleukin-18), NGAL (neutrophil gelatinase-associated lipocalin), Netrin-1, liver type fatty acid binding protein (L-FABP), urinary exosomes, and TIMP2 (insulin-like growth factor -binding protein 7)/IGFBP7 (insulin-like growth factor binding protein 7), also known as NephroCheck®, the first FDA-approved biomarker testing platform to detect acute kidney injury (AKI) in patients.

2
  1. 1.
    Liu S, Liu J, Liu D, Wang X, Yang R. Inhibition of Poly-(ADP-Ribose) Polymerase Protects the Kidney in a Canine Model of Endotoxic Shock. Nephron. 2015;130(4):281-292. https://www.ncbi.nlm.nih.gov/pubmed/26184635.
  2. 2.
    Griffin B, Faubel S, Edelstein C. Biomarkers of drug-induced kidney toxicity. Ther Drug Monit. December 2018. https://www.ncbi.nlm.nih.gov/pubmed/30531196.

Literature Curation with PubTator Functionality

PubTator is a Web-based tool for accelerating manual literature curation (e.g. annotating biological entities and their relationships) through the use of advanced text-mining techniques. As an all-in-one system, PubTator provides one-stop service for annotating PubMed citations.

PubMed.mineR has a PubTator function. The PubTator function uses a PMID as entry and delivers results regarding chemicals, diseases, genes, and mutations, if they are referenced in the article. We are going to use the article by Griffin (see article 2 above, PIMD: 30531196) Let’s try it and see what hppens:

# Run PubTator function on PIMD 30531196 and save results on pubtator_output:
pubtator_output <- pubtator_function(30531196)
# Print PubTator output for chemicals, diseases, genes, and mutations:
pubtator_output$Chemicals
pubtator_output$Diseases
pubtator_output$Genes
pubtator_output$Mutations

Results are here:

There are many other pubmed.mineR functionalities. I encourage the reader to explore them and comment on the comments section of this post.

Exploration of other R packages.
Articles Published by Year and Word Cloud

This section is inspired on the code presented here.

library(RISmed)
library(dplyr)
library(ggplot2)
library(tidytext)
library(wordcloud)
result <- EUtilsSummary(“(drug induced) AND acute kidney injury”,
type = “esearch”,
db = “pubmed”,
datetype = “pdat”,
retmax = 30000,
mindate = 1960,
maxdate = 2019)
fetch <- EUtilsGet(result, type = “efetch”, db = “pubmed”)

abstracts <- data.frame(title = fetch@ArticleTitle,
abstract = fetch@AbstractText,
journal = fetch@Title,
DOI = fetch@PMID,
year = fetch@YearPubmed)
abstracts <- abstracts %>% mutate(abstract = as.character(abstract))
abstracts %>%
head()
abstracts %>%
group_by(year) %>%
count() %>%
filter(year > 1959) %>%
ggplot(aes(year, n)) +
geom_point() +
geom_line() +
labs(title = “Pubmed articles with search terms (drug induced) AND acute kidney injury \n1960-2019″, hjust = 0.5,
y = “Articles”)
cloud <- abstracts %>%
unnest_tokens(word, abstract) %>%
anti_join(stop_words) %>%
count(word, sort = TRUE)
cloud %>%
with(wordcloud(word, n, min.freq = 15, max.words = 500, colors = brewer.pal(8, “Dark2”)), scale = c(8,.3), per.rot = 0.4)

This is the first of a series of posts analyzing text mining applications for PubMed. The second one explores knowledge graphs and semantic analytics.

Top 7 Predictive Model Applications in Drug Safety and Pharmacovigilance

by 3 Comments

As drug safety and pharmacovigilance organizations develop more sophisticated data analytics capabilities, they are starting to move from basic descriptive analysis towards predictive analysis and the development of predictive models. Predictive analytics uses existing information to make predictions of future outcomes or future trends in all areas of Medicine and Health Care1.

The importance of being one step ahead of (adverse) events is most clearly seen in the framework of signal detection, and of the identification and characterization of individuals with a specific risk for developing an adverse event after the exposure to a medicine, both in clinical development2,3 and in post-marketing settings4.

Identification of risks from spontaneous reports

Predictive modeling can be used for the identification of previously unrecognized risks of medicines in pharmacovigilance reports. A nice example of this use is VigiRank, a data-driven predictive model for emerging safety signals, which has been shown to outperform disproportionality analysis alone in real world pharmacovigilance signal detection5. VigiRank is to be applied in VigiBase, in which predictive models have been proven useful to detect safety signals that were eventually validated, in pediatric populations.6

Evaluation of unexpected increase in reporting frequency

Similarly, the European Medicines Agency developed an algorithm to detect unexpected increases in frequencies of reports, in particular quality defects, medication errors, and cases of abuse or misuse. The algorithm applied to the EudraVigilance database showed encouraging results7.

Risk prediction of adverse experiences after exposure to a drug

Predictive models have been also used to predict the relationship between exposure to an investigational medicinal product and the risk of adverse events. For example, Niebecker8 characterized the relationship between exposure to afatinib and diarrhea and rash/acne adverse event trajectories, with the final goal of developing a modeling framework to allow prospective comparison of dosing strategies and study designs with respect to safety. In another other example, predictive models have been used for the prediction of adverse reactions after administration of rituximab in patients with hematologic malignancies9.

Different approaches to predictive analysis have been taken, depending on the specific machine learning tool applied. Machine learning has been used to predict the probability of adverse event occurrence at the time of drug prescribing, using a neural network model.10

Predictive models in clinical development and postmarket signal detection

Other authors developed a model to quantify whether safety signals observed in first-in-human studies were likely the result of chance or the compound under investigation. The model quantifies how likely an event is due to chance, conditionally on the characteristics of the subject and the study11.

The combination of different predictive modeling techniques like random forest, L1 regularized logistic regression, support vector machine, and neural models were successfully applied to detect signals arising from laboratory-event-related adverse drug reactions. The authors combined features from each of the modeling techniques into a machine learning model. The application of this model to an electronic health record environment was considered satisfactory for signal detection purposes12.

Supervised machine learning signal detection methods have been tested for the identification of adverse drug reactions. In the world of medication dispensing data, sequence symmetry analysis (SSA) has been used to detect signals of adverse drug reactions. This precise study shows how a gradient boost classifier complements well SSA13.

Specific subpopulations like hospitalized patients

Predictive analysis and model development shows interesting uses in the evaluation of risks as in this case, where the authors used mathematical models to determine the probability of adverse drug experiences in the surgical setting at the time of hospital admission, identifying the patients that are at a higher risk of an adverse drug experience during the hospital stay14. In another study focused on drug safety in hospitals, the authors perform a systematic review of predictive risk models for adverse drug events during hospitalization15.

Prediction of hepatotoxicity and interactions

To predict drug-induced hepatotoxicity based on gene expression and toxicology data, by means of a multi-dose computational model16.

Use of predictive models for the prediction of adverse drug reactions induced by drug-drug interactions17.

Predictive models for comparative safety

Leonard CE et al. utilized a Cox proportional hazard model to identify comparative safety differences among 3 sulfonylureas and the risk of sudden cardiac arrest and ventricular arrhythmia18.

  1. 1.
    Alanazi H, Abdullah A, Qureshi K. A Critical Review for Developing Accurate and Dynamic Predictive Models Using Machine Learning Methods in Medicine and Health Care. J Med Syst. 2017;41(4):69. https://www.ncbi.nlm.nih.gov/pubmed/28285459.
  2. 2.
    Federer C, Yoo M, Tan A. Big Data Mining and Adverse Event Pattern Analysis in Clinical Drug Trials. Assay Drug Dev Technol. 2016;14(10):557-566. https://www.ncbi.nlm.nih.gov/pubmed/27631620.
  3. 3.
    Poleksic A, Xie L. Predicting serious rare adverse reactions of novel chemicals. Bioinformatics. 2018;34(16):2835-2842. https://www.ncbi.nlm.nih.gov/pubmed/29617731.
  4. 4.
    Ventola C. Big Data and Pharmacovigilance: Data Mining for Adverse Drug Events and Interactions. P T. 2018;43(6):340-351. https://www.ncbi.nlm.nih.gov/pubmed/29896033.
  5. 5.
    Caster O, Sandberg L, Bergvall T, Watson S, Norén G. vigiRank for statistical signal detection in pharmacovigilance: First results from prospective real-world use. Pharmacoepidemiol Drug Saf. 2017;26(8):1006-1010. https://www.ncbi.nlm.nih.gov/pubmed/28653790.
  6. 6.
    Star K, Sandberg L, Bergvall T, Choonara I, Caduff-Janosa P, Edwards I. Paediatric safety signals identified in VigiBase: Methods and results from Uppsala Monitoring Centre. Pharmacoepidemiol Drug Saf. February 2019. https://www.ncbi.nlm.nih.gov/pubmed/30767342.
  7. 7.
    Pinheiro L, Candore G, Zaccaria C, Slattery J, Arlett P. An algorithm to detect unexpected increases in frequency of reports of adverse events in EudraVigilance. Pharmacoepidemiol Drug Saf. 2018;27(1):38-45. https://www.ncbi.nlm.nih.gov/pubmed/29143393.
  8. 8.
    Niebecker R, Maas H, Staab A, Freiwald M, Karlsson M. Modelling Exposure-Driven Adverse Event Time Courses in Oncology Exemplified by Afatinib. CPT Pharmacometrics Syst Pharmacol. January 2019. https://www.ncbi.nlm.nih.gov/pubmed/30681293.
  9. 9.
    D’Arena G, Simeon V, Laurenti L, et al. Adverse drug reactions after intravenous rituximab infusion are more common in hematologic malignancies than in autoimmune disorders and can be predicted by the combination of few clinical and laboratory parameters: results from a retrospective, multicenter study of 374 patients. Leuk Lymphoma. 2017;58(11):2633-2641. https://www.ncbi.nlm.nih.gov/pubmed/28367662.
  10. 10.
    Kasatkin D, Bogomolov Y, Spirin N. [Steps to personalized therapy of multiple sclerosis: predicting safety of treatment using mathematical modeling]. Zh Nevrol Psikhiatr Im S S Korsakova. 2018;118(8. Vyp. 2):70-76. https://www.ncbi.nlm.nih.gov/pubmed/30160671.
  11. 11.
    Clayton G, Schachter A, Magnusson B, Li Y, Colin L. How Often Do Safety Signals Occur by Chance in First-in-Human Trials? Clin Transl Sci. 2018;11(5):471-476. https://www.ncbi.nlm.nih.gov/pubmed/29702733.
  12. 12.
    Jeong E, Park N, Choi Y, Park R, Yoon D. Machine learning model combining features from algorithms with different analytical methodologies to detect laboratory-event-related adverse drug reaction signals. PLoS One. 2018;13(11):e0207749. https://www.ncbi.nlm.nih.gov/pubmed/30462745.
  13. 13.
    Hoang T, Liu J, Roughead E, Pratt N, Li J. Supervised signal detection for adverse drug reactions in medication dispensing data. Comput Methods Programs Biomed. 2018;161:25-38. https://www.ncbi.nlm.nih.gov/pubmed/29852965.
  14. 14.
    Bos J, Kalkman G, Groenewoud H, et al. Prediction of clinically relevant adverse drug events in surgical patients. PLoS One. 2018;13(8):e0201645. https://www.ncbi.nlm.nih.gov/pubmed/30138343.
  15. 15.
    Falconer N, Barras M, Cottrell N. Systematic review of predictive risk models for adverse drug events in hospitalized patients. Br J Clin Pharmacol. 2018;84(5):846-864. https://www.ncbi.nlm.nih.gov/pubmed/29337387.
  16. 16.
    Su R, Wu H, Xu B, Liu X, Wei L. Developing a Multi-Dose Computational Model for Drug-induced Hepatotoxicity Prediction based on Toxicogenomics Data. IEEE/ACM Trans Comput Biol Bioinform. July 2018. https://www.ncbi.nlm.nih.gov/pubmed/30040651.
  17. 17.
    Liu R, AbdulHameed M, Kumar K, Yu X, Wallqvist A, Reifman J. Data-driven prediction of adverse drug reactions induced by drug-drug interactions. BMC Pharmacol Toxicol. 2017;18(1):44. https://www.ncbi.nlm.nih.gov/pubmed/28595649.
  18. 18.
    Leonard C, Brensinger C, Aquilante C, et al. Comparative Safety of Sulfonylureas and the Risk of Sudden Cardiac Arrest and Ventricular Arrhythmia. Diabetes Care. 2018;41(4):713-722. https://www.ncbi.nlm.nih.gov/pubmed/29437823.

The Pharmacovigilance of the Future: Prospective, Proactive, and Predictive

by 2 Comments

Peter J Pitts, President of the Center for Medicine in the Public Interest, and Hervé Le Louet, President of CIOMS, have just published an intellectually-stimulating essay on the future of pharmacovigilance entitled “Advancing Drug Safety Through Prospective Pharmacovigilance“. The complete reference of the article is: Pitts PJ, Le Louet H. Ther Innov Regul Sci 2018; https://doi.org/10.1177/2168479018766887.

First, the authors point out that we are entering a new era in drug development. To support that statement, they refer to how the FDA is transforming its way of thinking. On the FDA guidelines on collaborative approach for drug development for pediatric rare diseases, the agency proposes new design types for rare diseases, utilizing the example of Gaucher disease. The proposed study design features include: double-blind, controlled, randomized, multi-center, multi-arm, multi-company noninferiority or superiority trial to evaluate the efficacy and safety of product A, B, C…

Other innovative approaches found in the FDA guideline are those related to the use of modeling and simulation to optimize pediatric studies, as for example to predict the effect of a drug in children based on previously known performance in adults, particularly to inform the dosing rationale.

Small frequency of the disease or the outcome under study should never be an excuse for the weaknesses of a study design. As we were taught when studying Epidemiology, if you don’t have enough cases in your center, then you should try a multi-center study. Now, the next frontier is, not only multi-center studies, but multi-company studies.

The pharmacovigilance paradigm is changing and evolving very fast, keeping up with all the new developments in artificial intelligence (AI), the analysis of real world data to obtain real world evidence, and the multiple, really diverse sources of safety information that are available today. According to the authors:

Artificial intelligence will facilitate what the pharmacovigilance ecosystem lacks today – coordinated and efficient systems for developing actionable evidence on safety and effectiveness

The field of artificial intelligence is evolving so rapidly, that I’m convinced we will pretty soon face the paradox of needing AI help for human intelligence to understand what AI is delivering.

To me, the most important point of this paper relies on the subtle comparison between what I would call the ‘old’ pharmacovigilance, which is reactive and non-anticipatory, and the ‘new’ pharmacovigilance, which is proactive in continuously evaluating the benefit-risk profile of a product, elaborating predictive models giving place to predictive pharmacovigilance.

I cannot finish my review without mentioning the most interesting and intriguing section of the paper “Inventing the Pharmacovigilance Future“. In this section, the authors present brilliant ideas they very probably can help to put into practice. I would like to highlight their suggestion of “an international effort under the tripartite chairmanship of the WHO, the ICH, and the CIOMS, to investigate, debate and develop prototype programs for drugs approved via expedited review pathways, based on more sensitive premarket metrics of risk pontential”. And the last, and most intriguing of the concepts presented in this paper, is the Real World Pharmacovigilance Score (RWPS), a baseline prediction of likely adverse events based on projected volume and specific clinical use. Many questions I have about RWPS are not responded in the paper: how is it calculated, do you have any example of application in ‘real world’? I wish they will publish a paper on this matter.

I recommend you to read the essay, eye opening and intellectually challenging.

Disproportional Recording vs Disproportional Reporting

by Leave a Comment

Signals of Disproportional Recording – Seriously?

I have read with great interest a paper recently published in Drug Safety journal (Zhou X, Douglas IJ, Shen R, Bate A. Signal Detection for Recently Approved Products: Adapting and Evaluating Self-Controlled Case Series Method Using a US Claims and UK Electronic Medical Records Database. Drug Saf. https://doi.org/10.1007/s40264-017-0626-y). As always, I read with attention all papers authored by Andrew Bate and his team.

Something in the paper that called my attention was the concept of “Signals of Disproportional Recording (SDRs)“. I read it first in the abstract, and my first thought was that it was an error, and the authors were actually referring to Signals of Disproportionate Reporting (SDRs). Signals of disproportionate reporting are understood as statistical associations between medicinal products and adverse events i.e. drug-event pairs. When a SDR is identified for a medicinal product, this adverse event is reported relatively more frequently in association with this medicinal product than with other medicinal products (Practical Aspects of Signal Detection in Pharmacovigilance : Report of CIOMS Working Group VIII. CIOMS, Geneva, 2010).

Of course, it was not a mistake. The Analysis Methods section of the paper explains it clearly:

Incidence rate ratios (IRRs) are calculated by comparing the rate of events in a given post-exposure period (risk period) with the rate of events in unexposed periods absent of the exposure (all other observed times). In a signal screening framework, statistical uncertainty is examined based on the 95% confidence interval (CI) of the IRR estimates. Specifically, when the lower bound of the 95% CI of the IRR estimate is > 1, this is considered a positive finding and is a Signal of Disproportional Recording (SDR) analogous to SDRs in spontaneous reporting, which are findings of potential interest that have not undergone clinical review to be considered signals of suspected causality.

But I wanted to know more about this new concept. Who was the first one to use it on a scientific communication? I did a little research and found one reference for the same concept: A. Bate. Tuning Epidemiological Study Design Methods for Exploratory Data Analysis in Real World Data. Abstract presented in 15th ISOP Annual Meeting “Cubism in Pharmacovigilance” Prague, Czech Republic 27-30 October, 2015.

So, it is clear that the concept of signal of disproportional recording has been first used by Andrew Bate. The concept is brilliant. The finding of an adverse event-drug pair in an electronic health record happens because someone recorded it, independently of whether or not is was also reported.

Even though secondary use of electronic healthcare records (EHR) and insurance claims data for hypothesis testing has occurred for many decades, signal detection activities to identify potential drug safety issues has historically focused primarily on spontaneous reports. But there is an increasing interest on using EHR for signal detection in pharmacovigilance. Electronic health records exhibit special characteristics (longitudinal nature, partially unstructured data) that is requiring to adapt old analytics to this new framework, and even create new methods and concepts (Zorych I, Madigan D, Ryan P, Bates A. Disproportionality methods for pharmacovigilance in longitudinal observational databases. Stat Methods Med Res 2013; 22(1):39-56). That is, the analytic techniques used for the analysis of drug-event (or drug-outcome) pairs in spontaneous reporting of adverse events, are not directly applicable to EHR and claims data. Creative thinking and new research are welcome in this area.

The paper is interesting, not only for using “disproportional recording”, which is anecdotal. It explores how to adapt and evaluate the self-controlled case series method and its use in claims and electronic medical records databases, for the challenging aspect of signal detection in the framework of  recently approved products. You will find an interesting discussion on the appropriate risk period selection method, which may be different in drug safety signal detection, than in formal hypothesis-testing studies.

The authors conclude that self-controlled case series method may be useful for safety signal detection in EHRs, and that early identification of previously unknown safety signals may be possible shortly after a new product is launched. Performance of this method varies by the nature of both exposure and event pair and their anticipated association.