[StATS]: PharmaIQ talks (December 6, 2006) Category: Data mining
I attended the conference “Signal Detection and Data Mining” sponsored by PharmaIQ. Here are some notes I took during some of the talks.
Wolfgang Schurman from Schering AG presented a talk, “Developing Successful Long-Term Drug Safety Strategies for an Uncertain Future”.
Most regulatory changes in the review of drugs occurred as the result of medical disasters, such as the death of ten children from a contaminated tetanus vaccine in 1902, and the birth defects caused by Thalidomide in the 1950s,
Some of the recent regulations such as 726/2004 and 2001/83/EC speak to new safety reporting requirements.
The IOM report “The Future of Drug Safety: Promoting and Protecting the Health of the Public” came up with four conclusions:
- we have a safety crisis
- we recognize that all drug companies want to change
- we need to sort out who is responsible for what
- we need to improve communication
There is an uncertain future because new products have a widely unknown safety profile, new and stricter regulations, more inspections, a huge number of cases, huge databases but with poor quality. We live in an era of instant communication because of the Internet and we need to be proactive when possible and respond immediately when unexpected problems occur.
Goals are to provide safe medicine, avoid safety crises, avoid unjustified recalls, avoid legal claims, and avoid negative inspection outcomes. All of this must be done while limiting the inevitable increase in cost and resources.
Pharmacovigilance staff need to be adequately resourced, staffed, and trained. They need a thorough knowledge of the literature, epidemiology principles, and the company’s products. The department needs to be independent and report directly to a board member rather than be subordinate to marketing or medical affairs. You also need an optimal IT environment (e.g., e-reporting, image storing, and document management system).
Dr. Schurman criticized the 7/15 day reporting timelines as being unrealistic in a global environment with multiple organizations involved with research. He also stressed the need to harmonize and simplify regulations. You also need to concentrate on the important cases rather than get lost in a sea of information.
William Maier, Director of Epidemiology at Elan Ltd, talked about “A Combinatorial Approach to Signal Detection for Respiratory Adverse Reactions”
He summarized the new EU risk management plan (www.emea.eu.int/pdfs/human/euleg/9626805en.pdf).
Dr. Maier’s approach was what to do when a signal is detected. Where can epidemiology help you?
- Event classification-are you only seeing the severe tip of an epidemic?
- Severity of the problem-what is the predicted prognosis?
- Extrapolating to a larger population-how many people are actually affected?
- Incorporating other data sources-can you predict risk using data from other drugs?
- Evaluating other risk factors-is it a real association or is it spurious?.
- Evaluation of risk prevention activities-did they really work?
The objective is very critical. Are you trying to generate signals or testing a formal hypothesis?
Dr. Maier offered an interesting case study: a novel pill based therapy is in Phase III trials. Two unusual side effects are noted-patients who suffer from a rare condition called Churg-Strauss Syndrome (CSS). To investigate this further, he reviewed various databases to characterize the background rate of CSS in the general population, to note that all cases of CSS occur in patients with asthma, and that certain existing drugs for treating asthma have an increased risk for CSS compared to other drugs. There are many limitations to the inferences that you can draw from these databases, and Dr. Maier outlined these limitations.
Ulrich Vogel from Boehringer Ingelheim talked about “Best Practices for Combining Manual and Medical Opinion With Statistical Analysis.”
He applied a general sociological concept known as “the wisdom of crowds”. He cited work by a famous statistician, Sir Francis Galton who witnessed a crowd at a county fair estimating the weight of an ox on display. The crowd included both cattle experts and lay persons. Each participant wrote their estimate of weight on a card. Dr. Galton was trying to show the mediocrity of a crowd estimate, but when he computed the average of all the estimates, it was closer to the true weight than even the best of the individual guesses.
The key criteria for the wisdom of crowds to work is that you need a true diversity of opinion, independence of individual opinions, the ability to draw on local knowledge, and<U+FFFD> some mechanism for creating a consensus estimate.
CIOMS III presented the five axis model for evaluation of safety signals: mechanism, temporal relationship, dose relationship, reproducibility, and distinction from background rate.
If you get a signal, before you investigate further, ask four questions.
Is it evaluable? For example, the limitations of a database may be so severe that a signal is effectively uninformative.
Is it homogenous? For example, a complaint of weight gain might represent a mix of increased appetite and water retention.
How does it break down? For example, is there a specific demographic group that predominantly produces a set of adverse event reports.
How conclusive is your result?
Spontaneous reports do have many limitations, but they do deliver the perceived profile. They tell you what the public, the regulators, and the media are thinking.
Cross database comparisons are valuable because they increase diversity and appease concerns about limitations of an individual database.
Nawab Qizilbash from Oxon Clinical Epidemiology gave a talk, “Signal Matching: Managing and Monitoring Databases.”
His outline covered the following topics: key factors in choosing a database, comparing accessible databases, matching signal types to the appropriate database, requirements for properly using databases, and maintaining the knowledge and training to make informed decisions.
Internal data sources include company trials (as summarized by meta-analysis) and spontaneous reports. External sources of data include spontaneous reports, safes data, government statistics, hospital-based data, large medical practice databases, large medical and/or pharmacy claims databases, population based studies, patient registrics, patient based studies, and surveys.
The method of detection depends greatly on the frequency of events. Clinical trials can detect rates of 1 in 10, in 100, or maybe 1 in 1,000. Registries are better at detecting more rare events, such as 1 in 5,000, but to detect really rare events you need epidemiological methods (such as case-control designs) or spontaneous reports.
If you are choosing a database, you need to understand your objective (detection, investigation, or monitoring), your target population, the rarity and severity of the event, the comparison group, completeness of ascertainment, validity of the source, data quality, size of the database, timing, costs, and geography.
Differences in geographical prescribing practices are an important consideration. Your consideratons change if your drug is novel or a member of an existing class of drugs. You should also factor in whether your drug is on the market or still in development.
External spontaneous reporting databases include the FDA AERS, and WHO-UMC (76 countries).
Epidemiological databases can be split into medical practice databases, claims databases, and prospective patient registries. Data could be aggregated or at the patient-level. It can be cross-sectional or longitudinal. Some examples of these databases are the General Practice Research Database (UK), THIN (UK), Medplus (UK, France, Germany), and Thales (France). The U.S. has a large number of HMO databases that are useful for pharmacovigilance.
He highlighted an interesting example of an association that was noted with prescription of a statin (presumably for heart disease) and reduced rates of dementia. A re-analysis showed a strange dose response pattern, though. The protective effect was strongest in those patients who had been taking the drug the least amount of time and the effect more or less disappeared among patients taking the drug for several years. This is an example of prescription bias. If a patient comes into a doctor’s office showing signs of memory problems and forgetfulness, the doctor may be reluctant to add on a new drug, especially if it is for prevention of a future illness rather than the treatment of a current illness.
The expertise needed for a good safety review include an epidemiologist with significant clinical experience, a programmer who is not afraid of dirty data, and a statistician who can appreciate the special issues of observational data.
Pauline Gerritsen-van Schieveen from Astellas presented a talk, “Risk Management and Signal Detection in the Real Life of a Middle Size Company.”
She stressed the need to start risk assessment before the first-in-man study. You can identify potential issues from the pre-clinical data. This assessment is done by the toxicologist. There is also a competitor analysis (what are the safety problems of other compounds in the same therapeutic class).
For every identified or potential or theoretical risk, a risk minimization plan is written. Every risk needs systematic follow-up. Every bit of new information needs to be assessed to check for new potential issues or for relevant information on previously identified issues. If new potential issues are identified, the risk minimization plan is updated appropriately.
Some possible updates are changes in inclusion/exclusion criteria, or a change in safety parameters or in the frequency of sampling. Some data may require immediate action (e.g., fatal or life-threatening adverse events). Risk management processes need a direct link with crisis management.
Dr. Gerritsen-van Scieveen’s company has seen some major changes after the creation of a risk management group. There was a company wide update of terminology to reflect appreciation of risk management issues, closer cooperation with the exploratory development group, an update of quality system documents, and the reservation of more time for risk management. The company also started the development of an EMEA mandated risk management plan very early in the process. The company is also considering that more time ought to be spent on post marketing pharmacovigilance and less time in clinical development.
Inspectors are common in the pharmaceutical industry and they are looking for specific things in risk management. First they ask for a rational process for determining the frequency of safety evaluation-once a year is not enough. Second, they are looking for documentation of the risk management process and proof that the documented process is actually being followed.
It’s hard to take good notes, so I apologize to the speakers if I did not accurately portray their messages.
This page was written by Steve Simon while working at Children’s Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children’s Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at
for pages similar to this one at with general help resources. You can also browse Children’s Mercy Hospital website. Need more information? I have a page reproducing it here as a service, as it is no longer available on the Hospital. Although I do not hold the copyright for this material, I am This page was written by Steve Simon while working at Children’s Mercy