IQWiG: Reliable assessment of drugs is only possible on the basis of clinical study reports (CSRs)

In 2012 researchers from the German Institute for Quality and Efficiency in Health Care (IQWiG) presented a study in the BMJ analysing information sources used in 16 health technology assessment (HTA) reports of drugs (“benefit assessments”). This study clearly demonstrated that publicly available sources, such as scientific journals and entries posted in trial registries (“registry reports”), contained far less information on methods and outcomes of clinical trials than non-public CSRs prepared by pharmaceutical companies.

In a second article published today in PLOS Medicine, the IQWiG researchers now show that if, instead of only assessing selected outcomes as in the first study, all patient-relevant outcomes of the clinical trials are assessed, the information deficit in the publicly available sources is even greater.

Data analysed for more than 1000 outcomes

All HTA reports of drugs completed by IQWiG between 2006 and 2011 also formed the basis for the new study. The authors included those trials from the HTA reports for which the pharmaceutical companies had provided complete CSRs to IQWiG. Publicly available information in scientific journals and trial registries was available for 86 out of 101 of these trials, so that the information provided in all 3 sources could be compared. The trials contained data on more than 1000 patient-relevant outcomes such as mortality or disease symptoms.

Huge difference in the information provided

IQWiG assessed whether the results on the patient-relevant outcomes in the trials were “completely” or “incompletely” reported. The difference in the information provided was immense: Whereas 86% of all outcomes were fully reported in unpublished CSRs, the corresponding number was only 39% for combined publicly available sources. Likewise, negative effects on patients (“harm outcomes”) such as serious adverse events or treatment discontinuations were reported far less often in the publicly available sources (27 to 72% versus 84 to 92%, depending on the harm outcome investigated).

Make CSRs publicly accessible

Beate Wieseler, Head of the Drug Assessment Department, comments on this first comprehensive quantification of the information gain through full CSRs: “The publicly available journal articles and registry entries thus report less than half of outcomes of clinical trials comprehensively. At the same time, with CSRs documents are available that provide complete information on methods and outcomes. The consequence can only be: all CSRs must be made publicly accessible. One should note that IQWiG is already in a privileged position due to its legal remit to conduct benefit assessments. Other researchers and physicians wishing to be fully informed about the advantages and disadvantages of an intervention have even more difficulties in gaining access to data.

Although the proportion of clinical trials published as scientific articles or registry reports has increased in the past few years, this is unfortunately not accompanied by more complete reporting of patient-relevant outcomes. Large information gaps still remain and we cannot even guess how large these gaps are in other types of drug trials or in trials of non-drug interventions.”

Need for legislation

Beate Wieseler further points out: “The plan by the European Medicines Agency (EMA) to make all clinical trial data submitted for marketing authorization publicly available from January 2014 onwards can therefore only be a first step. A central repository is required, also including data from older trials, as such trials investigate drugs widely used in current medical practice. These data would probably shed a totally new light on several drugs and their position in their therapeutic area. However, a voluntary commitment by the pharmaceutical industry, which would like to decide on a case-by-case basis which data it discloses, is insufficient. We are hoping for legislation and continue to support the All Trials Campaign (see below) so that the problem stays on the agenda.”