August 11, 2014; Silver Spring, MD – Highlights – Draft

Executive Highlights

The FDA held a Part 15 Public Hearing yesterday on the topic of interim data disclosure from ongoing cardiovascular outcomes trials (CVOTs), which are used to support new drug approval as per the FDA’s 2008 CV Guidance. There was plenty to say on that topic, especially given the difficulty that the FDA’s current lack of explicit guidance has caused – consequences have included Sanofi’s withdrawal of its NDA for Lyxumia (lixisenatide). Some of the most important commentary from the day, in our view, was on the appropriateness of the overall 2008 CV Guidance – presentations from yours truly at Close Concerns and diaTribe helped raise this point, as did talks by other audience members including the ADA’s Dr. Robert Ratner. Although the FDA panel seemed laser focused (appropriately so) for most of the day on the more specific issue of improving the current practice of fully public interim data disclosure (the core topic of the hearing), we were encouraged to hear panelist Dr. John Jenkins acknowledge that “it may be time to revisit the basis of cardiovascular studies for diabetes drugs” and to hear FDA biostatistician Dr. Lisa LaVange share that the FDA is aware of the concern that CV safety requirements might be driving companies away from diabetes research – we hope this presages a more extensive conversation to come on the appropriateness of the CV Guidance, although we imagine that the agency will certainly first deal with the narrower issue of interim data disclosure.

In the presentations and discussion on the issue of early data disclosure from CVOTs, a key question that emerged was what different stakeholders needed to know in terms of interim data. A thought-provoking proposal from AstraZeneca involved the transmittal of a simple thumbs-up/thumbs-down decision (as determined by the FDA’s hazard ratio thresholds) from a CVOT’s data monitoring committee (DMC) to the FDA, but Dr. LaVange (along with some speakers from academia) argued that the agency and clinical community would likely require more granular data before considering approval. Dr. Steve Nissen (Cleveland Clinic, Cleveland, OH), architect of the 2008 CV Guidance, suggested that more detailed interim data could be sent to a highly firewalled and very select group within the sponsor company that would use that information to handle regulatory filings. Among the solutions discussed, no very clear winner emerged, as each alternative came with its own set of trade-offs. The consensus in the room was that the FDA could help the current situation by providing more explicit guidance on how interim data should or should not be disclosed to support approval while still preserving trial integrity, although agreeing on the details of that guidance will require more discussion. Read on below for our coverage of the day’s key presentations and discussions – we have ordered the concluding remarks before the audience presentations, as Dr. LaVange’s summary served as an excellent summary of the day’s key findings.

Detailed Discussion & Commentary

Opening Presentations

A Two-Stage Approach to Ruling Out an Excess Amount of Risk

Mat Soukup, PhD (FDA, Silver Spring, MD)

FDA statistician Dr. Mat Soukup provided an overview of the design of the FDA’s 2008 Cardiovascular Guidance for diabetes drugs. He delineated the two stages of the process – stage 1 requires excluding a hazard ratio of 1.8 based on 122 events (which could be done through interim data from an ongoing CVOT), while stage 2 requires excluding a hazard ratio of 1.3 based on 611 events. He emphasized that findings from stage 1 are “unstable,” and that “caution should be advised in interpreting such findings.” Although he acknowledged that the 2008 CV Guidance and its implementation involves unique statistical and operational challenges, he stated that the focus of the day’s hearing is on maintaining the confidentiality of stage 1 data. In a follow-up comment, FDA biostatistics lead Dr. Lisa LaVange noted that the FDA is not used to being privy to interim analyses from ongoing trials, and that diabetes now represents a unique paradigm for the Agency.

• Dr. Soukup noted that sponsors can provide data to meet the CV Guidance requirements in a number of ways. He grouped the options into four main categories:
  
  • Approach #1: Sponsors would conduct a meta-analysis of phase 2/3 results to support stage 1. Stage 2 would depend on a large outcomes trial.
  • Approach #2: Sponsors would use an outcomes trial to support stage 1, and then a meta-analysis of that trial and a second outcomes trial to support stage 2.
  • Approach #3: Sponsors would use a single trial, powered for stage 2, for both stages 1 and 2. Stage 1 would be supported by an interim analysis from the trial.
  • Approach #4: Sponsors would use a meta-analysis of phase 2/3 studies plus an interim analysis of an ongoing outcomes trial for stage 1. Stage 2 would be supported by an analysis of the completed outcomes trial.

Concluding Remarks

Summary and Next Steps

Lisa LaVange, PhD (FDA, Bethesda, MD)

Dr. Lisa LaVange closed the hearing by providing an overview of her major conclusions from the day’s proceedings. Her remarks were followed by several additional comments from the panel, including a statement from Dr. John Jenkins that “it may be time to revisit the basis of cardiovascular studies for diabetes drugs.”

• Dr. LaVange noted that none of the speakers had commented on the FDA’s policy of redacting interim data, although at least one comment submitted to the docket expressed concerns with that approach, and said she would “take that as general agreement” with the current policy. A recent example of redacted data was the FDA’s process for evaluating Takeda’s Nesina (alogliptin).
• Her main high-level conclusions from the hearing were: (i) That the issue of data confidentiality in CVOTs results from trying to balance the need for getting new therapies to patients in a timely manner with the need to collect adequate data in order to answer questions about cardiovascular safety, and (ii) that stakeholders would benefit from more explicit FDA guidance regarding procedures for appropriate disclosure of interim results.
• The FDA has heard “loud and clear” from speakers today that requiring two separate CVOTs, one pre-approval and one post-approval, would not be desirable, due to both the extra resources needed to conduct two trials and continuing concerns about difficulties in recruitment for the second trial if the outcome of the first is made public.
• Notably, Dr. LaVange acknowledged that “we have heard” the voices of patients today and tomorrow demanding new therapies and the concern that cardiovascular safety requirements are driving companies away from diabetes research.
• The FDA is unlikely to adopt a “DMC-only” approach to interim analysis. She cautioned that not all sponsors have access to a DMC with the likes of Dr. Tom Fleming or other experienced statisticians, and that as “we are talking about full approval,” she does not believe that only knowing whether the boundary of a 1.8 hazard ratio was met without knowing any other details of the interim analysis would be sufficient for regulators to make a decision.
• Dr. Robert Temple reminded the audience that the enrollment of high-risk patient populations in CVOTs is no accident. He acknowledged that enrolling only patients with a longer duration of disease could affect the generalizability of results to the larger patient population, but he emphasized that these trials are “designed to see if drugs are toxic,” and that it is necessary to enroll the sickest patients in order to accrue enough events for a sufficiently powered trial
• Dr. John Jenkins acknowledged the suggestion by many speakers that “it may be time to revisit the basis of cardiovascular studies for diabetes drugs.” While he did not elaborate in detail, the FDA has clearly heard the call to reevaluate whether large CVOTs should be required across the board for approval of diabetes drugs, and we hope this message will lead to a broader conversation on the 2008 Guidance in the relatively near future.

Audience Presentations

Steven Nissen, MD (Cleveland Clinic, Cleveland, OH)

Cleveland Clinic cardiologist and FDA 2008 CV Guidance co-architect Dr. Steve Nissen took the podium for a relatively lengthy half hour time slot to both defend the current overarching FDA policy for cardiovascular safety evaluation and to suggest ways to circumvent the problems posed by public data disclosure from ongoing trials – the presentation featured some very clear, strong opinions. He stated that the FDA’s CV Guidance does not have “one shred of disadvantage” and argued for the need to “fight to preserve this paradigm.” In his view, CVOTs increase the chance that a diabetes drug will be able to prove CV superiority – in our view, although CVOTs are bigger and longer than more standard phase 3 trials, the opposing force is that the FDA policy may slow the flow of new therapies. Turning to the topic of the hearing, Dr. Nissen believes that regulators need access to more interim data than a thumbs-up/thumbs-down regarding the 1.8 hazard ratio threshold, but that very strict guidelines are needed to ensure that interim data is not disclosed to anyone beyond a tightly firewalled group within the sponsor company. Although the plan he suggested was comprehensive in terms of plugging possible data leaks to the public, as a complete package we worry that it may be quite a burden for industry. The FDA panel seemed quite intrigued by Dr. Nissen’s plan, and asked him a number of questions following his presentation.

• Early in his presentation, Dr. Nissen noted that there is currently no clear guidance on what happens if a drug that meets the initial 1.8 hazard ratio threshold for approval does not meet the 1.3 hazard ratio threshold at the completion of the CVOT. He steered clear of making any suggestions on this issue, but noted that it would be important to have clearer guidance from the FDA on that point. We imagine withdrawing the drug from the market would be one possibility, although further trials could be another alternative. 
• Dr. Nissen extolled what he sees as the virtues of the FDA’s 2008 CV Guidance. He argued that with CVOTs, there is a greater chance that a drug will be able to demonstrate CV superiority – we would note that although CVOTs are more likely to demonstrate superiority than a standard phase 3 program, with the way CVOTs are being executed now (high-risk patients, relatively short duration) the chances are much lower. He stated emphatically that there is “not one shred of disadvantage” to the CV guidance, and asserted “we have to fight to preserve this paradigm.” CVOTs, in his view, are delivering valuable data already (he cited the heart failure signal seen in SAVOR for AZ’s Onglyza [saxagliptin]), and the flow of this data will only increase in coming years.
• According to Dr. Nissen, the possibility of a drug being approved after meeting the first CV risk threshold (1.8) is a “convenience,” and excluding a 30% increase in risk should be the ultimate goal. Using this mindset, it follows that compromising the ongoing CVOT means that the FDA will be missing vital safety data on the drug, which could throw its approved status into question.
• Dr. Nissen used a set of examples to illustrate how the disclosure of MACE point estimates from ongoing trials could lead to misinterpretation. Although an interim MACE hazard ratio point estimate of 1.22 would meet the guidelines for first-stage approvability, Dr. Nissen noted that in the public domain, that data would lead to marked reductions in adherence and recruitment. A different, but also devastating, set of outcomes is possible if the interim data is positive. He also suggested that trial sponsors might potentially modify a trial (by changing enrollment or any number of other parameters) if they learned of interim data.
• Dr. Nissen believes that disclosing data only to a select group within the sponsor company can provide sufficient information to the FDA while maintaining sufficient confidentiality – he laid out a comprehensive plan for how to set up the firewalled group. Some of the recommendations were quite logical (having a pre-determined interim data agreement between the sponsor, primary investigator, and data monitoring committee), although in totality the package of recommendations could be quite a lot for sponsors to implement.
• Dr. Nissen reminded the audience that a strict policy on interim data disclosure in the US is of no use without global cooperation, as disclosure of data by another country would undo all attempts to maintain trial integrity.   

Questions and Answers

Q: Earlier on you had two slides on two different situations, one in which the trial showed a benefit, and one where there was an increased hazard ratio. Can you talk more about what a sponsor might do following public data disclosure that could undermine trial integrity?

A: The problem is that we have to rely upon the integrity of sponsors. When they say that they have a firewalled team that is not allowed to interact with the people making trial decisions, at some level we have to trust them. It is like everything else in life – if someone really wants to break the rules, then they will break the rules. The agency should require a signed statement committing that no communication with the firewalled group has taken place or will take place. In the end, you can’t legislate integrity, but you can’t do that with academic investigators either. We can at least set the standards and lay out the principles of what the agency expects, and if anyone breaks the rules, the agency would be correct in keeping their feet to the fire.

Dr. Patrick Archdeacon: When trying to set up a blinded group within a sponsor company, there can be unforeseen complications as personnel transition between roles. The alternative that John Jenkins proposed was that only the FDA sees the granular data, and that the company remains entirely blinded, so that there is less to worry about.

A: That is a very interesting idea, and Dr. Jenkins, I admit that I had not thought of that. There is some history of private communications between DMCs and the Agency – I think it’s worth thinking about. The difficulty I have is that the decision by the company to file is made in the context of the totality of the information, of which the CVOT would be only part of that decision. I made the prediction that companies would not have the audacity to file if they had a hazard ratio close to 1.3. Having access allows companies to integrate what they know about other benefits and the CV trial, and then the unblinded team would be able to say whether or not they are going to file. It respects the rights of the sponsors to make decision about what is in the best interests of their companies. When you take them out of that process, I have some discomfort, but the idea is worth thinking about.

Dr. Robert Temple: Could you elaborate on the things that the people running the trial could do if they had access to interim data?

A: Biological plausibility is a term used in pharma that I hate, because it is easy to come up with a biologically plausible reason for something after the fact. For example, if in phase 3 development a sponsor saw an evidence of favorable trends, then they might have the tantalizing possibility of doing a trial that was big enough to show superiority, or by increasing the sample size of the ongoing trial. Similarly, and this is much more nefarious, if there is a trend towards harm, the best way to make a signal of harm go away is to have poor quality data in the trial. Having lots of dropouts makes the data converge at 1.0. I’m not saying that anyone would do that, but you’re never going to know. If someone is trending in the wrong direction, it might influence how hard they work to keep people in the trial, or to make sure that there are very high quality standards.   

Thomas Fleming, PhD (University of Washington, Seattle, WA)

Dr. Thomas Fleming, a statistician with extensive experience serving on Data Monitoring Committees (DMCs), argued that disclosure of interim results can seriously compromise the integrity of clinical trials and that it is essential for the FDA to provide clearer guidance about interim disclosure procedures for CVOTs. He reviewed several trials in which interim results suggested very different conclusions than the final results ultimately did (for example, the hazard ratio from a trial in the 1990s comparing a novel HIV therapy to the standard of care was 2.08 when the first set of interim results were analyzed but had dropped to 1.00 by the completion of the trial). Dr. Fleming asserted (correctly, in our view) that disclosure of such potentially misleading interim data (e.g. point estimates and confidence intervals for CV hazard ratios) could seriously affect the public perception of a drug’s profile and consequently alter the behavior of trial participants. However, in the case of CVOTs, he believes it would be possible to publicly disclose only that a hazard ratio of 1.8 (the current pre-approval standard) has been ruled out without damaging the integrity of the trial. He emphasized the need for clear, consistent FDA guidance specifying who would have access to more detailed interim data when approval is being considered; possibilities include (i) only the DMC; (ii) the FDA and a small core group from the sponsor that could make a judgement about whether to pursue regulatory approval at that time; and (iii) the FDA and a designated “unblinded” team from the drug’s sponsor who would not be involved in future trial conduct, which Dr. Fleming believes would be acceptable only if clear procedures to maintain firewalls are put in place.

Questions and Answers

Dr. LaVange: You used the term “public equipoise” and had an informative slide about what can break down with enrollment. “Public” is a big term, it could include the people in the study, physicians running the study, the sponsor, other sponsors, the investment community – there are lots of people in “the public.” Are there gradations of equipoise? The people in the study need to be the most blind. And what can the role of the DMC be in an ongoing study to assess this? We put drugs on the market based on interim data and the study has to keep going. What kinds of assessments do we need and who oversees that?

Dr. Fleming: In terms of maintaining public equipoise, we need to have confidence that while we’ve set up a procedure that facilitates earlier potential filing, we must be confident that a trial whose principal issue is ruling out a hazard ratio of 1.3 is successfully completed. Equipoise and perceived equipoise are key. Confidentiality needs to be maintained for patients in the trial, future patients in the trial, and caregivers; the sponsor’s blinded team needs access to information that doesn’t alter equipoise, as in a superiority trial. Saying you didn’t cross the boundary leaves you with a sense of equipoise. Indicating the DMC has released data allows us to maintain equipoise. It’s not well defined who gets access; that’s what we need the FDA, the DMC, and the sponsor to work through. The principle is that it should be the smallest core group possible that is integral to regulatory filing, and those people need to be firewalled from everyone else. That’s what I mean by equipoise in the global public.

The DMC can play a role because they have complete access to interim data. In the traditional sense of an ongoing efficacy trial, we don’t keep everyone blinded. The DMC is unblinded, and I’ve never been aware of anyone on a DMC who has ever led to release or unblinding of results. We can do this. The DMC could take on added responsibility to determine whether a 122 event, 1.8 hazard ratio is ruled out, and they would release data to the unblinded sponsor team to share with regulators. The DMC will continue in its role in protecting interests and trial integrity by monitoring the trial. We need to ensure there is a data access plan and performance standards, so we can get indirect insights. As the trial is continuing, is the enrollment rate adequate? Are we avoiding loss of adherence or cross-ins? We can carefully monitor these; they’re indirect but informative. Regulatory authorities have a huge influence. The FDA can have a favorable influence in providing insight about how to do this properly. We’re relying on your leadership to successfully complete this.

Q: You gave two choices [regarding who has access to interim data], and I can imagine option A [the DMC has sole access] where we don’t see the data is not the most attractive for us. Do you think if we do option B [unblinded sponsor team reports data to the FDA] properly with caveats, that would be ok?

A: I agree, A is a wonderful option for maintaining confidentiality. We fully understand it has controversial aspects with regard to how much insight the regulatory authorities need. We need guidance on whether that would be feasible. Assuming it wouldn’t be, B becomes an alternative approach, which has been the most widely taken. I agree, I believe approach B can be done with integrity. I don’t believe it will occur with a passive approach; it needs to be proactive and pre-specified. What is the purpose of releasing the data? Who will get access? How to we educate them and achieve accountability so when access is given, we’re not putting confidentiality from the broader public at risk. It can be done, but it would take an active approach. Guidance from the FDA on your views will be invaluable.

Q: Could there be a possibility for something in between? What is the feasibility of the DMC and the CRO from a sponsor releasing data to the Agency but only telling the sponsor that the boundary has been met? The sponsor themselves wouldn’t see point estimates, confidence intervals, or other data. Is that feasible?

A: That’s another creative variation. I’ve been impressed with the commitment of sponsors to doing this right. I want to hear more from sponsors; it’s likely sponsors would consider that acceptable because it enhances their ability to ensure confidentiality and reduces complexities. If sponsors find it acceptable, I would consider it an important variation that would still, unlike option A, allow the Agency full access while restricting it only to the Agency.

Q: That helps to address the point about a Data Access Plan. If only an unblinded team from the sponsor would have access, that becomes harder to define in smaller companies. If the sponsor never sees the data but we work it out so the CRO and DMC can submit, that would avoid problems within the sponsor where one person may have multiple hats.

A: Going back 15 years when data monitoring was growing in frequency, when the FDA issued its guidance on DMCs, they specified the need for an independent statistician outside the company. There was lots of angst about that initially; people said we were taking control away from company, but there were important benefits to such an approach. Industry has been very creative in implementing DMCs, and has come to accept that recommendation as near standard. If the FDA had guidance saying there are variations for this, but the approach that’s appealing is where the independent CRO gets data released for filing, that would be difficult, but it has upsides. If that was recommended as the ideal approach, it could become more commonly used. You’re right – for a smaller company, option B is challenging, I do think just because it’s difficult doesn’t mean we can’t do it. I’ve been impressed by sponsors’ commitment to doing the right thing. With proper guidance to an acceptable approach, we can be successful.

Robert Ratner, MD (American Diabetes Association, Alexandria, VA)

After the more statistics-centric presentation from Dr. Fleming, the ADA’s Dr. Robert Ratner drew upon his extensive clinical trial experience to bring the conversation “back to reality,” in terms of what scientists and clinicians deal with on the ground. He commented both on the specific issue of investigator equipoise in the face of interim data disclosure as well as on big-picture questions about the appropriateness of the FDA 2008 CV Guidance. On the latter issue, he noted that landmark studies such as the DCCT and UKPDS required decades to demonstrate a benefit, and that improvements in the care of CVD through medications like statins may wash out the effects of diabetes medications. He also pointed out that there are over 100,000 patients enrolled in CVOTs now, and questioned the ethics of randomizing this large a population of patients in clinical trials. Regarding the topic of interim data disclosure, Dr. Ratner used the example of the safety concerns over rosiglitazone and the resultant discontinuation of the TIDE trial to show that pre-existing data can damage the real and perceived equipoise of investigators in another trial (in the case of TIDE, a trial that was seen as very important). Every single patient who volunteers to be in a clinical trial, Dr. Ratner stated, is incredibly brave, putting his or her trust in clinicians without knowing exactly what the consequences will be. Dr. Ratner suggested that the least we can do is avoid damaging the quality of ongoing trials; the most we can do is ask questions about the ethics of requiring those trials in the first place.

Kelly Close, MBA (Close Concerns & diaTribe, San Francisco, CA)

Drawing upon her years of experience as a patient and at the helm of Close Concerns, Ms. Kelly Close took the podium to discuss the possible risks of interim data disclosure as well as some broader points regarding CVOTs. Using the example of the retinopathy incidence from the DCCT (which was elevated after one year with intensive control but drastically reduced in the longer term), Ms. Close noted that interim findings can be potentially misleading. While acknowledging that the DCCT retinopathy allusion may not be the best example due to potential confounders (e.g., differences in biology, study objectives, level of evidence, decision algorithm, etc.), the point that in extreme cases, deciding on trials using early data could conceivably create a problem by preventing researchers from working toward valuable decisions that are able to be attained only over the long term. She also expressed concern that J&J had lost the ability to test for superiority in the CANVAS trial for Invokana (canagliflozin), forcing the company to initiate another trial. She presented a range of possible solutions to the interim data disclosure issue, including conditional approvals in low-risk patient subgroups without CV outcomes data, semi-closed FDA Advisory Committee meetings, fully pre-approval CVOTs tied to extended patent life for diabetes drugs, and collecting CV outcomes data earlier in clinical development. She provided a balanced list of pros and cons for each, but the ultimate takeaway was that all of the options involve some level of drawbacks, a fact that itself raises questions about the FDA’s overarching CV Guidance.

• Ms. Close next discussed a few reasons why the FDA 2008 CV Guidance might not be having the consequences that its architects wanted it to have, including:
  
  • CVOTs are typically not representative of the general diabetes patient population, enrolling patients at very high cardiovascular risk;
  • The average length of follow up in current CVOTs may be too short to draw reliable conclusions;
  •  The wide use of drug classes such as statins in CVOTs may wash out the more modest effect of glucose lowering agents on CV adverse events;
  • The 2008 CVOT Guidance appears to be putting a damper on innovation in diabetes, as demonstrated by the FDA decision on Novo Nordisk’s Tresiba (insulin degludec) delaying the approval of the potentially transformational GLP-1 agonist/basal insulin combination IDegLira (Xultophy; insulin degludec/liraglutide);
  • Perhaps as a result, it appears that companies (such as BMS) are leaving diabetes.
  • Barriers to research and innovation could be one of many factors slowing the flow of young scientists to diabetes and endocrinology.

John Adler, PhLic (AstraZeneca, Oslo, Sweden)

AZ statistician for metabolic disease Dr. John Adler gave one of the day’s few audience presentations from industry – we applaud AZ’s bold move to not only provide comments during the hearing, but to put forth a possible solution (one that we thought was fairly balanced) for how to address the topic of interim data disclosure. The solution, which Dr. Adler termed the Pragmatic Approach, would involve a letter being sent from the DMC of the ongoing trial stating that the drug either met or did not meet the FDA’s threshold for approvability. This policy would be accompanied by measures to ensure high trial quality and efforts to protect against selection bias in the ongoing trial (perhaps by completing enrollment by the time of submission). Agreeing to be largely left out of the loop of data disclosure represented a concession from AZ, but the key question here is whether the FDA would be satisfied a simple thumbs-up/thumbs-down from the DMC. Remarks made by the FDA’s Dr. Lisa LaVange later in the day (see below) lead us to believe that it might not be.

Manu Venkat (diaTribe & Close Concerns, San Francisco, CA)

Mr. Manu Venkat closed out the day’s audience presentations with a discussion of patient factors and considerations. The cornerstone of the talk was a survey of nearly 3,000 patients conducted by the market research company dQ&A. The survey placed patients in the middle of a CVOT following public disclosure of interim data demonstrating a slight increase or decrease in the incidence of adverse cardiovascular events – in both cases in the scenario, the drug was granted approval. Nearly half of patients would consider withdrawing from the trial or would definitely withdraw if the drug demonstrated a CV benefit, while 75% would at least consider withdrawing if there was a slight increase in risk. These data support the assertion that early public data disclosure from a trial can drastically alter the enrollment dynamics for that trial. Mr. Venkat concluded the presentation by discussing possible future steps, and by calling upon the FDA to more broadly consider the appropriateness of the CV Guidance in addition to fine tuning the interim data disclosure policy, emphasizing that the policies of today stand to impact patients decades from now.

Walter Offen, PhD (AbbVie, North Chicago, IL)

Dr. Walter Offen echoed Dr. Fleming’s concerns from the beginning of the day about the consequences of interim data disclosure in CVOTs, saying there is a high risk that misleading interim results could modify the behavior of trial participants and investigators. He argued that a paradigm of two separate trials (one pre-approval and one post-approval) would not resolve the issue, as results from the first trial would be likely to influence perception and behavior in the second trial. He also pointed out that organizations in other countries would not be bound by confidentiality agreements with the FDA and could easily render those agreements moot, as any data disclosed internationally would quickly reach the public in the US. Dr. Offen repeated Dr. Fleming’s call for clearer guidance from the FDA on what types of interim data can be disclosed and how companies should handle the issue internally. Notably, he also encouraged the FDA to “engage in an open dialogue on the CVOT requirement for every diabetes drug regardless of biological plausibility,” suggesting that alternative post-market assessment methods may be sufficient in many cases.

Questions and Answers

Q: What’s your position on Dr. Fleming’s proposal for limiting access to those needed to submit the application? People involved in the business of the company and in the ongoing trial wouldn’t know.

A: We do concur with having a detailed access plan and documenting who needed access to which data. There would be different levels of unblinding. Someone like a statistician would have access to patient-level data, and that would have to be noted. Others would have access to hazard ratios and confidence intervals. That makes a lot of sense; we wouldn’t just firewall, we need to know who has access.

Q: You noted that there are problems associated with conducting two separate trials, that it would not be that different from an interim analysis. Tom hinted that’s less important, and there has been some view that it would better to separate the two trials. You don’t think there’s much difference?

A: Not only is there not much difference, but there’s potential harm. Many patients are still ongoing, and if you terminate, you lose information. You force most patients into being censored, and you don’t know if they would’ve had a cardiovascular event. If the point estimate is 1.26, and that gets out to the public when you’re trying to start a new trial, everything that Tom said could affect conduct would apply in this situation.

Q: Disclosure anywhere is disclosure everywhere, and different agencies have different considerations. In your group, did you discuss recommendations based on the global effect?

A: We didn’t discuss it as a group, but listening to speakers, it occurred to me I think this is a consideration. If we know a particular country will publish the hazard ratio, it’s incumbent on the sponsor to think, should we not submit to that country until the trial is over? It’s in the sponsor’s best interests; it’s a risk to submit knowing they’ll open the data to everyone, since that will risk their ability to complete the trial.

Q: After Dr. Fleming’s talk, the panel pointed out an alternative [where the FDA would have access to interim data but the sponsor would not]. That would prevent CVOTs or details from being discussed at Advisory Committee meetings. Do you have a position on that?

A: We didn’t discuss the Advisory Committee aspect, but I believe it’s the same answer as divulging to the public. The public can be anyone. If something was said at an Advisory Committee meeting, unless it’s closed, that wouldn’t be advisable.

Q: Since there would be an asymmetry of information between the sponsor and regulator, what would happen is no discussion at the Advisory Committee, and the FDA, after hearing the Advisory Committee, would later weigh their own opinion. Is it a concern for industry that the FDA would have a second phase of consideration?

A: I don’t know exactly what PhRMA would say. You’re talking about the intermediate option where nobody at the sponsor sees the data, and the DMC sends it to the FDA. Our position is not fond of that. To make a submission, the company needs to look at the whole data. A sponsor might see other risk factors and side effects where they wouldn’t want to submit. The information has to come to a limited number of people at the sponsor. If it was a closed session, the unblinded people from the sponsor could be there.

Q: One implication would be that the FDA would make a decision based on information the sponsor didn’t have. I just want to know what your comfort level is with that.

A: We’re generally not comfortable.

Q: You said something about deciding what people in the company would need access. Can you clarify, beyond saying we met the boundary and providing full data to an unblinded team who prepares the submission, can you provide thoughts on whether other people in the company, like the management team, would be aware of data like the point estimate and the confidence interval or just whether they met the boundary?

A: Management would only know that we’ve met the boundary; senior management has no reason to see detailed data. People like the head of diabetes research and others could have access. The Data Access Plan would identify who has access to what level of data and why. The group doesn’t need to be large; we need to identify who on the CRO has access, and that number should be small.

Q: For example, should the people responsible for making the decision to continue pursuing approval or people who raise money to complete the trial be aware of anything beyond whether we met the boundary.

A: I don’t think so. The people putting together the submission would.

Andrew Emmett (Biotechnology Industry Organization, Washington, DC)

Mr. Andrew Emmett took the podium to represent BIO, which in turn represents a broad community of biotechnology companies, academic institutions, and related organizations. He agreed with previous speakers that disclosure of detailed interim data from CVOTs would undermine trial integrity and potentially have implications for global drug development if other agencies do not require the same level of confidentiality as the FDA. He stressed the need for more consistent guidelines on specific data disclosure procedures, specific criteria to identify sources of risk, and methods to ascertain whether bias has been introduced as a result of interim data disclosure. He contributed to the day’s consensus opinion that conducting two separate trials would not resolve the issue of bias, assuming both trials had the same hypothesis and were conducted in a similar population, but he suggested that conducting a separate post-approval study in a different population with a different but still informative hypothesis could be a potential way to obtain information while maintaining trial integrity. Mr. Emmett joined the chorus of speakers urging the FDA to revisit its blanket mandate for large CVOTs for all diabetes drugs; he said the decision should be made on a case-by-case basis based on biological plausibility and that in many cases, real-world data from sources like the FDA’s Sentinel network would be sufficient to address a drug’s safety once it is on the market.   

Questions and Answers

Dr. LaVange: You talked about mitigating the impact of public disclosure by finding criteria to identify sources of bias and ascertain where bias is introduced. The devil is in the details. Have you pursued any ideas?

A: We’ve discussed it; developing prospective criteria is a critical aspect. It’s challenging to do, and we need to discuss it further.

Q: In Dr. Nissen’s presentation describing an ideal Data Access Plan, there were two level, one of which was that people within the company involved in marketing, investor relations, or business interests should have no access. What do you think is the feasibility of that level of restricted access for small companies? They need to conduct business, raise money for completion of the trial, do SEC filings, etc. How feasible would it be for a small company to have people involved not know the data?

A: 90% of BIO’s membership is small companies, without a product on the market, often pulling on venture capital for funding. They have limited staff, and many members have to wear a number of hats. The Chief Scientific Officer could be involved in the trial, in regulatory filings, and in management. It can be managed, but it’s more of a management and governance challenge. It involves restricting people involved in the regulatory filing from key business decisions and ongoing trial conduct; you can have appropriate firewalls. We believe it can be managed regardless of size.

Q: Is it clear that that would be preferable to having asymmetry of information, where the FDA has access and companies don’t?

A: That’s an interesting point about asymmetry. We’ve assessed whether the data could be communicated by the DMC, but we haven’t discussed in depth the asymmetry of releasing detailed data to the FDA and not the sponsors. There are some concerns about impacting companies’ ability to make informed decisions, and it could influence discussions with the FDA.

Jenkins: I want to ask Dr. Fleming the same question I asked Dr. Nissen: is it possible to test the integrity of trial after a broader disclosure has happened?

Dr. Fleming: That’s a very relevant point. If data is released to the unblinded team for the sole purpose of facilitating a filing, there is a risk that broader dissemination could occur. As Dr. Nissen said, we’re relying on a commitment to maintaining integrity. If we wish to pursue the concept of using a 122-event trial to facilitate a decision, there’s a necessity for some level of risk, but we’re minimizing it. We will all in the academic community and industry benefit from FDA guidance clearly laying out expectations. We have some ability to monitor this. As members of the DMC, we get insight about interactions within the company, and we develop a sense of whether the procedures in the plan are followed. We’re also following the performance standards document, enrollment, adherence, the level of cross-in – we’re looking carefully, and we continue to look post-marketing after the decision. If those standards are met, that provides an important level of reassurance about integrity. It’s not proof and it’s not absolute, but there are insights we would gain.

Charles Hennekens, MD (American Association of Clinical Endocrinologists, Jacksonville, FL)

Dr. Charles Hennekens echoed the concerns of several other panelists regarding the risk of bias resulting from interim data disclosure in CVOTs. He focused on the principle of equipoise – the lack of certainty regarding the benefits of a new therapy over existing therapy that allows investigators to randomize patients to either without bias. He explained that though it is appropriate to maintain such uncertainty in the face of incomplete data, many patients and physicians may lose equipoise if interim results suggest that a drug leads to favorable outcomes. Dr. Hennekens believes that only DMCs should have access to unblinded interim data, and that disclosure of point estimates and confidence intervals during a trial will undoubtedly compromise integrity. He also criticized the use of meta-analyses of small trials for anything other than formulating hypotheses, as the risk of bias is very high – “if you torture the data enough, they’ll confess.”

Jonathan Seltzer, MD (ACI Clinical, Washington, DC)

Despite the difficulties posed by interim data disclosure, Dr. Jonathan Seltzer raised the point that in some cases, companies, researchers, and the FDA have the duty to disclose pertinent information about approved products to the public. However, he pointed out that “doctors aren’t good at statistics,” and expressed skepticism that consumers of interim data, including physicians, patients, attorneys, and payers, will be able to draw meaningful conclusions from such technical information. He advocated a “DMC-plus” approach, in which the DMC would bear sole responsibility for analyzing interim data but would be required to provide a description and rationale for their surveillance strategy as well as prespecified criteria for allowing a trial to continue. Dr. Seltzer also suggested that DMCs should involve a wider range of stakeholders, including representatives of patients, payers, and practicing physicians in addition to expert physicians and statisticians. He summarized the prevailing attitude regarding DMCs by comparing them to the black boxes in airplanes – “nobody knows what they do, but we have faith in the process.”

Questions and Answers

Q: In the new DMC-plus process, what would be the regulators’ role? Would there be any change?

A: It’s reasonable to say regulators can sit in on the process. Regulatory thinking is an important part of the DMC process. There are a lot of little companies out there that have people on DMCs with very little experience. There could be a regulator on the DMC who’s not involved with approval.

Steven Marso, MD (University of Texas Southwestern Medical Center, Dallas, TX)

Dr. Steven Marso advocated for a “less is more” approach to interim data disclosure, in which as few people as possible are exposed to as little data as possible and there are serious consequences for not maintaining confidentiality. He also discussed the importance of designing trials to minimize the duration of the post-interim phase (the phase following interim data disclosure of any kind), and particularly minimizing the duration of the “post-interim, post-Agency” phase, when information has been publicly disclosed and the potential for bias is highest. Dr. Marso echoed many previous speakers’ call for more explicit guidance from the FDA regarding interim data disclosure.

Questions and Answers

Dr. John Jenkins: Do you advocate that the criteria should be to delay interim analysis until enrollment has completed, as opposed to basing it on the number of events?

A: It would be ideal if the interim analysis and enrollment could complete around the same time. That’s a concept to discuss further. In lieu of that, I would advocate a trial design where the number of events for interim analysis would likely accrue at the time of planned enrollment. Enrollment efficiency becomes a critical issue for interim analysis. I absolutely favor that interim and enrollment happen at the same time. That negates concern over selection bias, which is one way to mitigate overall bias.

Q: You recognize a shared responsibility between the sponsor and the Agency and say it’s critical that the sponsor have the minimum number of people unblinded. What is your opinion on number of unblinded sponsor employees being zero? Is that still sharing of responsibility? What’s the absolute role for the sponsor?

A: I agree with Dr. Nissen that the decision for sponsors to submit data is larger than just cardiovascular outcome data, but it is a vital piece of information. You should ask sponsors their opinion, but if I put my sponsor hat on, I’d be uncomfortable making a strategic decision and submitting without knowing the point estimate and confidence interval. I hear people like Tom Fleming talk and I have to translate that theory to the operational level. For me, the DMC role would be expanded or the FDA would have to be comfortable with summary-level data. If you look at an NDA, it’s much bigger in scope than DMCs are used to submitting. I’d ask the Agency if they would be willing to have an abbreviated dataset to review. If they had to hire that work out to a third party writer or CRO, that’s an extension of the sponsor. I would argue for involvement of the sponsor because of the decision to submit and the scope of the DMC’s work. Other academics may disagree, but if that’s the case, the Agency would have to accept less quantity.

Dr. Jenkins: The conundrum is that we hear a lot about the unreliability of the point estimate in an interim analysis, but we all seem to want to look at it to make submission or regulatory decisions. Is that valid? That point estimate is highly unreliable – Dr. Fleming showed a two-fold adverse finding becoming neutral by the end of a trial. I suspect sponsors would be more favorably inclined to submit with a point estimate of 0.9 vs. 1.25, and that regulators would be more inclined to approve. It unavoidably influences behavior and decisions. Are we talking out of both sides of our mouths?

A: We are. Dr. Nissen was my mentor, and he said you need the courage of your convictions. Following the courage of our convictions would mean the sponsor and the Agency would not know the point estimates, and they would submit and approve based on the DMC general commitment. That’s a great strategy, but it won’t fly because we don’t have the courage of our convictions, and because I’m not convinced Tom Fleming is always right.

Q: The charge to the DMC is an ethical question about equipoise, which is different than overall risk/benefit. We take on a charge that’s different from their original role, and we have to look at small subgroups. We do stuff with the data that DMCs don’t do. So I think it’s somewhat valid, and perhaps there’s a role for someone.

A: As long as the Agency, sponsors, and academia recognize that a subset of a subset of the first interim results look highly unstable. To predicate decisions on that is challenging, but it’s human nature to want to know.

Q: The company could put a specific point estimate that would allow the trial to go forward in DMC charter.

A: That’s a discussion to have with statisticians. We don’t want to adapt the design of trial based on interim results.

Dr. Jenkins: We discussed that on the way back for lunch, and how that would effectively change the boundary. Now the boundary would be smaller than 1.8, probably 1.3. It’s a false construct to tell the DMC to do a 1.8 analysis but not to refer it to the Agency unless the point estimate is less than 1.1. You’ve changed the boundary.

Q: What I hear you saying is you don’t think companies want to let someone else do this, at least that’s what a lot of people have said. There’s been some enthusiasm for the idea of the DMC seeing the data and no one from company seeing it, but you’re skeptical about the company being entirely blinded/blindsided. What are your thoughts about how many people in the company would see the data? There’s a business decision, which means it’s not just the people running the trial but the people deciding whether to continue pursuing approval.

A: If you look at Steve’s slide, there’s high overlap in my thinking about who needs to know. It’s a practical solution because I believe companies want to know data as much as regulatory authorities. The list is bigger than an academician would be comfortable with. If you look at Steve’s list, it’s a long list, bigger than I would love. Whether chief executives need to know – if I ran global drug company and CEO, I can’t answer whether I would want to know for submitting. I just don’t have the insight to know what drives those issues. From a pragmatic clinical trialist, I could be convinced, but people would need to take pledge of silence and we would have firewalls.

-- by Emily Regier, Manu Venkat, and Kelly Close