American College of Cardiology

STEVENSON AND KORMOS, ET AL., MECHANICAL CARDIAC SUPPORT 2000
JACC Vol. 37, No. 1, January 2001:340-70

IV. Establishing Efficacy for Devices: Ethical and Practical Challenges

A. Therapies for Life-threatening Illness

The life-saving potential, procedural risks and costs associated with mechanical circulatory support for patients with end-stage heart failure mandate the thoughtful development of a basis of evidence for efficacy, safety and cost-effectiveness. The Medical Device Amendments of the Food, Drug, and Cosmetic Act require that new devices be “safe and effective” before they can be marketed and that this evidence be provided through “well-controlled scientific studies” or through “valid scientific evidence”.⁸⁰ Mechanical assist devices fall into the highest of three risk categories defined by the Amendments; class III being life supporting or sustaining and having substantial importance in preventing impairment of health or having a potential to incur risk of injury or illness. For these devices, the sponsor must conduct clinical trials before the FDA grants marketing approval through a so-called PMA decision. Incremental changes to already marketed devices may be approved through a supplemental PMA. Selection of the research design for evaluating a specific mechanical circulatory support device must reflect: 1) the nature of the medical device innovation, 2) the severity of illness of the patients, and 3) the timing within the regulatory approval process (i.e., pre- and post-marketing observations).

The devices under imminent consideration are designed for patients with advanced stages of heart disease. Duration of observation is more limited when severity of illness is higher, as in current populations with acute or chronic refractory class IV heart failure. Knowledge of the grim natural history at this stage increases allowance for consideration of therapies available outside of the device investigational protocol. In other areas of life-threatening illnesses, such as cancer and AIDS, limitations in life expectancy have led to attempts to look at alternative research designs for approving new regimens of care, which would minimize the ethical conflicts of offering only one “active” treatment arm.⁸¹ Under these conditions, efforts have also focused on trying to shorten the pre-marketing clinical trial and FDA review processes, lessening the level of evidence necessary for safety and efficacy PMA, while shifting more emphasis to rigorous post-marketing studies.

B. Differences Between Development of Drugs and Devices

By comparison to pharmaceutical innovation, device innovation is more incremental and iterative in nature, as has been the case for LVADs. Both before and after approval for clinical indications, these devices have undergone continuous modification of drivelines, electronic controllers, alarms, connectors, vents, conduits and power supply systems. In the initial stage, this process merits a determination of the initial feasibility without a control arm for devices not previously tested in humans. For drugs, this has often been a dose-ranging study with non-mortality end points such as hemodynamics or exercise capacity. Further benefits of the initial testing phase for any therapy include the defining of promising study end points and the estimation of the sample size required to show a clinically significant benefit. Perhaps even more so for devices than for drugs, premature entry into a clinical trial phase invites the risk of failure or, at least, the need for redesign and retesting.

The relationship between cause and effect is generally more transparent for devices than for drug therapies. Both good and bad results of device implantation are often evident within hours or days, compared with longer and more modest effects over years during the recent drug trials in mild-to-moderate heart failure. It is less likely that the benefit or harm of devices can be masked or mimicked by the natural history of heart failure. The attribution of outcomes may thus be somewhat less prone to bias for devices than for drugs.

The transparent effects of devices also inform both patient and physician with regard to treatment arm in a randomized trial. Even if it were acceptable to perform sham surgery, the physical characteristics of the device would challenge provision of a placebo. This is a major difference between trials of devices and trials of drugs, in which patients on a placebo often assume that they are receiving active and “best” therapy. In addition, treatment is in general difficult or impossible to withdraw for recipients of support devices, by contrast with the trivial nature of withdrawal from a drug study. The cost of developing, manufacturing and ensuring quality of devices is vastly higher for devices than for drugs. Many innovative devices are developed in small companies without previous product revenue to support clinical trials. The total cost per patient is more than an order of magnitude higher than for drugs. The higher costs are balanced in part by the higher expected magnitude of benefit, such that calculated sample sizes are proportionately lower than for drug trials. The expertise and experience required for successful device implantation restrict the eligible sites in trials of devices. These restrictions also limit the generalizability of results after approval, when use extends to centers with less expertise.

The sum of evidence guiding therapy for drugs is dominated by evidence from the large trials completed prior to drug approval. Once approved, it is difficult to identify use and attribute effects of any particular drug because of the variability of prescription and adherence in complex regimens of other medications. For this reason, post-marketing surveillance provides limited information regarding drugs for heart failure except for non-cardiovascular side effects. By contrast, the very complexity and undisguised impact of devices render their use and outcomes easier to track, as long as appropriate registries are maintained. The cumulative body of evidence guiding the ultimate use of devices may in the final analysis be weighted more heavily by information gained after initial approval.

C. The Potential for “Breakthrough” Devices

It is possible that initial studies in the future could identify a therapy with such obvious impact that it would be considered a “breakthrough” for a population with otherwise high early mortality. In this case it would be neither necessary nor ethical to perform a prospective trial with a control group in this population. Freedman acknowledges: “In the rare case when the first evidence of a novel therapy’s superiority would be entirely convincing to the clinical community, equipoise is already disturbed”.⁸² As was pointed out by Norman Shumway, the pioneer of cardiac transplantation, no randomized trial of cardiac transplantation has even been conducted, and it is likely that none will ever be. In retrospect, cardiac transplantation was thus a breakthrough. Early mortality was high, but transplantation was considered to represent a major advance over the presumed imminent mortality of the initial recipients. Current mechanical support devices as bridge to transplantation were in fact recognized as effective for this purpose and accepted with only contemporary cohort data. In part, because of the differences described in the preceding text, such a breakthrough in the near future appears more likely for a device for heart failure than for a drug.

Most new therapies do not achieve breakthrough status during preliminary testing but fall somewhere along the spectrum before approval (Fig. 1). Short of an unequivocal breakthrough, there may be some therapies that are not yet approved but are nonetheless considered by experienced clinicians to be sufficiently effective that an RCT is not acceptable. When this is recognized, clinical equipoise is absent, and a randomized clinical trial cannot ethically be performed. The best way to bridge this gap and expedite regulatory approval of effective therapies has not yet been determined for any of the life-threatening diseases.

It is important to recognize that no new technology is likely to represent a breakthrough for every population considered. Even for a device promising 80% six-month survival for patients with end-stage heart failure, the design of trials would remain relevant when extending the technology to those populations with lesser severity of illness in whom the benefit of the device could not be assumed.

D. Ethical Considerations Governing Trials of Mechanical Circulatory Support

1. Requirement for clinical equipoise. The ethical basis of randomized clinical trials in general has been debated.^83,84 On one hand, a physician has a responsibility to an individual patient to provide the best care possible, and a randomized treatment would not allow the clinician to provide the perceived best care. On the other hand, it has been argued that without robust, clinical evidence from well-designed trials, physicians cannot decide what is best care, and indeed, physicians’ perceptions of optimal treatment have at times been shown to be wrong.⁸⁴ When the question is one that is appropriately addressed by a randomized clinical trial, a fundamental task for investigators is to understand the ethical and scientific principles.

The ethical conduct for clinical trials of a new therapy rests on a fundamental tenet: the therapy has the promise of some benefit, but its efficacy to achieve this benefit is unknown and the new therapy always carries some risk. Clinical trial ethics demand genuine uncertainty over whether the treatment arm is superior or inferior to the control arm. Equipoise, the principle of uncertainty regarding the merits of two or more treatments,⁸² is required of the investigators to conduct ethical research. If an investigator believes that one treatment has been proven to be superior to another, then the ethical basis for the RCT is lost and the investigator may not ethically randomize his or her patient to the inferior treatment. However, investigators generally have some bias about which treatment is “best,” which has led to considerable debate about what is truly required for an investigator to maintain equipoise. “Theoretical equipoise” has been described as an odd and ethically irrelevant state that could exist only when the clinical data supporting two treatments is essentially equal. Theoretical equipoise is fragile; it is easily disturbed by new data, and it may be inappropriately sensitive to the investigator’s perceptions of trial outcomes. A more insightful understanding of equipoise, Freedman proposes, is that of “clinical equipoise,” in which genuine debate and uncertainty exist in the clinical community regarding a new treatment or intervention. Evidence must be present to support both sides, and for new treatments with little or no preliminary data, opinion must exist both for and against such a new treatment. Clinical equipoise accommodates even decided treatment preferences by individual clinician investigators during the conduct of a clinical trial if widely spread debate exists between clinicians, and clinical equipoise remains until convincing evidence has been formally presented, reviewed and widely accepted by the medical community at large.⁸² The clinical equipoise paradigm has been extended recently by the suggestion that the physician investigator, as part of the subject recruitment, divulge his or her treatment preference.⁸⁵ It is possible that this may cause greater numbers of patients to take the “best medical advice” from their physicians, with the result that fewer patients may enroll in trials, particularly of therapies available elsewhere. However, this potential conflict may be mitigated by a careful and complete presentation of the scientific merit for the trial, including evidence both for and against the investigational treatment, which forms the ethical basis for the study design and conduct.

It is not ethical to do a trial that is unlikely to provide adequate information. The research protocol must be properly designed to test the new approach. Because of the potential for harm, the question being addressed must be one that is medically important. There needs to be proper matching of the active and control interventions to the patient group being studied. The trial must also be feasible, with adequate resources available to properly conduct and complete the trial. The trial must be able to measure the end points chosen and generate useful data. Finally, it must actively monitor for known and unknown adverse effects, and it must be approved by an institutional review board whose major mandate is to protect the rights and safeguard the welfare of human research subjects. The approved protocol must be thoroughly presented to a subject and accompanied by a written consent form. In the most commonly used design, the subject then decides whether or not to participate in the clinical trial and, if he or she agrees to participate, provides voluntary consent and is randomized.

2. Ethical issues in patient selection for mechanical circulatory support. Which aspects of RCTs for circulatory support devices merit special ethical consideration? Because these devices are currently designed to intervene for life-threatening heart failure, one ethical challenge is the question of whether any imminently terminally ill patients should be entered into RCTs. It has been suggested in the oncology literature that such recruitment for otherwise unavailable therapy may have aspects of coercion.⁸³ Several points, however, emerge in support of enrollment. First, some patients seek clinical trial participation. They may receive purpose and device satisfaction from participation in a research protocol prior to death. They may provide themselves with more comprehensive care. Their participation may ensure that they will not be abandoned, and their interaction with clinical trial staff may yield greater comfort. Second, and more specific to trials of end-stage heart failure, defining the “imminently terminally ill” condition for patients is extremely difficult if not impossible, as described in the preceding text. For clinical trials of surgically implanted devices, it may be unwise to recruit and randomize a truly moribund patient, because the higher operative risks may obviate any clinical benefit and may jeopardize the clinical trial end points. If the recruitment of such a patient flirts with medical futility, it may also be ethically questionable because it may jeopardize meaningful end points contributed by other subjects. As the severity of both natural illness and operative risk shift down, as described above, the more appropriate operative candidates for device therapy also have a greater likelihood that enhanced medical therapy, perhaps including outpatient inotropic therapy, may provide months of survival with some reasonable quality of life outside of the hospital.⁸⁶ The difficulty in making accurate predictions of life expectancy for presumed end-stage heart failure, in combination with the unknown risk/benefit outcome with mechanical circulatory support, provide the most persuasive foundation for clinical equipoise regarding randomized clinical trials of current circulatory support devices.

Allocation of mechanical circulatory support also raises a question of whether it is ethical to restrict a novel but unproven technology to a certain group of people. Left ventricular assist devices have been approved by the FDA only for use as bridging devices for heart transplant recipients. The current randomized clinical LVAD trial restricts the study population to those with advanced heart failure who require but do not qualify for cardiac transplantation.¹³ The ethics of this issue have been extensively reviewed.⁸⁷ Clinical trials demand that subjects be selected so that some benefit from an LVAD intervention can be demonstrated, thereby benefiting the trial and other patients in the trial. Left ventricular assist device therapy has been seen as inferior to cardiac transplantation; therefore, potential cardiac transplant patients may reasonably be excluded from a destination therapy trial because investigators are not ethically mandated to offer an inferior treatment.⁸⁷

3. Ethical issues surrounding randomization. When an appropriate candidate has been identified, randomization in a trial of mechanical circulatory support poses unique challenges if subjects may be randomized to receive a device or conventional therapy consisting primarily of drug treatment. Such fundamentally different treatment approaches—one surgical and the other medical—have been associated with substantial subject and investigator treatment bias and ambivalence about random treatment assignments. This bias is of special significance for a fatal disease, as previously noted for cancer patients.⁸¹ Patients may passionately favor the new device technology, or they may sshrink from a mechanical approach that requires a life-threatening operative intervention. Such fears are magnified by the nature of device surgery, which makes “treatment withdrawal” difficult, unlikely and inadvisable, by contrast with pharmaceutical trials. Technical considerations that prevent blinding of either investigator or patient to treatment selection remove an otherwise powerful antidote to investigator and subject bias. Such concerns have created considerable difficulty in recruiting patients for the first randomized LVAD clinical trial. Finally, for physician investigators, attaining and maintaining clinical equipoise throughout a randomized clinical trial between dramatically different treatment options may be inherently problematic.

A major conflict arises for clinician investigators who then perceive an obligation to provide device treatment, if in light of the new and extensive information provided as part of the consent process, the patient has concluded that the device therapy may be life-saving and is clearly in his or her interest for survival. The investigator must rightfully acknowledge that the dilemma of a patient’s requesting one arm of a randomized trial would be less likely to arise if comprehensive information had not been provided during recruitment efforts. Increasingly, however, patients arrive with a preconceived notion of their imminent mortality and a favorable impression of the device therapy that has been disseminated through the media prior to patient recruitment. Anecdotal reports indicate that this situation has occurred—and understandably so, considering the nature of the designated population, which suffers the chronic low cardiac output syndrome and faces death over days, weeks or months. The patient with far-advanced disease may perceive that a successful device implant, although not guaranteed, may provide some reasonable chance to survive with improved quality of life. Does the scientific community, as investigators, linger at equipoise longer than they would as these patients?

What is an appropriate response from the investigator to a potential study subject who requests the device therapy arm rather than randomization? One generic response might be that the presentation by the investigator may not have been appropriately balanced. Although this generic comment is highly relevant to most clinical trial protocols, certain patients and circumstances may make this outcome unavoidable for mechanical cardiac assist device trials. It may in fact not be possible to adequately transmit information from which patients could provide a truly informed consent to a complex trial with outcomes that are outside any of their known experiences. Should a patient be permitted to choose the device therapy arm?

Similar issues have been raised in drug development for AIDS.⁸³ Alternative trial designs to include patient preferences⁸⁸ have been proposed. Such trials might conceivably lessen conflicts with patient preferences and perhaps enhance recruitment, with greater generalizability of outcomes,⁸⁹ as described in the following text. It has been argued that most patients in clinical trials are likely to have preferences anyway, which may influence outcomes.⁹⁰ However, such trials may increase cost and compromise scientific integrity of the data.⁸⁸ Ethically, it does not appear mandatory that a patient be offered the perceived superior “treatment arm” preference as long as clinical equipoise is present.

4. Ethical issues after randomization. For patients who do proceed with trial participation to randomization, anecdotal reports of patients randomized to the control arm without device suggest some may be despondent and feel that they have been “sentenced to death.” Such responses give rise to two concerns. First, it is possible that a patient’s preference for the treatment not received may influence his or her own quality and length of life and bias the outcome of a device trial, which preferentially enrolls patients who prefer active treatment. That patient preferences may have an important impact on the outcomes of randomized clinical trials has been postulated, but little data exist in this area.⁹¹ Depression has been well-documented to lead to worse outcomes with chronic illness. Expert clinicians know well that a significant loss and the consequent despondency can precipitate decompensation in an otherwise stable HF patient; it is conceivable that such an emotional blow as to miss a randomization to a perceived life-saving device might be life-threatening in itself. If patient despair occurs in significant numbers, the resultant drop out or loss-to-follow-up and patient defection to receive investigational therapies elsewhere could prevent meaningful comparison of the treatment arms. Such experiences challenge the otherwise persuasive Freedman position of “clinical equipoise.” There may be both ethical and practical rationale for considering some controlled circumstances in which devices could be provided for “compassionate use” during the course of a trial (see “Design of Clinical Trials for Mechanical Circulatory Support” below).

5. Future ethical issues for equipoise. To date, the Freedman concept of clinical equipoise has been appropriate and attainable for an RCT for mechanical circulatory support, granted that reasonable and serious debate has existed about which treatment may be superior and comprehensive longitudinal clinical data have not been available in non-transplant patients. With the anticipated rapid progress of mechanical circulatory support development and additional clinical trials, considerable effort will be required to maintain clinical equipoise. Although clinical equipoise provides a powerful basis for assessing the ethical conduct of proposed controlled clinical trials, the mechanisms by which clinical equipoise moves ahead to reach a new ethical basis is poorly defined for specific issues, perhaps particularly so for rapidly evolving device innovations. Our current society receives broad but shallow information, with immediate reports of clinical trial results and patient testimonials on the front pages of national newspapers. Both professionals and the public are challenged to discern knowledge from information and to know what is right for now; that is, to decide the basis for clinical equipoise. As we assess new generations of mechanical support devices, how will our present ethical basis be challenged, and for what reason and by whom will our ethical basis be shifted? Will it be led by governmental agencies, industry, investors, clinical investigators and patients reading news reports, or by other groups? Perhaps an objective, expert multidisciplinary group would be helpful in identifying and resolving the ethical dimensions of clinical trials of assist devices.

E. Design of Clinical Trials for Mechanical Circulatory Support

Over time, a wide variety of methods (clinical trials, quasi-experimental techniques, decision analysis, economic analysis and meta-analysis) have evolved to assess outcomes of new therapies. Those that involve primary data collection can be differentiated by whether or not reliable techniques were used at the data acquisition stage to control for variables that can limit the identification of cause and effect relationships between the intervention and outcome of benefit or harm.

1. Randomized clinical trials. The prospective randomized clinical trial is the consummate clinical experiment designed to minimize ambiguity in the interpretation of study results by striving for equality between comparison groups at the time of their assembly. It is widely regarded as the most powerful and sensitive tool for comparing therapeutic interventions.⁸⁵ As discussed above, this experience has derived largely from trials of drugs for mild-to-moderate HF. Despite the theoretical strengths of the method, and its pivotal importance in trials of pharmaceutical agents in HF, there are daunting challenges in applying randomized clinical trials to the evaluation of potentially life-saving devices for end-stage heart failure. Many of these challenges arise from the differences between drugs and devices as detailed above, particularly with regard to the ethical issues arising from the inability to blind the patient or physician to the treatment arm. The unique nature of these challenges was discussed in detail in the preceding section. It should be emphasized, however, that knowledge of the treatment assignment has immediate practical implications also because the patient’s preferences for a device or for no device may compromise both enrollment in, and adherence to, the treatment assignment. In one of the original trials of therapy for AIDS, blood tests were positive for the investigational therapy in 9% of the patients in the placebo arm, indicating off-protocol drug acquisition.⁹²

Interpretation of outcomes is also influenced by knowledge of the treatment arm. Sham operations are very controversial^91,92 and would not be compatible with the palpable and audible function of current mechanical devices. Expectations by patients and physicians may influence the recognition of complications, the intensity of other therapies and perhaps even survival. Important study end points also include the subjective assessment of symptoms and quality of life. Even exercise performance, ostensibly more objective, is influenced by the expectations of patients and physicians.

Measuring survival in trials that compare devices to medical therapies presents methodological concerns different from those presented when comparing similar therapies. When device therapy involves a high up-front operative risk, with a subsequently reduced mortality compared with controls, the survival curves are likely to cross. Analyzing the differences between such curves depends on the analytical method chosen and the time frame of the analysis. Most analyses such as the log-rank and Wilcoxon methods average risk over the follow-up period. Extending or reducing the follow-up time then has the potential to reverse the order of relative efficacy, because more or less weight will be given to the respective mortality in the perioperative period. Moreover, crossing survival curves imply lack of a consistent proportional relationship in the relative mortality of the two treatments. This violates the basic assumption in using proportional hazard methods, which have been the standard for survival analysis procedures.

Special needs in cancer and AIDS research have affected a number of advances in clinical trial methodology by employing statistical methods that permit not only more rapid and sensitive evaluation of toxicity but also adjustments in design based on the interim outcome experience within a trial.⁸¹ Further successful community-based strategies, particularly in the testing of anti-AIDS interventions, have overcome problems with patient recruitment, treatment and development of appropriate informed consent. Understanding of the special challenges involved in evaluating mechanical support will be necessary in the development of novel trial designs that lower obstacles while preserving the advantages offered by the randomized clinical trial.

Financial impediments have affected the conduct of VAD clinical trials profoundly. The issue of funding is central because device companies are often innovative organizations with limited cash reserves and few sources of income. Shrinking budgets for academic centers limit their resources in the face of the increased time required to prepare documents for institutional review boards, screen patients and provide detailed data for studies with limited enrollment. Moreover, the unreimbursed costs of the surgical procedure and recovery are substantial. Cutbacks in health care reimbursement prevent hospitals from continuing to support such visible programs internally as “loss leaders.” These disincentives to patient enrollment ultimately increase the overall duration and cost of the study.

The decision by the executive branch of the federal government to begin reimbursing the routine treatment costs of Medicare patients enrolled in clinical trials is an important step in the right direction. Beyond payment for routine costs, the concept of conditional coverage is increasingly advocated, in which insurers (such as Health Care Financing Administration [HCFA]) support the costs of patient treatment associated with both arms of a well-designed clinical trial, while the sponsors (e.g., National Institutes of Health or Industry) cover the costs of conducting the research. There is strong support from this conference for such conditional coverage.

2. The REMATCH trial. Despite the above limitations, an RCT to determine the impact of a mechanical circulatory support device on outcomes with end-stage heart failure is nearing completion. The ongoing REMATCH trial compares the ThermoCardio System implantable LVAD as “destination therapy” with optimal medical therapy in patients who are not candidates for transplantation,¹³ using the criteria defined above. Initiation and enrollment into this study have been delayed for both centers and patients by many of the issues described. Sufficient patients have been randomized, however, to reach meaningful conclusions. If a survival benefit is proven for this device in this population, future control groups for destination therapy may be receiving this device or receiving continued medical therapy if they have established contraindications to its placement. Even if no statistically significant benefit is demonstrated in the mechanical device-supported patients, the information obtained from both standard therapy and the assist device arms will influence device testing and population selection for future clinical device trials.

3. Modifications of the randomized controlled trial for mechanical circulatory support devices.
a. Option of later “compassionate” use of device. It should be re-emphasized that the gold standard methodology for deriving firm information regarding the impact of the treatment on outcomes remains the randomized, double-blinded, placebo-controlled trial, with hard, well-defined primary end points of major clinical importance.²³ It should also be recognized, however, that surgical interventions in patients with advanced illness may not appropriately lend themselves to all aspects of this methodologic gold standard, such as blinding to treatment. In designing trials for such interventions, one should begin by seeking to implement the ideal design and to deviate from the ideal only as is practically necessary. It is essential to take into account the impact of trial design modifications on the resulting data before drawing conclusions regarding the treatment effect.

Future design of a trial in which a circulatory device is compared with medical therapy might include a later offer of “compassionate cross-over” for interested patients. This would technically not be a “cross-over” trial because patients with HF would not routinely have the option to cross back from device to medical therapy and the patients receiving a device after randomization to the control arm would not be analyzed with the original device cohort. Provision of the device could be offered after a predetermined time period during which early survival and intermediate-term functional data would be obtained. Alternatively or additionally, the demonstration of certain pre-established criteria of disease progression could be considered as a surrogate end point, after which the device would be offered compassionately, recognizing that the operative risk might be higher at this time than at the time of randomization. The option of receiving a device in the future would offer hope to patients disappointed by initial assignment to no device. In addition to reducing some of the ethical concerns, this provision might actually render a more valid comparison of the two arms, by realigning the incentives for both physicians and patients to persevere through the control period without the device. It would hopefully decrease the risk of losing patients to follow-up as they seek this therapy in a less supervised setting elsewhere. For many of the reasons discussed above, these increased options would be expected to enhance enrollment and adherence to follow-up. This potential increase in enrollment needs to be balanced with the increase in sample size required to determine clinically significant differences.

b. Potential influence of initial patient preference. The ability of a patient to select a particular modality of therapy in a clinical trial may not only significantly enhance enrollment but also potentially influence the outcomes after treatment.^88–90 This argues for examining the preferences of patients as a factor that might influence the end point of the trial. One way of accomplishing this is to measure patient preferences for treatment assignment immediately before randomization and, if they are related to the primary end point, to use the results to adjust the primary comparison. A partially randomized design would give patients the option to either become part of a traditional randomized trial or take the therapy of their choosing. In a trial of two interventions, this results in four arms. The comparison of the two randomized arms offers the information of a standard RCT. Absolute confirmation regarding device outcome and complications is available for the patients choosing the device therapy, although there is no parallel control group. Comparisons between the randomized and nonrandomized arms, which must be treated as observational study, would give some indication of the effect of patient preferences on outcome.

4. Comparison of non-randomized cohorts. Alternative designs may be considered when the RCT is not considered appropriate, such as for established devices that incorporate limited improvements. It is also conceivable that cohort studies may be found acceptable when initial evidence of efficacy has persuaded the clinical community away from equipoise but has not yet led to formal device approval (Fig. 1). Cohort studies have employed both historical and prospective controls. With RCTs at the top of the hierarchy of research design, there are various levels of descending rigor for observational reports, all of which are susceptible to considerable bias. Controlling for selection bias can be improved by: 1) restriction of inclusion criteria to define relatively homogeneous cohorts with some loss of generalizability; 2) matching, such that each patient in one cohort is paired with one or more patients with a similar baseline profile for a limited number of key prognostic factors, which need to be better defined for advanced HF; 3) stratifying—comparing rates within subgroups with clinical characteristics that put them at the same risk of the outcome event, which can be done only for a few characteristics before statistical power is lost; and/or 4) adjusting for difference in clinical characteristics between the cohorts, using regression techniques. Unfortunately, none of these can control completely for the factors that led to the provision of a therapy to one patient and not to another, if the therapy was potentially available for both. An interesting example is the comparison of patients who received implantable LVADs as bridges to cardiac transplantation and those in the same centers who did not, for reasons attributed to device availability. This indicated a major benefit from devices used as bridges to transplantation, for which they were subsequently approved. However, generalization of the results to non-transplant candidates predicted a substantial benefit that was not borne out in the randomized pilot trial.⁵² Meta-analyses of observational trials have in some cases predicted the results of well-designed randomized trials ^93,94 but in other cases have been contradicted and supplanted by such trials.95 It has been suggested that “when recruitment of patients for an RCT is exceptionally difficult, threatening to make the sample of patients unrepresentative, neither reliance on RCTs nor reliance on observational studies is wholly satisfactory”.^95,96

a. Historical controls. There is a paucity of large “clinically rich” datasets in patients with class III and class IV heart failure. There is also little data on the components of medical therapy for truly class IV CHF patients. The Flolan International Randomized Survival Trial (FIRST),⁹⁷ examining the use of the vasodilator epoprostenol, and the recent Outcomes of a Prospective Trial of Intravenous Milrinone for Exacerbations of Chronic Heart Failure (OPTIME CHF) trial,⁹⁸ examining the use of milrinone during HF hospitalization, demonstrated high mortality regardless of medical treatment. The Pre-Transplant Research Database⁵¹ demonstrated high mortality in patients hospitalized or on intravenous inotropic agents, with mortality of only 13% for other patients awaiting transplantation (a younger population with fewer co-morbidities than patients currently considered for implantable devices). When concluded, the REMATCH trial will provide unique information on approximately 70 such patients receiving optimal medical therapy, and for a brief period, it will represent the most current data available. Historical controls provide useful information that requires interpretation in the context of the original reasons for data collection. Medical therapy is in a dynamic state, so reference to databases previously obtained may provide general guidance but is unlikely to sufficiently validate a new therapy unless it is in the breakthrough realm.

b. Prospective controls. Some of the problems of historical controls can be addressed by assembling the control cohort prospectively, along with the “experimental” cohort group. Once patients have qualified for participation in the study, their assignment to a particular cohort will depend on the goals of the study. Patient assignment, however, must be made in light of the need to establish cohorts that are equally constituted with respect to the risk for the primary measure of outcome. Despite the use of restriction, matching and stratification, cohorts are rarely evenly matched, and comparisons between the cohorts require analytical adjustment to account for differences in baseline patient characteristics.

i. Timed graduation from control cohort to active therapy. One approach is to enroll patients formally for a fixed time period before the device is implanted. This provides a brief period during which early mortality for the population can be determined. There is reason to suspect, however, that the patients dying during this interval were at initially higher risk than those surviving the observation interval preceding implantation. Alternatively, the period of delay may lead to clinical deterioration that increases the operative risk to a higher level than it was at the time of enrollment. Several factors thus render the initial cohort different from the group later undergoing device implantation.

ii. Patient preference cohort studies. A patient preference study (a prospective cohort study allowing patients to choose which therapy they want) may be of considerable appeal to patients (see preceding text). Those patients selecting their preferred treatment rather than randomization would constitute the preference cohorts. Depending on the planned comparisons, patients might also be given the option to cross over to the newer therapy after specific early end points if their opinions change and the change is technically feasible. This type of trial might greatly enhance recruitment because eligible patients with end-stage HF who fear a device may be more willing to allow themselves to be followed in the medical treatment arm. Such patients are currently not likely to be enrolled in any device trials. Similarly, many patients who would be reluctant to enroll in a trial because they might have only a 50% chance of being assigned to a device would now enroll. The fundamental drawback to this design is the possibility that self-selection of a particular therapy is, in some way, associated with the primary measure of outcome, making the groups unequal at baseline. This has not been determined.

iii. Risk-based allocation cohort studies. One approach that is being investigated for breast cancer therapy is to allocate therapy in clinical trials based on risk assessment, such that those patients deemed at greater risk of dying from the underlying disease would receive the experimental therapy and those at less risk would receive standard therapy.⁹⁹ The treatment effect is measured by comparing the observed results of the experimental group with a projection of the effect of standard treatment on the experimental group, based on a mathematical model. The model would be derived from observations made on the control group. Although this type of trial design is only now being examined, it may provide a novel method for studying the use of VADs in patients with complex heart failure. For investigating therapies of advanced HF, this trial design would be hindered by the limitations of our ability to identify risk profiles and predict outcomes in advanced HF.

F. The Vital Importance of Registries

1. Outcomes database for advanced heart failure. The growing national burden of advanced heart failure argues for the establishment of an ongoing registry at a number of institutions that would include information regarding therapies and outcomes. The large heart failure databases that have generated new mechanistic hypotheses have been of mild-to-moderate heart failure rather than the more severe heart failure responsible for most of the morbidity and mortality associated with this diagnosis. The complexity of this condition, with multiple etiologies, co-morbidities, therapies and modes of death, poses greater challenges to risk profiling and modelling than those encountered with specific cancers or AIDS. Despite the prevalence of advanced HF, however, there have been no national resources devoted to collaborative efforts to assemble such data.

There are several scientific and societal reasons for a greater commitment to this population. A registry of advanced heart failure would accelerate progress in developing mechanical circulatory support and other new therapies. Greater confidence in our ability to identify high-risk populations would accelerate the recognition of devices in the breakthrough realm. Indications for specific populations could be more readily defined. By virtue of its larger size, a registry offers a better opportunity for matching characteristics of an experimental group with a cohort of controls selected from the dataset. Moreover, a registry would support the development of a regression model that can be used to adjust for differences in assembled cohorts, multivariate regression modeling being the major technique employed for diminishing bias in cohort comparisons. The design of RCTs would be streamlined by better selection of target populations and prediction of event rates.

2. Registries for implantable devices. There is now broad consensus that there should be a mandatory registry for all implantable mechanical circulatory support devices. The impact and implications of device approval and acceptance are much greater than for those of any pharmacologic component of the medical regimen. The number of devices and patients that form the basis of approval is of necessity relatively small, and extensive further experience is required to optimize the clinical utility of new devices. The current consensus is that further development of implanted circulatory devices without plans for such a registry is unethical.

The same factors of technical complexity—cost outlays for the device and consoles, requirements for site expertise and the transparent impact of devices—that hinder large randomized trials prior to device approval may in fact facilitate ongoing surveillance after device release. In recent years, there has been increased attention to the potential of post-marketing studies to accelerate the process of approval. By contrast with pharmaceutical therapies, which are easier to study before approval and harder to supervise afterward, mechanical circulatory support devices may be supported by a weight of evidence distributed differently between pre- and post-approval experiences.

Past experience with all manufacturers has, however, demonstrated the numerous limitations of a voluntary registry, including a lack of uniform criteria for device insertion, variable surgical experience, incomplete data submission at all time points, cost issues and proprietary/marketing issues. There is nonetheless strong precedence for maintaining registries for implanted valves and pacing devices. Device manufacturers as well as health care providers must report information indicating that a device may have caused or contributed to a death or serious injury. In the case of high-risk devices, companies must keep records of patients with implanted devices. It should be possible to require specific baseline data collection on patients with mechanical assist devices after device approval if that stipulation is formally linked to the initial approval of the device.

In addition to patient survival data, regulatory agencies are likely to require post-approval clinical studies to expand on specific components of the safety profile for devices, such as infections or thromboembolic events and documented device failures and replacement. It is not known to what extent a mandatory registry can require specific detailed data, but a registry would provide a useful common denominator as a template. While post-marketing studies have generally used observational methods, the concomitant development of improved registries both for devices and advanced HF should allow more sophisticated modeling to determine relative outcomes of devices versus medical strategies. If there are numerous post-marketing studies that address the same issue, meta-analyses can be used to statistically combine the results of these individual studies to a degree justified by the similarity of devices. This form of analysis can help to resolve uncertainty when studies disagree as well as to answer questions that were not posed at the start of the individual studies. Moreover, it can improve estimates of the magnitude of therapeutic benefits and risks. Compared with trials of drugs and drug classes, meta-analysis has perhaps been underutilized for the analysis of the effects of mechanical assist devices.

It is unclear how the responsibility of supporting such registries should be allocated between industry and governmental agencies. The greater challenge is presented by the larger and more diffuse population with advanced HF, for whom there is no industry incentive to support systematic recording of outcomes. There are currently a number of proposals in the process of submission to direct and maintain a registry of implantable devices.