full-text ►
Guidance Documents (Medical Devices and Radiation-Emitting Products) > Draft Guidance for Industry and Food and Drug Administration Staff - Factors to Consider when Making Benefit-Risk Determinations in Medical Device Premarket Review: - Enviado mediante la barra Google

Draft Guidance for Industry and Food and Drug Administration Staff - Factors to Consider when Making Benefit-Risk Determinations in Medical Device Premarket Review

¹This guidance document is being distributed for comment purposes only.
Document issued on: August 15, 2011

You should submit comments and suggestions regarding this draft document within 90 days of publication in the Federal Register of the notice announcing the availability of the draft guidance. Submit written comments to the Division of Dockets Management (HFA-305), Food and Drug Administration, 5630 Fishers Lane, rm. 1061, Rockville, MD 20852. Submit electronic comments to http://www.regulations.gov². Identify all comments with the docket number listed in the notice of availability that publishes in the Federal Register.
For questions regarding this document, contact Rachel Turow at 301-796-5094 or by electronic mail at Rachel.Turow@fda.hhs.gov. For questions regarding this document concerning devices regulated by CBER, contact the Office of Communication, Outreach and Development (OCOD) by calling 1-800-835-4709 or 301-827-1800.

Department of Health and Human Services
Food and Drug Administration
Center for Devices and Radiological Health
Center for Biologics Evaluation and Research

Preface

Additional Copies

Additional copies are available from the Internet. You may also send an e-mail request to dsmica@fda.hhs.gov to receive an electronic copy of the guidance or send a fax request to 301-827-8149 to receive a hard copy. Please use the document number 1772 to identify the guidance you are requesting.

Introduction
Scope
Background
- 3.1 The Statutory Standard for Safety and Effectiveness
- 3.2 Types of Scientific Evidence
- 3.3 Benefit-Risk Determinations
Factors FDA Considers in Making Benefit-Risk Determinations
- 4.1 Measures for Effectiveness of Devices
- 4.2 Measures for Safety of Devices
- 4.3 Additional Factors for Weighing Benefits and Risks of Devices
Examples of Benefit-Risk Determinations
- 5.1 Hypothetical Examples
- 5.2 Examples Based on Actual FDA Benefit-Risk Determinations

Appendix A - Worksheet for Benefit-Risk Determinations

Draft Guidance for Industry and Food and Drug Administration Staff

Factors to Consider when Making Benefit-Risk Determinations in Medical Device Premarket Review

This draft guidance, when finalized, will represent the Food and Drug Administration's (FDA's) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance.

1. Introduction

FDA has developed this draft guidance document to provide greater clarity for FDA reviewers and industry regarding the factors FDA considers when making benefit-risk determinations during the premarket review process for certain medical devices. FDA believes that the uniform application of the factors listed in this guidance document will improve the predictability, consistency, and transparency of the premarket review process.
FDA's guidance documents, including this one, do not establish legally enforceable responsibilities. Instead, guidance documents describe the Agency's current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidance documents means that something is suggested or recommended, but not required.

2. Scope

This guidance document explains the factors that FDA considers when making benefit-risk determinations in the premarket review of certain medical devices. The processes discussed in this guidance are applicable to devices subject to premarket approval (PMA) applications and, in limited cases, devices subject to premarket notification (510(k)) requirements. This guidance applies to both diagnostic devices and therapeutic devices. Although guidance is not binding, the concepts and factors described herein generally capture how benefit-risk determinations are made by FDA during the premarket review process.

3. Background

3.1 The Statutory Standard for Safety and Effectiveness

Under §513(a) of the Federal Food, Drug & Cosmetic Act (“the FD&C Act”), FDA determines whether PMA applications provide a “reasonable assurance of safety and effectiveness” by “weighing any probable benefit to health from the use of the device against any probable risk of injury or illness from such use”. To aid in this process, PMA applicants submit valid scientific evidence, including one or more clinical investigations where appropriate, which FDA reviews to determine whether “the device will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the labeling of the device.”¹ FDA staff review the data submitted as part of the PMA and determine – based on a number of factors – if the data support the claims made by the sponsor for the device, i.e., intended use and/or indications for use, and data analysis demonstrates that the benefits of the device outweigh its risks. This process may also occur on a case-by-case basis in the review of 510(k) devices when there are differences between the target device and the predicate device that can adversely affect the safety and/or effectiveness of the target device.²

3.2 Types of Scientific Evidence

Medical devices can be evaluated using clinical and non-clinical testing methods. Clinical testing methods for medical devices can include, when appropriate, randomized clinical trials in the appropriate target population, partially controlled studies, studies without matched controls, well-documented case histories conducted by qualified experts, reports of significant human experience, and testing on clinically derived human specimens. Non-clinical testing methods can encompass an array of methods including bench testing for product safety/reliability, animal and cell-based studies, and computer simulations. These tests characterize mechanical and electrical properties of the devices that include wear, tensile strength, compression, flow rate, burst pressure, biocompatibility, toxicity, electromagnetic compatibility (EMC), sterility, stability/shelf life data, software validation, and testing of synthetic samples, including cell lines. For in vitro diagnostics, analytical validation, or assurance that the technical measurement aspect of the product is accurate and reliable, is also considered as part of a demonstration of reasonable assurance of safety and effectiveness. The data derived from any clinical and/or non-clinical testing are taken into account during the premarket review process and FDA’s benefit-risk determination.
Although a great deal of emphasis is placed on the importance of clinical data in demonstrating the safety and effectiveness of a medical device, non-clinical data also can be critical to understanding device safety and effectiveness. Medical devices often have attributes that cannot be tested using clinical methods alone and that play a major role in the safety or effectiveness of the device. For example:

For certain types of invasive diagnostic catheters, the only way to identify and diagnose disease in the specimens they gather is through examining the tissue under a microscope. In some cases, clinical studies may provide supportive evidence with respect to the safety and effectiveness of these devices, i.e., evidence that the device can be inserted safely into the patient and perform as expected. However, it may be sufficient to use non-clinical methods to determine some of the characteristics of the device, without conducting more expensive, difficult, and perhaps invasive clinical trials. As a result, non-clinical testing can be used to evaluate the diagnostic effectiveness (accuracy, precision, etc.) of the device. This testing can be performed using histopathology methods on non-clinical samples where the device is tested on tissue samples from animals or cadavers that are then evaluated under a microscope to determine the properties of that tissue. For these devices, even though clinical testing was performed, the results of the non-clinical histopathology testing may also influence FDA's benefit-risk determination.
In the case of some implants, the most robust long-term evidence comes from engineering tests that are able to challenge the device under worst-case conditions, test the device to failure and simulate many years of use. In contrast, clinical studies are usually limited in duration of follow-up, and, as a result, may be less informative with respect to the long-term performance of the device. In this case, the results of engineering testing may significantly influence FDA's benefit-risk determination independent of the clinical findings.

Both clinical and non-clinical data can play a role in the FDA’s benefit-risk determinations, and the factors discussed in this guidance are informed by both types of data.

3.3 Benefit-Risk Determinations

There are many factors that go into weighing the probable benefit of a device versus its probable risk. These factors include, among others, whether the device is a first-of-a-kind treatment or diagnostic, whether the device provides significant improvement in diagnosis and patient management of a serious disease, how known risks of the device can be mitigated, reliability of the study, whether there are multiple studies and the strength of those studies, what amount of risk the target population will tolerate in light of the condition being treated or diagnosed and the probable benefit of the device, and whether there are alternate treatments or diagnostic techniques available.
The factors FDA considers as part of the benefit-risk determination are explained in detail below. We also give examples of how the factors interrelate and how they may end up affecting FDA’s decisions. By providing greater clarity about FDA’s decision-making process, we hope to improve the predictability, consistency, and transparency of the review process for applicable devices.
We have also included for public comment a proposed draft of a worksheet that reviewers would use in making benefit-risk determinations. The worksheet is attached as Appendix A to this guidance. We believe that the level of documentation proposed by the worksheet would be very helpful to maintaining the consistency of review across different review divisions and better assuring that an appropriate decision is reached. In addition, by documenting reviewers’ thought processes as part of the review file and, in certain cases, the publicly available summary of our decision, sponsors will have a better idea of the basis for FDA’s decisions and gain a greater understanding of what factors were considered in reaching an approval or clearance decision. After receiving feedback from stakeholders and FDA staff, we may integrate use of this or a similar worksheet as part of the premarket review process. However, because the weighting of the factors for a type of device may change over time – such as a device no longer being a first-of-a-kind or the only available treatment as new therapies are approved – the benefit-risk determination for a specific device at one point in time may no longer represent the proper weighting of the factors for a similar type of device in the future.

4. Factors FDA Considers in Making Benefit-Risk Determinations

The factors described below are considered within the intended use of the device, including the target population.

4.1 Measures for Effectiveness of Devices³

Extent of the probable benefit(s) : The extent of the probable benefit(s) is measured by taking into account the following factors:

The type of benefit(s) – examples include but are not limited to the device’s impact on clinical management of the patient, the patient’s physical health and patient satisfaction in the target population, and can range from significantly improving patient management or reducing the probability of death, to aiding in some improvement of management or reducing the probability of loss of function, to providing relief from minor symptoms. For diagnostics, benefits may be measured according to the public health impact of identifying and preventing the spread of disease. Other benefits of diagnostics include earlier diagnosis of disease and identification of patients more likely to respond to a given therapy.
The magnitude of the benefit(s) in the individual patient – the magnitude measures the size of the benefit. We often measure benefit along a scale or according to specific endpoints or criteria (types of benefits). The change in the patient’s condition or their clinical management as measured on that scale, or as determined by an improvement or worsening of the endpoint, is what allows us to determine the magnitude of the benefit for an individual patient.
The probability of the patient experiencing a benefit – based on the data provided, it is sometimes possible to predict which patients may experience a benefit but sometimes this cannot be well predicted. A benefit may only be experienced by a small portion of patients in the target population, or a benefit may occur frequently in patients throughout the target population. It is also possible that different patient subgroups will experience different benefits or different levels of the same benefit.
The duration of effect(s) (i.e. how long the benefit lasts for the patient) – some treatments are curative, whereas, some may need to be repeated frequently over the patient’s lifetime. Treatments that are curative may be considered to have greater benefit than treatments that must be repeated because repetition may introduce greater risk or the benefit experienced may diminish each time the treatment is repeated.

4.2 Measures for Safety of Devices⁴

Extent of the probable risk(s)/harm(s): The extent of the probable risk(s)/harm(s) is measured by taking into account the following factors:

Number, severity, and types of harmful events associated with the use of the device:

Device-related serious adverse events – an injury or illness that is life-threatening, results in permanent impairment or damage to the body, or requires medical or surgical intervention to prevent permanent harm to the body.⁵
Device-related non-serious adverse events – those events that result directly from use of the device and that do not meet the criteria for classification as a serious adverse event.
Procedure-related or indirect harms – harm to the patient that would not be considered a serious or non-serious adverse event, and that indirectly results from use of the device.For example,breast implants have serious (e.g., capsular contraction) and non-serious (e.g., scar) adverse events associated with them. If a patient chooses to have her physician remove her breast implants, she may experience negative aesthetic consequences, such as deformation of her body, which would be an indirect harm from having had breast implants. This event is not necessarily considered an adverse event associated with breast implants, but is nonetheless an indirect harm resulting from the treatment. Similarly, risks associated with the collection of human biological materials would factor into the benefit-risk assessment.

Probability of a harmful event – the percent of the intended population that would expect to experience a harmful event.
Duration of harmful events (i.e. how long the adverse consequences last) – some devices can cause temporary, minor harm; some devices can cause repeated but reversible harm; and other devices can cause permanent, debilitating injury.
Risk from false-positive or false-negative for diagnostics – if a diagnostic device gives a false-positive result, the patient may receive an unnecessary treatment and incur all the risks that accompany that treatment, or may be incorrectly diagnosed with a serious disease. If a diagnostic device gives a false-negative, the patient may not receive an effective treatment and will miss out on the benefits it would confer, or may not be diagnosed with the correct disease or condition.

We also consider the number of different types of harmful events that can potentially result from using the device and the severity of their aggregated effect. When multiple harmful events occur at once, they have a greater aggregated effect. For example, there may be a harmful event that is considered minor when it occurs on its own, but, when it occurs along with other harmful events, the aggregated effect on the patient can be substantial.

4.3 Additional Factors for Weighing Probable Benefits and Risks of Devices

Uncertainty - there is never 100% certainty when determining reasonable assurance of safety and effectiveness of a device, nor need there be to show that the probable benefits outweigh the probable risks. However, the degree of certainty of the benefits and risks of a device is a factor we consider when making benefit-risk determinations. For example, when the probable benefits are large, less certainty regarding the probable risks may be acceptable to support the approval of the device. On the other hand, factors such as poor trial design, conduct, or analysis can make the data produced by the study unreliable. Furthermore, the repeatability of the study results, the results of other similar studies and whether the study is the first of its kind or a standalone investigation can all influence the level of certainty. In addition, the generalizability of the trial results to the intended treatment and user population is important. For example, if the device requires in-depth user training or specialization, the results of the clinical study may not be generalized to a wider physician population. Likewise, if the device is intended to diagnose a disease in a subpopulation, it may not be useful in the general population.
Characterization of the disease – the treated or diagnosed condition, its clinical manifestation, how it affects the patients who have it, how and whether a diagnosed condition is treated, and the condition’s natural history and progression (i.e., does it get progressively better or worse for the patient) are all important factors to consider when characterizing disease.
Patient tolerance for risk - if the risks are identifiable and definable, different patients will have a different risk tolerance and will make a decision as to whether the risks are acceptable. For diagnostic devices, the availability and nature of the treatment for the condition diagnosed may affect patients’ tolerance for risk. Different factors can influence patient risk tolerance, including:

Disease severity – patients suffering from very severe diseases (i.e. those that are life-threatening) may tolerate more risk for devices used in treatment. For diagnostic devices, individuals with severe diseases may be more adverse to the risk of a false negative.
Disease chronicity – some patients with chronic diseases who have adapted to their illness and minimized its interference with their daily lives may tolerate less risk and require risky devices to deliver a greater treatment benefit; whereas, patients who have suffered from a debilitating chronic illness over a long period of time may tolerate higher risk to gain less benefit.
Availability of alternative treatment/diagnostic options (also see below) – if there are no other treatment/diagnostic options available, patients may tolerate more risk for even a small amount of benefit.

Availability of alternative treatments or diagnostics – when making benefit-risk determinations, FDA considers whether other treatments or diagnostics, including non-device products, have been approved or cleared for the intended condition and patient population, how effective are they, and what known risks they pose; what off-label uses of approved or cleared devices, other products or procedures are available, if there are no devices or other products approved or cleared for the intended condition and patient population, and their benefit-risk profiles; and how well the alternatives address the needs of patients and providers. For example, if a new device has a very small benefit and there is significant uncertainty about that benefit, we may still approve the product if there are no available alternative treatments or diagnostics and the risk profile is acceptable.
Risk mitigation – the use of mitigations, when appropriate, can minimize the likelihood of a harmful event occurring. The most common form of risk mitigation is to include warnings in labeling, or to restrict the indication to a more limited use. Another important consideration in risk mitigation is the type of intervention required to address the harm. Some harms may require surgery; whereas, others may require only minimal intervention. Thus, even if there is a high likelihood of harm and the harm resulting from the risk is severe, it may be easily mitigated with a non-invasive treatment. Finally, some harms can be mitigated through changing device design features.
Novelty of technology – devices representing or incorporating new technologies, especially those that are first-of-a-kind, may provide a less than optimal benefit, but may also offer advantages that did not previously exist. With subsequent iterations of the device its benefit-risk profile may improve, the expected level of safety and effectiveness may increase, and later versions may offer significant advantages over the initial device. In these circumstances, we may approve a device with less benefit or more risk than would be generally tolerated for more established technologies, particularly where providers and patients have limited alternatives available, to facilitate patient access and encourage innovation.

5. Examples of Benefit-Risk Determinations

The examples below are hypothetical or simplified and are only offered for illustrative purposes. The decisions described in these examples are not predictive of future FDA decisions, rather they are hypothetical outcomes and are only intended to demonstrate how FDA considers the factors described in this guidance when making benefit-risk determinations. Similar scenarios or devices may result in different approval outcomes depending on the individual characteristics of a particular device.

5.1 Hypothetical Examples

Example 1

An implantable device is developed to treat a severe condition. There are alternative treatments available for the condition, but they are only effective for certain subgroups of patients and all of them pose significant risks.
The device is studied in a pivotal clinical trial with a design where all patients are implanted with the device but the device is only turned on in half of them. After completion of the trial, inactive devices can be turned on. The primary endpoint for the trial is the magnitude of the benefit, i.e., the trial is designed to measure how well the device reduces the patient’s symptoms.
The results of the pivotal clinical trial revealed the following:
Effectiveness: The probability that a patient would experience a substantial benefit was only approximately 15%. However, for the small portion of patients who did respond, their symptoms were significantly reduced and their quality of life was significantly improved due to improved mobility. As a general matter patients with this disease who are able to maintain good mobility longer tend to have a longer life expectancy. However, the duration of the benefit cannot be determined because the patients were only followed for one year.
Safety: There is a very low occurrence of harmful events after device implantation. However, all implanted devices that require a surgical procedure carry with them their own set of risks. In addition, permanent implants pose additional risks, namely, they typically remain with the patient for life and may be difficult or impossible to remove. Even in cases where the device is deactivated, it remains implanted and a risk of device fracture, mechanical failure, or an adverse biological response to the device remains. Finally, the surgery to implant the device is not routine and carries with it high risks.
Additional Factors
Uncertainty : The results of the pivotal clinical trial are unreliable because both treatment and control groups suffered significant patient lost-to-follow-up (the clinical status of many patients in the study was not known at its conclusion); therefore, it is difficult to tell if patient symptoms improved because of the device action, the placebo effect, or the effect of the surgery itself.
Patient Tolerance for Risk : Patients who are willing to take the risk of having the device implanted even for a small probability of benefit because they have no other treatment options and their symptoms are severe.
Risk Mitigation : The surgery to implant the device is risky, but the risks can be mitigated by requiring the device be implanted by a skilled surgeon.
Approval/Non-Approval Considerations: The device effectiveness is limited because the probability that a patient will experience a benefit is low (15%). In this case, FDA does not have the option to limit the use of the device to only those patients who are most likely to experience a benefit because there is no way to determine the subset of patients who would definitely experience the benefit. Moreover, this type of permanent implantable device poses significant risks and there is moderate uncertainty associated with the trial results. However, for those patients who do experience a benefit, symptom relief and improvement in quality of life is impressive and the risks, although substantial, could be somewhat mitigated through limiting the device use to specialized surgeons. Finally, the device treats a severe and chronic disease for which there are few, if any, alternative treatments. Therefore, FDA is likely to approve the device.

Example 2

A revolutionary device that replaces a patient’s memory is developed to treat Alzheimer’s, dementia, and other memory disorders. The device is permanently implantable and the patient must undergo a brain resection in order for the device to work properly. The device functions by downloading all of a patient’s memories onto a computer chip. Once the device is implanted, any residual memory the patient retained is no longer accessible to the patient
Effectiveness: A clinical trial of the device showed significant improvement in patients who were in the early stages of dementia and minimal improvement in patients who were in more advanced stages. Patients who received implanted devices when the majority of their memory was intact experienced the greatest benefit.
Safety: The surgery to implant the device is highly risky and must be done at certified centers of excellence by specially trained neurosurgeons. Even with these procedural restrictions, there is an 8% risk of mortality from the surgery alone. In addition, adverse events include partial paralysis, loss of vision, loss of motor skills, and slurred speech. Non-serious adverse events include personality shifts, mood swings, vertigo and insomnia.
Additional Factors
Uncertainty : The number of patients eligible and willing to enroll in the trial was small, but the data were robust and the trial was well-designed and conducted. The results of the trial are generalizable across all patients because different patients experienced different levels of benefit depending upon their stage of memory loss.
Patient Tolerance for Risk : Because of the disruptive effect on patients’ quality of life from diseases like Alzheimer’s, dementia, and other conditions that are associated with severe memory loss, patients (and their families) suffering from these diseases often have a very high tolerance for risk. In this case, it is likely that family members responsible for patients who are at more advanced stages of their illness and therefore experiencing more severe symptoms will have a higher risk tolerance than those at the beginning stages of disease.
Availability of Alternative Treatments or Diagnostics : There are currently no alternative treatments available.
Risk Mitigation : The risks associated with this device are great. The risks associated with implantation can be somewhat mitigated by limiting implantation to a highly select group of surgeons, but the risks associated with personality changes cannot be mitigated or predicted.
Novelty of Technology : This technology is a game-changer. There is no other similar technology like it in the world. It is likely that future improvements of the device may allow treatment of many other conditions that affect cognitive function. Also, there are no other treatments that can give patients the level of benefit that this device confers.
The device has a confirmed, substantial benefit for a defined and predictable subset of patients and a moderate benefit for another defined and predictable subset. Even though the clinical trial was small, the quality of the data is good and there is little uncertainty about the results. The risks associated with the device are great and can be mitigated by advising that only highly trained physicians can implant the device. And, because patients experience the greatest benefit when the device is implanted earlier, they must expose themselves to the risks for a longer period of time in order to reap the greatest benefit; therefore, the patients who stand to benefit most also have a higher risk tolerance. Many patients who suffer from memory disorders are willing to try anything to preserve their memories and quality of life. The fact that there are no alternative treatments for this condition and a known benefit with this treatment is also compelling. Furthermore, the risks are known and quantifiable. Therefore, this device, although risky, is approvable. The decision as to whether or not to implant the device is a matter of patient/family member preference and medical opinion. FDA is likely to approve the device knowing that patients (or responsible family members) can give informed consent to the risks the device poses and only a very small group of skilled physicians will be able to implant the device.

Example 3

A new in vitro diagnostic device (IVD) claims to provide early detection of any type of cancer in the general healthy population by testing patients’ blood. The device is studied in a diverse population of patients with cancer, and correctly identifies patients that have cancer 95% of the time.
Effectiveness: The IVD detects cancer well in patients who are already known to have cancer, but who likely have different characteristics than the general population in which early detection might be beneficial, because they may have more advanced cancer, they may have already received treatment, and they may have widely different demographic features (e.g., cancer patients are on average older than the general healthy population in which early detection would have greatest benefit). The IVD is effective, but not in the population that would most benefit from its use.
Safety: The device gave mostly correct results in patients who were known to have cancer, but was not tested in the intended use population, i.e. the general healthy population. If the device were to deliver a substantial proportion of false results in the intended use population, normal healthy people might be exposed to further invasive procedures or radiation, resulting in significant harm. Or, people might actually have cancer and the test would not detect it, thus preventing them from receiving beneficial treatment, or discouraging their physicians from monitoring them for cancer.
Additional factors:
Uncertain Diagnosis : Although the test could detect cancers, it could not report what type of cancer; therefore, much additional diagnostic work-up, some of it invasive, would be required for patients that receive a positive test result.
Patient Tolerance for Risk : Although most seemingly healthy people would probably like to know if they had early stages of certain cancers, a large proportion of false results would not be tolerated because patients’ health at the stage of testing is good. For very aggressive cancers for which the patient was known to be at risk due to exposure or family history, more risk might be tolerable.
Uncertainty about Cancer Natural History : It is believed that some cancers resolve without treatment, through various immune and other natural mechanisms. It is not possible to know which early cancers will progress and which will go away on their own. Aggressive treatment of early cancers may result in more morbidity than not treating.
This IVD device had the general benefit of detecting cancers anywhere in the body, but the study was not designed to address the intended use population of generally healthy people. The differences in test performance in patients who do not have obvious cancer (the intended use population) are unknown. The information yielded by the test is not specific enough to allow a definitive diagnosis without considerable further work-up. Given the risk of serious adverse events resulting from attempts to find the cancer, as well as the likely high risk of false results (overtreatment or failure to treat), and the uncertain benefit of early detection for certain cancers, FDA is not likely to approve the device because it was not demonstrated to be effective in the intended population and it has serious side effects without adequate available mitigations.

5.2 Examples Based on Actual FDA Benefit-Risk Determinations

A device to treat a very rare virus was tested in a clinical trial that was designed to show superiority to standard antiviral treatment. When the trial results were revealed, it was clear the sponsor had incorrectly designed the trial and the results could only demonstrate with significant uncertainty that the device performed as well as standard treatment, but not better. However, use of the device did not have harmful effects as severe as those associated with antiviral treatment, and neither treatment was curative. The viral infection was rapidly progressive and terminal, so the patients had very little time to live after they were diagnosed. FDA approved this device because it gave patients access to a treatment that appeared to be equivalent to the standard of care (with some uncertainty remaining), but that did not cause the same severity of side effects.
Ultrasound techniques have been developed to destroy masses such as kidney stones, cysts and fibroid tumors without the need for surgery. The effectiveness of these techniques is less than that of surgery, but the risks are far lower. Additionally, if the device proves ineffective for certain patients, surgery remains an option. Thus, for non-invasive therapies that do not interfere with the effectiveness of subsequent, more-invasive treatments that have been proven to be effective, the lower level of benefit can outweigh the risks, which are greatly reduced.
A permanently implanted cardiovascular monitoring device is intended to diagnose heart failure. The device is studied and shows that its use reduces the number of days the patient is hospitalized for heart failure by about three. However, the implantation procedure for the device requires that the patient be hospitalized for two days. There are similar devices on the market that provide a similar level of benefit as this device that do not require an implantation procedure. FDA determined that the benefit of saving one day of hospitalization does not outweigh the risk of complication from the surgery needed to implant the device.
Thermal imaging has been posed as an alternative to mammography to detect breast cancer. These devices propose to produce a thermal image of tissue that has been recommended for biopsy to determine if it really should be biopsied, i.e., was it likely to be cancer. For these types of diagnostic devices to be approvable, they must be able to detect possible cancers with a level of specificity that overcomes the risk of a false negative. Even with risk mitigation strategies, such as restrictions on labeling or patient populations, this device had too much uncertainty in the clinical data to be approvable.
A permanent birth control device can be placed in a woman’s reproductive system through the vagina using a specialized delivery catheter. This device is a permanent implant and is not intended to be removed. Explant of the device would require surgery. Clinical data show that the device is effective in preventing pregnancy over a two-year period in women and the safety data show a low incidence of adverse clinical events. However, study results also show that there are several cases where the physician had difficulty correctly placing the device. In addition, the device was noted to be fractured on a follow-up x-ray in a few study patients. Given the uncertainty of the long-term impact of the device, the possibility of device fracture (which was not predicted in any of the bench and animal testing), and the safety and effectiveness of alternative therapies, the device was deemed to be non-approvable for the intended patient population.
An implanted device offers a unique design feature in comparison to the standard of care used to treat similar conditions. While the current standard of care works very well, it has limitations associated with hindering the mobility of the patient; whereas, the novel implanted device does not affect patient mobility. Based upon the effectiveness data from the clinical study, the device demonstrates that it has significantly improved functional outcomes in comparison to the current standard of care. However, from a safety perspective, the device did present different adverse events than the current standard of care. It is believed that the risks can be appropriately mitigated with training of surgical professionals and labeling. In the event the implant was to fail over time, the clinician could also resort to the current standard of care. In this situation, despite the higher risk of harmful events, the potential benefits outweighed the risks and the device was approved.

Appendix A
Worksheet for Benefit-Risk Determinations

Factor	Questions to Consider	Notes
Measures for Effectiveness of Devices (the extent of the benefit)
Type of benefit(s)	What primary endpoints or surrogate endpoints were evaluated? What key secondary endpoints or surrogate endpoints were evaluated? Which endpoints or surrogate endpoints were not assessed in the trials but would have been relevant? Would their outcome change your decision?
Magnitude of the benefit(s)	For each primary and secondary endpoint or surrogate endpoints evaluated: What was the magnitude of each treatment effect? Was this magnitude clinically significant?
Probability of the patient experiencing a benefit	How did the benefits evaluated vary across sub-populations? (Note specific subpopulations, nature of difference and any known reasons for these differences.) Was there a variation in public health benefit for different populations?
Duration of effect(s)	Could you determine the duration, if relevant, of each treatment effect, including primary and secondary endpoints? If so, what was it?
Metrics for Safety of Devices (the extent of harmful events)
Severity and types of harmful events (events and consequences):
Serious adverse events	What are the serious adverse events for this product?
Non-serious adverse events	What are the non-serious adverse events for this product?
Other harms	What other harms may result from the use of this product?
Probability of a harmful event	What is the incidence of each harmful event in the study population? How much uncertainty is in that estimate? (i.e., are you concerned that the incidence of the harmful event in the study population may not represent the true incidence in the intended patient population?) How does the incidence of harmful events vary by subpopulation?
Duration of harmful events	How long does the harmful event last? Is the harmful event reversible? What type of intervention is required to address the harmful event?
Risk of false-positive or false-negative for diagnostics
Additional Factors for Weighing Benefits and Risks of Devices
Uncertainty:
Quality of the study design	How robust were the data?
Quality of the conduct of the study
Robustness of the analysis of the study results
Generalizability of results (e.g., training of MDs, learning curve)
Patient tolerance for risk	Is there any current literature that assesses patient tolerance for risk in light of different levels of perceived benefit to treat the condition in question?
Disease severity
Disease chronicity
Availability of alternative treatments or diagnostics	What other therapies are available for this condition? How effective are the alternative treatments? How does their effectiveness vary by sub-population? How well-tolerated are the alternative therapies? How does their tolerance vary by sub-population?
Risk mitigation	Could you identify ways to mitigate the risks such as using product labeling, establishing education programs, providing add-on therapy, etc?
Novelty of technology	How well is the medical need this device addresses being met by currently available therapies?

Summary of the Benefit(s)	Summary of the Risk(s)	Summary of Other Factors

Conclusions Do the probable benefits outweigh the probable risks?

¹ In addition to §513(a), the criteria for establishing safety and effectiveness of a device are set forth in 21 CFR 860.7. Subsection (b)(1) notes, “In determining the safety and effectiveness of a device … the Commissioner and the classification panels will consider the following, among other relevant factors…The probable benefit to health from the use of the device weighed against any probable injury or illness from such use.” (21 CFR 860.7(b)).
To make this determination, “the agency relies upon only valid scientific evidence.” (21 CFR 860.7(c)(1)). Valid scientific evidence is defined as “evidence from well-controlled investigations, partially controlled studies, studies and objective trials without matched controls, well-documented case histories conducted by qualified experts, and reports of significant human experience with a marketed device, from which it can fairly and responsibly be concluded by qualified experts that there is reasonable assurance of the safety and effectiveness of a device under its conditions of use.” (21 CFR 860.7(c)(2)).
A reasonable assurance of safety occurs when “it can be determined, based upon valid scientific evidence, that the probable benefits … outweigh any probable risks.” The evidence of which is demonstrated by establishing, “the absence of unreasonable risk of illness or injury associated with the use of the device for its intended uses and conditions of use.” (21 CFR 860.7(d)(1).
Similarly, a reasonable assurance of effectiveness occurs when “it can be determined, based upon valid scientific evidence … the use of the device for its intended uses … will provide clinically significant results.” (21 CFR 860.7(e)(1)). The evidence of which is demonstrated principally through “well-controlled investigations” (see 21 CFR 860.7(e)(2)) as defined in 21 CFR 860.7(f)
² Under §513(a)(2)(C) of the FD&C Act, FDA may weigh “any probable benefit to health from the use of the device against any probable risk of injury or illness from such use” not only in determining there is a reasonable assurance of safety and effectiveness for a device evaluated under a PMA, but also in determining substantial equivalence under §513(i).
For purposes of this guidance the term “studies” is equivalent to the term “investigations.”
³ 21 CFR 860.7(e) “There is reasonable assurance that a device is effective when it can be determined, based upon valid scientific evidence, that in a significant portion of the target population, the use of the device for its intended uses and conditions of use, when accompanied by adequate directions for use and warnings against unsafe use, will provide clinically significant results”, as demonstrated by well-controlled investigations.
⁴ 21 CFR 860.7(d) “There is reasonable assurance that a device is safe when it can be determined, based upon valid scientific evidence, that the probable benefits to health from use of the device for its intended uses and conditions of use, when accompanied by adequate directions and warnings against unsafe use, outweigh any probable risks. The valid scientific evidence used to determine the safety of a device shall adequately demonstrate the absence of unreasonable risk of illness or injury associated with the use of the device for its intended uses and conditions of use.”
⁵ 21 CFR 803.3.