Discussion Topic:
The 2008 Draft FDA guidance on the ISE provided in this session provides a lot of detail on pooled analyses (also referred to in the guidance as “meta-analyses”).  Based on the information in this guidance, what are the benefits of pooling efficacy data across studies? What might be some of the inherent problems in pooling data (including factors that would prevent pooling)?

Student #1 – Cheryl
1. According to the 2008 draft guidance “Integrated Summary of Effectiveness”, there are several benefits to pooling the data. One benefit to pooling data is when individual studies are expected to lack power when analyzing patient subgroups such as demographics or geographical regions. Another benefit is when looking at the dose response relationship. This would be helpful when the various studies used different dose levels. The third benefit identified in the 2008 draft guidance is when looking at the drug effect on secondary endpoints.

2. One of the biggest problems with pooling data is that it is often not thought about until after analysis is complete and the potential for bias enters the equation. It is better to prepare for pooling when the protocol is written. Decisions on how the data should be analyzed should be decided at this point to avoid any bias. The 2008 draft guidance notes that in the case of studies with “drastic heterogeneity”, pooling is not suggested.

Reference
U.S. Department of Health and Human Service (2008). Guidance for Industry: Integrated summary of effectiveness. Retrieved from https://blackboard.gwu.edu/@@/DF4CC84DB1DC3B669A40111C8524B95B/courses/1/32538_201203/content/_4695972_1/embedded/2008%20ISE%20Draft%20Guidance.pdf

1.    Benefits:
1.    Pooling is beneficial when looking at patient subgroups that an individual study might lack power in.
2.    When looking at dose-response relationships, especially when individual studies were conducted using different doses.
3.    When assessing the drug effects on a secondary endpoint or on a component of a composite endpoint.
4.    Pooled analyses also can be beneficial in evaluating time to effect and response rates when the primary endpoint is continuous.
2.    Inherent problems:  The decision to pool data is often times made after the fact, there is a potential for bias. It is necessary for careful planning when writing a protocol to plan for pooling to reduce biases.

Reference:
U.S. Department of Health and Human Service (2008). Guidance for Industry: Integrated summary of effectiveness.
Student #3 – Ann
According to our readings this week, “Integrated Summary of Effectiveness” there are a few main benefits to using meta-analysis (pooled analyses) from more than one clinical trial. If meta-analysis is used, then it should be mentioned in the ISE as to whether it was planned before the protocol was written or an after thought.
1. Assessment of dose response using different doses in demographic sub-groups.
2. Assessment of drug effects on (a.) secondary endpoint or (b.) a component of composite endpoint.
3. When it comes to examining several patient subgroups, such as demographics, etiology or the severity of the disease, geographical regions, if and when individual studies may lack power.
4. In a continous variable primary endpoint, meta-analysis can be used in evaluating time to effect and response rates.
5. Results can be used in a Phase IV commitments.

When to avoid meta-analysis:
1. Early decisions should be made before results are known to avoid biased results.
2. State whether including or excluding studies based on their outcomes will be biased.
3. Severe heterogenity should avoid pooling studies.
4. Run a randomized study.

References:
U.S. Department of Health and Human Services (2008). Guidance for Industry:Intergrated summary of effectiveness. Retrieved from:
https://blackboardgwu.edu.@/DF4CC84DB1DC3B669A40111C8524B95B/courses/1/32538_201203/content/_4695972_1/embedded/2008%20ISE%20DRAFT%20GUIDANCE.pdf
Student #4 – Alisa
Statistical issues should be summarized study by study as well as collectively. There should be an analysis of the similarity of efficacy for subjects in different regions either study-by-study, subsets of studies, or pooled analyses.  Pooling of efficacy data across studies can be beneficial in the following manner:

???????Pooling data of individual studies can collectively provide support for the claimed effectiveness of the study drug.

???????Pooling data is beneficial when examining various sub-groups such as demographics, etiology, and severity of disease or geographical regions. In particular where individual studies lack the expected influence.

???????Pooling data can provide useful information when assessing the drug’s effects on secondary endpoint or a composite of a composite endpoint.

?Pooling data is beneficial when assessing the dose-response relationship, in particular when studies use different doses and demographic sub-groups.

The Guidance for Industry: Integrated summary of effectiveness, (2008) further notes that pooled results can also be used to design future trials including phase IV trials. Additionally a meta-analysis that does not have positive individual studies as part of the pooled analysis is not like to be accepted as support for a primary endpoint.

The Guidance for Industry: Integrated summary of effectiveness, (2008) noted that generally, when individual studies fail to show effectiveness based on their planned analyses, a meta-analysis would not provide persuasive evidence of effectiveness. The decision to pool data usually is made after the fact and is potentially biased.  The decision to pool data should be made before results are known to avoid bias. It is important that if pooled analysis of the clinical study is performed, it should be clearly stated in the ISE whether it was performed according to a predefined protocol or if it was post hoc. The following factors can affect pooling and need careful consideration: Pooling studies with drastic heterogeneity and studies with different allocation ratios.
Reference

U.S. Department of Health and Human Services (2008). Guidance for Industry: Integrated summary of effectiveness. Retrieved from http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm079803.pdf