Analysis of Multiple End Points in Clinical trials

1. Analysis of Multiple End Points in Clinical trials By Trusha Patel and Sirisha Davuluri

2. Article Details “An efficient method for accommodating potentially underpowered primary endpoints” • By Jianjun (David) Liand Devan V. Mehrotra”

3. Contents • Introduction • Procedure 1: PAAS Method • Procedure 2: 4A Method • Independent Primary Endpoints • Correlated Primary Endpoints • Performance of both Procedures (Probability of achieving positive trial) • Conclusion • Appendix • References • Questions?

4. Introduction Clinical trials generally classify the endpoints into primary, secondary and exploratory types. Primary Endpoints: • Primary endpoints address primary objectives of the trial. They are usually few but are clinically most relevant to the disease and the treatment under study. They assess the main clinical benefits of the treatment.

5. Introduction Secondary Endpoints: • Secondary endpoints characterize extra benefits of the treatment under study after it has been demonstrated that the primary endpoints show clinically meaningful benefits of the treatment. Explanatory Endpoints: • Exploratory endpoints are usually not prospectively planned and are generally not rigorously evaluated like primary and secondary endpoints.

6. Introduction Different approaches have been used to specify a clinical decision rule for trials that have more than one primary endpoint. Significant results can be required for each of several primary endpoints to consider a trial ‘‘positive.”

7. At least One Procedure • To demonstrate the treatment’s superiority on at least one endpoint. • If each multiple endpoint is independently clinical relevant, the multiple endpoint problem can be formulated as a multiple testing problem, and the trial is declared positive if at least one significant effect is detected. • When significant results are required for more than one but not all of multiple primary endpoints for a trial to be considered positive, correction for multiplicity is also necessary, and this must take into account the total number of endpoints and the number required for the trial to be considered positive.

8. Different Methods • Weighted Bonferroni procedure • Prospective Alpha Allocation Scheme (PAAS Method) • Adaptive Alpha Allocation Approach (4A Method) • Bonferroni-type parametric procedure • Fallback-type parametric procedure For this project, we are mainly focusing on PAAS Method and 4A Method by considering two Primary Endpoints (A : well powered, B : potentially underpowered).

9. Procedure 1: PAAS Method Procedure: • Moye’(2000) proposed the ‘‘prospective alpha allocation scheme” for preserving Type I error rates at acceptable levels when there are multiple endpoints. • Assume that p-values for the individual endpoints are independent. • The experiment-wise type I error rate = αshould be capped, say at 0.05, and the fraction of the Type I error rate α allocating α1* = α − ε, ε= 0.01 to endpoint A α2* = 1 - (1−α)/(1−α1*) to endpoint B

10. PAAS Method Advantages: • PAAS is a simple and appealing method. • Prospectively allocating alpha in this method preserves the experiment-wise Type I error rate and makes it possible to consider a treatment efficacious when the null hypothesis is not rejected for the primary endpoint but is rejected for one or more of the secondary endpoints. Disadvantage: The prospective alpha allocation scheme preserves the experiment-wise Type I error rate at a higher rate than is customarily accepted.

11. Procedure 2: 4A Method Procedure: • Li and Mehrotra (2008) proposed a multiple testing procedure, which they referred to as the adaptive alpha allocation approach or 4A procedure. • Consider a clinical trial with m endpoints and assume that the endpoints are grouped into two families. The first family includes m1 endpoints that are adequately powered and the second family includes m2 potentially underpowered endpoints (m1 + m2 = m).

12. 4A Method - Independent Primary Endpoints Let’s consider 2 multiple points for illustration of the procedure: • Let pAand pBdenote the p-value for endpoint A and endpoint B, respectively. • In the article, they assume that p-values are two-tailed (to mimic common practice) and that ‘statistically significant’ results are in the direction of interest; however all the methods discussed can also be used with one-tailed p-values. • In their proposed adaptive alpha allocation approach, endpoint A is tested at the pre specified level α1 = α − ε, and endpoint B is tested at the adaptive level.

13. 4A Method - Independent Primary Endpoints • If A achieves statistical significance (i.e. pA≤α1), then B is tested at level α2(pA,α1, α) = α; • If A fails to achieve statistical significance (i.e. pA>α1), then B is tested at level 0 <α2(pA,α1,α) ≤ α1, which is close (or equal) to α1if pA is not much greater than α1, but approaches zero as pA increases. • Sample values of α2 as a function of observed pA are provided in Table I (ρ = 0 column) • Table I shows the alpha allocation for Primary endpoint B.

14. Table I

15. Extending 4A to the scenario of three or more independent primary endpoints • Let there be M primary end points out of which the first m are assumed to be powered by type A endpoints and the rest of the M-m are underpowered or type B endpoints. • Here, they suppose, the p-values for the type A endpoints are tested for statistical significance at an overall level α1= α – εusing Hochberg’s method. • The p-values for the type B primary endpoints are then measured at and overall adaptive level α2(p(m), α1, α,m) using Hochberg’s method

16. Extending 4A to the scenario of three or more independent primary endpoints Note : (3) and (4) are generalizations of (1) and (2); setting m=1 in (3) and (4) leads to (1) and (2), respectively. Advantage: The remaining endpoints are tested at a generally higher significance level, which improves their power.

17. 4A Method- Correlated Primary Endpoints Procedure: • If the endpoints are correlated, then formula (1) and (3) cannot be used directly because the FWER may be inflated. Therefore, an adjustment may be needed in the formulas presented for correlated endpoints. • The 4A procedure is readily implemented using Table I if ρis known. • For the case of unknown ρ, See Appendix-I. Extending 4A to the relatively uncommon scenario of M>2 primary endpoints is notably more challenging in the case of correlated endpoints compared with independent endpoints.

18. Performance of both Procedures (Probability of achieving Positive Trial) Let’s compare the performances of the PAAS and 4A methods for our motivating scenario of two primary endpoints: A-well powered and B-underpowered. Goal: To improve the probability of correctly achieving a positive trial, i.e. of rejecting at least one of the two null hypotheses, while ensuring that the family wise type I error rate is at most α. Table A summarizes the probabilities of achieving a positive trial using the PAAS and 4A methods when the two endpoints are either independent (ρ=0) or correlated (ρ=0.5); results are based on 10,000,000 simulations. (See Table III in Appendix-II)

19. Table A

20. Performance of both Procedures (Probability of achieving Positive Trial) Case 1-7: Primary endpoint A: Adequately Powered Primary endpoint B: Underpowered Case 8-17: Primary endpoint A: Underpowered Primary endpoint B: Underpowered For PAAS method, α1* = 0.04 and α2* = 0.010; and For 4A method, α1= 0.04 and α2≡α2(pA,α1,α,ρ) is calculated adaptively as described before.

21. Results • Use of 4A method increased the probability of achieving a positive trial compared with PAAS. • ForIndependent endpoints, the absolute gain in power ranged from 0.9 % (case 7; 77.9 versus 78.8%) to 3.7% (case 8; 78.8 versus 82.5%) compared with PAAS. • For Correlated endpoints A and B (ρ=0.5); the power for 4A was numerically higher than that for PAAS, For example, the power gains of 4A ranged from 0.1 per cent (case 7; 77.4 versus 77.5 per cent) to 2.9 per cent (case 8, case 14) compared with PAAS.

22. Results • The probability of achieving a positive trial by pre specifying both A and B as primary endpoints and using 4A for the analysis is about as good as (cases 7 and 13) or notably better than (cases 1 and 8) the marginal power for endpoint A. • In other words, 4A enables us to accommodate the underpowered endpoint B in the primary family while essentially preserving or substantially enhancing the likelihood of achieving a positive trial compared with the strategy of using only A as the primary endpoint with α=0.05.

23. Conclusion • There is no optimal strategy for the use of significance testing with multiple endpoints, especially when dealing with clinically persuasive endpoints that may be underpowered, but based on the simulation results that are provided in the article, we can conclude that 4A method performs better than PAAS method. • The power advantage of 4A over PAAS was greater when the marginal power for endpoint B was lower (higher), regardless of whether the endpoints were independent or correlated.

24. Appendix-I

25. Appendix-II

26. References • http://www.ncbi.nlm.nih.gov/pubmed/18759248 • Multiple Testing Problems in Pharmaceutical Statistics, by Alex Dmitrienko, AjitTamhane and Frank Bretz.

27. Questions