Testing Dose-Response with Multivariate Ordinal Data

1. Testing Dose-Response with Multivariate Ordinal Data Bernhard Klingenberg Asst. Prof. of Statistics Williams College, MA Paper available at www.williams.edu/~bklingen In Collaboration with Aldo Solari, Luigi Salmaso and Fortunato Pesarin, University of Padova

2. Introduction Safety and Toxicity Data Notation and hypothesis of interest Stochastic Ordering Theorem (SMH IJD) Testing SMH Simple Test statistics Permutation Approach Step-down methods for indiv. endpoint significance Increase power Outline • Example • Parallel, 5 dose group study with rats (8 rats per dose group) • 25 Adverse Events from exposure to Perchlorethylene

3. Introduction: Safety and Toxicity Safety and Toxicity Data: • To capture large number of possible manifestations of a dose (exposure) effect on safety or toxicity: Multiple Endpoints • One such collection of endpoints to evaluate neurophysiological effects: Functional Observational Battery (FOB) • Others: Drug Safety, Disease progression

4. Introduction: FOB • Goal: Evaluation of neurophysiological effects to a toxin (Perchlorethylene) • Data1: • 2 groups (No exposure vs.1.5g/kg exposure) • 8 rats in each group • Each evaluated at 25 endpoints (various effects), grouped into 6 domains • Response ordinal, on a scale from 1 (no effect) to 4 (most severe reaction) 1 Moser (1986) Journal of the American College of Toxicology

5. Introduction: FOB

6. Introduction: Notation • k-dimensional response vectors: ControlTreatment • Random Sample ControlTreatment • Hypothesis of interest: “No dose effect” st d

7. Introduction: No Toxicity d • “ ”: For all response sequences ControlTreatment • “ “: Stochastically larger 2 ControlTreatment • Note: Rejection of H0 does not lead to H1 st 2 Marschall & Olkin (1979) Inequalities: Theory of Majorization and Its Applications

8. SMH • Usually only interested if k margins are equal or not. I.e., for each adverse event , • Def.: Simultaneous Marginal Homogeneity (SMH) 3: Vector of marginal probabilities are equal under the two exposures, for all adverse events simultaneously 3 Agresti and Klingenberg (2005) JRSS C, Klingenberg and Agresti (2006), Biometrics

9. Lacrimation Lacrimation Arousal Arousal SMH SMH with just two adverse events ControlTreatment

10. SMH • Theorem: • Prior assumption plausible when dealing with adverse events data (increase in exposure shift towards higher outcome categories) IJD SMH Cumulative marginal inhomogeneity:

11. Testing SMH • Consequence of Theorem: If prior assumption plausible, can use permutation approach to test hypothesis of SMH • Test for SMH: Modeling approach via cumulative logits (proportional odds form) 4 • Estimation (ML, conditional ML, GEE,…) computationally impossible, Asymptotics invalid 3 4 Han, Catalano, Senchaudhuri, Metha (2004) Exact Analysis of Dose Response for Multiple Correlated Binary Outcomes, Biometrics.

12. Testing SMH • Let • Simple test statistic: Standardized differences in marginal sample proportions • given by (from multinomial assumption):

13. Testing SMH • To take advantage of ordinal nature: Consider scoring function • Let be score matrix • Look at difference in mean scores: • Estimate covariance matrix 4

14. Testing SMH • Test statistic for sparse data, ignoring correlation among adverse events: with • This gives global test of safety/toxicity • Permutation approach: 16!/(8!8!) = 12870 possible permutations, many leading to identical values of • Advantage of permutation approach: Incorporates dependence by resampling entire vectors; exact significance levels

15. Computation with equally spaced scores: Note: Testing SMH • Example: Arousal Endpoint

16. observed Testing SMH • Permutation Distribution: Asympt. Distr. Perm. Distr.

17. Testing SMH • Identifying which individual adverse events are significant leads to multiple hypotheses testing: • Use test statistic (standardized mean score difference) for individual tests • Multiplicity adjustments via step-down approach of Westfall &Young (1993), using distribution of maximum test statistic

18. Testing SMH • Permutation Distribution: Perm. Distr. Observed maximum

19. Testing SMH

20. Testing SMH • How sensitive are results to assigned scores? • Consider the scores that maximize (obtainable via isotonic regression; data-driven) • Appropriate for safety/toxicity data; maximizes the contrast btw. the mean score differences Equally spaced scores: With

21. Testing SMH

22. Testing Domains Domain effects? • Some endpoints may measure similar effects • Multiplicity adjustment at the endpoint level may be too conservative, leading to some false negatives • Adjusted P-value for domain less than or equal to smallest adjusted P-value within domain • “Proof”: Let endpoint h be in the first domain Dom1:

23. Testing Domains Important Consequence (Robustness Property): • Consonant domain test statistic: • Reject only (at domain level) if at least one endpoint within domain significant • If no significant endpoint, domain also not significant For domain significance, it is irrelevant how many, potentially non-significant endpoints are grouped into a domain! * * Provided the same test statistic is used for all intersection hypotheses

24. Testing Domains • Dissonant domain test statistic: • Accumulate effects over endpoints within domain • Even though no individual endpoint is significant, several marginally significant ones can result in significant domain P-value

25. Summary • Testing dose-response for multivariate ordinal data • Correlated ordinal responses (typical for toxicity or safety data) are often sparse and imbalanced use permutation approach • Instead of modeling dose-response, we focused on testing SMH vs. stochastic ordering • SMH IJD, but SMH IJD under • Test statistic: zh= (difference in mean scores) / s.e., for each endpoint, assuming working independence • s.e. derived from multinomial model and estimated under SMH • Multiplicity adjusted P-values for each endpoint from Westfall and Young’s step-down procedure st

26. Conclusions for FOB • Neuromuscular domain showed significant effect • Domain P-value 0.003 with dissonant test • Domain P-value 0.025 with consonant test • Adverse events in neuromuscular domain that show increased toxicity at the 1.5 g/kg exposure level when compared to control • Sensorimotor domain also shows increased level of toxicity, although no individual adverse event is significant

27. Did not show • How methods extend to several dose levels • Effect of discreteness (used mid P-values throughout) • Combining P-values instead of test statistics, with functions other than the maximum Thank you and Go Gators