Gene Expression Profiling

1. Gene Expression Profiling

2. Good Microarray Studies Have Clear Objectives • Class Comparison (gene finding) • Find genes whose expression differs among predetermined classes • Class Prediction • Prediction of predetermined class using information from gene expression profile • Response vs no response • Class Discovery • Discover clusters of specimens having similar expression profiles • Discover clusters of genes having similar expression profiles

3. Class Comparison and Class Prediction • Not clustering problems • Supervised methods

4. Levels of Replication • Technical replicates • RNA sample divided into multiple aliquots • Biological replicates • Multiple subjects • Multiple animals • Replication of the tissue culture experiment

5. Comparing classes or developing classifiers requires independent biological replicates. The power of statistical methods for microarray data depends on the number of biological replicates

6. Microarray Platforms • Single label arrays • Affymetrix GeneChips • Dual label arrays • Common reference design • Other designs

7. Common Reference Design A1 A2 B1 B2 RED R R R R GREEN Array 1 Array 2 Array 3 Array 4 Ai = ith specimen from class A Bi = ith specimen from class B R = aliquot from reference pool

8. The reference generally serves to control variation in the size of corresponding spots on different arrays and variation in sample distribution over the slide. • The reference provides a relative measure of expression for a given gene in a given sample that is less variable than an absolute measure. • The reference is not the object of comparison.

10. Dye swap technical replicates of the same two rna samples are not necessary with the common reference design • For two-label direct comparison designs for comparing two classes, dye bias is of concern and dye swaps may be needed.

11. Class ComparisonGene Finding

13. Controlling for Multiple Comparisons • Bonferroni type procedures control the probability of making any false positive errors • Overly conservative for the context of DNA microarray studies

14. Simple Procedures • Control expected the number of false discoveries by testing each gene for differential expression between classes using a stringent significance level • expected number of false discoveries in testing G genes with significance threshold p* is G p* • e.g. To limit of 10 false discoveries in 10,000 comparisons, conduct each test at p<0.001 level • Control FDR • Expected proportion of false discoveries among the genes declared differentially expressed • Benjamini-Hochberg procedure • FDR = G p* / #(p p*)

15. Additional Procedures • Multivariate permutation tests • Korn et al. Stat Med 26:4428,2007 • SAM - Significance Analysis of Microarrays • Advantages • Distribution-free • even if they use t statistics • Preserve/exploit correlation among tests by permuting each profile as a unit • More effective than univariate permutation tests especially with limited number of samples

16. Randomized Variance t-testWright G.W. and Simon R. Bioinformatics19:2448-2455,2003 • Pr(-2=x) = xa-1exp(-x/b)/(a)ba

18. Class Prediction

19. Components of Class Prediction • Feature (gene) selection • Which genes will be included in the model • Select model type • E.g. Diagonal linear discriminant analysis, Nearest-Neighbor, … • Fitting parameters (regression coefficients) for model • Selecting value of tuning parameters

20. Feature Selection • Genes that are differentially expressed among the classes at a significance level  (e.g. 0.01) • The  level is selected only to control the number of genes in the model • For class comparison false discovery rate is important • For class prediction, predictive accuracy is important

21. Complex Gene Selection • Small subset of genes which together give most accurate predictions • Genetic algorithms • Little evidence that complex feature selection is useful in microarray problems

22. Linear Classifiers for Two Classes

23. Linear Classifiers for Two Classes • Fisher linear discriminant analysis • Requires estimating correlations among all genes selected for model • Diagonal linear discriminant analysis (DLDA) assumes features are uncorrelated • Compound covariate predictor (Radmacher) and Golub’s method are similar to DLDA in that they can be viewed as weighted voting of univariate classifiers

24. Linear Classifiers for Two Classes • Compound covariate predictor Instead of for DLDA

25. Linear Classifiers for Two Classes • Support vector machines with inner product kernel are linear classifiers with weights determined to separate the classes with a hyperplane that minimizes the length of the weight vector

26. Support Vector Machine

27. Other Linear Methods • Perceptrons • Principal component regression • Supervised principal component regression • Partial least squares • Stepwise logistic regression

28. Other Simple Methods • Nearest neighbor classification • Nearest k-neighbors • Nearest centroid classification • Shrunken centroid classification

29. Nearest Neighbor Classifier • To classify a sample in the validation set, determine it’s nearest neighbor in the training set; i.e. which sample in the training set is its gene expression profile is most similar to. • Similarity measure used is based on genes selected as being univariately differentially expressed between the classes • Correlation similarity or Euclidean distance generally used • Classify the sample as being in the same class as it’s nearest neighbor in the training set

30. Nearest Centroid Classifier • For a training set of data, select the genes that are informative for distinguishing the classes • Compute the average expression profile (centroid) of the informative genes in each class • Classify a sample in the validation set based on which centroid in the training set it’s gene expression profile is most similar to.

31. When p>>n The Linear Model is Too Complex • It is always possible to find a set of features and a weight vector for which the classification error on the training set is zero. • It may be unrealistic to expect that there is sufficient data available to train more complex non-linear classifiers

32. Other Methods • Top-scoring pairs • Claim that it gives accurate prediction with few pairs because pairs of genes are selected to work well together • Random Forest • Very popular in machine learning community • Complex classifier

33. Comparative studies indicate that linear methods and nearest neighbor type methods often work as well or better than more complex methods for microarray problems because they avoid over-fitting the data.

37. Evaluating a Classifier • Fit of a model to the same data used to develop it is no evidence of prediction accuracy for independent data • Goodness of fit vs prediction accuracy

39. Class Prediction • A classifier is not a set of genes • Testing whether analysis of independent data results in selection of the same set of genes is not an appropriate test of predictive accuracy of a classifier • The classification of independent data should be accurate. There are many reasons why the classifier may be unstable. The classification should not be unstable.

42. Hazard ratios and statistical significance levels are not appropriate measures of prediction accuracy • A hazard ratio is a measure of association • Large values of HR may correspond to small improvement in prediction accuracy • Kaplan-Meier curves for predicted risk groups within strata defined by standard prognostic variables provide more information about improvement in prediction accuracy • Time dependent ROC curves within strata defined by standard prognostic factors can also be useful

44. Time-Dependent ROC Curve • M(b) = binary marker based on threshold b • PPV = prob{ST|M(b)=1} • NPV = prob{S<T|M(b)=0} • ROC Curve is Sensitivity vs 1-Specificity as a function of b • Sensitivity = prob{M(b)=1|ST} • Specificity = prob{M(b)=0|S<T}

46. Validation of a Predictor • Internal validation • Re-substitution estimate • Very biased • Split-sample validation • Cross-validation • Independent data validation

48. Split-Sample Evaluation • Split your data into a training set and a test set • Randomly (e.g. 2:1) • By center • Training-set • Used to select features, select model type, determine parameters and cut-off thresholds • Test-set • Withheld until a single model is fully specified using the training-set. • Fully specified model is applied to the expression profiles in the test-set to predict class labels. • Number of errors is counted

49. Leave-one-out Cross Validation • Leave-one-out cross-validation simulates the process of separately developing a model on one set of data and predicting for a test set of data not used in developing the model

50. Leave-one-out Cross Validation • Omit sample 1 • Develop multivariate classifier from scratch on training set with sample 1 omitted • Predict class for sample 1 and record whether prediction is correct