1 / 38

Large Scale Data Integration

Large Scale Data Integration. Curtis Huttenhower Sequence and Expression 01-24-08. Functional Relationships. Two genes that work together to achieve similar cellular goals are functionally related Proteins that co-complex: ribosomal, polymerase, ORC, etc. etc. A TF and its target

garrett-dickerson
Télécharger la présentation

Large Scale Data Integration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Large Scale Data Integration Curtis Huttenhower Sequence and Expression 01-24-08

  2. Functional Relationships • Two genes that work together to achieve similar cellular goals are functionally related • Proteins that co-complex: ribosomal, polymerase, ORC, etc. etc. • A TF and its target • Two enzymes catalyzing different steps in the same metabolic pathway • A membrane-bound receptor and a downstream phos. target • etc. etc. etc. • Genes that do really different stuff are considered to be functionally unrelated • Anything else is neither • Genes with unknown function • Genes in similar but non-identical pathways

  3. Functional Relationships • How can we tell? • These databases classify every gene pair into one of three groups: • Functionally related • Unrelated • Neither

  4. Data • Well, as long as we’re talking about gene pairs, how do pairs of genes act in data? MANY MANY microarrays Correlation Colocalization Yes/No Shared miRNA sites Yes/No High Two-hybrid Yes/No Same chromosome band Yes/No Low Conserved TF sites Yes/No Affinity Yes/No

  5. Data • These fall into two general categories: • Yes/no (binary or 0/1) • Continuous (numerical scores) Each dataset turns into a set of gene pairs labeled with small integers (or nothing). G1 G2 0 G1 G3 1 G1 G4 - G2 G3 - G2 G4 0 G3 G4 1 Two-hybrid Yes/No G1 G2 0.9 G1 G3 0.75 G1 G4 0.1 G2 G3 -0.1 G2 G4 0.2 G3 G4 -0.5 G1 G2 4 G1 G3 3 G1 G4 1 G2 G3 0 G2 G4 2 G3 G4 0 Binning Microarrays Correlation

  6. Integration • Now, for each gene pair… We have an “answer” indicating whether it’s a related pair G1 G2 1 G1 G2 0 G1 G2 1 G1 G2 - G1 G2 3 DS1 DS2 DS3 DSN And we have a bunch of datasets contributing their opinions (i.e. experimental results)

  7. Integration • Let’s look at each dataset individually: Our answers know about a bunch of unrelated genes. G1 G3 0 G3 G4 0 G9 G14 0 G10 G11 0 … G1 G2 1 G1 G3 1 G4 G7 1 G10 G12 1 … And a bunch of related genes. Within each dataset, some subset of these pairs have values: DS1 G1 G3 0 G3 G4 - G9 G14 1 G10 G11 0 … G1 G2 1 G1 G3 0 G4 G7 - G10 G12 1 …

  8. Integration • Let’s look at each dataset individually: Our answers know about a bunch of unrelated genes. G1 G3 0 G3 G4 0 G9 G14 0 G10 G11 0 … G1 G2 1 G1 G3 1 G4 G7 1 G10 G12 1 … And a bunch of related genes. Within each dataset, some subset of these pairs have values: DS2 G1 G3 - G3 G4 0 G9 G14 0 G10 G11 - … G1 G2 - G1 G3 1 G4 G7 1 G10 G12 1 …

  9. Integration • Within each dataset, let’s count up the number of times each value occurs for each type of gene pair (related or unrelated): G1 G3 0 G3 G4 0 G9 G14 0 G10 G11 0 … G1 G2 1 G1 G3 1 G4 G7 1 G10 G12 1 … Functionally related? No Yes 96 9 0 Dataset value DS1 13 36 1 G1 G3 0 G3 G4 - G9 G14 1 G10 G11 0 … G1 G2 1 G1 G3 0 G4 G7 - G10 G12 1 …

  10. Integration • Within each dataset, let’s count up the number of times each value occurs for each type of gene pair (related or unrelated): G1 G3 0 G3 G4 0 G9 G14 0 G10 G11 0 … G1 G2 1 G1 G3 1 G4 G7 1 G10 G12 1 … Functionally related? No Yes 0.9 0.2 0 Dataset value DS1 0.1 0.8 1 G1 G3 0 G3 G4 - G9 G14 1 G10 G11 0 … G1 G2 1 G1 G3 0 G4 G7 - G10 G12 1 …

  11. Integration • Each dataset is now represented by two probability distributions: • One for related gene pairs, one for unrelated Related genes are more likely to bind Related Related genes are more likely to be highly correlated Unrelated Prob. Prob. This is particularly noticeable for continuous datasets, where these represent correlations. 0 1 0 1 2 3 4 5 DS1 value DS5 value

  12. Integration • In the best case, datasets look like these: • In the worst case, they look like these: Related Prob. Prob. Unrelated 0 1 0 1 Prob. Prob. Prob. 0 1 0 1 0 1

  13. Integration • The variation in a dataset’s probability distribution indicates how informative it is. • Some microarrays might look like these: Even if genes are highly correlated, it doesn’t mean anything, because unrelated genes are also correlated. Prob. 0 1 2 3 4 5 Everything’s really correlated! We can actually correct microarrays like this during preprocessing. Prob. 0 1 2 3 4 5

  14. Prediction Ok, so what? • Given what we know about some genes, we’ve learned something about datasets: For each dataset Di, we know P(Di = d | FR) • What we want to know is, given some data, what can we predict about unknown genes?

  15. Prediction • We know: • And we want to know: The probability of a gene pair being functionally related P(FR) The probability of each dataset containing some value P(D = d) The probability of each value given a relationship (or not) P(D = d | FR) The probability that new genes are related given some data P(FR | D = d)

  16. Prediction • Enter Thomas Bayes: • Who established Bayes’ theorem:

  17. Prediction For each new gene pair, we can find its probability of being functionally related Each dataset is weighted according to how informative we’ve calculated it to be <insert math here> Datasets with no data for a particular gene pair are ignored And importantly for us, this all happens very quickly, regardless of the number of genes or datasets

  18. Prediction • The result is that we produce a probability of functional relationship for each gene pair: G1 G2 0.9 G1 G3 0.75 G1 G4 0.1 G2 G3 0.3 G2 G4 0.2 … • Which in turns translates into a fully connected interaction network: Only the most confident edges are typically shown

  19. Context Specificity • We can do even better! • This process lets us figure out how much to “trust” each dataset. • But datasets can give better (or worse) results in particular biological areas: Microarrays for ribosomal gene pairs Microarrays for all gene pairs Prob. Prob. 0 1 2 3 4 5 0 1 2 3 4 5

  20. Context Specificity • So we don’t just learn one probability distribution per dataset. Prob. 0 1

  21. Context Specificity • So we don’t just learn one probability distribution per dataset. • We learn one probability distribution per dataset per biological process of interest! Carbon metabolism Prob. Translation Autophagy 0 0 0 1 1 1 0 1

  22. Context Specificity • So we don’t just learn one probability distribution per dataset. • We learn one probability distribution per datasetper biological process of interest! • This means that for each gene pair, we can predict a different probability of relationship per process of interest. Carbon metabolism Prob. Translation Autophagy 0 0 0 1 1 1 0 1 G1 G2 0.9 G1 G3 0.75 G1 G4 0.1 G2 G3 0.3 G2 G4 0.2 … G1 G2 0.1 G1 G3 0.2 G1 G4 0.75 G2 G3 0.15 G2 G4 0.9 … G1 G2 0.15 G1 G3 0.1 G1 G4 0.9 G2 G3 0.2 G2 G4 0.25 …

  23. Context Specificity • So we don’t just learn one probability distribution per dataset. • We learn one probability distribution per datasetper biological process of interest! • This means that for each gene pair, we can predict a different probability of relationship per process of interest. • Which in turn produces different interaction networks for each process. Carbon metabolism Prob. Translation Autophagy 0 0 0 1 1 1 0 1 G1 G2 0.9 G1 G3 0.75 G1 G4 0.1 G2 G3 0.3 G2 G4 0.2 … G1 G2 0.1 G1 G3 0.2 G1 G4 0.75 G2 G3 0.15 G2 G4 0.9 … G1 G2 0.15 G1 G3 0.1 G1 G4 0.9 G2 G3 0.2 G2 G4 0.25 …

  24. Predicting Gene Function Ok, so what? • We can now dig through these networks for interesting things: • YFG, its interaction partners, and their processes • Each edge represents specific datasets/publications • Dense clusters (new functional modules) • Areas that change a lot from process to process • What known disease genes are doing • Find relationships between TFs and their targets • Predicting function for uncharacterized genes

  25. Predicting Gene Function • Suppose we have a whole interaction network for autophagy: • How do we predict new genes involved in the process? • Look at stuff “around” the known autophagy genes!

  26. Predicting Gene Function • The bioPIXIE algorithm: • Given a network and some query genes, find the other genes most strongly connected to the whole query

  27. Predicting Gene Function • The bioPIXIE algorithm: • Given a network and some query genes, find the other genes most strongly connected to the whole query G1: 0.5 + 0.5 + 0.1 = 1.1

  28. Predicting Gene Function • The bioPIXIE algorithm: • Given a network and some query genes, find the other genes most strongly connected to the whole query G1: 0.5 + 0.5 + 0.1 = 1.1 G2: 0.9 + 0.9 + 0.5 = 2.3

  29. Predicting Gene Function • The bioPIXIE algorithm: • Given a network and some query genes, find the other genes most strongly connected to the whole query G1: 0.5 + 0.5 + 0.1 = 1.1 G2: 0.9 + 0.9 + 0.5 = 2.3 …

  30. Predicting Gene Function • The bioPIXIE algorithm: • Given a network and some query genes, find the other genes most strongly connected to the whole query G1: 0.5 + 0.5 + 0.1 = 1.1 G2: 0.9 + 0.9 + 0.5 = 2.3 … Then display the genes with the best scores and the strongest edges connecting them

  31. Predicting Gene Function • The ratio algorithm: • Given a network and some query genes, find the other genes most specifically connected to the whole query

  32. Predicting Gene Function • The ratio algorithm: • Given a network and some query genes, find the other genes most specifically connected to the whole query G1: = 0.8

  33. Predicting Gene Function • The ratio algorithm: • Given a network and some query genes, find the other genes most specifically connected to the whole query G1: = 0.8 G2: = 1.5

  34. Predicting Gene Function • The ratio algorithm: • Given a network and some query genes, find the other genes most specifically connected to the whole query G1: = 0.8 G2: = 1.5 … Then display the genes with the best scores and the strongest edges connecting them

  35. Predicting Gene Function • These can differ a lot, particularly for “hubby” genes!

  36. Predicting Gene Function • These can differ a lot, particularly for “hubby” genes! bioPIXIE G1: 0.9 + 0.5 = 1.4 G2: 0.5 + 0.5 = 1.0 Ratio G1: = 1.2 This difference is exacerbated when the query isn’t itself strongly connected, since it makes it easy for hubby genes to dominate bioPIXIE’s results. G2: = 1.7

  37. Predicting Gene Function • How is this relevant to us? • Suppose we ask about just a few genes • If they’re not internally consistent in the data, bioPIXIE’s results are mostly hubs • This usually means that each predicted gene is only related to one or two of the query genes

  38. Predicting Gene Function • This is a problem in the human genome, where our prior knowledge is relatively limited • The ratio algorithm generates predictions that are targeted towards the commonalities of the query:

More Related