1 / 19

Potential Errors in GO annotations in MGI Database

Potential Errors in GO annotations in MGI Database. Carson Andorf, Drena Dobbs, Vasant Honavar Artificial Intelligence Research Laboratory Department of Computer Science Bioinformatics and Computational Biology Program Center for Computational Intelligence, Learning, & Discovery

bryce
Télécharger la présentation

Potential Errors in GO annotations in MGI Database

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Potential Errors in GO annotations in MGI Database Carson Andorf, Drena Dobbs, Vasant Honavar Artificial Intelligence Research Laboratory Department of Computer Science Bioinformatics and Computational Biology Program Center for Computational Intelligence, Learning, & Discovery Iowa State University honavar@cs.iastate.edu Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  2. Summary • Nearly 95 percent of the GO annotations returned by AmiGO for a set of mouse protein kinases are inconsistent with the UniProt annotations of their human homologs • A large fraction of these inconsistent annotations have the RCA evidence code • Further investigation suggests that the annotations returned by AmiGO might be incorrect Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  3. Background • Automated methods for protein function annotation • Allow high-throughput annotation of thousands of sequences • Increase the risk of error propagation • Potential sources of errors - noisy training data - error in the algorithm - mislabeling of data - poor general performance by classifiers - a simple clerical error - human errors Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  4. Data set: Human and Mouse Protein Kinases GO:0003674 : molecular_function ( 121801 ) - GO:0003824 : catalytic activity ( 41632 ) - GO:0016740 : transferase activity ( 13210 ) - GO:0016301 : kinase activity ( 5613 ) - GO:0004672 : protein kinase activity ( 3415 ) - GO:0004674 : protein serine/threonine kinase activity( 2077 ) - GO:0004713 : protein-tyrosine kinase activity( 771 ) Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  5. Protein Kinases • Protein Kinases are among the most well-studied proteins • Protein serine/threonine phosphorylation regulates virtually every signaling pathway in the eukaryotic cell • Tyrosine phosphorylation is modulates key biological events associated with development and disease • cancer, diabetes, and inflammation • Accurate annotation of protein kinases is extremely important Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  6. Method • Machine learning algorithm HDTree • (Andorf et al., in press) • Combines classifiers based on • k-gram amino acid composition of proteins and • homology (PSI-BLAST) Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  7. Experiment I • Data: 244 mouse and 330 human protein sequences • GO families GO0004674 (Serine/Threonine Kinase) and • GO0004713 (Tyrosine Kinase) • Use annotations returned by AmiGO as reference labels (without filtering based on evidence codes) • 71 mouse and 233 human proteins are labeled with GO0004674 • 106 mouse and 90 human proteins are labeled with GO0004713 • 67 mouse and 7 human proteins had both labels • Train classifier on human data and test on human data • Train classifier on human data and test on mouse data Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  8. Initial results • Classifier trained on human data and tested on human data (10-fold cross validation) had excellent performance • 89.1% accuracy with a 0.85 correlation coefficient • Classifier trained on human data and tested on mouse data had terrible performance • 15.1% accuracy and a -0.42 correlation coefficient • Why? Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  9. First observation: Discrepancy in Distribution Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  10. Second Observation: Evidence codes • 211 of the 244 mouse protein kinases had a RCA (inferred from reviewed computational analysis) evidence code • Of the 33 mouse proteins that did not have a RCA evidence code, 28 proteins were classified correctly by the classifier trained on human proteins (85%) • Question: What is special about the 211 mouse proteins with GO function labels with RCA evidence code? Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  11. Third Observation: Source of RCA annotations • Annotations returned by AmiGO came from from the Mouse Genome Informatics Database (MGI) • The MGI annotations came from the Fantom2 (Functional Annotation of Mouse) Database • Each of the 211 mouse proteins had at least one RCA from FANTOM Consortium and the RIKEN Genome Exploration Research Group (Okazaki et al, Nature, 420, 563-573, 2002) Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  12. Fourth Observation: Inconsistency between UniProt and AmiGO RCA • AmiGO RCA annotations for 201 of the 211 mouse proteins were inconsistent with UniProt annotations Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  13. Fifth Observation: Distribution of Annotations Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  14. Experiment II • Train classifier using Human proteins and test on mouse proteins • Use UniProt labels instead of AmiGO labels as reference • Accuracy on mouse proteins: 97%. • 205 of the 211 proteins that were mislabeled with respect to AmiGO reference labels were correctly labeled with respect to UniProt reference labels • A search of the Mouse Kinome Database shows that the majority of the mouse kinases have a human ortholog with sequence similarity greater then 90% (154 of the 244 proteins we used) Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  15. Sixth Observation Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  16. Tentative Conclusions • There is no reason to expect that the relative distribution of the Ser/Thr kinsases and Tyr kinases in human and mouse would be dissimilar • The annotations returned by AmiGO for the 211 mouse protein kinases (nearly 95% of the 244 mouse protein kinases) deserve a closer examination Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  17. Postscript I • To the best of our knowledge, the problematic annotations with RCA evidence code originated in Okazaki et al. (Okazaki et al, Nature, 420, 563-573, 2002) • They were propagated to • MGI through the Fantom2 (Functional Annotation of Mouse) Database and • From MGI to AmiGO • Question: How far did these annotations propagate? Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  18. Postscript II 136 rat protein kinase annotations retrieved using AmiGO • Had ISS (inferred based on sequence or structural similarity) evidence code • Had functions assigned based on one of the 201 potentially incorrectly annotated mouse proteins • 94 Ser/Thr kinase proteins mislabeled as either a Tyr kinase or dual specific • 42 Tyr kinase proteins assigned as a Ser/Thr kinase or a dual specific 337 mouse and rat proteins annotations are probably incorrect Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

  19. Conclusions • If our analysis is correct, it would mean that we have serendipitously discovered potential errors in annotations • Others (e.g., Dolan et al, Bioinformatics 21, 136-143, 2005) have noted the need for annotation consistency checks • Our discovery further underscores the need for better procedures for • Multiple checks for consistency of annotations – especially in the case of annotations with RCA and ISS evidence codes • Better methods for tracking propagation of annotations across databases Research Supported in part by NIH GM066387 Vasant Honavar, 2006.

More Related