1 / 27

Monthly Program Update January 12, 2012 Andrew J. Buckler, MS Principal Investigator

Monthly Program Update January 12, 2012 Andrew J. Buckler, MS Principal Investigator. With Funding Support provided by National Institute of Standards and Technology. Agenda. Monthly snapshot in Jira (including status of installation at NIST) QIBA 3A project snapshot

malana
Télécharger la présentation

Monthly Program Update January 12, 2012 Andrew J. Buckler, MS Principal Investigator

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Monthly Program Update January 12, 2012 Andrew J. Buckler, MS Principal Investigator With Funding Support provided by National Institute of Standards and Technology

  2. Agenda • Monthly snapshot in Jira • (including status of installation at NIST) • QIBA 3A project snapshot • Theoretical development • Architecture and SW stack 2

  3. BSD-2 license • Domain is • www.qi-bench.org. • Landing page provides • Access to prototypes, • Repositories for download and development, • Acknowledgements, • Jiraissue tracking, and • Documentation Go to the site and work With the apps and Jira 3

  4. QIBA 3A Project Snapshot(recalling that this is a testbed for us) 4 4

  5. Primary Basic structure of the challenges Secondary Investigation n Investigation Investigation Investigation 1 • First one: • Presently in pilot phase, • Using StudyDescription method • Used batch scripting with reference method to aid data curation • 10-12 participants (about 20 QI-Bench users) • First participant data received • Analysis plan using N-way ANOVA in R started • Pivotal phase starting with batch assisted curation • Will be transitioning to database schema for metadata (gradually away from spreadsheet) Pilot Pilot Pilot Pilot • Defined set of data • Defined challenge • Defined test set policy Pivotal Pivotal Pivotal Pivotal Train Train Train Train Test Test Test Test 5 5 5

  6. Relative performance is computed according to descriptive statistics • We determine a group value for each of the descriptive statistics, e.g., as the mean plus 1 stdev (or as wide as we think wise). • Results presented using radar plots 6 6

  7. In this example, the new proposed method does not perform well enough to be considered a valid method since it falls outside the group values. 7 7

  8. In this example, the new proposed method is seen to perform within group values and may even help pave the way for an improved claim. 8 8

  9. Theoretical Developmentprogress re: utilization of logical and statistical inference at each of two levels, technical performance of assay methods, and qualification of biomarker in specific clinical context 9 9

  10. Another way to look at what needs to happen RDF Triple Store obtained_by CT Volumetry CT used_for measure_of Therapeutic Efficacy Tumor growth Specify Execute Formulate Feedback Reference Data Sets Analyze QIBO Feedback Y=β0..n+β1(QIB)+β2T+ eij 10 10 10

  11. Specify: Establish a logical specification and setup terms for mathematical analysis RDF Triple Store • Functionality: • Establish means to semantically labeling imaging biomarker data with emphasis on representing both the clinical context in which an imaging biomarker is used as well as the specifics of the imaging protocol used to acquire the images. • Set up the logistic regression model: • Precisely specify dependant variable • Account for covariates • Enumerate independent variables and error terms (sources of variability) • Establish database for collection of terms. • Method: • Provide GUI to traverse the QIBO concepts according to their relationships and create statements represented as RDF triples and stored in an RDF store. • Each set of RDF triples will be stored as a “profile.” • Relationship strength initialized based on prior estimates (if available) obtained_by CT Volumetry CT used_for measure_of Therapeutic Efficacy Tumor growth Specify QIBO 11

  12. Ontologies supporting Specify • Extend the QIBO to link to existing established ontologies • leverage BFO upper ontology to align different ontologies • convert portions of BRIDG and LSDAM to ontology models in OWL • Automated conversion would done in two steps: • convert current Sparx Enterprise Architect XMI EMF UML format • export resulting EMF UML into a RDF/OWL representation using TopBraid Composer 12

  13. Formulate: advanced query framework made possible by Specify RDF Triple Store • allow users to select the profiles (or set of RDF triplets) created in Specify, execute a query and retrieve the results in various forms. • assemble/transform the set of RDF triples to SPARQL queries: • form an uninterrupted chain linking the instance of the input class from the ontology to the desired output class • formulate/invoke necessary SPARQL queries against the web services deployed in SADI framework. • interface with the query engine and will have offline (asynchronous) query execution capability. • results to be exportable as serialized objects (RDF/XML and CSV) obtained_by CT Volumetry CT used_for measure_of Therapeutic Efficacy Tumor growth Formulate Reference Data Sets 13

  14. Data Services supporting Formulate • wrap existing data services such as NBIA, caArray, caTissue, AIM and PODS using Semantic Automated Discovery and Integration (SADI) • this is enabled by metadata available through the UML representations of the models exposed by these services and CDE annotations available for them through caDSR. • describe service I/O semantically using the extended version of QIBO • service registry of SADI will help the automated composition of computer-interpretable queries by the query engine. • example: “there is a service that returns Biological Subjects that has undergone certain Biological Interventions” 14

  15. Analyze: Use annotation and image markup to support statistical inference • Support Clinical Performance assessment (i.e., in addition to current Technical Performance) • Outcome studies • Integrated genomic/proteomic correlation studies • Group studies for biomarker qualification • (set up a basic multiple regression analysis, e.g.) Intent to treat analysis of the primary outcome via covariance model of the general form (QIBt)=β0..n+β1(QIB0)+β2T+ eijwhere QIBt and QIB0 are the QIB at a time after treatment and at randomization respectively, T is a treatment group indicator, and β0..n, β1, and β2 are model parameters. β2 represents the effect of treatment and its estimate is the difference between group means on the log scale, after adjustment for any imbalance between the groups in log QIB. The error terms in the model, eij, are assumed mutually independent and normally distributed. Depending on the nature of the QIB, the log transformation may be used instead of the direct value. Likewise calculations may be performed using z scores with corresponding conversion with raw values. 15

  16. Examples of output at biomarker (above the assay level) To inform thresholding Hypothetical model of dynamic biomarkers of the Alzheimer's pathological cascade Jack et al., 2010 To substantiate surrogacy (or its weaker form of “activity”) From Jack 1999. Note: W-score is the relative score of the measured HC volume corrected for intracranial volume and compared to age and sex adjusted normals. 16 16 16 16

  17. Architecture and SW StackSo what is a cohesive architecture that maximizes leverage of best thinking, existing touchpoints, and stays current over time? 17 17

  18. caB2B, NBIA, PODS data elements, DICOM query tools. STDM standard of CDISC into repositories like FDA’s Janus. MVT portion of AVT, re-useable library of R scripts. QIBO, AIM, RadLex/ Snomed/ NCIt; built using Ruby on Rails. MIDAS, BatchMake, Condor Grid; built using Zend on PHP. 18

  19. MVT: Reasonable framework, but many gaps • There are multiple possibilities to deploy it as a web application, some of which we’ve considered: • Re-implement the existing implementation to use GWT in place of Swing, inclusive of both the XIPHost as well as MVT components, retaining the WG23 concept. • Re-implement only those parts necessary to perform the needed MVT functions using GWT with enough data handling to do so but without doing everything necessary to retain the full XIPHost capability. • Leverage the GUI design concept but otherwise implement without starting from the Swing code. • In all cases, there is the secondary design alternative of introducing a RESTful web service layer explicitly or not. • (By the way, just for fun, I performed a conversion of the current Swing code to Ajax using AjaxSwing.  I got most of AVT working over the web with minimal effort, but this isn’t a serious alternative because AjaxSwing has a license fee.  I did it because I wanted to see how easy such a path would be.  It’s an interesting capability! But irrelevant in the end.) 19

  20. XIP XIP Application Client access MIDDLEWARE XIP IDE Service access AIM RadLex Inventor Application Modules WG 23 System Services PLUG NCI WG 23 System Services SOCKET VTK ITK AIMTK other CaBIG DICOM SERVICES (DCMTK) OTHER SERVICES GRID CLIENT SERVICES EVS Protégé DICOM XIP App HW WG23 Service Host Grid Data Service Grid Analytical Service NCIA AIM Data Service OS IVI Middleware caGrid DICOM Image Sources caDSR, EVS, RadLex, AIM ontology, etc DICOM Services Pros: optimized for DICOM, works with workstationsCons: hard to create web apps, not optimized for semantic web 20 20 20

  21. Alternative architectural form… • SW Stack • J2SE (J2EE compliant) • MySQL • caGrid • Globus • Application: • JBoss • caCore With pros and cons “opposite” that of the XIP based architecture 21 21 21

  22. XIP XIP Application Client access MIDDLEWARE XIP IDE Service access AIM RadLex Inventor Application Modules WG 23 System Services PLUG NCI WG 23 System Services SOCKET VTK ITK AIMTK other CaBIG DICOM SERVICES (DCMTK) OTHER SERVICES GRID CLIENT SERVICES EVS Protégé DICOM XIP App HW WG23 Service Host Grid Data Service Grid Analytical Service NCIA AIM Data Service OS IVI Middleware caGrid DICOM Image Sources caDSR, EVS, RadLex, AIM ontology, etc DICOM Services Functionality view annotated with architecture MIDAS Client When annotation and markup has already been done MIDAS C++ API MIDAS Web API MIDAS Visualization MIDAS Compute Server Annotation and markup Reference data sets MIDAS e-journal Publication DB MIDAS Data Server MIDAS Core Apache PostGreSQL File System RIS worklist items AIM-enabled (e.g., ClearCanvas) workstation DICOM Q/R 22 22 22

  23. OS First step to rationalizing architecture: mash them together and see what falls out XIP Application XIP IDE XIP Client access MIDDLEWARE Inventor Application Modules WG 23 System Services PLUG Service access AIM WG 23 System Services SOCKET VTK ITK AIMTK other RadLex DICOM SERVICES (DCMTK) OTHER SERVICES GRID CLIENT SERVICES NCI EVS Protégé CaBIG XIP App DICOM HW WG23 Service Host Grid Data Service Grid Analytical Service NCIA AIM Data Service This is an ongoing discussion. More to come! IVI Middleware SADI framework (e.g., wrapped caGrid) DICOM Image Sources caDSR, EVS, RadLex, AIM ontology, etc DICOM Services 23 23 23

  24. 24

  25. Value proposition of QI-Bench • Efficiently collect and exploit evidence establishing standards for optimized quantitative imaging: • Users want confidence in the read-outs • Pharma wants to use them as endpoints • Device/SW companies want to market products that produce them without huge costs • Public wants to trust the decisions that they contribute to • By providing a verification framework to develop precompetitive specifications and support test harnesses to curate and utilize reference data • Doing so as an accessible and open resource facilitates collaboration among diverse stakeholders 25

  26. Summary:QI-Bench Contributions • We make it practical to increase the magnitude of data for increased statistical significance. • We provide practical means to grapple with massive data sets. • We address the problem of efficient use of resources to assess limits of generalizability. • We make formal specification accessible to diverse groups of experts that are not skilled or interested in knowledge engineering. • We map both medical as well as technical domain expertise into representations well suited to emerging capabilities of the semantic web. • We enable a mechanism to assess compliance with standards or requirements within specific contexts for use. • We take a “toolbox” approach to statistical analysis. • We provide the capability in a manner which is accessible to varying levels of collaborative models, from individual companies or institutions to larger consortia or public-private partnerships to fully open public access. 26

  27. QI-BenchStructure / Acknowledgements • Prime: BBMSC (Andrew Buckler, Gary Wernsing, Mike Sperling, Matt Ouellette) • Co-Investigators • Kitware (Rick Avila, Patrick Reynolds, JulienJomier, Mike Grauer) • Stanford (David Paik, Tiffany Ting Liu) • Financial support as well as technical content: NIST (Mary Brady, Alden Dima, Guillaume Radde) • Collaborators / Colleagues / Idea Contributors • FDA (Nick Petrick, Marios Gavrielides) • UCLA (Grace Kim) • UMD (Eliot Siegel, Joe Chen, Ganesh Saiprasad) • VUmc (Otto Hoekstra) • Northwestern (Pat Mongkolwat) • Georgetown (Baris Suzek) • Industry • Pharma: Novartis (Stefan Baumann), Merck (Richard Baumgartner) • Device/Software: Definiens (Maria Athelogou), Claron Technologies (Ingmar Bitter) • Coordinating Programs • RSNA QIBA (e.g., Dan Sullivan, Binsheng Zhao) • Under consideration: CTMM TraIT (Andre Dekker, JeroenBelien) 27

More Related