1 / 6

RDA Data Fabric (DF) Interest Group Peter Wittenburg & Gary Berg-Cross

RDA Data Fabric (DF) Interest Group Peter Wittenburg & Gary Berg-Cross. DF IG Goals

chancellor-dale
Télécharger la présentation

RDA Data Fabric (DF) Interest Group Peter Wittenburg & Gary Berg-Cross

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RDA Data Fabric (DF) Interest Group Peter Wittenburg & Gary Berg-Cross • DF IG Goals • The goal is to develop alternative views, components and aspects of the DF concept and related infrastructure. Conceptualizations be discussed to come to an agreed RDA view on how the evolving DF landscape can be productively described. • As part of this, essential DF components and their interrelation need to be identified and defined. • Some of the existing RDA groups including metadata WGs and IGs are working on DF components and need to be positioned in such a landscape. • New working groups need to be defined to work on identified components and interfaces. Introduction to Group Coming to a reproducible data science is a high priority. Only highly automated self-documenting procedures respecting proper data organization principles will overcome current barriers. The Data Fabric group needs to work out directions in the belief that integrated data fabrics are a critical component of infrastructures paving the way to reproducible science. The idea for this IG emerged from the discussions amongst the chairs of various RDA WGs Characteristics of a Data Fabric: We are just beginning to scout out thelandscape of data fabric . In one view it is a minimalistic set of infrastructure and service requirements by which services can plug into (belong to) the defined  fabric. In a data fabric we see how the separate components, developed separately, can be made to work together, this means that for different sets of components the data fabric will be different. We note, strongly, that it is meant as a descriptive/conceptual way to deal with the interrelation between many components, rather than prescriptive (like you would have with an architecture). This diagram provides a high-level view of possible actions within a Data Fabric running from raw data to increasingly documented data that has been enriched and analyzed creating referable and citable data. As shown publications are part of this Data Fabric since they are often used for data mining and other analysis.

  2. Data Fabric IGDFOrganizing Chairs: Gary Berg-Cross & Peter Wittenburg

  3. Data Foundation and TerminologyDFTChairs: Gary Berg-Cross, Raphael Ritz, Peter Wittenburg

  4. Some early statements on Data Fabric & Components • Infrastructure Component View • (after Reagan Moore) • A data fabric is the set of software and hardware infrastructure components that are used to manage data, information, and knowledge. • When an enterprise implements a data management solution, one of multiple types of DFs infrastructure is typically chosen to enable the: • Data management –enterprise to build a data repository, manage an information catalog, & enforce management policies • Data analysis –enterprise to process a data collection, apply analysis tools, and automate a processing pipeline. • Data preservation –enterprise to build reference collections and knowledge bases that comprise the intellectual capital, while managing technology evolution • Data publication –discovery and access of data collections. • Data sharing – controlled sharing of a data collection, shared analysis workflows, and information catalogs. Data Fabric Service View (after Beth Plale) A DF should: Be self-documenting – a service contributes to the lifecycle of data objects it handles and must keep track of the scientifically relevant actions it performs on those data objects.   The resulting log files are periodically be sent to a provenance consolidator.  Track data objects through its service processing using one of the well-known object identifier schemes Identify itself as one type of service as drawn from an RDA- agreed upon list of service types.  Implement an interface to a publish-subscribe system which serves as the Data Fabric Control mechanism. • Data Object View • (after Peter Wittenburg) • The data fabric covers a domain of registered digital objects (DO) that are stored in well managed repositories. • DOs are associated with metadata describing its creation context and history (provenance). • The Data Fabric covers a domain of registered software components (workflows, services) that are in fact a special class of DOs. • Actions on DOs may be guided by abstract policies that are explicit and thus auditable. • There can be multiple data fabric implementations that should be highly interoperable.

  5. People and Plans Who The DF group is a joint initiative of the first 5 working groups who understood that all what they started needs to be seen as part of a bigger plan full of dynamics. They also realized that the work which they started needs to be maintained to meet new requirements. Others interested in the goal of improving our data creation machine is welcome to participate in the DF group. Initiating Members Rebecca Koskela, Keith Jefferey, Jane Greenberg, Reagan Moore, Rainer Stotzka, Tim Delauro, Tobias Weigel, Raphael Ritz, Gary Berg-Cross, Peter Wittenburg, DaanBroeder, Larry Lannom, Beth Plale, Juan Bicarregui, Herman Stehouwer • The suggested Data Fabric IG is planned as a forum to discuss these alternative views, components and aspects of the DF concept. • To be discussed: • What is the agreed RDA view on a Data Fabric. • How the outputs from the RDA working groups fit in the DF concept and how they relate to each other and to various related WGs and IGs within the RDA. • Which further activities are required to push the data fabric concept ahead. • Continuation and initialization of working group activities related to the DF. • Improving the uptake of the WG outputs by communicating them as a coherent whole within the DF concept. Status & Planned Outcomes The DF group will first work on an initial position paper to characterize the landscape of components and interfaces that have the potential to realize a reproducible data science. Such a landscape will change over years where components will be replaced by others requiring that the position paper will need to be amended frequently. The DF group will be a continuous source for identifying new requirements and barriers that need to be addressed by new RDA groups, in particular working groups. Updating the results will be an interesting but worthwhile challenge. Prepared for the 4th RDA Plenary in Amsterdam, Sept. 2014,

More Related