A Data Management Life-Cycle

1. A Data Management Life-Cycle By David Ferderer Project Chief Chris Skinner Contractor Greg Gunther Contractor dferdere@usgs.gov

2. Presentation Outline • USGS Landscape • Life-Cycle Model and Strategy • Component Descriptions (Skinner) • Demonstration (Gunther) • Conclusions and Future Directions

3. USGS Landscape - Energy Program • What We Do • Provides Science-Based Energy Assessments • Organization Issues • Regional Centers and Competitive Funding Process • Multiple Project Areas, Applications, Data Types, and Platforms • Information Issues • Technology and Data Explosion • Access, Delivery, and Archive Requirements • Diverse Client and Product Needs • Policy and Mandates

4. USGS Landscape - Central Energy Team • 125 Full and Part-Time Employees • Independent Thinkers and Researchers • Multiple Application Platforms • UNIX (ArcInfo 8, ArcView 3x, SDE 3, ORACLE 8, EarthVision, Seismic, PETROMOD) • PC/NT (ArcInfo 8, ArcView 3x, Geographix) • Centralized and Distributed Data Storage • 100mb Fast Ethernet Network

5. Central Energy Team “Information” Shift Information Services Data Management Project Life-Cycle GIS Integration

6. Life-Cycle Model and Strategy • Life-Cycle Model (Conceptual) • A Series of Processes and Utilities that Manage the Flow of Data to Information, Products, and Knowledge • Life-Cycle Implementation Strategy (Actual) • Processes are Translated into the Find, Get, Use, Deliver, and Maintain Strategy • Strategy Defines Tasks, Components, and Deliverables

7. Implementation Strategy • DM Finds Internal and External Data Resources • DM Gets the Data Organized, Documented, and Accessible to Team Projects • Projects Use the Data and Other Resources in Research • DM Assists Projects in Delivering Products to Public • DM Maintains the System and Upgrades Components

8. Strategy Components and Utilities (Internal USGS) Team Data Library Archive Library Inventory Database Metadata Utilities CD-ROM Templates Data Processing Utilities Hypermedia Publications Project Design Intranet Resources Find External Data and Information Get Data Organized Find Internal Data and Information (Archive and Reuse) Data Life-Cycle Maintain (Upgrades and Documentation) Deliver Data and Knowledge to Projects and the Public Use Data and Other Resources In Research Projects

9. Team Data Library Team Data Library • Centralized Storage • Team Data Resources (primarily spatial) • Theme and Sub-Theme Organization • Standardized • Naming Conventions • Directory Structure • Storage Formats (e00, shape, SDE) • Common Data Projection (geographic) • Metadata • Browse Graphics

10. Archive Library Team Archive Library • Offline Storage of Team Data Resources • Contains • Publications • USGS Digital Data Products (DLG, DEM, DOQ) • Team Archives • Standardized File Names and Directory Structure

11. Inventory Database Inventory Database • MS Access Database Tracking Team’s Data Holdings • Contains • 60 Information Fields (10 Required) in 21 Tables • 28 Fields Corresponding to FGDC Metadata Elements • Inventoried 4600 Datasets and 680 Archives (> 500 GB)

12. Inventory Database Inventory Database • Features • Tracks Multiple Types of Data (Spatial, Text, Graphic and Tabular) • Separately Tracks Archives, Publications, and Individual datasets • Automatic Loading and Editing Scripts • Serves as the Engine to DART…

13. DART • Data Access, Retrieval, and Tracking System • Easy Access to Team Data Resources via Web Browsers • Customized Search and Browse of Archives, Publications, and Datasets • Direct Data and Metadata Download to User’s Desktop • Object-Oriented Application • Java Server Pages on ServeletExec 3.1 • Stay Tuned for the Demonstration!

14. Metadata Utilities Metadata Utilities • Web-Based Metadata Entry and Creation System • Users Generate, Modify, and Save Compliant Metadata Output to the Desktop • Provides a Simplified and Comprehensive Online Help System • Contains • Links to Other Metadata Tools and Resources • Library of Metadata

15. Hypermedia Publications CD-ROM Templates Data Processing Utilities Project Design Other Data Management Products • Data Processing and Automation Utilities • Portal to ‘How-To’, AMLs, and FAQ Documents Residing in the Team and On the WWW • Project and Workspace Design Recommendations • Templates Promote Efficient Work-Flow, Data Organization, Archives, and Rapid Publication • CD-ROM Templates and Hypermedia Distribution

16. Maintenance • DM Provides Continual Maintenance and Upgrades of System Components • Develop Publications and Documentation • User Manuals • Formal Component Documentation • Templates, Guidelines, and Policies • Fact Sheets and Bulletins

17. Demonstration Greg Gunther

18. System Summary • Easy Access to Datasets • Generate Metadata Quickly and Easily • Find External Data with Over 1000 WWW Links • Simplify Data Processing Tasks • Organizes Projects with Workspace Templates • Streamlines CDROM Publications • Provides One-Stop Shopping For Shared Internal Resources

19. Future Directions • Increase Inventory Effort • Integrate GeoDatabase Model (ArcGIS) for Proprietary Datasets • Formalize Metadata Extension to FDGC Standard • Streamline Product Delivery - Implement IMS • Publish Documented Tools and Utilities • Implement Enterprise Architecture and Planning

20. Future Architecture Enterprise Planning* Getting Started Planning & Initiatives Current Systems BusinessProcesses Where We Are Today Data Architecture IS/IT Architecture GIS &Application Architecture Where We Want To Be Implementation and Migration Plans Plan To The Future *Modified from Spewak Model

21. Conclusions – What We Have Learned Data Management: • It’s ESSENTIAL for Survival But Needs to be Promoted • Distributed Projects REQUIRE Data Centralization • Projects RARELY Account for Data Management Planning and Costs • Data Stewardship MUST Begin at the Onset of Projects • The Terms EASY and USEFUL - Lead to Implementation • Component Model Must be FLEXIBLE to Adapt to Technology Trends

22. The End And The Beginning Of a New Cycle…