Hydrologic Data and Modeling: Towards Hydrologic Information Science

1. Hydrologic Data and Modeling: Towards Hydrologic Information Science David R. Maidment Center for Research in Water Resources University of Texas at Austin

2. Hydrologic Data and Modeling • New knowledge in hydrology • Hydrologic data • Hydrologic modeling • Hydrologic information systems

3. Hydrologic Data and Modeling • New knowledge in hydrology • Hydrologic data • Hydrologic modeling • Hydrologic information systems

4. By deduction from existing knowledge By experiment in a laboratory By observation of the natural environment How is new knowledge discovered? After completing the Handbook of Hydrology in 1993, I asked myself the question: how is new knowledge discovered in hydrology? I concluded:

5. Deduction is the classical path of mathematical physics Given a set of axioms Then by a logical process Derive a new principle or equation In hydrology, the St Venant equations for open channel flow and Richard’s equation for unsaturated flow in soils were derived in this way. Deduction – Isaac Newton Three laws of motion and law of gravitation http://en.wikipedia.org/wiki/Isaac_Newton (1687)

6. Experiment is the classical path of laboratory science – a simplified view of the natural world is replicated under controlled conditions In hydrology, Darcy’s law for flow in a porous medium was found this way. Experiment – Louis Pasteur Pasteur showed that microorganisms cause disease & discovered vaccination Foundations of scientific medicine http://en.wikipedia.org/wiki/Louis_Pasteur

7. Observation – direct viewing and characterization of patterns and phenomena in the natural environment In hydrology, Horton discovered stream scaling laws by interpretation of stream maps Observation – Charles Darwin Published Nov 24, 1859 Most accessible book of great scientific imagination ever written

8. Conclusion for Hydrology • Deduction and experiment are important, but hydrology is primarily an observational science • discharge, water quality, groundwater, measurement data collected to support this.

9. Scientific progress occurs continuously, but there are great eras of synthesis – many developments happening at once that fuse into knowledge and fundamentally change the science Great Eras of Synthesis 2020 Hydrology (synthesis of water observations leads to knowledge synthesis) 2000 1980 Geology (observations of seafloor magnetism lead to plate tectonics) 1960 1940 1920 Physics (relativity, structure of the atom, quantum mechanics) 1900

10. Hydrologic Science It is as important to represent hydrologic environments precisely with data as it is to represent hydrologic processes with equations Physical laws and principles (Mass, momentum, energy, chemistry) Hydrologic Process Science (Equations, simulation models, prediction) Hydrologic conditions (Fluxes, flows, concentrations) Hydrologic Information Science (Observations, data models, visualization Hydrologic environment (Physical earth)

11. Hydrologic Data and Modeling • New knowledge in hydrology • Hydrologic data • Hydrologic modeling • Hydrologic information systems

12. CUAHSI Member Institutions 122 Universities as of July 2008 (and CSIRO!)

13. HIS Team and Collaborators • University of Texas at Austin – David Maidment, Tim Whiteaker, Ernest To, Bryan Enslein, Kate Marney • San Diego Supercomputer Center – Ilya Zaslavsky, David Valentine, Tom Whitenack • Utah State University – David Tarboton, Jeff Horsburgh, Kim Schreuders, Justin Berger • Drexel University – Michael Piasecki, Yoori Choi • University of South Carolina – Jon Goodall, Tony Castronova • CUAHSI Program Office – Rick Hooper, David Kirschtel, Conrad Matiuk • National Science Foundation Grant EAR-0413265

14. HIS Goals • Data Access– providing better access to a large volume of high quality hydrologic data; • Hydrologic Observatories– storing and synthesizing hydrologic data for a region; • Hydrologic Science– providing a stronger hydrologic information infrastructure; • Hydrologic Education– bringing more hydrologic data into the classroom.

15. HIS Overview Report • Summarizes the conceptual framework, methodology, and application tools for HIS version 1.1 • Shows how to develop and publish a CUAHSI Water Data Service • Available at: http://his.cuahsi.org/documents/HISOverview.pdf

16. Water Data Water quantity and quality Soil water Rainfall & Snow Modeling Meteorology Remote sensing

17. Water Data Web Sites

18. HTML as a Web Language Text and Pictures in Web Browser HyperText Markup Language <head> <meta http-equiv="content-type" content="text/html; charset=utf-8" /> <title>Vermont EPSCoR</title> <link rel="stylesheet" href="epscor.css" type="text/css" media="all" /> <!-- <script type='text/javascript' language='javascript‘ src='Presets.inc.php'>--> </head>

19. WaterML as a Web Language Discharge of the San Marcos River at Luling, TX June 28 - July 18, 2002 Streamflow data in WaterML language

20. Services-Oriented Architecture for Water Data • Links geographically distributed information servers through internet • Web Services Description Language (WSDL from W3C) • We designed WaterMLas a web services language for water data • Functions for computer to computer interaction HIS Servers in the WATERS Network HIS Central at San Diego Supercomputer Center Web Services

21. Get Data WaterML HIS Central National Water Metadata Catalog Get Metadata

22. CUAHSI Point Observation Data Services • Data Loading • Put data into the CUAHSI Observations Data Model • Data Publishing • Provide web services access to the data • Data Indexing • Summarize the data in a centralized cataloging system

23. CUAHSI Point Observation Data Services • Data Loading • Put data into the CUAHSI Observations Data Model • Data Publishing • Provide web services access to the data • Data Indexing • Summarize the data in a centralized cataloging system

24. Data Values – indexed by “What-where-when” Time, T t “When” A data value vi (s,t) “Where” s Space, S Vi “What” Variables, V

25. Data Values Table Time, T t vi (s,t) s Space, S Vi Variables, V

26. Observations Data Model Horsburgh, J. S., D. G. Tarboton, D. R. Maidment and I. Zaslavsky, (2008), "A Relational Model for Environmental and Water Resources Data," Water Resour. Res., 44: W05406, doi:10.1029/2007WR006392.

27. HIS Implementation in WATERS Network Information System National Hydrologic Information Server San Diego Supercomputer Center • 11 WATERS Network test bed projects • 16 ODM instances (some test beds have more than one ODM instance) • Data from 1246 sites, of these, 167 sites are operated by WATERS investigators

28. CUAHSI Point Observation Data Services • Data Loading • Put data into the CUAHSI Observations Data Model • Data Publishing • Provide web services access to the data • Data Indexing • Summarize the data in a centralized cataloging system

29. Point Observations Information Model Utah State Univ Data Source Little Bear River Network GetSites Little Bear River at Mendon Rd Sites GetSiteInfo GetVariableInfo Dissolved Oxygen Variables GetValues 9.78 mg/L, 1 October 2007, 5PM Values {Value, Time, Metadata} • A data source operates an observation network • A network is a set of observation sites • A site is a point location where one or more variables are measured • A variable is a property describing the flow or quality of water • A value is an observation of a variable at a particular time • A metadata quantity provides additional information about the value

30. Publishing an ODM Water Data Service Texas A&M Corpus Christi Utah State University University of Florida Assemble Data From Different Sources ODM Data Loader Ingest data using ODM Data Loader WaterML Load Newly Formatted Data into ODM Tables in MS SQL/Server Observations Data Model (ODM) USU ODM UFL ODM TAMUCC ODM Wrap ODM with WaterML Web Services for Online Publication

31. Publishing a Hybrid Water Data Service Snotel Metadata are Transferred to the ODM WaterML Snotel DataValues Snotel METADATA ODM Web Services can both Query the ODM for Metadata and use a Web Scraper for Data Values Snotel Water Data Service Get Values from: Metadata From: ODM Database in San Diego, CA Snotel Web Site in Portland, OR http://river.sdsc.edu/snotel/cuahsi_1_0.asmx?WSDL Calling the WSDL Returns Metadata and Data Values as if from the same Database

32. Locations Variable Codes Date Ranges WaterML and WaterOneFlow Penn State Data GetSiteInfo GetVariableInfo GetValues Utah State Data NWIS WaterML Data WaterOneFlow Web Service Data Repositories Client EXTRACT TRANSFORM LOAD WaterML is an XML language for communicating water data WaterOneFlow is a set of web services based on WaterML

33. Set of query functions Returns data in WaterML WaterOneFlow

34. CUAHSI Point Observation Data Services • Data Loading • Put data into the CUAHSI Observations Data Model • Data Publishing • Provide web services access to the data • Data Indexing • Summarize the data in a centralized cataloging system

35. Data Series – Metadata description Time End Date Time, t2 There are C measurements of Variable Vi at Site Sj from time t1 to time t2 Count, C Begin Date Time, t1 Site, Sj Space Variable, Vi Variables

36. Series Catalog Time Sj End Date Time, t2 Vi Count, C Begin Date Time, t1 Site, Sj Space Variable, Vi Variables t1 t2 C

37. Texas Hydrologic Information System Sponsored by the Texas Water Development Board and using CUAHSI technology for state and local data sources (using state funding)

39. CUAHSI National Water Metadata Catalog • Indexes: • 50 observation networks • 1.75 million sites • 8.38 million time series • 342 million data values NWIS STORET TCEQ

40. request return return request NAWQA request return return request NAM-12 request return NWIS request return request return return request NARR Data Searching • Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them Searching each data source separately Michael Piasecki Drexel University

41. NAWQA NWIS NARR HODM Semantic Mediation Searching all data sources collectively GetValues GetValues GetValues GetValues generic request GetValues GetValues Michael Piasecki Drexel University GetValues GetValues

42. Hydroseekhttp://www.hydroseek.org Bora Beran, Drexel Supports search by location and type of data across multiple observation networks including NWIS and Storet

43. HydroTagger Ontology: A hierarchy of concepts Each Variable in your data is connected to a corresponding Concept

44. Data Sources NASA Storet Snotel Unidata NCDC Extract Academic NWIS Transform CUAHSI Web Services Excel Visual Basic ArcGIS Java Load Matlab Applications http://www.cuahsi.org/his/ Operational services

45. HydroExcel

46. HydroGET: An ArcGIS Web Service Client http://his.cuahsi.org/hydroget.html

47. Direct analysis from your favorite analysis environment. e.g. Matlab % create NWIS Class and an instance of the class createClassFromWsdl('http://water.sdsc.edu/wateroneflow/NWIS/DailyValues.asmx?WSDL'); WS = NWISDailyValues; % GetValues to get the data siteid='NWIS:02087500'; bdate='2002-09-30T00:00:00'; edate='2006-10-16T00:00:00'; variable='NWIS:00060'; valuesxml=GetValues(WS,siteid,variable,bdate,edate,'');

48. Synthesis and communication of the nation’s water data http://his.cuahsi.org Government Water Data Academic Water Data National Water Metadata Catalog Hydroseek WaterML

49. Hydrologic Data and Modeling • New knowledge in hydrology • Hydrologic data • Hydrologic modeling • Hydrologic information systems

50. Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive • Software standards for model linking • Uses model core as an “engine” • http://www.openMI.org