Integrating Instruments and Sensors into Grid Computing for Enhanced Real-Time Data Processing

1. Instruments and Sensors as Grid Services Donald F. (Rick) McMullen1 Kenneth Chiu2 John C. Huffman1 Kia Huffman1 Randall Bramley1 1Indiana University 2SUNY Binghamton

2. Motivations • Instruments and sensors are not well integrated into grids • Data is acquired, processed and stored before it “hits the grid”. • Need methodology for interacting with instruments and sensors in real time from grid applications • Some abstraction of sensor and instrument functionality is needed to make grid applications that use them more robust and flexible.

3. Goals • Integrate instruments and sensors (e.g. real-time data sources) into a Grid computing environment with Grid services interfaces. • Abstract instrument capabilities and functions to reduce data acquisition and analysis applications’ dependence on specialized knowledge about particular instruments. • Move production of metadata as close to instruments as possible and facilitate the automatic production of metadata. • Develop a standard, reusable methodology for “grid enabling” instruments. • Collaborate with scientists in academia and industry in a broad range of disciplines who either develop instruments or whose work depends on the details of using them.

4. Ground motion sensor array Electron Microscope Increasingbandwidth per sensor Traffic sensors X-Ray Crystallography Wireless ‘mote’sensor network Radio Telescope Increasing number of sensors Increasing real-time application • Simple 2-D analysis of instrument taxonomy • At lease five dimensions identified in more detailed analysis • Project must address enough points (classes) to assure breadth of applicability

5. Initial Applications • High brilliance X-ray crystallography • Large instrument application • Deeply integrated into bio and medical discovery research methodology • Mature analysis software and large user community • Robotic telescopes • Small numbers of sensors: CCD, environmental; some control aspects: filters, aiming, dome. • Global coordination needed for scheduling • Aggregation of disparate sensors into a “composite”instrument • Small sensors • Minimal memory and CPU • Wireless connectivity • Developing parallel project to use ad hoc/swarm networks for data collection for real-time simulation and prediction • Updating of data flows in response to sensor/network reconfiguration.

6. CIMA implementation targets Large scientific instruments Embedded sensorsand controllers verylargesystems, few elements verysmallsystems, manyelements PC104 industrialcontroller board Synchrotron beamline(APS/ALS) MICA Mote wireless sensor/controller board

7. MMSF Automated Telescope • Typical remote access automated observatory • System has 33 distinct “sensors”, 12 controllers • Open/close of roof based on a Polaris transparency monitor and rain detector – simple grid of wires detecting rain drops • Telescope direction and dome control • Filter selection and telescope focus • Liquid nitrogen fills of the CCD dewar jars ….

8. MMSF Observatory Features • Instruments producing data without units • Temperature, humidity cutoffs determined empirically as resistances, not degrees or % • Hierarchical and co-located instruments • Single platform holds three instruments, so orienting one changes orientation of all • Updates to equipment occurs frequently • Data transferred via 28k modem line –middleware needs to work locally, directly between instruments and sensors

9. Observatory Data Architecture • Control Data • object control • instrument control • Object Data (i.e., object of scientific interest) • Full spectrum from raw unitless data to derived data artifacts • Instrument package • system package data (multiple attributes output) • system sensor data (single attribute output) • nonsystem sensor data (weather data from NOAA) • calibration data • access protocols

10. X-Ray Crystallography Proxy Box InstrumentManager Portal InstrumentServices LAN WAN LAN DataArchive Non-grid service Grid service Persistent Non-persistent

11. LTER • Automatic updating of flows • Provenance • SOAP/ARTS (Antelope Real-Time System)

12. Further applications currently being explored • Electron Microscopy • U. Queensland EM group – regional scale, multiple instruments • - KISTI 2nd largest EM (after Osaka) Robotic telescopes - Bradford robotic telescope, Oxenhope observatory, Faulkes telescope project • Environmental monitoring • Water use • contaminants • Industrial monitoring and control, e.g. Train axles • - Ore trains – Km long, derailment is very expensive to fix • - Temperature sensors on axles monitor bearing status, anticipate wheel failure

13. Architecture

14. Common Instrument Middleware Architecture (CIMA) Elements • Schema for instrument functionality (and ontology for schema attributes); • Data model for representing instrument metrics and calibration; • A small, high performance, embeddable Web Services stack, initially in Java, including Proteus support for multi-protocol, multimodal transport; • Service implementation for accessing the instrument’s functionality and metrics via the Proteus-mediated interface; • Ability to dynamically insert new protocols into running instances • OGSA and WS-RF compliant functions to register with a location service, authenticate users, provide access control to instrument controls and data, send and receive events, and co-schedule the instrument into a Grid computing and storage context.

15. Overview

17. Example CIMA minimal knowledge bootstrap procedure CIMA instrument Application Globus init, user Proteus/SOAP calls authentication, and instrument lookup Service “ send description” Implementation (SI) returns Application parses “ RDF description” description of itself description for ports to and instrument read for calibration and voltage “ read calibration port” SI returns “ calibration” stored calibration “ read thermocouple port” SI calls controller “ thermocouple voltage” function to read Application reads and return voltage thermocouple voltage then computes and displays a temperature

18. Sensor Sensor ChannelSource ChannelSource Sensor ChannelSource ChannelSource Services • Instrument • get • set • Sensor • get • set • ChannelSource • get • set • register • ChannelSink • receive • get • set Instrument ChannelSink

19. Parcels • Wish to unify our data models, etc. • Toolkit must be application-independent as much as possible. • Attributes • Type (string) • Globally unique ID (string) • Encoding • CDATA, Binary, ASCII, Base64 • Location • Inline, URL, Other • Parcel Sets • Special data used for connectivity information.

20. Technologies

21. Technologies • Web services • XML, SOAP, WSDL, binary XML • Grid services • OGSA/OGSI, WS-RF, DAIS • Axis C++ gSOAP • Proteus (SC 2002) • XBS (HPC 2004) • Schema-specific parsing

22. Proteus Motivation • Web Services for scientific computing • SOAP performs well as a lingua franca • But suffers from performance problems for scientific data • Solution: establish initial communication with SOAP, and then switch to a faster protocol. • Grid intermediaries

23. Proteus Overview • Provides multiprotocol RMI system to applications • Can wrap existing protocol implementations with dynamic invocation • Facilitates use of SOAP as common language • Switch to faster protocols if supported by both sides. • Mediates between protocol providers and applications • Applications use Proteus client API • Providers use Proteus provider API • Allows a new provider to be added (at run-time) without changing application • Generic serialization/deserialization allows marshalling code to be reused for multiple protocols

24. Client Client Proteus Proteus ProviderA ProviderA ProviderB ProviderB Multiprotocol Process 1 Process 2 Protocol A Protocol B Network

25. Proteus APIs Application Client API Proteus Provider API Protocol A Provider Protocol B Provider

26. Schema-Specific Parsing • XML processing stages (conceptual) • Well-formedness • Lexical and syntactic, defined by core XML specification. • Validity • Conformance to a schema, mainly structural • Application

27. Application Application Application Validation Validation Validation Well-formedness Well-formedness Well-formedness Merging Stages Fully articulated Implicitly validated Schema-specific parsing User written Merged

28. Compiler-Based Approach C (fast) • Front-end parses schema into intermediate representation. • Back-end generates code from intermediate representation. • Intermediate representation is a generalized automata. XMLSchema IR Java RELAXNG C(low-pow)

29. Summary • Create standards for accessing broad spectrum instruments and sensors. • Incompatible components should still have some base level of interoperability.