MDSplus
MDSplus. Tom Fredian MIT Plasma Science and Fusion Center. What is MDSplus?. Data Acquisition System Remote Data Access System Data Visualization and Analysis System Data available via fortan, c, c++, java, idl,matlab, visual basic, labview, php, python Data Archival System. History.
MDSplus
E N D
Presentation Transcript
MDSplus Tom Fredian MIT Plasma Science and Fusion Center
What is MDSplus? • Data Acquisition System • Remote Data Access System • Data Visualization and Analysis System • Data available via fortan, c, c++, java, idl,matlab, visual basic, labview, php, python • Data Archival System
History • Designed and developed for use on Alcator C-Mod, RFX and ZTH between 1987-1991 to run on OpenVMS platform • Remote data access from wide variety of platforms added 1994 • Ported to Unix/Windows 1999 • Added Globus Security Infrastructure 2002 • Based on X.509 identity certificates
MDSplus Worldwide • MDSplus currently installed at over 30 experiments, fusion laboratories and universities. • Most widely used tool for accessing fusion related data. • Over 3000 downloads of the MDSplus software.
MDSplus • Data Acquisition • MDSplus typically used in “pulse” mode • DAQ Device support developed as needed • Remote data access • Simple efficient data access on the LAN • Secure authenticated data access on the WAN • Visualization Tools • dwscope & jScope • IDL, MATLAB, python, PHP interfaces (and more)
Data Storage • The information about each pulse is stored in a hierarchical structure. • Diagnostic / Data analysis setup • Machine Control • Task scheduling • Raw and Analyzed results • Calibration information • Data longevity • All of the data from a diagnostic is stored in an organized way. • This provides a mechanism for the data from a diagnostic to be self descriptive. • One set of tools can be used to operate on both data and setup information. • Concurrent multi-user read and write
Traverser Icons for Node usage Expandable Node Right button popup menu Command Window
Complex Data types • In addition to a wide selection of primitive data types (byte, word, long, float, double, complex, etc...) MDSPlus supports data types to store commonly used combinations of items. • Signal (scaled data, raw data, dimension_0 [,dimension_1 …]) • Subscripting both arrays and signals • With_units(item, units) • Range - start : end : increment • Action(dispatch, task) • Dispatch(1, ident, phase, when, notify) • etc…
Homogeneous Data Access Interface • MdsConnect,host[:port] • MdsOpen, tree, shot • var = MdsValue(expression [,args…]) • MdsPut, name, expression [,args…] • MdsClose [, tree, shot] • MdsDisconnect
Remote Access - details • Types of client/server connections • MdsConnect “thin client” • One server process per client connection • Server activated by inetd • Expression evaluation done in server process ctx • TreeOpen “thick client” • One server process per client per tree host • Server activated by inetd • Expression evaluation done in client process ctx
MDSplus “Data Gateway” • MDSplus is sometimes used as a gateway to other data stores. • Expression evaluation can call into local shared libraries to retrieve or put data.
Secure WAN Access • MDSplus network protocol (MDSIP) layered on top of Globus GSI using XIO • X509 Credentials used to authenticate both client and server. • Centralized management of access control via ROAM (Resource Oriented Authentication Manager)
ROAM used to solve authorization problem • Resource Oriented Authorization Manager (ROAM) provides a central location for authorization information • Administrators & other stakeholders can control access • Users can request access
ROAM data model is fundamental to a coherent picture of authorization in the grid • Oriented around resources: codes, databases, entire sites • If you have to sign a paper to get permission to use something, then it’s probably a resource • Resources have associated permissions • e.g. “execute” for a code, “access” for a site
People interact with ROAM through a web page • An interactive web page allows users and resource owners to request and grant permissions • Less error-prone than editing text files • No more grid-mapfiles! • Can take the place of mdsip.hosts for MDSplus
Potential use of MDSplus at FNAL • Transfer of CERN control data to FNAL • Push or pull data? • Client application running at CERN gathering data and writing it securely to MDSplus data storage at FNAL • MDSplus data server running at CERN with access to control data accessible securely via client at FNAL • How much data? How often? • MDSplus is currently “pulse” based. Not designed for “trending” data (yet). • Extensions for long pulse/continuous data to be explored either via the “Collaborative Technology” SCIDAC proposal or ITER projects. • What type/structure of data? How will it be accessed?
Other topics (if time) • Electronic Logbook (quick demo) • MDSplus in action (C-Mod) • ~2GB/shot (uncompressed) 15 minute rep-rate • 82,000 tree nodes in each data set (“directories” and “files”) • Expressions examples for gateways
Summary • MDSplus used at most fusion research sites. • Useful for handling some types of HEP data? • Next Steps? • http://www.mdsplus.org/ • http://www.fusiongrid.org/ • twf@psfc.mit.edu