1 / 25

Internet 2 Distributed Storage Infrastructure (I2-DSI)

Internet 2 Distributed Storage Infrastructure (I2-DSI). TERENA-NORDUnet Networking Conference 1999 7-10 June, Lund, Sweden Leif Abrahamsson, Ellemtel leif.abrahamsson@ellemtel.se. Re-create leading edge R&E network capability Enable new generation of applications

feryal
Télécharger la présentation

Internet 2 Distributed Storage Infrastructure (I2-DSI)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Internet2 Distributed Storage Infrastructure (I2-DSI) TERENA-NORDUnet Networking Conference 1999 7-10 June, Lund, Sweden Leif Abrahamsson, Ellemtel leif.abrahamsson@ellemtel.se

  2. Re-create leading edge R&E network capability Enable new generation of applications Integrate with production environment UCAID/Internet2 Internet2 Goals

  3. IPv6 Measurement Multicast Network Management Network Storage Internet2 Working Groups • Quality of Service • Routing • Security • Topology

  4. Internet2 Distributed Storage Infrastructure Update Micah Beck Univ. of Tennessee, Knoxville Bert Dempsey Univ. of North Carolina, Chapel Hill http://dsi.internet2.edu

  5. UT Knoxville / ICL M. Beck T. MooreM. Swany J. Talley UNC Chapel Hill /SILS B. Dempsey Paul. JonesD. Weiss Z. Xiao Site Managers NC Supercomputing Center Indiana University EROS Data Center Univ. of Hawaii at Manoa Ellemtel L. Abrahamsson C. Achouiantz P. Johansson IBM R Aiken G. Carpenter G. Goldszmidt J. Warren Sun Microsystems R. Chawla (Healtheon) Viagenie (with CANARIE) M. Blanchet F. Parent I2-DSI Technical Participants

  6. A Word From Our Sponsors • Cisco DNS redirection • Ellemtel engineering effort • IBM large storage & DCE servers • Novell storage & directory servers • Starburst reliable multicast software • StorageTek large storage servers • Sun design collaboration

  7. Single Server Model • High performance locally • Unacceptable performance across commodity backbone

  8. Relying on Wide Area QoS • High performance access with reserved bandwidth • Essential for real-time communication • Technically difficult, expensive, not generally available

  9. I2-DSI Model: Replicated Services • Clients access nearby server • Everyone gets performance • Local resources implement a global service

  10. replicated core delegated server local users I2-DSI Service Architecture • Replication • Rsync+, Omnicast, AFS/DFSNovell Replication • Resolution • Sonar DNS, Distributed Director • Delegation • Cache prefetch general users

  11. Internet Content Channels • A channel is a collection of content which can be transparently delivered to end user communities at a chosen (price,performance) point through a flexible, policy-based application of resources

  12. Server Channel Examples • Replicated Web Servers • APIs: Standard HTML, Active Server Pages • Channels: Web sites • Streaming Media • APIs: MPEG-2, proprietary file formats • Channels: collections of multimedia presentations • Executable content • APIs: Java byte code, Tcl, Perl • Channels: CGI programs

  13. Current Server Deployment

  14. I2-DSI Server IBM RS/6000 AIX Server 2 GB RAM 72 GB Disk / 900 GB Tape ADSM Hierarchical Storage Mgt.

  15. I2-DSI Server Operations • Project Operations Coordinator • Judi Talley, University of Tennessee at Knoxville • Site Managers • Dave Vernon, Indiana University • David Lassner, University of Hawaii at Manoa • Mark Johnson, NC Networking Initiative • Michael Rechtenbaugh, EROS Data Center

  16. Infrastructure Expansion • StorageTek • 2 PC/Linux Servers • 700GB disk, tape backup • Novell • 6 PC/NetWare Servers • 100GB disk • Smaller institutions or departments

  17. InfrastructureDevelopment Efforts • Proximity Resolution • Martin Swany: SonarDNS • Geoff Carpenter, German Goldszmidt: Narwhal (IBM) • Replication Mechanisms and Modeling • Bert Dempsey & students: • Debra Weiss: Batch rsync + multicast • Zhiwei Xiao: Network metrics and modeling • Channel Representation and Server • Leif Abrahamsson, Christophe Achouiantz, Patrik Johansson (Ellemtel)

  18. 10 applications Indiana: Digital music and media library UNC-CH: Instructional Management System San Jose State: Art history images Vanderbilt: zoomable medical images Viagenie: Network docs database Columbia: Earth sciences environment UNC-CH: Virtual Laboratories Ohio Supercomputer Center: High Volume Datasets CalTech: Globally Interconnected Databases Univ. of Kent: National Software Archive Red Hat: pan-Linux source distribution I2-DSI Applications WorkshopChapel Hill, NCMarch 4 &5, 1999

  19. I2-DSI Applications WorkshopChapel Hill, NCMarch 4 &5, 1999 • 4 technologies • Minnesota: Scalable Video • IBM Research: Multicast, Filter and Store • Moscow Ctr. for New Info. Tech. in Med. Ed.: Semantic Text Analysis • IBM Research: Narwhal Resolution Proxy • http://dsi.internet2.edu/apps99.html • Special issue of the Journal of Network and Computer Applications (Academic Press)

  20. Application Management Partner MetaLab.unc.edu • The site formerly known as SunSITE.unc.edu • Fearless Leader Paul Jones • “A cool, tall glass of sweet tea on a hot day.” • 2 M HTTP & 1/3 M FTP file transfers daily • Collections policy • teaching, research, or public service • use technology in innovative and unique ways • non-commercial or not-for-profit

  21. Application Strategy • Chose initial applications • Available or easily ported services • Low update demands • Port to an I2-DSI server • Our development effort is limited • App developers can have access to the servers • Distribute to homogeneous core • Derive service abstractions

  22. The Need for Channel Representation Standards locally interpreted files replicated files Origin Server Replicated Server Replicated Server proxy Web clients Standard-based Web traffic Replication of source files

  23. Replication Performance and Scalability Issues • Server placement • Server resources • Server description (metadata) • Server Channel description (metadata) • Object representation • Characterization of replication mechanisms • Channel-to-server mapping (subscription)

  24. Network Storage Symposium (NetStore ‘99) • Network Storage Symposium • Seattle, Oct 14-15, 1999 http://dsi.internet2.edu/netstore99 • Scope (look for CFP) • I2-DSI implementation • I2-DSI applications • Related networking projects • Storage technology

  25. Conclusions • A server platform is in place • Infrastructure development • Service abstractions (search, computation) • Publication and replication protocols • Portable representation and API • Heterogeneous servers • Six months to show results from initial application development efforts

More Related