New Ideas for Computing Environments

Networking Information Communication Engineering New Ideas for Computing Environments EnginFrame as a multi-purpose framework for computational and data Grids Institute of Physics - Prague - Czech Republic - 12 Dec 2002 e-mail: Alberto.Falzone@nice-italy.com info@nice-italy.com

OUTLINE • NICE Company Profile • IT Evolution and visions • Enginframe Architecture & Features • Service overview • Case Studies • Future Perspectives and Outlook • Summary and Conclusions

NICE Background • Distributed Resource Management • LSF, projects, consulting • System Administration • VENUS, consulting, support • Unix - Windows NT integration • Computing Portals • EnginFrame, projects and consulting • Web, Java, XML competence • Expertise in Citrix MetaFrame on NT & Unix

Research INFN, ENEA, ICTP, CASPUR, CILEA, CNR, Astronomical & Astrophysical Observatories Mechanical Ferrari, FIAT Avio, CRF, Comau, Iveco, ELASIS, Marelli, UTS, Teksid, Brembo, Lear Energy ABB, Ansaldo, Nuovo Pignone, ENEL, EniChem, Enterprise Oil, Agip BioTech Pharmacia, (ENEA) Aerospace Alenia, Galileo, IDS Electronics STMicroelectronics, Accent, Alcatel, Ericsson, Siemens Telecom Telecom Italia, Italtel, CSELT Education Scuola Normale Superiore di Pisa, Polit. Of Milan, Universities Partners Compaq/HP, IBM, NEC, SGI, Sun, Avaki Consulting Accenture, Elsag, ITGlobalValue NICE Customers/Partners

ITALY Main Office: Camerano Casasco ASTI Other sites (consultants): TURIN, MILAN, CATANIA Where We are

ICT evolution The two driving forces: • Semiconductors technology evolution • Moore’s Law = 2x increase every 18 months • Networking and Communication technologies • Communication speed growth is faster than Semiconductors’ evolution • Internet and World Wide Web

Pros Uniform access Easy Admin. Full Resource Allocation Control Cons High Costs Limited Scalability Fast obsolescence Single point of failure Mainframe-centric vision

Pros Costs Scalability Incremental growth Freedom for users Cons Partial utilization Limited local power Complex management Desktop-centric vision

Company Network Powerful Workstations Application Servers Dedicated Servers Linux Compute Clusters ... Network-centric vision • The Network is the Computer! • LSF (Resource Optimization & Analysis) • VENUS (System Administration) • FlowGuide (Job-flow Management) • Avaki (Data Management)

Clients Inter/Intranet Server(s) Standard Web Browser The Web-centric vision Dedicated Servers MetaFrame Globus/Grid Machines LSF Compute Farm

Powerful Workstations MetaFrame Servers Dedicated Servers Tomorrow? LSF Compute Farms Globus/Grid Machines ASP(s) on the Internet ... Computing Environment(s) Company Network Today

User Problems • Are Technical Computing environments getting more and more complex? • Do we need to be System Administrators to do our job? • Do we really need to type this command line?!? • Isn’t there an easier way to access computing resources?

System Administrator Problems • How do we manage this complexity? • How can we integrateheterogeneous Unix/Windows resources? • How can we reduce start-up costs for new applications and methodologies? • How can we cut training, ownership and maintenance costs?

Why EnginFrame? • Common problems: • Distributed Resource Managing solutions acceptance and usability • Client access: Windows-Unix integration • Policy enforcement • Collaboration with customers or partners • Data management on WAN • Pressure from Enterprise-wide projects • No expertise for Technical Computing problems among “traditional” web players

What is EnginFrame? • Inter/Intranet access to computing resources • Any-client can connect through any browser • Thin/Fat clients, appliances, PDA, Wireless, ... • High scalability & flexibility for any resource • Access an arbitrary number of HW & SW resources (clusters, stand-alone hosts, …) • Based on the existing command-line • Flexible in content presentation • Easy to use, setup and manage

EnginFrame (R)evolution 1.x - LAN Integration (1998) • Unix-Windows NT Integration • Simplified parallel and distributed computing, … • Developed in collaboration with Compaq 2.x - WAN MetaComputing (1999) • Standards convergence • Dynamic contents management 3.0 - Computing Portal Infrastructure

EnginFrame three tier model CLIENT Users “browse” the Computing resources as easily as a Web site. Standard Web browsers allow broad accessibility and very positive user experience WEB SERVER Collects Resource offers and creates HTML pages and forms to interact with. Web servers with servlet engine are becoming a standard RESOURCES Agents “publish” service offerings as XML and act on servers’ request XML can describe many different kinds of services (LSF clusters, standalone, …) WHY? HOW?

X / ICA connections XML Output HTML Rendering Browsing request Service request ASP on the Internet Computing Portal with EnginFrame Application Server MetaFrame + NFuse Clients Web Server EnginFrame GlobusAgent LSF Agent NFuse Agent ... EnginFrame Server and Repository Standard Web Browser GLOBUS Grids LSF Compute Farm

Web front-end Web Browser WebSphere App. & Data Integr. EnginFrame GUI applications Citrix ICA Client Citrix MetaFrame License Mgmt Analysis LicenseMiner* InfoMiner LicenseBroker PCC Intelligence Grid Mgmt PCC LSF/Globus System Admin S+C VENUS Data Management Avaki DataGrid / AFS Operating System Any OS UNIX, Linux UNIX, Linux, W2K Hardware PC, WS, Laptop, ... Server, Web Farm Server, WS, Rack, ... Client Web Server Resources NICEDeliverables

EnginFrame features • Built on standard technologies • Open to Grid standards (Globus, GridML) • Easily extensible to Open Grid Service Architecture (OGSA) specs • Interfaced with leading 3rd party tools • Flexible data management • Support for input/output data transfer • Support for remote access to centralized data • Native XML representation of data • Flexible security management • Support data encryption (via SSL server) • Authentication: NIS, Domain, AFS, Grid/Globus, etc.

EnginFrame service description • Describe service options, actions and information • Intuitive design • Fast and easy prototyping • Use your preferred XML editor SCREEN CAPTURE OR OTHER IMAGE

EnginFrame user experience • User logs in and provides his/her credentials • Browse through service offerings • Fills the required data form and (if needed) specify his/her input files • Submits his/her request for the job

Security • Open architecture to support different security policies • Directly supports HTTP authentication and plain-text authentication • Connection can be secured with SSL or VPN • Security plug-ins available for most common protocols (AFS, Globus GSI, Nfuse, …)

EnginFrame Benefits • Hide the complexity of Technical Computing environments • Instant migration to the Computing Portal paradigm from a command based world • Enhance resource Manageability for System and Application managers • Enable smooth NT-UNIX-appliance integration • Ease deployment of new software and methodologies • Automatic and transparent enforcement of company policies • Simplify resource Accessibility in the Virtual Workplace • Enhance collaboration with remote sites and partners • Protect your Intellectual Property • Shorten Time-To-Market/Time-To-User

Service overview • The basic building block of the portal is the Service • A Service is an XML representation of any Computing related action • a Service might be a FEM solver execution, a parallel Job based on MPI, a query to the LSF master to find my pending jobs, etc. • Services are designed focused on their users • A Service should be self-explaining • It should make life easier for the user

Service structure • The Service is basically described by • a service name • a service description or help (info) • the options requested to the user • the command (action) to be executed when the service is requested • None of these items is required • Service behavior highly depends on the presence of such items

XML terminology • Tag or Element - an item enclosed in <>, which is part of an XML document • e.g. <action>, <table> • Attribute - an option of a tag, which has a name and a value enclosed by single or double quotes • e.g. <service id=“myservice”> • Name space - an optional naming convention that groups tag related to a common context • e.g. <ef:action>, <ef:service>

Service Example <ef:service id="gzip"> <ef:name>gzip sample</ef:name> <ef:option id="level" label="Compression level" type="list"> <ef:option id="9">maximum</ef:option> <ef:option id="4">medium</ef:option> <ef:option id="0">none</ef:option> </ef:option> <ef:option id=”FILE" label="File to compress" type="file"/> <ef:action id="submit" label="Submit job"> EF_SPOOLER_NAME="gzip $file” export EF_SPOOLER_NAME ${EF_ROOT}/plugins/lsf/bin/bsub -o output.txt gzip -$level \"$FILE\” <ef:result type="text/xml"/></ef:action> </ef:service>

EnginFrame Agent EnginFrame Server XSLT Processor Service results are added Special tags are expanded Templates are applied To browser EF XML file EF Rendering pipeline • The rendering pipeline enriches and then transforms the EF XML files • Customization can be at Agent-level and/or through an XSLT style-sheet

Choosing the right layout • One very effective method is to select one existing page from your Intranet/Internet site • You need to identify a page where you can figure out the space for the service navigation bar and the service content area

Case study: Consolidation Company in the Defense sector • Different companies have merged into one • Sites and customers spread over WAN • Projects need common coordination and collaboration • No common design methodology • Duplicated licenses across different sites • Insufficient local resources • Limited communication

Solution: EnginFrame Centralized eDesign Services

Benefits • Savings in management cost and complexity • Simulation from anywhere, in any moment • No training needed • Application-independent Collaborative engineering • Project-specific cost control • Integration with project management, workflow and existing Intranet Portal (Oracle-based) • Full control over resource allocation • Unified and efficient resource administration

Chipset IP + + FirmWare Case Study: Intellectual Property • Scenario: • Accent is designing a chipset for Siemens • Siemens needs to test the firmware & software for this chipset • Problem: • Intellectual Property cannot be disclosed

Solution: EnginFrame Black-box for Firmware simulation

Secure HTTP Before & After Internet Firewall Internal users LSF Compute Farm

Benefits • Great savings in time, travel and logistic costs • Simulation from anywhere, in any moment • All existing infrastructure is left intact • no additional/dedicated HW and software • no administrative efforts to set-up a new farm • Intellectual property is fully protected • only specific files can be returned to the user • pre-coded simulation scripts • No training needed

Case Study: Time to User Full ASP set-up in 1 day!

Case Study: Data Exchange Collaboration problem in the supply chain • Many suppliers with different supported 3D models • Complex operations to convert files • Management not comfortable with uncontrolled data exchange • User friendliness

Solution: EnginFrame DataGate

Case study: a portal for DataGrid Problems: • Scientists only care about their applications and not about the technicalities to run them • They are too lazy to learn new command line interfaces • The middleware services and the ways to access them change very rapidly

The solution: EnginFrame GENIUS: The Grid Portal for DataGrid

Benefits • Independence from MiddleWare changes: • Transparent to the users • The user is guided in his/her choices • Independence from Layout changes: • The services mantain the same structure • New graphic look & feel can be easily implemented • Intuitive use

Development strategy • EnginFrame evolves at two levels • Architectural evolution • Project driven enhancements • Future technological focuses: • OGSA/WebServices • Enterprise/Science Portal integration • DB integration • B2B and workflow integration • 3D visualization integration

Roadmap • EnginFrame 3.3 - Q4 02 • Data management enhancements • Portlet integration • EnginFrame 3.5 - Q2 03 • Authorization system • Early DB integration • EnginFrame 4.0 - Q4 03 • WebServices & early B2B functions

New Ideas for Computing Environments