Metadata-rich Service Oriented Grid Portals

1. Metadata-rich Service Oriented Grid Portals Mehmet Nacar mnacar@cs.indiana.edu

2. Outline • Research Objectives • Background • Grid portals • Research issues • Architecture • Contributions Mehmet Nacar

3. Research Objectives I • Improve ease of Grid portal development • We will research problems in federating multiple portal instances • Integrating portals based on the same code (e.g. all GridSphere) • Integrating portals with different code bases • Integrating portals with non-portal services • identity management • authorization Mehmet Nacar

4. Research Objectives II • We will investigate problems turning portals into Grid Web services • It is the right architecture especially for Grid services • WSRP is the Web Service standard • WSRP must be coupled to other Web service standards for addressing and reliability • This will require an examination and implementation of SOAP handlers Mehmet Nacar

5. What is a portal? • Web based application provides • Personalization, single sign on and content aggregation • Web, science and educational portal • Yahoo, Google • CIMA, VLAB and LEAD • OneStart and Oncourse • Portal vs. portal framework • Jetspeed, GridSphere and Sakai frameworks • Modern portal framework vs. traditional portals • Jetspeed, GridSphere and uPortal • CGI-based portals, ReciprocalNet Mehmet Nacar

6. Yahoo portal Mehmet Nacar

7. Portal page Mehmet Nacar

8. Portlets • What is a portlet? • Building blocks of portals • Portlet examples • Calendar, chat, file transfer, weather and news. • We have developed Grid portlets for OGCE • Myproxy, job submission and GridFtp. • Portlet container • Provides lifecycle management of portlets • Defines window states and portlet modes • Equips packaging and deployment • JSR 168 specification defines • Portlet container • Portlet API Mehmet Nacar

9. Portal framework • Portal framework aggregates portlet contents • Portal services are portal capabilities • Portal services are evaluated in different aspects: • Administrative services • Account management, portlet management • User management services • User profile, layout management • There are also related portlets • Administrative portlets are account creation, portlet registry • User portlets are file transfer and calendar • Portal front page displays anonymous content with login portlet • What services involves when a user logs into the portal • Login service  Authorization Service  Layout Service Mehmet Nacar

10. Problem statement • Portals and portlets are standardized but this opens new problem areas, particularly for Grid portals • Federating multiple portal instances built from JSR 168 compatible frameworks. • We must decouple portal “capabilities” from the container and make them into interoperable Web services. • Examples include identity management, authorization and logging. • Portlets and portals themselves can become Web services that can be federated. • But we must show these can be combined with other Web service registration, discovery, reliability, and others. Mehmet Nacar

11. Grid portals • What makes a portal Grid-enabled? • Portlets provide access to Grid services • Portal server leverage Grid credentials to access Grid services • Provide capabilities/portlets • GridFTP, job submission and job monitoring • What tools and services are available to build Grid portals? • Java COG: Grid client programming API • MyProxy: credential management service • OGCE: collection of portlets and tools for building portlets • GAMA and PURSe: integrated Grid account management • We will examine two motivating examples • VLAB: traditional job submission portal for computational chemistry • CIMA: scientific instrument and data portal Mehmet Nacar

12. VLAB Portal Mehmet Nacar

13. VLAB: The Virtual Laboratory for Earth and Planetary Materials • Primarily a traditional job submission, monitoring, and management portal. • Collaborative Grid services and portals support computational material science. • Component based Grid portlet development makes application development easier. • VLAB Challenges: • Grid Portlets must be easy to develop using component libraries. • HTML <form> actions in Grid portals typically have several steps: • Stage data files in and out of the desired remote host. • Run one or more executables. • Keep track of job progress • Store all of the information as “job pedigree” for reproducibility. Mehmet Nacar

14. Research issue: Grid components • Grid portlet applications require dynamic interaction and fine-grained components. • Portlets themselves need to be built out of components • Grid services mostly use request/response paradigm • Grid portlets use web forms heavily • Compared to wikis, blogs, RSS-driven news portals, which have a different problem of content management. • Grid widgets can provide components for: • Proxy credential • GridFtp operations • Job submissions, multi-staged jobs • Using widgets as tag libraries help to encapsulate reusable Grid components Mehmet Nacar

15. Grid Tag Libraries • Grid tag libraries use the Java COG Kit abstractions • Grid tags simplify association of composite Grid actions • And increase reuse of code • There are associated custom JSF tag extensions we’ve developed: • <o:proxy/>, <o:task/>, <o:taskGraph/>, <o:taskAdd/> and <o:contextStore/> • Taskgraph supports multi-staged jobs <f:verbatim> <o:taskGraph id="myGraph" method="#{taskgraph.test}" binding="#{taskgraph.taskgr}" > <o:proxy id=“myproxy" method=“#{proxy.create}" /> <o:task id="task1" method="#{task.create}" type="FileTransferInput" /> <o:task id="task2" method="#{task.create}" type="JobSubmit" /> <o:task id="task3" method="#{task.create}" type="FileTransferOutput" /> <o:taskAdd id="taskadd1" name="task1" depends="task2" method="taskgraph.add" /> <o:taskAdd id="taskadd1" name="task2" depends="task3" method="taskgraph.add" /> </o:taskGraph> </f:verbatim> Mehmet Nacar

16. CIMA Crystallography portal • CIMA picture snapshot Mehmet Nacar

17. CIMA (Common Instrument Middleware Architecture) • Primarily a data portal to online instruments • Crystallographers collect data in participating laboratories and collaborate on samples. • Portlets have to access data with group privileges. Mehmet Nacar

18. Research issue: federating portal instances • CIMA and LSST Dark Energy Survey portal presents this problem: • Multiple CIMA portal deployment instances are associated with different labs (IUMSC, Purdue, University of Sydney etc) • Need to solve problem of portal identity federation. • Recall identity management is key portal capability. • Other problems (such as federated authorization) flow from this. • Portal frameworks like GridSphere, uPortal and Jetspeed only support local “capabilities”. This approach has drawbacks: • Portal services tightly coupled to portal container and cannot being reused. • Portal frameworks have proprietary services. • Authorization capabilities of the portals define access rights for portlets not for contents • These “capabilities” should be full-fledged Web services • This architecture greatly simplifies the federation problem Mehmet Nacar

19. Federating multiple portals I • Requirements • Browser-based login is required for portals • Each site maintains its own portal server • Single sign on across the portal servers • Federation can be done providing: • Several portals have the same code base but run different instances • Integrating portals with different code bases • Integrating portals with non-portal services, which have identity and authorization that is external to the portal framework. Mehmet Nacar

20. Federating multiple portals II Mehmet Nacar

21. Identity service • Identity service issues security tokes for users • Identity service support interfaces including: • Issuing identity/security tokens • Authenticating security tokens for redirection • Interfaces for administrative tasks including: • Adding, removing, editing users Mehmet Nacar

22. Federating multiple portals III Mehmet Nacar

23. Integrating portals with services • Mediator is a Web service • Administration of Web services through portal Mehmet Nacar

24. How WSRP fits into this architecture? • Handles Grid job management problems • WSRP provides Grid Web services Mehmet Nacar

25. Implications • Provides standard way for portals to use external services • Coordination of portal services is handled outside of portal framework • More services can be added dynamically • Service interfaces are extensible • New features can be added to services • Enables lightweight portal frameworks (minimum local capabilities) • Migration of portlets and services between portal frameworks will be easier • Administration of Web services through portal Mehmet Nacar

26. Does this architecture scale well? • There will be some overhead • Local capabilities vs. services • In long term portal servers will be overloaded • The architecture turns out to be advantageous • Adaptable to for new features • Simplifies migration across portal frameworks • Services can be replicated • Identity service could run on several locations Mehmet Nacar

27. Milestones • Implementation of identity, logging and authorization services using WS technologies. • Design of mediator service • Mediator needs to coordinate services • This will require the investigation of SOAP handlers and their proper pipelining. • Managing SOAP header processors rather than services. • Integration of reliable and secure WSRP requires two handlers • WS-RM reliability handler. • WS-Security handler Mehmet Nacar

28. Contributions of this thesis • Simplification of portlet development for Grid applications through reusable libraries. • Portlets are “coarse-grained” components. • Tag libraries for portlets are “fine-grained” components. • Design, development and validation of service-oriented portal architecture • Extends the current field • Limited by close coupling of portal core services to specific frameworks • Our approach provides a natural way to federate multiple portal instances and to integrate portal services with other Grid services. • We will examine the extremes of portal scalability in this approach. • This framework will be a demanding test of Web Service standards. • Requires integration of several Web services standards • Integration of several standards: registration, security, reliability and others. • SOAP header processing is done by intermediaries • Intermediary coordination is similar to but distinct from Web Service workflow standards such as BPEL. Mehmet Nacar