The ‘Zen’ of Web Services

1. The ‘Zen’ of Web Services David Gristwood davidgri@microsoft.com blogs.msdn.com/David_Gristwood

2. Agenda • Core Principles • Messaging • Metadata • Interoperability • WS-* Standards

3. 1 - Introduction • Web has been a phenomenal success at enabling simple computer/human interactions • Key factor in the success of HTTP and HTML is its relative simplicity • Web services take many ideas and principles of Web and apply them to computer/computer interactions

4. Web services protocol stack • HTTP/HTML "read-mostly" • Content often static/highly cacheable • Web services architecture designed for highly dynamic program-to-program interactions • Many kinds of distributed systems may be implemented • Synchronous & asynchronous messaging systems, distributed computational clusters, mobile-networked systems, grid systems, peer-to-peer environments… • Web services protocol stack to be much more general purpose

5. Core Principles • Vision: new generation of applications based on autonomous web services • Core principles of web services architecture: • Message orientation • Protocol composability • Autonomous services • Managed transparency • Protocol-based integration

6. a) Message orientation • Web services communicate using messages • Place significant emphasis on how individual messages formed/processed • Unlike DCOM/(O)RPC where this is largely invisible • Messages are atomic unit of communication • Can choose to surface as RPC style developer model

7. (Message orientation) • Assumes SOAP lowest layer in protocol stack • Isolates message transfer from transport details • Ideally, protocol-specific bindings do not leak into application semantics • De-couple historical HTTP offerings • HTTP strict request/reply doesn’t lend itself well to exchange patterns that are easy in TCP such as full duplex, etc

8. (Message orientation) • Need to span multiple network nodes • E.g. access checks, content-based routing, or application-specific validation • Message may need to traverse multiple message transports • SMPT, HTTP, TCP, MSMQ, etc • Security needed beyond single point to point protocols / singe trust domain • WS-Security vs SSL

9. b) Protocol Composability • Protocols compose when they can be used either independently or combination • Many domain-specific protocols are effectively “silos” • Protocol designers find themselves coining new mechanisms for dealing with security, reliability, etc • E.g. DCE, CORBA, etc

10. (Protocol Composability) • Protocol suite designed as family of composable protocols • Each defines a fine-grained unit of functionality, e.g. signing and sealing • SOAP's architecture anticipates composition via header mechanism • No cost to applications that do not use specific protocol • New protocols can be introduced at any time to complement existing ones

11. c) Autonomous Services • Web services are autonomous agents • development, deployment, operation, management, and security all vary independently to service's consumer • “Forced independence" has ramifications that permeate the architecture • Extensibility • New versions of SOAP not needed • Security - trust and federation • Control over resources managed • Recycle when not needed, e.g. lease lifetimes, subscriptions, etc

12. (Autonomous Services) • Versioning • SOAP provides a protocol evolution model based on headers • Headers are expected to be added or removed over the lifetime of a given protocol • New headers carry upgrade policy • Headers that cannot be safely ignored are annotated with a mustUnderstand attribute

13. d) Managed Transparency • All implementation details are private to a service • Message-oriented façade provides insulation • Opacity is critical to service autonomy • Requires publicly visible aspects transparent to the outside world • Machine-readable contracts • Essential for creating rich ecosystem for tools & execution environments

14. e) Protocol-Based Integration • Assume no shared OS, no shared virtual machine, no shared programming language or abstraction • Reflects increasing emphasis on SOA • Evolution of component software, distributed objects, and message-oriented middleware

15. 2 - Messaging • Web services rely on: • XML for basic underlying data model • SOAP for message processing / data model • WS-Addressing for addressing services • Independent of transport

16. Discovery UDDI Description WSDL Packaging SOAP Application Data XML Protocol Internet Application Core Web Service Stack

17. XML and the Infoset • Common understanding of exactly what constitutes a message required • XML document defined by XML Information Set (‘Infoset’) • Abstract data model • Information items maps to elements, attributes, etc, • Foundation of all XML specifications • XML Schema, XML Query, XSLT

18. SOAP • SOAP designed to reduce the engineering cost • lowest-cost technology has the best chance of gaining universal acceptance • A SOAP message is an XML document information item that contains three elements: • <Envelope> • <Header> • <Body> • Separate application specific payload from protocol information

19. SOAP POST /StockQuote HTTP/1.1 Host: www.stockquoteserver.com Content-Type: text/xml; charset="utf-8" Content-Length: nnnn SOAPAction: "Some-URI" <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/> <SOAP-ENV:Header> <t:Transaction xmlns:t="some-URI" SOAP-ENV:mustUnderstand="1"> 5 </t:Transaction> </SOAP-ENV:Header> <SOAP-ENV:Body> <m:GetLastTradePrice xmlns:m="Some-URI"> <symbol>DEF</symbol> </m:GetLastTradePrice> </SOAP-ENV:Body> </SOAP-ENV:Envelope>

20. XML and Binary Data • Infoset encoding allows traditional text and opaque binary data • W3C XML-binary Optimized Packaging (XOP) format uses multi-part MIME to allow raw binary data in XML 1.0 document • SOAP Message Transmission Optimization Method (MTOM) specifies how to bind this format to SOAP • XOP and MTOM are the preferred approach for mixing raw binary with text-based XML • MTOM became W3C Proposed Recommendation Nov 2004

21. XOP <soap:Envelope xmlns:soap='http://www.w3.org/2003/05/soap-envelope' … <soap:Body> <m:data xmlns:m='http://example.org/stuff'> <m:photo xmlmime:contentType='image/png'>/aWKKapGGy= </m:photo> </m:data> MIME-Version: 1.0 Content-Type: Multipart/Related;boundary=MIME_boundary; … --MIME_boundary Content-Type: application/xop+xml; charset=UTF-8; type="application/soap+xml; action=\"ProcessData\"“ … <soap:Envelope <soap:Body> <m:data xmlns:m='http://example.org/stuff'> <m:photo xmlmime:contentType='image/png'><xop:Include xmlns:xop='http://www.w3.org/2004/08/xop/include' href='cid:http://example.org/me.png'/> </m:photo> … --MIME_boundary Content-Type: image/png Content-Transfer-Encoding: binary // binary octets

22. Message Exchange Patterns • SOAP supports various message exchange patterns as required by apps • Synchronous request/response • Most popular, RPC style • Asynchronous request/response • Explicit message correlation becomes mandatory • One-to-many broadcast • Publish/subscribe

23. XML XML Publisher-SubscriberAsynchronous Messaging Subscriber Publisher Connection is ended Event occurs Publisher sends a message to the Subscriber Subscriber registers with Publisher

24. WS-Addressing • Needed for messaging carried across multiple transports • WS-Addressing defines three sets of SOAP header blocks for this purpose • Action header block: processing of a message • MessageID/RelatesTo: identify/correlate messages • To/ReplyTo/FaultTo: identify agents to process the message and its replies • Endpoints key aspect of WS-Addressing • Finer-grained addressing than just a URI • Comprise base address & optional sets of reference properties & reference parameters

25. WS-Addressing <S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing"> <S:Header> <wsa:MessageID> uuid:6B29FC40-CA47-1067-B31D-00DD010662DA </wsa:MessageID> <wsa:ReplyTo> <wsa:Address>http://business456.example/client1</wsa:Address> </wsa:ReplyTo> <wsa:To>http://fabrikam123.example/Purchasing</wsa:To> <wsa:Action>http://fabrikam123.example/SubmitPO</wsa:Action> </S:Header> <S:Body> ... </S:Body> </S:Envelope>

26. 3 - Metadata • Metadata enables interoperability • Describes message interchange formats the service can support • Describes valid message exchange patterns of a service • Describes capabilities and requirements of a service • Concept of “Service Policy”

27. WSDL • Web Service Description Language (WSDL) first widely adopted mechanism • Describes basic characteristics of web service • Basic message patterns • Abstract contract for a service • Ports and bindings to concrete transports and physical deployment information • WSDL file can specify base elements required for a program to interact with a web service • Several tools available to read WSDL file and generate the code required

28. WSDL <message name='quoteRequest'> <part name='body' element='quote-schema-ns:stockName'/> </message> <messagename='quoteResponse'> <part name='body' element='quote-schema-ns:stockPrice'/> </message> <portType name='quotePortType'> <operation name='getQuote'> <input message='quote-wsdl-ns:quoteRequest'/> <output message='quote-wsdl-ns:quoteResponse'/> </operation> </portType> <binding name='quoteBinding' type='quote-wsdl-ns:quotePortType'> <operation name='getQuote'> <soap:operation soapAction="http://example.com/stockQuoteAction"/> <input> <soap:body part='body' use='literal'/> </input> <output> <soap:body part='body' use='literal'/> </output> </operation> </binding> <service name='stockService'> <port name='stockPort' binding='quote-wsdl-ns:quoteBinding'> <soap:address location='http://example.com/quotes/'/> </port> </service>

29. Beyond WSDL • WSDL not sufficient to describe all aspects of a web service. E.g. • Operational characteristics: The service is available only between 9 a.m. and 5 p.m. • Security characteristics: Kerberos tickets are required for access to the service • First generation Web services must exchange metadata out of band using proprietary protocols • This issue addressed by WS-Policy

30. 4 - Interoperability • The main promise of Web services • Problems occur through ambiguity, choice, misinterpretation, tool support, etc • Needs to be a managed process • Testing, conformance, etc • Main body is Web Services Interoperability Organization (WS-I)

31. Formed February 2002 Basic Profile 1.1 August, 2004 Basic Profile 1.0 August 2003 Draft Basic Security Profile August 2004 Testing tools available WS-I : The Industry is Aligned

32. Interoperability Profiles • Profile is a set of guidelines beyond the core protocols • Necessary because of specification's general-purpose design • Resolve ambiguities to ensure all implementations process SOAP messages in same way • The WS-I Basic Profile • Basic Profile 1.0 first profile published by Web Services-Interoperability Organization (WS-I) • Interoperable use of SOAP 1.1 & WSDL 1.0

33. WS-* Specifications Timeline http://msdn.microsoft.com/webservices/understanding/specs

34. WS-* Specification Process Specification Published Feedback and Interop Workshops Revise spec Standards Org WS-I • Comprehensive end-to-end approach • Process reconciles conflicting goals • Quality of engineering • Time to market • Breadth of industry support

35. Web Services Architecture Applications & Application Infrastructure Connected Applications BusinessProcess … Management Security Reliability Transactions Metadata Foundation Messaging XML … HTTP TCP SMTP Transports

36. WS-Discovery • Web service discovery is key enabler for automating connections to services • i.e. without human intervention • Mirrors two most common approaches to finding information in a computer system: • Looking in a well-known location - Universal Description, Discovery, and Integration protocol (UDDI) • Web services explicitly announce their arrival and departure

37. WS-ReliableMessaging • Many conditions may interrupt exchange of messages • Especially HTTP • Multiple transport-layer connections • Messages may be lost, duplicated or reordered • WS-ReliableMessaging enables reliable delivery based on delivery assurance characteristics: • At-Least-Once Delivery • Each message is delivered at least one time • At-Most-Once Delivery • Duplicate messages will not be delivered • In-Order Delivery • Messages are delivered in the same order they were sent

38. WS-Coordination • N-way coordination protocols require a designated coordinator • “shepherd” a unit of work through a number of cooperating services • E.g. services not all connected at same time • WS-Coordination specification defines coordination framework to support scenarios where explicit coordinators are required • Coordination Context SOAP header block uniquely identify piece of joint work to be undertaken

39. WS-AtomicTransaction • Specifies traditional ACID transactions for Web services • Within context of atomic transaction coordination type • Completion protocol, and two variants of a Two-Phase Commit protocol • Prepare, Commit and Abort • Volatile 2PC and Durable 2PC

40. WS-Eventing • Event notification alternative to polling • Publish/Subscribe model • Defines operations an event source provides that allow subscriptions to be created / managed • Flexible in way topics are defined, organized, & discovered • Expiration time for subscription to manage lease • Renewal and unsubscribe operations • Source can terminate subscription any time

41. WS-Management • Discover/Navigate presence of management resources • Settings can be set, containers enumerated, etc • Scale from small devices to data centres / distributed system • Access via endpoint of agent that provides access to resource • Key used to define specific resource • Supports notification • Batched, Pull and Trap

42. Security • Security has to be built based on explicit agreed-upon mechanisms / assumptions • Yet flexible enough to support wide variety of security policies required by different organizations • Must go beyond point-to-point protocols • Otherwise, how to trust intermediaries? • End-to-end message security is richer alternative to point-to-point transport-level security • supports the loosely coupled, federated, multi transport, etc • No new security mechanisms were invented to satisfy the needs of SOAP-based messaging • Existing approaches to distributed systems security have proved sufficient

43. WS-Security • WS-Security specifies mechanisms for • message integrity / confidentiality • single message authentication • Specifies how a cryptographic signature is represented with specific parts of message • allows arbitrary well-formed fragments of the message to have separate signatures • Confidentiality is achieved through the encryption of well-formed fragments of the message • Authentication achieved using digital signatures

44. WS-Trust • Complements WS-Security with protocols for requesting, issuing and brokering security tokens • Define operations to acquire, issue, renew, validate security tokens • System administrators determine initial trust relationships • E.g. a given service is a trusted root service

45. WS-SecureConversation • Some mechanisms for message authentication / confidentiality are computationally expensive • Okay for one off message, but for exchange of many messages more efficient mechanism needed • WS-SecureConversation defines a security context between two communicating parties • based on shared secrets, such as symmetric encryption • security context is shared for the lifetime of a session

46. WS-SecurityPolicy • Complements WS-Security by specifying security policy assertions in a language conformant to WS-Policy • Six assertions relate to • security tokens, message integrity, message confidentiality, message visibility to SOAP intermediaries, constraints on the security header, age of a message • E.g. a policy assertion may require that all messages be signed using public keys from a given authority

47. WS-Federation • Identities are valid within a trust domain but most likely meaningless in other trust domains • mechanisms needed to validate identities • WS-Federation defines mechanisms to enable identity, account, attribute, authentication, and authorization information sharing across trust domains • extends the WS-Trust model to allow attributes and pseudonyms to be integrated into the token issuance mechanism • E.g. Joe may be known at work as joe@companya.com, but also have other identities, such as joe_bloggs@hotmail.com and josb@cornell.edu - federation systems can determine that each of these identities is the same Joe

48. Resources • An Introduction to the Web Services Architecture and Its Specifications msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwebsrv/html/introWSA.asp

49. Questions?