The Concurrency And Coordination Runtime And Decentralized Software Services Toolkit

1. The Concurrency And Coordination Runtime And Decentralized Software Services Toolkit  George Chrysanthakopoulos Software Architect Microsoft Corporation

2. Announcing CCR And DSS Toolkit 2008 • Enables developers to more easily create loosely-coupled concurrent and distributed applications • Allows early adopters to use the technologies today for building commercial and non-commercial applications • Provides early adopters access to select technologies today; transitioning to Microsoft’s .NET Framework in future

3. CCR/DSS Initially applied in robotics • Robots are complex and heavily depend on coordination between software components • The application (robot solving a problem) is a composition of hardware and software, developed by different people • Performance, determinism are critical • Concurrency, isolation • Re-use enables progress

4. CCR/DSS Toolkit At A Glance Runtime Authoring Tools Services • Visual Programming Language • Visual Configuration and Deployment • Visual Studio templates • Coordination and Concurrency runtime (CCR) • Decentralized Software Services (DSS) • Samples and tutorials • Infrastructure services

5. CCR/DSS Enterprise CustomersBeyond robotics! ASP.NET page handling, IO coordination Mail sorting system Event processing for security systems

6. Concurrency And Coordination Runtime (CCR)

7. Why CCR? • Concurrency • Process many tasks (load-balance across cores) • Scalability, Responsiveness • Exploit latency • Coordination • Exercise control without blocking threads • Orchestrate asynchronous operations • New mechanism to handle failure for concurrent, asynchronous code • Runtime • Advanced scheduler with fairness, throttling • Extensible primitives

8. customer Siemens Infrastructure Logistics Inc.

9. Customer Use CaseSiemens Automation • Siemens postal automation handles majority of worldwide mail volume • CCR is now basis of next generation blackboard system • AI agents use OCR and database lookup to determine mail source/destination • Publication/subscription systems that requires high throughput, isolation between components • New version is scalable, readable, extensible • Quick adoption – Took few days, one dev, to get CCR based solution up and running

10. Customer Use CaseSiemens case study quotes • “We deal in milliseconds and microseconds, and there are not very many commercial products that we can take off the shelf and integrate into our product that can meet our demanding performance criteria…” • “We dropped the CCR code into our application and within a few hours we were able to establish the basic mechanics and flow for our AI agents to work with the Blackboard Framework” HamidSalemizadeh, Director of Engineering, Reading & Coding, Siemens Infrastructure Logistics Inc. David Hudspeth, Software Engineer, Siemens Infrastructure Logistics Inc.

11. customer Tyco Software House

12. Customer Use CaseTyco • Tyco event management system used from small stores to the White House • Next generation now uses CCR at its core • Fast – CCR solution uses only 1 thread per core and is twice as fast • Responsive – Multiple dispatcher queues and fair scheduling prevent starvation of low frequency events • Robust – Causalities simplify failure handling, increase robustness • Quick adoption: Within a week CCR was integrated

13. Customer Use CaseTyco case study quotes • “With CCR, we see linear scaling. If you double the number of processors, you will see a doubling in performance. That is impressive” • “I came back from the Microsoft conference, and two days later I had removed our thread pool code and we were running with CCR. You can’t beat that” Stephen Tarmey, Architect, Tyco International’s Software House

14. CCR Programming Model • Asynchronous in-process message passing • No explicit threads, locks, semaphores! • Task scheduled based on message availability • Data-dependency scheduler • Models concurrency • Coordination primitives (join, choice, …) • Composition of data-driven components • Iterative tasks • Express sequential control flow of asynch. tasks

15. Dispatcher schedules items from its queues round-robin to run in its threads Post places a message on the port Enqueue work item Port Dispatcher Scheduler picks next work item to execute Arbiter is activated on queue Thread calls handler with message as arg Creates work item from message and handler Arbiter checks whether it can consume the message Arbiter DispatcherQueues Threads Handler Arbiter is attached to port

16. There can be many of everything Port Dispatcher Arbiter DispatcherQueues Threads Handler Handler Handler

17. CCR Coordination PrimitivesExposed via Arbiter methods Code-Scheduling (non port-specific) Single-Port Primitives Multi-Port Primitives FromHandler Single Item Receiver Join (Logical AND) FromIterator Handler Multi-Item Receiver Choice (Logical OR) Interleave (Reader/Writer) Multi-Port Receiver

18. Hello, World! Dispatcher uses fixed number of threads, schedules from queues in round-robin var dispatcher = new Dispatcher(0,”default”); varqueue = new DispatcherQueue(dispatcher); var port = new Port<string>(); port.Post("Hello, World!"); Arbiter.Activate(queue, port.Receive(message => Console.WriteLine(message) ) ); Port: Building block for sending and receiving messages Post: Queues a message Receive coordination primitive Task: Delegate that handles the message (message is consumed) Port on which to receive the message

19. Hello, World!Iterator version void Start() { var queue = new DispatcherQueue(); var port = new Port<string>(); port.Post("Hello"); port.Post(“World”); Arbiter.Activate(queue, new IterativeTask(() => Hello(port) ) ); } IEnumerator<ITask> Hello(Port<string> port) { for(vari=0; i<5; i++) { yield return port.Receive(); var message = (string)port; Console.WriteLine(message); } Lambda captures arguments for iterator Loops around asynchronous operations are easy Schedule iterative task Yield execution until receive is satisfied Item retrieved after yield

20. CCR InteropAsynchronous Pattern (Begin/End) IEnumerator<ITask> CcrReadFileAsync(string file) { var result = new Port<IAsyncResult>(); using (varfs= new FileStream(file,…,FileOptions.Asynchronous)) { varbuf= new byte[fs.Length]; fs.BeginRead(buf, 0, buf.Length, result.Post, null); yield return result.Receive(); varar = (IAsyncResult)resultPort.Test(); try { fs.EndRead(ar); ProcessData(buf); } catch { // handle exception } } } Instead of passing a delegate, stream will post AR to port Retrieve AR result from port Complete operation

21. CCR InteropUsing adapters for common APIs static IEnumerator<ITask> CopyStream(Stream source, Stream dest) { varbuffer = new byte[4096]; intread = 0; do { PortSet<int,Exception> readResult = StreamAdapter.Read( source, buffer, 0, buffer.Length); yield return readResult.Choice(); varexception = (Exception)readResult; if (exception != null) yield break; read = (int)readResult; varwriteResult = StreamAdapter.Write(buffer,0,read); yield return writeResult.Choice(); } while (…) } CCR Adapter for Stream class Yield to result outcomes Retrieve success outcome (bytes read) Proceed to asynchronous write

22. demo CCR Concurrent Processing

23. Decentralized Software Services(DSS)

24. Why DSS? • Robust • Deep isolation (data and execution) • Contain and manage failure • Uniform concurrency model • Composable • Protocol and runtime support to create, manage, deploy, data driven applications • Runtime and tool support for service bindings • Publication/subscription integrated with structured state manipulation • Observable • Service is a living document addressable through URIs • Consistent mechanism for configuring, managing and controlling access

25. DSSP Protocol DSSP HTTP Microsoft Open Specification Promise

26. Service State – A Live Document <DriveState   <Connected>true</Connected>   <DistanceBetweenWheels>0.112</DistanceBetweenWheels>   <LeftWheel> …   </LeftWheel>   <RightWheel> …   </RightWheel>   <PollingFrequencyMs>80</PollingFrequencyMs>   <TimeStamp>2007-10-10T13:07:45.5195866-07:00</TimeStamp> </DriveState> Flexible UI Service Orchestration

27. Service As A Unit Of Composition • Service Properties • Identity (URI) • Structured State • Composition through partnering • Uniform Behavior • Deep isolation in execution and data • State retrieval and manipulation • Service creation and Termination • Notifications are coupled to state changes

28. Six Steps To A DSS Service

29. Define Data TypesExample state and operation types • [DataContract] • class State • { • [DataMember] • public List<Record> {get; set;} • } • [DataContract] • class Record • { • [DataMember] • public string Name {get; set;} • [DataMember] • public int Age{get; set;} • } • // operation • class Query : Query<Record,Record>{} • // operation port • class OperationsPort : PortSet<Query,Get,Update> {}

30. Service ImplementationDeclare partnerships, operation port Declarative, dynamic composition, annotations picked up by visual editor [Contract(Contract.Identifier)] class RecordKeeperService : DsspServiceBase { [Partner(“PeopleLookup”, Policy = PartnerCreationPolicy.UseExistingOrCreate, Optional = false)] peopleLookup.OperationsPort _peopleLookupPort = peopleLookup.OperationsPort(); [ServicePort(“/recordkeeper”,AllowMultipleInstances = true] OperationsPort _mainPort; protected override void Start() { base.Start(); } } Attach handlers to operation port, publish instance URI in service directory

31. Implement ServiceTypical service handlers [ServiceHandler(ServiceHandlerBehavior.Concurrent)] public IEnumerator<ITask> QueryHandler(Query queryOp) { var response = FindItem(queryOp.Body); queryOp.ResponsePort.Post(response); yield break; } [ServiceHandler(ServiceHandlerBehavior.Exclusive)] public IEnumerator<ITask> UpdateHandler(Update updateOp) { QueryAgequeryAgeOp; yield return _peopleLookupPort.Query(out queryAgeOp); int age = (int) queryAgeOp.ResponsePort; UpdateRecord(age, updateOp.Body.Name); updateOp.ResponsePort.Post(new UpdateResponse()); } CCR Interleave is iterator-aware, guaranteeing atomicity across asynchronous steps

32. Dealing With Partial FailureCausalities Create causality at root of execution graph Failure occurs on one of the side branches Deal with error as message at origin of execution

33. demo DSS/CCRLogSync Sample

34. Log Sync’ing Demo • Create 1000 services and get their state

35. Visual Programming LanguageOrchestrating DSS services with dataflow • Use any DSS service • Connect services • Notifications • Request/Response • Control and logic • Data transformation • Control dataflow Notification Available DSS services Service instance Request Response

36. DSS Manifest Editor IBuilding applications from components • Choose services to instantiate • Partner services using design time declarations • Resolve generic services • Local or remote • Set initial configuration • Edit initial service state document • Create deployment

37. DSS Manifest Editor IIDistributed applications • Multiple execution nodes • A “node” is an OS process • Several nodes per machine or across machines • Partner services across domains • Create deployment • Automatic distributed startup for testing

38. demo DSS/CCRVision Processing

39. Summary • New product addressing distributed, concurrent applications • Enterprise customers are using it now • Lightweight concurrency runtime that can be dropped in existing code • Lightweight, observable service framework

40. Learn more • http://www.microsoft.com/ccrdss • Download toolkit • Documentation (tutorials, videos, user guide) • Channel9 Videos • Case studies • Community Forum

41. Evals & Recordings Please fill out your evaluation for this session at: This session will be available as a recording at: www.microsoftpdc.com

42. Q&A Please use the microphones provided

43. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.