Power of Powerpoint:

1. Power of Powerpoint:

2. Living in an Agile World- Überleben auf der ÜberholspurKeynote, OOP 2003 Michael Stal

3. Agenda Goal of this talk: To illustrate agility from different perspectives: • Definition of Agility • Motivation behind Agility • Agile Architecture • Agile Technology • Agile Organization • Summary

4. What the Hell is Agility? • According to my search engine agility is about • Dog Training or • Older People • According to many software engineers it is about • eXtreme Programming and competing software engineering methodologies • So, what is the right answer, anyway?

5. One View of Agility • "Agility is a core competency - it is the capability to respond in a satisfactory manner to change, uncertainty and unpredictability in the wider business environment.“ • "Agility is a next generation manufacturing concept. It belongs in the 21st century and does not fully exist in any company today. The challenge for the future is to create agile companies using technologies, organisational forms and people to develop a new form of manufacturing that transcends existing mind sets that are becoming increasingly dominated by the dogma of lean production." • "If agility does not scare you, then you have not fully understood what it is about.“ All quotations taken from Paul T. Kidd, next generation manufacturing expert

6. Agility and Software Development • If we follow this definition, agility is an important issue in our business: • Software developers are definitely facing change, uncertainty, and unpredictability! • We all work in companies that are driven by competition, increasing market requirements and higher speed of technology and product innovation. • Thus, we are driven by agility. • As an example, let us take a look at software development in the industry.

7. Industry and Business -A Look into the Crystal Ball • Smart Traffic: Smart cooperation of vehicles, goods, infrastructures to enable fast and seamless transportation. • E-Government: Administration on the Web. • Ubiquitous Services: Smart services that can be consumed from anywhere using any device. • Intelligent Home.

8. Forces • Web- and Net-based Integration of Applications, Services, and Systems • Quality of Service issues such as availability, resource and time constraints, ... • Deployment of Automated and Autonomous Systems • Smart, ubiquitious Services that share and distribute context • Heterogeneous Data • Software as Service

9. ... And Consequences • Developers should not make too many assumptions about usage contexts and environments a priori. • Developers don‘t live on an island and must integrate legacy code and other peer systems. • Programmers need to develop and deploy their software on different heterogeneous machines and devices. • Running applications are reconfigured instead of recompiled. • Software Engineering must cope with change requests in Internet time.

10. Multiple Aspects increase complexity of Software Engineering • Today‘s Forces: • Global markets lead to strong competition • Time-to-market and costs must be reduced • Quality is a conditio-sine-qua-non • Cyle times are extremely fast • Software as Service and Software as (Web-based) Integration Platform • To meet these requirements multiple aspects have to be considered that are strongly interdependent: • Efficient organizations to steer software development effectively • Appropriate, stable but extensible software architecture to meet today‘s and tomorrow‘s requirements • Right technologies to implement the software architecture Technologies Architecture Organization

11. Agile Architecture "I conclude that there are two ways of constructing a software design: One way is to make it so simple there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies." --- Tony Hoare, Turing Award Lecture, 1980

12. Architecture Does Matter • „Software entropy“ should be maximized, i.e., loosely coupling between peers, decentralized information access, reflective approaches („Just-in-Time Integration“). • Software should be partitioned into composable services that can be spontaneously connected and orchestrated by business/technical processes (component-based software). • Software means to connect people and devices. • Legacy code must be integrated in order to protect investments. • It is important to leverage architectural concepts.

13. Flavors of Architecture • Today‘s and even more tomorrow‘s Software Architecture is • Multi-Tier, to logically and physically allow to distribute functionality, and to separate concerns; • Component-based, to enable integration, easy usage of vertical services (persistence, transactions, security), scalability, and re-use; • service-oriented, to support arbitrary devices (pervasive computing), to enable the deployment of value-addes services; • aspect-oriented, to ease separation of cross-cutting concerns from business logic and to increase manageability and flexibility; • (at least partially) decentral, to assure fault tolerance and allow to integrate autonomous devices and services; • open (and web-based), because heterogeneity is predominant everywhere so that integration aspects will become one of the most important issues.

14. Architect‘s Toolset to reach these Goals • Processes, such as the Unified Software Development Process or MDA. • Notations, such as UML. • Technologies, such as frameworks and distributed object computing. • Enabling techniques like separation ofconcerns; and programming paradigms, such as object technology. • Each approach addresses a specific aspect in the process of constructing high-quality software architectures. Frameworks OO UML DOC Unified Software Development Process

15. Is this sufficient? • However, all these concepts do not tell how to solve a very specific problem that arises during the construction of a particular software system. • For example, how to keep two cooperating components consistent to each other, or how to organize the event handling in a distributed system? • Solving such problems is still left to developer’s intuition and experience in software construction.

16. Patterns • Experts mostly do not invent solutions. • When experts work on a particular problem it is unusual for them to tackle it by inventing a new solution that is completely distinct from existing ones. • Confronted with a ‘new’ problem they often remember a similar one they once solved successfully and adapt the ‘old’ solution to the new context. • Experts think in Problem/Solution pairs. • Reusing ideas keeps development smart and agile, wasting time does not!

17. Let’s take a look at a well-known pattern

18. Observer (GoF) Problem Changing the internal state of a component may cause inconsistencies to the state of other components. How can we restore consistency effectively in a way that: the information provider must not depend on information consumers; andthe information consumers that depend on the information provider must not be known a priori. Observer (1) House1 Door_A; Window_1a Window_1b ... depends on depends on is presented as

19. Solution-Structure Implement a change propagation mechanism between the information provider, the subject, and the information consumers, the observers. The Subject maintains a registry of observers and notifies allregistered observers about changes to its state. An Observer declares an update function to be called by the subject’s changepropagation mechanism. Concrete Observers implement the update function in a system-specific manner. ConcreteObserver update doSomething * Subject Observer state observerList update attach detach notify setData getData state = X; notify(); s->getData() for all observersin observerList do notify(); Observer (2)

20. Solution-Dynamics The Observers register with the subject’s change propagation mechanism. A Client modifies the subject’s data. The Subject starts its change propagation mechanism. The Subject calls the update function ofevery registered observer. The Observers retrieve the changed data from the subject and update themselves. Subject Observer 1 Observer 2 attach(this) attach(this) setData notify update getData update getData Observer (3)

21. Benefits Defined handling of dependenciesbetween otherwise strongly coupled components Support for dynamic configuration of a subject with observers Adding new observers does not affect the subject or the change propagation mechanism Liabilities Indirection Unnecessary updates Update cascades Observer (4)

22. A Pattern presents a concrete solution schema for recurring design problems; documents proven design experience; specifies concepts above the level of individual classes and objects; describes structure and behavior of cooperating objects; provides a common vocabulary and concept understanding; and addresses specific quality properties of the problem’s solution. Thus patterns are a perfect means for agile software development. ConcreteObserver House1 Door_A; Window_1a Window_1b ... update doSomething depends on depends on is presented as Properties of Patterns Problem Observer Solution Subject Observer Subject Observer 1 Observer 2 * state observerList update attach(this) attach detach notify setData getData attach(this) notify setData update state = X; notify(); getData update s->getData() getData for all observersin observerList do notify();

23. Delegate Service Locator Session Facade Transfer Object DataAccessObject Best Practice Patterns • No patterns is an island. • Using whole pattern systems that cover best practices in a given domain such as Core J2EE Patterns helps to minimize integration and implementation efforts (thus increasing architectural agility):

24. Patterns for Architectural Analysis • It is often forgotten that patterns do not only help to build new software architectures, but • Patterns are also an appropriate means to understand exising software architectures. • Take a look at your favourite platform (Java, .NET) and reverse engineer it by mining its underlying principles. • In practice, software architects need to do both: • They leverage patterns to design new functionality and • they reverse engineer the patterns behind legacy code for integration purposes

25. Quality is a First Class Object • We cannot sacrifice quality for agility. • For software projects it is surprisingly difficult to achieve quality. • Quality is nothing others take care for after you finished your project! • There are two important aspects: • Does the architecture meet the functional requirements? • Does it also meet its non-functional requirements such as performance, security, scalability? • Quality Assurance starts before the project starts and it never ends. Programmer's Drinking Song (sung to the tune of ``100 Bottles of Beer'‘):   99 little bugs in the code,   99 bugs in the code,   fix one bug, compile it again,   101 little bugs in the code.   101 little bugs in the code....(Repeat until BUGS = 0)     -- Anonymous

26. Aspect-Oriented Software Architecture • How can we keep our systems flexible and easily reconfigurable? We can reach this goal by taking separation of concerns to an even higher level • Non-functional properties are typical candidates for the emergence of cross-cutting concerns • Cross-cutting concerns define aspects of a software system that are not localizable and therefore spread across the system such as security aspects • While OO helps to decompose functionality (separation of concerns), it doesn‘t help to factor out cross-cutting concerns • Generative techniques and Aspect-oriented Programming are potential candidates that come to our rescue

27. The Need for Integration • Software Users • often use systems that consist of heterogeneous islands. • need fast RoI for IT investments. • Smart integration helps reducing the costs. • Integration solutions require: • Architecture with strong separation of concerns (component-based, multiple Tiers). • Appropriate technologies that offer maturity, acceptance, support, and productivity.

28. Levels of Integration • Unfortunately, there are different layers of integration. • We must cope with almost all of them. Client Integration Web Integration Workflow Integration Persistence Integration Service Integration Interoperability of Data & Communication Platform Integration

29. Integration Pyramid • Until now for each integration layer software engineers tried to follow the best-of-breed approach. However this approach might lead to heterogeneity and even increase complexity . • A uniform and cross-cutting techical solution leads to a much better integration solution because this approach spans all layers. • Today‘s platform such as Java and .NET are already targeting this multi-layer integrating problem. They hide heterogeneity by a homogeneous stack of APIs. • Integration between Java and .NET through Web technologies/XML. Client Business and Web Layer Middle Layer System-Level Protocol Layer

30. Agile Technology “The most likely way for the world to be destroyed, most experts agree, is by accident. That’s where we come in. We’re computer professionals. We cause accidents." --- Nathaniel Borenstein, Programming as if People Mattered

31. Technology does also matter • Necessary to choose the right software technologies: IDEs, CASE-Tools, CM-Tools, Test-Tools, Middleware. • The best architecture and the best process will not help if you choose the wrong technologies. Some technologies are more like shooting youself in the foot. • Take integration as an example. You‘ve got a choice of integration platforms: • Microsoft .NET • J2EE • Web + XML • CORBA • EAI • Think „Service-based“ not just „Web-based“!

32. Case Study: Selecting the right Middleware • For integration and communication, middleware solutions are essential • There are different kinds of communication middleware: • Object-oriented Middleware (e.g., CORBA, COM, RMI) • Message-Oriented Middleware (e.g., JMS, MSMQ) • Transaction Monitors (e.g., Tuxedo) • Middleware decisions depend on the concrete requirements of your application domain. If different approaches are appropriate, you may • Choose the best-of-breed approach, or use • Hybrid solutions (e.g., EJB, .NET+MSMQ+COM+)

33. The Web reveals Limitations • Lacking interoperability between middleware solutions. • Insufficient Web integration. • No programmatic access. WebServer FRONTEND BACKEND HTTP TUNNEL

34. User E-Shop Inventory Web Xpress Transport Co Credit Cart Company CustomerObj login create checkPreferences displayItems selectItem checkAvailability updatePage pay checkCredit checkCredit displayOrderInfo accept handleOrder sendOrder logoff fini Future Vision: Service-Oriented Approach www.e-shop.com www.credit.com www.transport.com Browser @ Home

35. The Web is the Computer • The Web is an excellent integration platform since it relies on open standards. • Using the Web we can establish an architecture that • supports web-based inter-application communication paradigms, • provides location transparency, • allows to add, exchange, or remove services dynamically, • hides system details from the developer.

36. Solution Architecture: The Web as a global middleware Message exchange Message exchange Broker 1 * 1 Client-side Proxy Server-side Proxy * register_server find_server establish_communication marshal unmarshal receive_result service(proxy) marshal unmarshal dispatch receive_request * calls * calls 1 1 Client Service call_service_p start_task start_up main_loop service(impl.)

37. Step 1 – Providing a Transport Protocol • A protocol defines syntax, semantics and order of messages exchanged between peers. SOAP is the right protocol. <soap:Envelope> <soap:Header> <transaction> <soap:mustUnderstandstand=„true“ xmlns=„http://tx.com“> <id> 12345678 </id> </transaction> <soap:Header> <soap:Body> <m:getPhoneNumber> <theName> Bill Gates </theName> <m:/getPhoneNumber> </soap:Body> </soap:Envelope>

38. Step 2: Defining a Service Description Language • A description language must be available to define Web service interfaces and how to invoke them. • We can use XML to • specify deployment information • and structural information. • That is what the Web Service Description Language (WSDL) is all about. • WSDL addresses syntax, not semantics!

39. 1. search for web service 2. return service URL 3. read service description 5. forward request to service 4. send request using SOAP over HTTP 7. send result using SOAP over HTTP 6. return result Step 3: Defining a global Service Broker UDDI • UDDI is a global registry • Registration possible at any node • Registrations replicated at daily basis • Common SOAP protocol used Publisher API Inquiry API Description of Web Service (WSDL) Web Server Client Web Service

40. Client Proxy Server Proxy Step 4: All you need are Generators • Introduce tools that generate glue for connecting you with the Web ORB, i.e., • Client Proxies for connecting the client with services. • Server Proxies for seamless service deployment. WSDL

41. Web Service User/Provider Mainframe Integration Layer Micro/Macro Services Frontend Layer (Web Server) Backend Server Virtual Machine Legacy Workflow Engine Core Services (Calendar, Preferences, Transactions, ...) Service Context (Who, Where, When, Why, ....) Service Description (WSDL) Service Description, Discovery, Integration (UDDI) Web-based and -related Protocols (HTTP, XML, SMTP, ...) Web-based Service Frameworks Core Elements of Web Frameworks

42. The Problem with Web-based Service Frameworks Andrew Tanenbaum: „The best thing about standards is that there are so many to choose from“

43. Agile Technologies • Although there are many technologies to choose from, there are technologies that support agility and some that do not. • Examples for agile technologies: • Java and .NET • Web Services • Web and XML Standards • Peer to Peer • Pervasive Computing • Ad Hoc Technologies (Plug‘n Play) • CORBA • Extensible and scriptable IDEs

44. Agile Organization “In any great organization it is far, far safer to be wrong with the majority than to be right alone.” John Kenneth Galbraith

45. Organizational Aspects • Experience and research have shown that organizational issues have a fundamental impact on • software architectures and • development processes. • Thus, it is important to optimize your organization in order to optimize your products. • Organizations need to be agile, i.e., effective and efficient: “Efficiency is doing things right, effectiveness is doing the right things.” Peter Drucker

46. Agility and Engineering • To be agile your engineering process should • embrace change (Kent Beck) • support an incremental approach with each increment being a small mini project that moves you closer to the final product. • strongly rely on testing • involve the users and their requirements without giving away control:

47. Top 10 Problems in Software Projects What would you do if you were an agent and had to sabotage your competitors‘s projects? • No clear responsibilities within teams • Missing, contradicting, and dynamically changing requirements • No communication or too much communication • Inadequate processes • Wrong technologies • Unexperienced developers • Underestimation of software architecture activities • No or insufficient quality assurance • Unrealistic time frames • Wrong tool set In an agile world these issues prove to be even more painful, because they are being turned to Agile Problems

48. Summary • Today‘s businesses are agile. As a consequence, software projects must also be agile. • Hence, • Use agile processes • Leverage agile architectures • Select appropriate technologies • Consider Web protocols for integration • Don‘t forget to make your organization agile • Don‘t believe in Gurus and Jedis (telling us for years how to ease software development pain) • And don‘t always wait for TNT (The Next Technology) to save your projects! Most of the constituents for agile software development are already invented. • This summary in only one sentence: Agility denotes a Holistic Approach

49. Last but not least : The Art of User Manuals "If you cannot read this, please ask the flight attendant for assistance." United Airlines Flight Safety Brochure