Building BI Environments on a Budget

1. Building BI Environments on a Budget Eric Vallo BI Solutions Architect

2. My Point of Reference • Formerly a BI architect for two large enterprise customers • Now a BI architect for a company with less than 2,000 employees • Successfully completed a Crystal Enterprise 10/Business Objects 6.5 migration to XI R2 • Now challenged with big requirements but limited financial resources to implement a Business Objects environment

3. Introduction • Constructed in a robust, Services Oriented Architecture (SOA), Business Objects provides endless configuration options to satisfy business requirements • Budget constraints are always a consideration when building out an environment, but don’t necessarily have to be a limiting factor

4. Project Scope vs. Budget • In past lives, big BI projects typically came with big budgets • Working with a smaller customer, smaller project sizes result in dramatically smaller budgets • The outcome ultimately has to be the same, but the approach is entirely different

5. Typical Requirements • Satisfy all the reporting requirements • Provide appropriate performance levels in accordance with service level agreements • Have some type of redundancy built into the architecture • Ensure that the environment can scale with the growth of the reporting needs

6. Deconstructing the Layers of the Architecture

7. High Level Architecture • Start by thinking of the BO architecture in a very abstract form • Simply put, your architecture can be sliced into distinct layers • Each layer comes with its own capabilities to achieve your organizations goals for capability and availability Presentation Layer BI Layer File Repository Database Repository

8. Presentation Layer • The Presentation Layer is physically identified by the web server environment that renders the final content to report consumers • Handles all of the HTTP traffic to and from the report consumers • Also accessible via external applications doing SDK development • Typical deployments are on .NET (IIS) or Java (Tomcat, WebLogic, WebSphere, etc)

9. BI Layer • The BI Layer is where Business Objects actually resides • Handles communication between all layers and services • Manages security and content

10. File Repository • The File Repository is where Business Objects content is physically stored • Structure is a file system, organized in folders based on unique IDs in the Business Objects system

11. Database Repository • The Database Repository, known as the Central Management Server database • The metadata database that defines all objects, security, etc.

12. Constructing the Environment

13. A Case Study • Very small deployment, currently one server running two CPUs • Small user base less than 1,000 users • Java web application shop • Incrementally design an environment that provides fault tolerance and can still scale beyond 1,000 users

14. Presentation Layer Design • Deploy the Presentation Layer on physically separate hardware from the BI Layer, File Repository, and Database Layer • Great candidate for Linux based servers • Great candidate for VMware • Easily load balanced using industry standard hardware • Doesn’t incur additional Business Objects license expenses

15. Load Balancing Deep Dive • Assumption: Most organizations today employ some type of hardware or software load balancing technology • Prior knowledge with Load Balancers is not necessarily required to include in architecture • Generally architected with primary and redundant load balancers • Creates a virtual IP address for your clients to access • Make friends with your Network Engineer, because he/she holds the keys to this capability

16. Load Balancing Deep Dive • Leverage “sticky” sessions • Lower the polling for failed servers • Lower the retries to detect a server failure • Successfully proven to minimize impact to online users in the event there is a node failure • Enables you to easily add web servers if the load is too great across your existing servers • This is what makes scaling with VMWare attractive!

17. VMWare Deep Dive • Constructed correctly, VMWare can provide the ability to rapidly scale an environment • Clustered virtual infrastructure allows for a VM to run on any hardware in the physical cluster • Resource scheduling evaluates load real time and moves VMs to server with least load – grid computing concepts • Fault tolerance – if the physical server fails, the time to migrate the server state is approximately 12 seconds • Physical server maintenance does not impact the VM in a clustered environment • Ease of upgrades to allocate CPU/Disk/Memory to a VM • Hot backups can be performed to restore server state • Additional VMs can literally be added within an hour

18. BI Layer Design • Again, deploy on physically separate hardware from the other layers • Typically deployed on Unix based or Wintel architecture • Great candidate to evaluate on Linux! I am! • Most important part of the architecture to ensure is on physical hardware • Load balancing already built into Business Objects

19. BI Layer Redundancy BO licenses aren’t cheap. How do I get redundancy? Solution: Active/Passive Configuration

20. BI Layer Design • Hardware is cheap, so get an identical server to the primary Business Objects server for redundancy • Consider Linux for a lower cost environment configuration if necessary skill sets are present • Install it just like it were part of the cluster and test it • When 100% operational, stop all services on the passive server • Utilize 3rd party monitoring such as OpenView to monitor system and fail over when required

21. Making it Fail Over “Automagically” • Health check examples might include CPU, Memory, or Disk utilization, or even checking the availability of your server’s logon page • If failure is detected, invoke script to enable services on your passive server • Use the monitoring tool to throw an alert that your environment has been failed over via email/pager • Minimizes the impact to users on the platform while you fail over to the passive server and keeps your operation running

22. Leverage Multi-Core Architecture • Assume any new server will include multi-core technology, resulting in a relatively new approach to licensing • Secondary cores will typically deliver between 50-75% of the performance of the primary core • Resulting CPU licensing is 50% the cost of a full CPU license per second core • 2 CPU x 2 Cores Each = 3 CPU license requirement • Look for hardware that allows you to disable the second core on each chip This can effectively reduce the number of CPUs purchased up front to two • Enables an incremental CPU increase on a server of one CPU before additional servers/licenses may be required • 2 CPU x 1 Core Each = 2 CPU license requirement • Same concept holds true whether purchasing a two, four, or even an eight CPU server

23. Strategies in Licensing • Consider named user licensing vs. CPU based licensing for the initial environment construction • Ensure any master agreement in place is flexible enough to permit conversion of named to CPU licenses • Remember to profile users for creators vs. consumers, so you can differentiate licensing between Business Objects Enterprise and Web Intelligence licenses

24. Where To Next? • Proactively monitor your server performance down to the service level (CMS, Tomcat, Web Intelligence, etc.) • If capacity is available on existing components of the cluster, scale up first before scaling out • Stretch the limits from the default BO settings to allocate additional resources to existing processes • Add additional job servers • Add additional report servers • As projects roll in, incrementally increase spending to augment the environment to upgrade servers, add more servers, add BOBJ licenses, etc. • The architecture picture painted so far means that you can gradually add components to each layer to meet your goals

25. File Repository Server Design • Again, deploy on physically separate hardware from the other layers • Can be deployed on pretty much any hardware that has a file system which can be recognized by your Business Objects servers • Another great candidate for VMWare • Great opportunity to piggy back on any Enterprise storage solution

26. Enterprise Storage • This is the right time to make friends with your Storage Manager • Ensure your file server has the right connectivity, typically fiber, to hook into a SAN solution • This provides higher availability of your file system • The File Repository Server <> a huge server • It just needs to be able to handle the I/O to the SAN solution

27. VMWare Revisited • High availability through rapid replication in a clustered environment • Illustrates grid computing concepts • Rapid expansion capabilities • High probability that it can avoid outage impacts altogether with its failover capabilities

28. Database Repository Design • Again, deploy on physically separate hardware from the other layers • A vast array of database technologies are supported here • Pick one that your organization has expertise with • Be aware that database high availability solutions can be extremely expensive • The best strategy on a tight budget: • Piggy back on an existing database environment • Ensure a strong backup and recovery strategy

29. Bonus – External Access • Through the use of a DMZ/Proxy, external traffic can easily leverage the same architecture • The proxy handles encryption and transmission to external clients • If external users are non-Active Directory accounts, LDAP or BOE security can easily be leveraged here as well • Whether you open up your BO environment externally all depends on the needs of the business • Also consider Citrix for external access to the platform

30. RecommendationsandConclusions

31. Recommended Approach • Understand all Enterprise Resources that are available (Load Balancing, VMWare, Enterprise Storage, Databases) • Get control of the BI Layer • Scale up before you scale out, if possible • Add a second server to the cluster - By its nature, requires you to implement a shared file system for your File Repository Server • Redistribute the HTTP traffic to stand alone servers • Quickest bang for the buck by moving services such as Tomcat to a stand alone server • Evaluate if a high availability database instance is present in your organization in order to leverage its strengths • Bottom line: a small environment can exhibit all the strengths in stability as a large scale environment

32. Conclusions • The environment can scale up or out • Scale up has already occurred for Crystal Reports Job Server and Web Intelligence Report Server • We have increased to this configuration in response to increased demand in both users and scheduled reporting • For my organization, this environment currently supports • 200+ recurring Crystal Reports • 600 Web Intelligence users • Early stages of SharePoint integration • Early stages of integration with WebSphere Portal Server • Live Office and Query as a Web Service coming soon • Uses Active Directory for Internal employees • LDAP authentication coming soon for external access

33. Questions? Eric Vallo BI Solutions Architect eric.vallo@gmail.com