Sitecore Online Marketing Suite

1. Sitecore Online Marketing Suite Technical Deep Dive Timothy Ward, Lead Solution Architect for Australia and New Zealand tiw@sitecore.net

2. Webinar Agenda Technical Introduction API Introduction Architecture Technical Concepts Extending the System Reporting FAQ�s The next step...

3. Just a few notes� We will have time for questions at the end of the presentation. The webinar will be recorded and the slides will be available for all to use. Feel free to email me questions about the presentation (tiw@sitecore.net) Code examples used throughout the demo will not be made available for public use. Like the name suggests, we are in a Technical Deep Dive. Some basic understanding of the OMS is necessary to get full understanding of all the concepts being presented.

4. System Requirements System Requirements for OMS database � For the database collecting OMS data, the following is recommended Dedicated physical server A minimum of one quad core CPU recommended A minimum of 8GB RAM recommended 1 disk for OS, 2 disks for Database, 1 disk for TempDb plus RAID according to MS SQL Server recommendations. Windows Server 2003 or 2008 64 bit recommended. MS SQL Server 2008 or MS SQL Server 2005 64-bit recommended MS SQL Server 2008 is strongly recommended due to improvements in Query Plans and memory usage. We emphasize the importance of a relevant Maintenance Plan in order to keep indexes tuned. (we will soon show you how to build this plan)

5. Environmental Settings Options for the collection of OMS data on high traffic web-sites: Use MS SQL Server clustering Best practice: Place the OMS database on a separate SQL Server/cluster and the core/master/web databases on a separate SQL Server/cluster. Larger hardware facilities such as the disk subsystem / NAS / SSD / CPUs / memory (400 users browsing your site will average 4mb per minute) Consider separating SQL Server data collection from the SQL analysis server Use e.g. daily replication to move live data to the analysis system Remove analytics-only indexes from OMS database Consider limiting storage of uninteresting behavioral statistics. For example, avoid saving requests from confirmed robot visitors. Consider dividing the collection of analytics data for clusters of web farm servers (e.g. 5 servers share one OMS write-repository)

6. Failover setup with one analytics database

7. Distributed setup with two segments

8. Creating a Maintenance Plan

9. Environment Considerations High CPU load on SQL Server�On a busy SQL Server with a large OMS database (>10GB) and many inserts. The CPU can be overloaded if the indexes in the OMS database is very fragmented. Please rebuild your indexes frequently (see 1.�Create a Maintenance Plan�above) Discard Robots�Activate 'Sitecore.Analytics.ExcludeRobots.config' to discard Robot visits to be written into the database. This can also be configured manually based on IP address.�Please request hotfix "Sitecore CMS 6.2.0 rev. 091012 Hotfix 324339-1" via�http://support.sitecore.net Remove Robot Data Backup your databases first. Run the SQL script�'Remove BOTs CleanDB.sql'�on your OMS database. Increase FillFactor (reserved percentage of free space on indexing) to 50% for the following tables: Page, PageEvent, Session, GlobalSession�To limit fragmentation on these busy tables. This will increase the size of the indexes! OMS database on the same SQL Server as Sitecore CMS databases�On a low traffic site you can consider having the OMS database on the same SQL Server as the other Sitecore databases. We strongly recommend to remove robot data frequently.

10. Architecture Implicitly = Cookies, email (links), actions on the web site (e.g. a order brochure). Credentials = Username and password Anonymous = Visitor without a cookieImplicitly = Cookies, email (links), actions on the web site (e.g. a order brochure). Credentials = Username and password Anonymous = Visitor without a cookie

11. Architecture Implicitly = Cookies, email (links), actions on the web site (e.g. a order brochure). Credentials = Username and password Anonymous = Visitor without a cookieImplicitly = Cookies, email (links), actions on the web site (e.g. a order brochure). Credentials = Username and password Anonymous = Visitor without a cookie

12. Architecture

13. Architecture Implicitly = Cookies, email (links), actions on the web site (e.g. a order brochure). Credentials = Username and password Anonymous = Visitor without a cookieImplicitly = Cookies, email (links), actions on the web site (e.g. a order brochure). Credentials = Username and password Anonymous = Visitor without a cookie

14. Architecture

15. Architecture Implicitly = Cookies, email (links), actions on the web site (e.g. a order brochure). Credentials = Username and password Anonymous = Visitor without a cookieImplicitly = Cookies, email (links), actions on the web site (e.g. a order brochure). Credentials = Username and password Anonymous = Visitor without a cookie

16. Architecture

17. Architecture Implicitly = Cookies, email (links), actions on the web site (e.g. a order brochure). Credentials = Username and password Anonymous = Visitor without a cookieImplicitly = Cookies, email (links), actions on the web site (e.g. a order brochure). Credentials = Username and password Anonymous = Visitor without a cookie

18. Architecture

19. Architecture

20. Architecture

21. DB Architecture The Session table is the most important table in the Analytics database. It is related to all other tables (apart from Status) and can be thought of as your point of access to all the information you want to retrieve from the database. All Web site visitors create a new session every time they visit the Web site and this is recorded in the Session table. Most reports need to access this table first before accessing information from the other tables. If you study any of the existing reports in Sitecore Analytics, you can see that the Session table appears in the majority of SQL queries. NOTE: There is a built in Database Diagram that comes shipped with the OMS that is available under the Database Diagrams Folder in SQL Management Studio called �dbo.Sitecore�.

22. Analytics API The Sitecore Analytics API is abstracted into its own assembly Sitecore.Analytics.dll Access the analytics database through the SitecoreAnalyticsDataContext item like so : SitecoreAnalyticsDataContext dataContext = return new Sitecore.Analytics.SitecoreAnalyticsDataContext(ConfigurationManager.ConnectionStrings["analytics"].ConnectionString); Get all pages which has been visited in the current SiteContext var pagesInCurrentSite = from session in dataContext.Sessions where session.MultiSite == website join page in dataContext.Pages on session.SessionId equals page.SessionId select page; Select the item from the database and count the hits var popularPages = from page in pagesInCurrentSite group page by page.ItemId into x select new PopulerItem { ItemId = new ID(x.Key), Count = x.Count() };

23. Analytics API Sort the items by how many hits it has received var popularPagesSorted = from popularPage in popularPages orderby popularPage.Count descending select popularPage; IEnumerable<PopulerItem> TopItems = popularPagesSorted.Take(NoOfItems); NOTE : This is based off the following type of item has already been initiated public class PopulerItem { public ID ItemId { get; set; } public int Count { get; set; } } NOTE : Almost all joins will be done with the Session table as everything links back to the session of the user.

24. Analytics API � Query Data Create a sublayout with the following code behind -> protected void Page_Load(object sender, EventArgs e) { IEnumerable<NiCam.Classes.GetPopularPages.PopulerItem> sessions = GetPopularPages.GetPopularPagesCall(Sitecore.Context.Site.Name, Sitecore.Context.Database, 5); SessionRepeater.DataSource = sessions; SessionRepeater.DataBind(); } protected void SessionRepeater_OnItemDataBound(object sender, RepeaterItemEventArgs e) { if (e.Item.ItemType == ListItemType.Item || e.Item.ItemType == ListItemType.AlternatingItem) { NiCam.Classes.GetPopularPages.PopulerItem popularItem = e.Item.DataItem as NiCam.Classes.GetPopularPages.PopulerItem; HyperLink hyperList = e.Item.FindControl("ItemHyperLink") as HyperLink; Sitecore.Data.ID itemId = popularItem.ItemId; Item item = Sitecore.Context.Database.GetItem(itemId); if (item == null) return; hyperList.Text = item["Title"]; hyperList.NavigateUrl = LinkManager.GetItemUrl(item); } }

25. Analytics API Add it in the Presentation Layout Details and you get

26. Analytics API � Tagging Tags are values that are added to the visitors global session (the object that encaptulates recurring visits). Tags are useful to apply filtering to visitors, or to visualize specific highligts in the Sitecore statistics. Tagging the global session results in the tag being printed in various reports, even though user have added the value on a prior session.

27. Analytics API � Tagging

28. Analytics API � Invoke tracking The Sitecore.Analytics.AnalyticsTracker object gives you the context tracking object which allows you to reach, modify and raise analytics for current page, previous page and next page. AnalyticsTracker.Current AnalyticsTracker.PreviousPage AnalyticsTracker.NextPage Each of these returns the AnalyticPage object which allows you to invoke analytics.

29. Analytics API � Invoke tracking Example: To trigger analytics on previous page.

30. Analytics Configuration

31. Analytics Configuration

32. Quick Reference Reminder

33. Create a new Condition

34. Create a new Condition

35. Filling out the new Condition Item

36. Filling out the Content Item

37. Create a Script Action

38. Create a new Action(in code)

39. Create a new Action (in code)�again

40. Create a new Conditional Rendering Rule

41. Create a new Action

42. Create a new Action

43. Let�s use the new Action

44. Bring it all together

45. Reporting

46. Sitecore Analytic Reports

47. Sitecore Analytic Reports

48. Sitecore Analytic Reports

49. Sitecore Analytic Reports

50. Creating a new report

51. Edit an existing report

52. Analytics and robot detection Sitecore have a method included to auto detect robots. Robots will usually not maintain session state, and if we experience a sessionless user with multiple requests, these will be classified as �Bot � Auto detected�. Such robots will be excluded from analytics reports unless manually re-classified.

53. Analytics and robot detection In order to enable autodetection, make sure to include the following web control on your layout: <sc:VisitorIdentification runat="server" />

54. MaxMind GeoIP Lookup Initiates a background thread which looks up the visitors domain name and geo information based on the IP number. Stores all looked GEO and DNS information. The background thread obtains all unresolved IP numbers and performs lookup. In multi-server setups, assign one server and one only to perform lookups. Sitecore/MaxMind LookUp Providers. string LookUpReturn = Sitecore.Analytics.Lookups.LookupManager.Provider.ReverseDnsLookup("127.0.0.1");

55. Analytics API Profile information is summarized throughout the visit and is committed to database when session ends. Visitor behavior and tracking is stored on every request. All storage requests are added to a queue being handled by a background thread. The queue max-size can be defined in the configuration file. Should the number of requests exceed the queue limit, new requests are ignored in order to preserve the performance of the server. Look at the Sitecore.Analytics.Pipelines.HttpRequest.StartDiagnostics and Sitecore.Analytics.Pipelines.HttpRequest.EndAnalytics

56. What can be achieved with the API?*The following points in no way reflect current or in development Sitecore functionality Heat Map Logging (Demonstration) � where are users clicking on your page? What controls are being used? What is the user experience when filling in forms? Mouse or Keyboard?

58. FAQ Q. Does OMS replace marketing automation like Eloqua and Marketo? A. For organizations not currently using marketing automation/campaign management tools Sitecore offers the same functionality, but with a deeper view into customer behavior and analytics. For those who have a need to keep existing solutions, OMS can run in parallel with online marketing/email campaign solutions.

59. FAQ Q. Does OMS collect data in all PageModes? A. No. If you are wondering why you are using the Page Editor and your analytic score aren�t showing up it is because the anayltics doesn�t work while in the Page Editor.

60. FAQ Q. Does OMS replace the need for products like Google analytics, WebTrends and Ominture? If they can�t be replaced, does OMS integrate with them? A. We expect that many customers will find that OMS completely fulfills their need for web analytics. However, for those customers who have a need to keep existing solutions, Sitecore OMS does integrate with Web analytic tools.

61. FAQ Q. How does the OMS remember who I am? A. Sitecore OMS is integrated in the Sitecore Application, which hosts the customer�s website. This means that Sitecore OMS can write a cookie out of the customer�s name e.g. NAME : SC_ANALYTICS_GLOBAL_COOKIE VALUE : 8A15C6FBC6504A7BAC64FFBC45B00CCE DOMAIN : www.sitecore.net PATH : / EXPIRES : 10-3-2011 16:08:52

62. FAQ Q. How is this different to other Analytic Applications? A. 3rd party analytic applications are placing its cookie on your machine from JavaScript which is on an external server (This can be blocked). Anything from using a proxy to turning off JavaScript for graceful degradation of your website will stop this cookie being sent and hence stop the analytics from realising that you are the same person.

63. FAQ Q. The OMS collects a lot of data, could this have an effect on performance? A. In essence, any application that stores and requests a lot of data will affect performance. There are tips for increasing this performance including turning robot detection off, maintenance plans and having a dedicated analytics server. More suggestions on next slide.

64. Suggestions Use MS SQL Server clustering Best practice: Place the OMS database on one SQL Server/cluster and the core/master/web databases on another SQL Server/cluster. Use larger hardware facilities such as the disk subsystem /NAS/SSD/CPUs/memory. Consider separating the SQL Server data collection from the SQL analysis server. Use regular (for example, daily) replication to move live data to the analysis system. Remove analytics-specific indexes from the OMS database. Consider limiting the storage of insignificant behavioral statistics. For example, avoid saving requests from visitors that have been identified as robots. Consider dividing the collection of the analytics data into clusters of Web farm servers (for example, every 5 servers could share one OMS write-repository)

65. Learning More Training OMS User Course http://www.sitecore.net/TrainingANZ.aspx Documentation on SDN Analytics Configuration Reference Rules Engine Cookbook Client Configuration Reference (updated for v6.1) Data Definition Reference (updated for v6.1) OMS Cookbook Report Designer

66. Questions? Asking a Question Please state your name and where you are from Please be aware that specific questions to your own implementations can be answered in email. The questions should be directed at global answers that all can benefit from. Some questions may need to be answered with more investigation.

67. The End Please send all OMS requests, questions and ideas to tiw@sitecore.net Sign up for the Developer Mailing list at http://scaug.sitecoreaustralia.com.au Australia's blog is available at http://sitecoreaustralia.wordpress.com Presentation will be available on request or via the Sitecore Australia blog.