Enhancing Search Engine Personalization: Integrating User Preferences and Collaborative Systems

1. Search Engines Personalization BrachaShapira bshapira@bgu.ac.il Ben-Gurion University

2. Personalization • “Personalization is the ability to provide content and services tailored to individuals based on knowledge about their preferences and behavior” [Paul Hagen, Forrester Research, 1999];

3. Acceptance of Personalization • Overall, the survey finds that interest in personalization continues to be strong with 78% of consumers expressing an interest in receiving some form of personalized product or content recommendations. ChoiceStream Research

4. Motivation for Search Engine Personalization • Trying to respond to the user needs rather than to her query • Improve ranking tailored to user’s specific needs • Resolve ambiguities • Mobile devices – smaller space for results – relevance is crucial

5. Search Engines Recommender Systems - Two sides of the same coin???? • Search Engines • Goal – answer users ad hoc queries • Input – user ad-hoc need defined as a query • Output- ranked items relevant to user need (based on her preferences???) • Recommender Systems • Goal – recommend services of items to user • Input - user preferences defined as a profile • Output - ranked items based on her preferences

6. Search Engines Personalization Methodsadopted from recommender systems • Collaborative filtering • User-based - Cross domain collaborative filtering is required??? • Content-based • Search history – quality of results???? • Collaborative content-based • Collaborate on similar queries • Context-based • Little research – difficult to evaluate • Locality, language, calendar • Social-based • Friends I trust relating to the query domain • Notion of trust, expertise

7. Marcol- a collaborative search engineBrachaShapira, Dan Melamed, Yuval Elovici • Based on collaborations on queries • Documents found relevant by users on similar queries are suggested to the current query • An economic model is integrated to motivate users to provide judgments.

8. MarCol Research MethodsSystem Architecture

9. MarCol Example

10. MarCol Example

11. MarCol Example

12. MarCol Example

13. Ranking reward: up to 3 MarCol Example

14. MarCol Ranking Algorithm • Step 1: Locate the set of queries most similar to the current user query. Where: – a (“short”) query submitted by a user u – the set of all (“long”) queries – the cosine similarity between and – a configurable similarity threshold

15. MarCol Ranking Algorithm • Step 2: Identifying the set of most relevant documents to the current user's query. Where: – the set of all documents that have been ranked relevant to queries in – a configurable similarity threshold

16. MarCol Ranking Algorithm • Step 3: Ranking the retrieved documents according to their relevance to the user query. The relevance of document to query : Where: – similarity between user query and the document. – similarity between user query and documents’ query . – the average relevance judgment assigned to the set of the documents for the query (measured in a 1..5 scale).

17. Experiment Results – first experimentSatisfaction • There is not a significant difference between the modes (p=0.822535) for a 99% confidence interval.

18. The properties of a pricing model • Cost is allocated for the use of evaluation, and users are compensated for providing evaluations. • The number of uses of a recommendation does not affect its cost (based on Avery et al. 1999). That value is expressed by the relevance of a document to users query and the number of evaluations provided for that document representing the credibility of calculated relevance. • Voluntary participation (based on Avery et al. 1999). The user decides whether he wants to provide evaluations. • The economic model favors early or initial evaluations. Therefore, a lower price is allocated for early and initial evaluations than for later ones and a higher reward is given for provision of initial and early evaluations than for later ones.

19. Cost of document Calculation • An item that has more evaluations has a higher price (until reaching upper limit). • An item that has few recommendations offers a higher reward for evaluation. • The price of an information item is relative to its relevance to the current users query. • The price is not affected by the number of information uses.

20. Document Cost Calculation – the price of document for a query Where: – the number of judgments – upper bound

21. Reward Calculation reward – is the amount of MarCol points that a user is awarded for providing an evaluation for document that was retrieved for query Reward Where: – the number of judgments – upper bound

22. Experiment Methods • Independent variable: • The only variable manipulated in the experiment is an existence of the economic model.

23. Experiment Methods • The following questions (tasks) were used (Turpin and Hersh 2001): • What tropical storms hurricanes and typhoons have caused property damages or loss of life? • What countries import Cuban sugar? • What countries other than the US and China have or have had a declining birth rate? • What are the latest developments in robotic technology and it use? • What countries have experienced an increase in tourism? • In what countries have tourists been subject to acts of violence causing bodily harm or death?

24. Experiment Procedure • There were six equal subgroups, while every subgroup was given its unique sequence of questions (a Latin square). • There were six sub stages; on each sub stage the participants were provided with a different question.

25. Experiment Results – first experimentPerformance • There is a significant difference between the modes (p≈0) for a 99% confidence interval.

26. Experiment Results – second experimentPerformance • There is a significant difference between the modes (p≈0) for a 99% confidence interval.

27. Experiment Results – first experimentParticipation • There is a significant difference between the modes (p=0.008204) for a 99% confidence interval.

28. Experiment Results – first experimentAccumulatedParticipation

29. Experiment Results – first experimentAccumulatedParticipation

30. Experiment Results – second experimentParticipation • There is a significant difference between the modes (p=0.000164) for a 99% confidence interval.

31. Experiment Results – second experimentAccumulatedParticipation

32. Experiment Results – first experimentSatisfaction • There is not a significant difference between the modes (p=0.822535) for a 99% confidence interval.

33. Experiment Results – second experimentSatisfaction • There is not a significant difference between the modes (p=0.746576) for a 99% confidence interval.

34. Summary of Results • User performance is significantly better when using MarCol mode. • The average superiority of is 6% in the first experiment, and 16% in the second. • The user performance superiority of MarCol increases as the task is more difficult. • User participation is significantly higher when using MarCol mode. • The average superiority of MarCol is 46% in the first experiment, and 96% in the second. • The user participation superiority of MarCol increases as the task is more difficult. • The participation grows constantly over time and so does the gap between the MarCol and MarCol Free modes in both experiments. • There is not any significant difference in user satisfaction between the modes.

35. Conclusions and Trends search engines personalization • Search engines already integrate personal ranking • Technology is yet to be developed to enahance personalization • Still needs evaluations to calibrate the degree of personalization • Privacy issues are to be considered