Research Process, Research Design and Questionnaires

1. Research Process, Research Design and Questionnaires

2. RESEARCH PROCESS Identify and Define Research Problem Theory / Practice Hypotheses / Conceptualization Research Design Data collection Data Analysis Findings In this workshop we talk about all of the steps in the research process except Data Analysis and Findings.

3. RESEARCH PROCESS – Research Problem What is a problem? . . . any situation where a gap exists between the actual and the desired state. A problem does not necessarily mean that something is seriously wrong. It could simply indicate an interest in improving an existing situation. Thus, problem definitions can include both existing problems in the current situation as well as the quest for idealistic states in the future.

4. RESEARCH PROCESS – Problem Identification • How are problems identified? • Observation – manager/researcher senses that changes are occurring, or that some new behaviors, attitudes, feelings, communication patterns, etc., are surfacing in one’s environment. The manager may not understand exactly what is happening, but can definitely sense that things are not what they should be. • Preliminary Data Collection – use of interviews, both unstructured and structured, to get an idea or “feel” for what is happening in the situation. • Literature Survey – a comprehensive review of the published and unpublished work from secondary sources of data in the areas related to the problem.

5. RESEARCH PROCESS – Problem Identification • A literature survey ensures that: • Important variables likely to influence the problem are not left out of the study. • A clearer idea emerges regarding what variables are most important to consider, why they are important, and how they should be investigated. • The problem is more accurately and precisely defined. • The interviews cover all important topics. • The research hypotheses are testable. • The research can be replicated. • One does not “reinvent the wheel”; that is, time is not wasted trying to rediscover something that is already known. • The problem to be investigated is perceived by the scientific community as relevant and significant.

6. RESEARCH PROCESS – Problem Identification • Typical Business Research Problems: • Training programs are not as effective as anticipated. • Sales volume of products/services is not increasing. • Balancing of accounting ledgers is becoming increasingly difficult. • The newly installed information system is not being used by the employees for whom it was designed. • Introduction of flexible work hours has created more problems than it has solved. • Anticipated results of a recent merger/acquisition have not been realized. • Inventory control systems are not effective. • Frequent interruptions in production. • Low employee morale. • Frequent customer complaints. • Installation of an MIS keeps getting delayed. • Ad campaign is not generating new sales prospects.

7. RESEARCH PROCESS – Problem Identification What are some business problems you are aware of or have confronted?

8. Problem Definition Steps: “A problem well defined is a problem half solved!” RESEARCH PROCESS – Problem Definition • Understand and define the complete problem. If more than one problem is identified, separate and prioritize them in terms of who and when they will be dealt with. • Identify and separate out measurable symptoms to determine root problem versus easily observable symptoms. For example, a manager may identify declining sales or lost market share as the problem, but the real problem may be bad advertising, low salesperson morale, or ineffective distribution. Similarly, low productivity may be a symptom of employee morale or motivation problems, or supervisor issues. • Determine the unit of analysis = individuals, households, businesses, objects (e.g., products, stores), geographic areas, etc., or some combination. • Determine the relevant variables, including specifying independent and dependent relationships, constructs, etc.

9. RESEARCH PROCESS – Problem Definition • Examples of Well-Defined problems: • Has the new packaging affected the sales of the product? • How do price and quality rate on consumers’ evaluation of products? • Is the effect of participative budgeting on performance moderated by control systems? • Does better automation lead to greater asset investment per dollar of output? • Has the new advertising message resulted in higher recall? • To what extent do the organizational structure and type of information systems account for the variance in the perceived effectiveness of managerial decision-making? • Will expansion of international operations result in an improvement in the firm’s image and value? • What are the effects of downsizing on the long-range growth patterns of companies? • What are the components of “quality of life”? • What are the specific factors to be considered in creating a data warehouse for a manufacturing company?

10. RESEARCH PROCESS – Definitions • Variable = the observable and measurable characteristics/attributes the researcher specifies, studies, and draws conclusions about. • Types of Variables: • Independent variable =also called a predictor variable, it is a variable or construct that influences or explains the dependent variable either in a positive or negative way. • Dependent variable =also known as a criterion variable, it is a variable or construct the researcher hopes to understand, explain and/or predict. • Moderator variable =a variable that has an effect on the independent – dependent variable relationship. The presence of a moderator variable modifies the original relationship between the independent and dependent variables by interacting with the independent variable to influence the strength of the relationship with the dependent variable. • Mediating variable =also known as an intervening variable, it is a variable that surfaces as a function of the independent variable and explains the relationship between the dependent and independent variables. Moderator variables specify when certain effects will occur whereas mediators speak to how or why such effects occur. Moreover, mediators explain how external events take on internal psychological significance.

11. RESEARCH PROCESS – Definitions continued . . . Measurement = is the process of determining the direction and intensity of feelings about persons, events, concepts, ideas, and/or objects of interest that are defined as being part of the business problem. As part of measurement, researchers use predetermined rules to assign numbers or labels to: (1) individuals’ attitudes, behaviors, characteristics, etc.; (2) objects’ features or attributes; and (3) any other phenomenon or event being investigated. Rules tell researchers how to assign numbers or labels; e.g., assign the numbers 1 to 7 to responses based on the intensity of an individual’s feelings, beliefs, etc. Measurement involves two processes: (1) identification/development of constructs; and (2) scale measurement. The first process involves identifying and defining what is to be measured, while the second process involves selecting the scale to measure the construct(s). Construct = also referred to as a concept, it is a abstract idea formed in the mind based on a set of facts or observations. The idea is a combination of a number of similar characteristics of the construct. Examples of constructs include: brand awareness, brand familiarity, purchase intentions, satisfaction, importance, trust, service quality, role ambiguity, etc. Scale measurement = using a set of symbols or numbers to represent the range of possible responses to a research question.

12. RESEARCH PROCESS – Constructs Examples of Constructs Investigated in Marketing: ConstructsOperational Description Brand Awareness Percentage of respondents that have heard of a designated brand; awareness could be either unaided or aided. Brand Attitudes The number of respondents and their intensity of feeling positive or negative toward a specific brand. Purchase Intentions The number of people planning to buy the specified object (e.g., product or service) within a designated time period. Importance of Factors To what extent do specific factors influence a person's purchase choice. Psychographics The attitudes, opinions, interests and lifestyle characteristics of individuals providing the information. Satisfaction How people evaluate their post-purchase consumption experience with a particular product, service or company.

13. “Role Ambiguity” Construct • Conceptual/theoretical definition = the difference between the information available to the person (actual knowledge) and that which is required for adequate performance of a role. • Operational definition = the amount of uncertainty an individual feels regarding job role responsibilities and expectations from supervisors, other employees and customers. • Measurement scale = consists of 45 items assessed using a 5-point scale, with category labels 1 = very certain, 2 = certain, 3 = neutral, 4 = uncertain, and 5 = very uncertain. • Examples of items: • How much freedom of action I am expected to have. • How I am expected to handle non-routine activities on the job. • The sheer amount of work I am expected to do. • To what extent my boss is open to hearing my point of view. • How satisfied my boss is with me. • How I am expected to interact with my customers. Source: Singh & Rhoads, JMR, August 1991, p. 328.

14. “Service Quality” Construct • Conceptual/theoretical definition = the difference between an individual’s expectations of service and their actual experiences. • Operational definition = how individuals react to their actual service experience with a company relative to their expectations that a company will possess certain service characteristics. • Measurement scale = consists of 82 items assessed using a 7-point scale, with category labels 1 = not at all essential to 7 = absolutely essential. • Examples of items: • Employees of excellent companies will give prompt service to customers. • Excellent companies will have the customers’ best interests at heart. • Excellent companies will perform services right the first time. • Employees of excellent companies will never be too busy to respond to • customer requests. • Excellent companies will give customers individual attention. • Materials associated with products and services of excellent companies • (such as pamphlets or statements) will be visually appealing . Source: Parasuraman, Zeithaml & Berry, JM, Fall 1985, p. 44.

15. RESEARCH PROCESS Identify and Define Research Problem Theory / Practice Hypotheses / Conceptualization Research Design Data collection Data Analysis Findings

16. RESEARCH PROCESS – Theory/Practice What is theory ??

17. RESEARCH PROCESS – Theory/Practice Theory = a systematic set of relationships providing a consistent and comprehensive explanation of a phenomenon. In practice, a theory is a researcher’s attempt to specify the entire set of dependence relationships explaining a particular set of outcomes. Theory is based on prior empirical research, past experiences and observations of behavior, attitudes, or other phenomena, and other theories that provide a perspective for developing possible relationships. Theory is used to prepare a theoretical framework for the research.

18. RESEARCH PROCESS Identify and Define Research Problem Theory / Practice Hypotheses / Conceptualization Research Design Data collection Data Analysis Findings

19. Hypotheses = preconceptions the researcher develops regarding the relationships represented in the data, typically based on theory, practice or previous research. RESEARCH PROCESS – Hypotheses Examples: “The average number of cups of coffee students drink during finals will be greater than the average they consume at other times.” “Younger, part-time employees of Samouel’s restaurant are more likely to search for a new job.”

20. RESEARCH PROCESS – Theoretical Framework Theoretical Framework = a written description that includes a conceptual model. It integrates all the information about the problem in a logical manner, describes the relationships among the variables, explains the theory underlying these relationships, and indicates the nature and direction of the relationships. The process of developing a theoretical framework involves conceptualization – which is a visual specification (conceptual model) of the theoretical basis of the relationships you would like to examine.

21. RESEARCH PROCESS – Theory/Practice • Basic Features of a Good Theoretical Framework: • The variables/constructs considered relevant to the study are clearly identified and labeled. • The discussion states how the variables/constructs are related to each other, e.g., dependent, independent, moderator, etc. • If possible, the nature (positive or negative) of the relationships as well as the direction is hypothesized on the basis of theory, previous research or researcher judgment. • There is a clear explanation of why you expect these relationships to exist. • A visual (schematic) diagram of the theoretical framework is prepared to clearly illustrate the hypothesized relationships.

22. RESEARCH PROCESS – Conceptual Models Price PurchaseLikelihood Independent Dependent Variable Variable Moderator Variable • Discount Level • Restrictions Price PurchaseLikelihood Independent Dependent Variable Variable

23. RESEARCH PROCESS – Conceptual Models Mediator Variable (full mediation) Perceived Value PurchaseLikelihood Price Mediator Variable (partial mediation) Perceived Value PurchaseLikelihood Price Independent Dependent Variable Variable

24. Group Exercise: Use the Samouel’s and Gino’s restaurant database variables to develop a theoretical framework/conceptual model of the relationships that could be examined. Consider and evaluate several models, but be prepared to report your most interesting or thought provoking model. Theoretical Framework – Conceptualization

25. Supervision Conceptual Models – Samouel’s Employee Database Employee Commitment Work Groups Compensation Supervision Potential Hypotheses: Commitment is positively related to supervision, work groups and compensation. Intention to Search is negatively related to supervision, work groups & compensation. Intention to Search Work Groups Compensation

26. Description of Customer Survey Variables GINO'S Samouel's Restaurant VS. Variable DescriptionVariable Type Restaurant Perceptions X1 Excellent Food Quality Metric X2 Attractive Interior Metric X3 Generous Portions Metric X4 Excellent Food Taste Metric X5 Good Value for the Money Metric X6 Friendly Employees Metric X7 Appears Clean & Neat Metric X8 Fun Place to Go Metric X9 Wide Variety of menu Items Metric X10 Reasonable Prices Metric X11 Courteous Employees Metric X12 Competent Employees Metric Selection Factor Rankings X13 Food Quality Nonmetric X14 Atmosphere Nonmetric X15 Prices Nonmetric X16 Employees Nonmetric Relationship Variables X17 Satisfaction Metric X18 Likely to Return in Future Metric X19 Recommend to Friend Metric X20 Frequency of Patronage Nonmetric X21 Length of Time a Customer Nonmetric Classification Variables X22 Gender Nonmetric X23 Age Nonmetric X24 Income Nonmetric X25 Competitor Nonmetric X26 Which AD Viewed (#1, 2 or 3) Nonmetric X27 AD Rating Metric X28 Respondents that Viewed Ads Nonmetric

27. Samouel's Restaurant Description of Employee Survey Variables Variable DescriptionVariable Type Work Environment Measures X1 I am paid fairly for the work I do. Metric X2 I am doing the kind of work I want. Metric X3 My supervisor gives credit an praise for work well done. Metric X4 There is a lot of cooperation among the members of my work group. Metric X5 My job allows me to learn new skills. Metric X6 My supervisor recognizes my potential. Metric X7 My work gives me a sense of accomplishment. Metric X8 My immediate work group functions as a team. Metric X9 My pay reflects the effort I put into doing my work. Metric X10 My supervisor is friendly and helpful. Metric X11 The members of my work group have the skills and/or training to do their job well. Metric X12 The benefits I receive are reasonable. Metric Relationship Measures X13 Loyalty – I have a sense of loyalty to Samouel’s restaurant. Metric X14 Effort – I am willing to put in a great deal of effort beyond that expected to help Samouel’s restaurant to be successful. Metric X15 Proud – I am proud to tell others that I work for Samouel’s restaurant. Metric Classification Variables X16 Intention to Search Metric X17 Length of Time an Employee Nonmetric X18 Work Type = Part-Time vs. Full-Time Nonmetric X19 Gender Nonmetric X20 Age Nonmetric X21 Performance Metric

28. RESEARCH PROCESS Identify and Define Research Problem Theory / Practice Hypotheses / Conceptualization Research Design Data collection Data Analysis Findings

29. RESEARCH DESIGN – Types Research Design Alternatives – Purpose: (1) Exploratory – to formulate the problem, develop hypotheses, identify constructs, establish priorities for research, refine ideas, clarify concepts, etc. (2) Descriptive – to describe characteristics of certain groups, estimate proportion of people in a population who behave in a given way, and to make directional predictions. (3) Causal – to provide evidence of the relationships between variables, the sequence in which events occur, and/or to eliminate other possible explanations.

30. Two Broad Approaches: Research Design – Approaches Qualitative. Quantitative.

31. Role of Qualitative Research: RESEARCH DESIGN • Search of academic, trade and professional literature (both traditional & Internet). • Use of interviews, brainstorming, focus groups. • Internalization of how others have undertaken both qualitative and quantitative research. • Use of existing questionnaires/constructs. Outcome of Qualitative Research: • Improve conceptualization. • Clarify research design, including data collection approach. • Draft questionnaire.

32. Role of Quantitative Research: RESEARCH DESIGN • Quantify data and generalize results from sample to population. • Facilitates examination of large number of representative cases. • Structured approach to data collection. • Enables extensive statistical analysis. Outcome of Quantitative Research: • Validation of qualitative research findings. • Confirmation of hypotheses, theories, etc. • Recommend final course of action.

33. RESEARCH PROCESS Identify and Define Research Problem Theory / Practice Hypotheses / Conceptualization Research Design Data collection Data Analysis Findings

34. DATA COLLECTION • Approaches: • Observation • Human • Mechanical/Electronic Devices • Surveys • Self-Completion • Mail/Overnight Delivery/Fax • Electronic • Interviewer-Administered • Face-to-Face – Home, Work, Mall, Focus Groups • Telephone

35. DATA COLLECTION • Selection of data collection approach? • Budget • Knowledge of issues – qualitative vs. quantitative • Respondent Participation • Taste Test; Ad Test • Card Sorts; Visual Scaling • Time Available

36. DATA COLLECTION • Types of Data: • Primary • Secondary

37. PRIMARY DATA • Primary Data Sources: • Informal discussions; brainstorming • Focus groups • Observational Methods • Structured & Unstructured Surveys • Experiments

38. Primary Data – Focus Groups • Focus Groups = bring a small group of people (10-12) together for an interactive, spontaneous discussion of a particular topic or concept. Discussion is led by a trained moderator and usually lasts 1 ½ hours. • Typical Objectives: • To identify and define problems. • To generate new ideas about products, services, delivery methods, etc. • To test advertising themes, positioning statements, company and product names, etc. • To discover new constructs and measurement methods. • To understand customer needs, wants, attitudes, behaviors, preferences and motives.

39. Primary Data • Factors Influencing Overall Mobile Phone Satisfaction • 20032002 • Features 27% 21% • Durability 23% 16% • Physical Design 19% 28% • Battery Function 16% 16% • Operation 15% 19% • 2004 Wireless Retail Sales Satisfaction Study • Sales Staff 44% • Price/Promotion 28% • Store Display 14% • Store Facility 14% • Source: J.D. Power and Associates, 2002, 2003 & 2004. These percentages typically are determined in quantitative surveys (descriptive research). These factors typically are identified in qualitative focus groups (exploratory research).

40. Hotel Selection Factors:1. Location 2. Past Experience 3. Recommendations or Friends and Family 4. Brand ReputationGuest Satisfaction Factors:1. Guest Room 2. Departure Process 3. Pre-Arrival/Arrival Experiences 4. Hotel Services 5. Food & Beverage servicesNote: the first three factors account for more than 70 percent of guest satisfaction ratings.Source: J.D. Power & Associates, August 21, 2001. Primary Data

41. Original Equipment Tire Satisfaction Study:1. Product Quality39% - Number of tires with a problem - Number of problems experienced - Number of original tires replaced 2. Long-Term Performance22% - Wear ability - Length of warranty - Overall reliability & dependability - Freedom from pull to left or right3. Situational Performance19% - Traction on wet roads - Traction at fast starts - Holds road well in emergencies - Lack of vibration at highway speeds - Overall safety - Overall ride at highway speeds4. Design14% - Road quietness - Style & appearance of sidewalls - Tread design - Size of tire matches size of vehicle 5. Winter Traction5%Source: J.D. Power & Associates, August 27, 2001. What is the construct in this study? Primary Data

42. PRIMARY DATA – Focus Groups Focus Groups: Some of my “best” experiences? Some of my “worst” experiences?

43. PRIMARY DATA – Observations • CONSIDERATIONS: • Methods – human/mechanical/electronic. • Useful where respondent cannot or will not • articulate the answer. • Cannot be used to measure thoughts, feelings, • attitudes, opinions, etc.

44. Purpose of Questionnaires: PRIMARY DATA – QUESTIONNAIRES • To obtain information that cannot be easily observed or is not already available in • written or electronic form. • Questionnaires enable researchers to measure concepts/constructs.

45. QUESTIONNAIRE DESIGN • Steps in Questionnaire Design: • Initial Considerations – problem, objectives, target population, sampling, etc. • Clarification of Concepts – select variables, constructs, measurement approach, etc. • Developing the Questionnaire • Length and sequence. • Types of questions. • Sources of questions. • Wording, coding, layout and instructions. • Pre-testing the Questionnaire. • Questionnaire Administration Planning.

46. Open-ended Questions = place no constraints on respondents; i.e., they are free to answer in their own words and to give whatever thoughts come to mind.Closed-ended Questions = respondent is given the option of choosing from a number of predetermined answers. QUESTIONNAIRE DESIGN • Two Types of Questions: • Open-ended • Closed-ended

47. Examples of Open-ended Questions:·How do you typically decide which restaurant you will eat at?· Which mutual funds have you been investing in for the past year?·How are your investment funds performing?·Do you think airport security is better now than it was six months ago? QUESTIONNAIRE DESIGN

48. Open-ended Questions QUESTIONNAIRE DESIGN • Typically used in exploratory/qualitative studies. • Typically used in personal interview surveys involving small samples. • Allows respondent freedom of response. • Respondent must be articulate and willing to spend time giving a full answer. • Data is in narrative form which can be time consuming and difficult to code and analyze. • Possible researcher bias in interpretation. • Narrative is analyzed using of content analysis. Software is available (e.g., NUD*IST).

49. Content Analysis Software: TextSmart is a software package that enables users to view, manipulate and automate the coding or categorization of responses to narative data. The ability to automate the examination and organization of narrative data is particularly helpful when a ‘large scale’ survey is undertaken. It can be used to analyze any textual data, and its output can be exported to SPSS for further analysis. For example, you can do correspondence analysis* on a contingency table from a TextSmart analysis. For more information about TextSmart and related SPSS products visit the WWW site www.spss.com. QSR NUD*IST stands for Non-Numerical Unstructured Data Indexing and Theorizing. It is a popular computer software package used by researchers to analyze text from focus group or interview transcripts, literary documents and so on. It examines non-textual data such as photographs, tape recordings, films and so on. Users can us it to index and link several documents in a structured way to produce categorical data in a form amenable to further analysis. NUD*IST output can be exported to software programs such as SPSS and Excel. For more information about QSR NUD*IST and its related product NVIVO visit their website (http://www.scolari.co.uk/qsr/qsr_n4.htm).

50. Closed-end Questions: QUESTIONNAIRE DESIGN • Single Answer • Multiple Answer • Rank Order • Numeric • Likert-Type Scales • Semantic Differential