Development and Validation of the Public Affairs Scale:

1. Development and Validation of the Public Affairs Scale: Implications for Research on Service Learning and Community Engagement

2. Chantal Levesque-BristolElizabeth Carmichael StrongJeffrey Cornelius-WhiteYogini DhopadeNicole Howland International Association for Research on Service Learning and Community Engagement Annual Conference – 2010

3. Public Affairs Mission at Missouri State • In 2007-2008 – At the beginning of the Public Affairs Scale Development • Re-definition and Focus of the Mission • Search for an Associate Provost for Student Development and Public Affairs • Operationalization of the public affairs mission at the department and college level unit of analysis

4. Pillars of the Public Affairs Mission • Community Engagement • Cultural Competence • Ethical Leadership

5. Community Engagement • Community Engagement is recognizing needs in the communities within which one belongs, then contributing knowledge and working with the community to meet those needs.  Community engagement requires extending beyond one's self for the betterment of the community - a process that fosters greater awareness and personal growth. 

6. Cultural Competence • Cultural Competence begins with cultural self-awareness and expands to knowledge of, respect for, and skills to engage with those of other cultures.  Culturally competent individuals respect multiple perspectives and are able to successfully negotiate cross-cultural differences.

7. Ethical Leadership • Ethical Leadership is striving for excellence and integrity as one continually develops ethical and moral reasoning while contributing to the common good.  Ethical leaders have the courage to live by their principles in all parts of their personal and professional lives (Center for Ethical Leadership, 2007).

8. Indicators of the Public Affairs Mission • Leader in the American Democracy Project • e-Citizenship initiative • Annual Public Affairs Conference • Internationally known speakers • Annual Public Affairs Week • Organized by students • Common Reader Programs and Themed Year-Long Activities • 2008-2009 Sustainability • 2009-2010 Evolving Economic Realities • 2010-2011 Leading in a Global Society • Citizenship and Service-Learning Office

9. Why develop the Public Affairs Scale? • Operationalization of the Public Affairs Mission – develop Public Affairs scale • Assessment of the development of the Public Affairs pillars in students-validity of scale • Outcomes of civically engaged students from perspectives of community partners and students

10. Method Sources of information examined Development procedure Participants Analytical Strategy

11. Sources of Information examined • Center for Ethical Leadership • American Speech-Language-Hearing Association • Alameda County Public Health Department • Association of American Colleges and Universities • Carnegie Foundation for the Advancement of Teaching • Center for Disease Control • Transcultural Nursing, • US Department of Health and Human Services: Health Resources and Services Administration.

12. Public Affairs Scale Development Procedure • Review of information and sources described above • Construction of 81 items to assess the three pillars of the mission • 30 items were created to assess Community Engagement • 22 items were created to assess Cultural Competence • 29 items were created to assess Ethical Leadership • Construct validity assessed with Socially Responsible Leadership Scale – Revised (Tyree, 1998) • previously validated instrument derived from the Social Change Model (Higher Education Research Institute, 1996),

13. Socially Responsible Leadership Scale • Consciousness of Self (individual) • Congruence (individual) • Commitment (individual) • Collaboration (group) • Common Purpose (group) • Controversy with Civility (group) • Citizenship (community/society) • Change (community/society)

14. Socially Responsible Leadership Scale • Change – believing in the importance of making a better world and a better society through some activity • Citizenship – believing in a process whereby an individual and/or a group become responsibly connected to the community and to society through some activity • Collaboration – working with others in a common effort • Commitment – having the energy that motivates an individual to serve and that drives the collective effort

15. Socially Responsible Leadership Scale • Common Purpose – having shared goals and values when working with others • Congruence – thinking, feeling, and behaving with consistency, genuineness, authenticity and honesty toward others • Consciousness of Self – being aware of the beliefs, values, attitudes and emotions that motivate a person to take action • Controversy with Civility – believing in two fundamental realities of any creative effort: (1) that differences in viewpoint are inevitable, and (2) that such differences must be aired openly but with civility

16. Participants • Mostly women (70.6%) • Age ranged from 18 to 59 with a mean age of 23 • Mostly Caucasians (80%) • Approximately 69% of them were undergraduate students • Mostly freshmen (35.7%) • Approximately 10% each of sophomores, juniors, and seniors. • The students came from all colleges with the College of Health and Human Services (28.6%) and the College of Education (23.1%) having the most participants.

17. Analytical Strategy • Original 81 items were subjected to an Exploratory Factor Analysis with 3 factors specified • Retained items that maintained conceptual breadth within the three areas. • Item reduction • Eliminating items not significantly loading (belonging) on their predicted factor • Eliminating items loading on the wrong factor • Eliminating items loading on two or three of the factors to a relatively equal degree (cross-loading).

18. Results Scale Structure and Reliability Scale Means and Standard Deviations Scale Means by Class Rank Correlations between the Pillars of the Public Affairs and Class Rank Correlations Between the Pillars of the Public Affairs and Social Responsibility

19. Scale Structure and Reliability • Scale is presented in packet of handouts • http://www.missouristate.edu/assets/fctl/Public_Affairs_Self-Report_Scale_Validation_11-16-09.pdf • Stable 3-factor Public Affairs Scale explaining 45% of the variance • Community Engagement (15 items) • a = .90 • Cultural Competence (10 items) • a = .75 • Ethical Leadership (15 items) • a = .92

20. Scale Means and Standard Deviations For the three pillars of the Public Affairs Scale

22. Scale Means by Class Rank For the three pillars of the Public Affairs Scale

24. Correlations between the Pillars of the Public Affairs and Class Rank • Community Engagement (r = .14, p < .05) • Cultural Competence (r = .17, p < .01).

25. Correlations Between the Pillars of the Public Affairs and Social Responsibility • All correlations significant at the p < .001 level. • Highest correlations were generally found with Ethical Leadership • Correlations were highest between the subscales of the Public Affairs and the Socially Responsible Leadership Scale that are conceptually most similar • Provides empirical validation for the Public Affairs Scale and Conceptualization • Correlations were high and significant without being too high to signify that the constructs were simply replicated in another scale.

26. Correlations Between the Pillars of the Public Affairs and Social Responsibility • Community Engagement and Citizenship (r = .706) • Ethical Leadership and Collaboration (r = .688) • Ethical Leadership and Congruence (r = .681) • Cultural Competence and Change (r = .570). • Cultural Competence and Controversy with Civility (r = .529).

27. Use of the Public Affairs Scale to Evaluate a First Year Service-Learning Program

28. Presentation Outline • Theoretical framework • Measurement strategy • Data analysis • Interpretation • Contributions to the field • 2010 research • Questions for future research

29. Theoretical framework • Service-learning intervention in the first-year experience course contributes positively to self-efficacy, leadership, interpersonal skills and engagement in the campus and community • Knapp, T., Fisher, B., & Levesque-Bristol, C. (2010) Service-Learning’s Impact on College Students’ Commitment to Future Civic Engagement, Self-Efficacy, and Social Empowerment, Journal of Community Practice, 18: 2, 233-251

30. Measurement Strategy • Introduction to University Life • One credit class • 107 sections • 4 service-learning sections • 203 students completed the Time 1 survey, and 173 completed Time 2 survey • 87 in the service-learning sections • 116 In the non service-learning sections • 135 completed both assessments

31. Measurement Tool Collaboration with the Faculty Center for Teaching and Learning • Public Affairs Self-Report Scale • Self efficacy • Five facets of Cognitive Communication Competence (planning, modeling, presence, reflection, and consequence) Survey includes 3 sections • Identification number • General information • 76 questions rated in Likert Scale Strongly Disagree to Strongly Agree Always True of Me to Never True to Me

32. Sample questions • I know I can make a difference in my community (self efficacy) • I am able to communicate effectively with people from different cultures (communication) • I am dependable and reliable (leadership) • I feel insecure about my ability to do things (contributing to campus and community)

33. Survey Administration • CASL staff members administered pre- and post-surveys during the first and last week of classes • 4 SL sections and 5 control sections • Staff members read a one page survey explanation • Surveys were gathered and data was entered into SPSS

34. Data Analysis • Mean comparisons between service-learning sections (N = 80) and non service-learning sections (N = 93) at time 2. • Mean comparisons for service-learning (N = 66) and non service-learning (N = 69) sections to evaluate change over time.

35. Mean Comparison SL Vs. Non SL

36. Change Over Time SL and Non SL

37. Interpretation of Findings • At the end of the semester, there were significant differences on all the public affairs subscales and self-efficacy • Students in the service learning sections reported greater levels of cultural competence, community engagement, ethical leadership, and self-efficacy • Marginally significant differences were observed for planning and reflection

38. Interpretation of Findings • Throughout the semester, change was more pronounced for students in the service learning sections • Cultural competence, community engagement and planning

39. Contributions to the field • Missouri State’s research underscores findings of other service-learning studies, i.e., Astin, & Sax, Eyler & Giles, Astin, Volelgesang, Ikeada & Yee, Bringle & Hatcher, in regard to self-efficacy, leadership and engagement • Students in service-learning sections reported greater levels of cultural competence, ethical leadership, and community engagement • Service-learning impacted the increase in cultural competence, one pillar of the Public Affairs Mission

40. Continued Research • Introduction to University Life (IDS 110) moved from a 1 hour course to a 2 hour course (GEP 101) • Nine SL sections available • 97 sections overall • Research continues this semester

41. Questions • How do service-learning experiences in the first year shape the rest of a student’s college career? • How do service-learning experiences in the first year impact a student’s commitment to service beyond college?

42. Overall Implications • Public Affairs Scale Development • Operationalization of the Public Affairs Mission – develop Public Affairs scale • Assessment of the development of the Public Affairs pillars in students-validity of scale • Outcomes of civically engaged students from perspectives of community partners and students • This last point is the goal of some future research • Emphasize continued reciprocity

43. We would love tohear from you! • Chantal Levesque-Bristol • clevesque@missouristate.edu • Elizabeth Carmichael Strong • elizabethstrong@missouristate.edu

44. Bibliography Astin, A.W.(1993). What matters in college? Four critical years revisited. San Francisco: Jossey-Bass. Astin, A. W., & Sax, L. J. (1998). How undergraduates are affected by service participation. Journal of College Student Development, 39(3), 251-263. Astin,A.W., Volelgesang, L.J., Ikeda, E.K., & Yee, J.A. (2000). How service-learning affects students. Higher Education Research Institute, University of California, Los Angeles. Bringle, R.G., & Hatcher, J.A. (1995). A service-learning curriculum for faculty. Michigan Journal of Community Service Learning, 2, 112-122. Eyler, J., & Giles, D.E. (1999). Where’s the learning in service-learning? San Francisco: Jossey Bass. Eyler, J., Giles, D. Stenson, C. & Gray, C. (2001). What We Know about The Effects of Service-Learning on College Students, Faculty, Institutions and Communities, 1993-2000. Third Edition. Scotts Valley, CA: Learn and Serve America National Service—Learning Clearinghouse. Tyree, T. M. (1998). Designing and instrument to measure socially responsible leadership using the social change model of leadership development. Unpublished doctoral dissertation, University of Maryland, College Park, MD. Zlotkowski, Edward. (2002). Service-Learning and The First-Year Experience: Preparing Students for Personal Success and Civic Responsibility (Monograph No. 34). Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition.