Did they get it or not?

1. Did they get it or not? Donna W. Bailey, RN, MN University of North Carolina at Chapel Hill

2. Acknowledgements • The opportunity to develop this research project was graciously facilitated and supported by Dr Sheila Englebardt. • Additional support for participation was provided by a Pre-Doctoral Fellowship NIH/NINR T32 NR07091.

3. Problem • Determining whether learning has occurred with course material where the primary interaction is dialogue between faculty and students using online discussion forums

4. Purpose • To explore the idea that discussion forum archives could be used to evaluate the use of concepts in dialogue as a form of behavioral change • To determine the feasibility of using a text analysis software to evaluate the student’s use of concepts in dialogue

5. Setting • University of North Carolina at Chapel Hill School of Nursing • Graduate nursing course: Health Care Informatics • Class Size: 27 students including students receiving academic credits and continuing education students

6. Assumption The use of concepts in student dialogue can be an indication that learning has occurred.

7. Course Overview • Structure of course • course outline • use of discussion forums • evaluation of discussion

8. Nursing informatics Data Information Knowledge Internet Project Management Change Computerized patient records Privacy Confidentiality Security Software applications Key Course Topics/Concepts

9. Orientation/WWW Change Management Project Management Data Elements Computerized Patient Record Applications Vendors Administrative Applications Staffing/Scheduling Privacy Confidentiality Security Future Discussion Cafe Discussion Forums

10. Participation • Total responses = 1370 over 15 weeks • Standard forums = 10 • Special forums = 4 • Range of responses in standard forums = 1 - 5 • Range of student responses in standard forums = 2 - 7

11. Method • Forums converted to text files and imported into Q.R.S.NUD*IST Vivo1.0 • Text search completed using course topics/concepts • Count of passages in discussion forums that included topic phrase or concept

12. Results • Topic/concept frequency was high in the discussion forum dedicated to the topic • Related topic/concepts were raised at lower frequencies in focus discussions where the related concept is commonly associated, I.e. privacy, confidentiality security in relation to CPR and applications

13. Results related to using text analysis software • Using text search software provided a quick way to scan a discussion forum for key topics/concepts • It assisted in in-depth analysis of dialogue by searching for other key words/phrases that familiarity with the discussion might surface as other key focal points in the discussion

14. Implications for using discussion forums • The strategy for course presentation involving discussion should be carefully planned to: • present key concepts • build on/relate to existing knowledge • structure subsequent discussions that reflect back on core concepts and require thoughtful integration into the next level of course material

15. Implications for using text analysis software • Access to and familiarity with using the program • Use of the software as adjunct tool to facilitate analysis and not the sole analytic method

16. Questions dbailey1@email.unc.edu donnaba@mindspring.com