Explorations in Internet-Based Collaborative Informatics Research: A Cognitive Evaluation

1. Explorations in Internet-Based Collaborative Informatics Research: A Cognitive Evaluation Vimla L. Patel, PhD, DSc Departments of Medical Informatics and Psychiatry College of Physicians and Surgeons Columbia University New York, New York

2. STANFORD COLUMBIA • Academic Research • Graduate Degree Program • Knowledge Representation • Advanced Server Technology • Decision-Support Systems • Knowledge Acquisition • Clinical Trials Workstations • Breast Mass Protocols • Academic Research and Practice • Graduate Degree Program • Clinical Information Systems • Medical Logic Standards • Decision-Support Systems • Clinical Workstations • Knowledge-Based Vocabularies • Cholesterol Guidelines Shortliffe Cimino Communication McGill Evaluation Studies Asynchronous Synchronous • Email - personal • Email Dist’n Lists • World Wide Web • Regular Mail • Telephone • Group Conference Calls • Video Conferencing • Face-to-Face Meetings Patel • Academic Research • Medical Informatics Training • Component-Based Software • Clinical Workstations • Integrated Guidelines • Decision-Support Systems • Knowledge Authoring Tools • Breast Cancer Guidelines • Academic Research and Practice • Integrated Clinical Training • Decision-Support Systems • Clinical Workstations • Integrated Guidelines • Clinical Information Systems • Electronic Medical Records • Flu Vaccine Guideline Greenes Barnett BWH - HARVARD MGH - HARVARD

3. Sources of Data • Electronic Interviews • Audio-Taped Conference Calls • Email • Current & Archived • Progress Reports • Papers, Presentations, and Proposals • Developments at Intermed WWW Sites • WWW Access Statistics • Access of Resources, Documents, etc.. • File Transfers Between Sites • Audiotapes of Mini Workshop(s)

4. Perspectives on Collaboration: E-Mail Interview PLEASE COMMENT ON THE FOLLOWING TWO STATEMENTS: • InterMed is a Highly Integrated Collaborative Endeavor with Largely Shared Assumptions. Work on Common Projects Require Shared Efforts and a High Degree of InterPersonal and Intergroup Integration • InterMed is a More Loosely Coupled Cooperative Project which Each Team Provides Shared Expertise and Resources towards some Common Goal and Towards Institution-Specific Goals. Progress is Largely a Function of Individual Intiative Efforts, which are Later Coordinated within the Larger InterMed Context.

5. Modes of Communication Best Suited To Specific Purposes PURPOSE DSG STANFORD COLUMBIA

6. Categories of Activity During Conference Call

7. Frequency of Guideline-Related Email Messages per Week 45 40 AMIA Fall Collaborative Submissions 35 Mini Workshop on Guidelines Vocabulary Guidelines Exp. 30 Intensive Work Begins on Guidelines Model/Detailed Agenda 25 Number of Emails 20 Representation Common Guidelines HPCC Meeting, 15 10 5 0 1 32 46 49 54 65 68 76 80 82 84 86 88 90 92 94 July 94 Jan 95 July 95 Dec 95 Feb 96 April 96 WEEKS FROM JULY 1994 TO APRIL 1996

8. Sequence of Goals in Progress Report In Progress Common Guideline Model Complete Specific Goals Prototypical Guideline/ WWW CGI Adapt And Display Guidelines on WWW July 94 April 96 MONTHS FROM JULY 1994 TO APRIL 1996

9. A B E F C D G H K J I L M Sociometric Analysis of Email Communication Patterns Stanford Columbia InterMed Central DSG MGH

10. A B E F C D InterMed Central K L M Email Communication Patterns (Jan-Feb 96) Episode 1: Vocabulary Server and Guidelines Stanford Columbia MGH

11. A B E F C D G InterMed Central H K J I Email Communication Patterns (Feb-Mar 96) Episode 2: Guidelines Representation Stanford Columbia MGH BWH

12. Conclusions • Significant Evolution in Communication Patterns • Greater Degrees of Integration • Focused Communication • Increased Use of Richer Representations • Diagrams, WWW • Integrated Plans and Coordinated Actions • Sustained Collaboration Requires Sustained Commitment • Different Modalities — Cognitive & Social Consequences • Learning Curve in Collaboration • Collaboration is difficult!