Enhancing Research with Grey Literature: Usage, Advantages, and Access

1. GreyWorks 2010Transparency Governs the Grey Landscape MODULE THREE: Demand-side Grey Literature

2. Measures Demand-side • Citations • References • Website logs • User statistics • Links • Etcetera Farace – General

3. Applications and Uses Examples: • Problem solving • Decision making • Policy development • Scholarly communication • Research and innovation • Product enhancement • Technology transfer • Etcetera Farace – General

4. Grey Literature Usage • Respondents were asked to name 2 documents used in the last month that were important to their work. • Participants provided document titles and sources which we then located. • 59% of documents listed were Grey Literature. • Many thought they could find all Grey Literature via traditional online services. Liddy et al., GL6 - Usage

5. On usage statistics • The big silence. • Confirms the DRIVER study 2007: • 70% of the repositories do log the statistical data on access but analysis and interpretation « in development » or « problematic ». • No public access to statistics but in some cases, information to authors (CCSD). Schopfel et al., GL10 – Usage Stats

6. Advantages of including ‘grey’ in a literature review • Access to wider variety of information • Reduces publication bias • Contain more local information- important for planning • Contain process information- important for policy makers Knight et al., GL6 - Use

7. Advantages of including ‘grey’ in a literature review • Reports not published in peer reviewed journals (e.g. UN, WHO, HTA) • Access to new research • World Wide Web (4-6 Billion web pages) • Contain valuable and unique information, which is not found elsewhere Knight et al., GL6 - Use

8. Conclusion • The identification of relevant studies in the grey literature and their inclusion in systematic reviews can be particularly time-consuming and difficult. • However, the inclusion of grey literature in systematic reviews will help to overcome some of the problems of publication bias and evidence for process and context • Aberdeen University is taking forward the development of the tools which can be used to review the grey literature in terms of best available evidence and evaluation mechanisms Knight et al., GL6 - Use

9. Karst Information Portal 86.2% 82.8% 81.0% 96.6% Use Grey Information Proceedings Theses/ Dissertations + Maps Trip/CaveReports • To the statement, • “Grey literature is a very important source of information to karst researchers …” • 70.9 % agree Chavez et al., GL8 - Use Top Four Grey Information Sources n=58

10. Karst Information Portal 75.5% Preservation & Access Most Difficult Grey Information Types to Access Where do you archive the grey information that you produce? Personal Archiving Space No Formal Archiving 28.3% Chavez et al., GL8 - Preservation n=49

11. 96.6% Use Grey Information n=58 86.2% Proceedings 82.8% Trip/Cave Reports 81.0% Theses/Dissertations & Maps 89.3% Produce Grey Information n=56 80.8% Proceedings 76.9% Trip/Cave Reports 65.4% Images & Speeches/Invited Talked Chavez et al., GL9 – Use and Production

12. KIP & Grey Literature A core function of KIP is to facilitate access to grey literature such as: Audience for KIP: • Geologists & geographers • Land managers & land planners • Maps • Insurance companies & home buyers • Measurements of cave notes • Biologists & cave enthusiasts • Notes on subterranean wildlife Chavez et al., GL9 – Access and Use

13. Analysis of GL in the changing context (2) IN FACT Why is it important to raise the profile of GL? Because grey literature is still useful… GL is more likely to: • report negative results • discuss studies that concluded prematurely • offer a much more comprehensive picture of the state of knowledge for any topic De Castro et al., GL8 - Use

14. Which HEP information system do you use the most? High Energy Physics … ? Gentil, GL9 - Use

15. Variations according to search Gentil, GL9 - Use

16. What feature is important in a HEP information system? NOT IMPORTANT VERY IMPORTANT Gentil, GL9 – User Demands

17. Conclusions • What did we learn? • Community-driven systems answer the needs of the community. • Listen to the users – they know what they want! • Increased importance of a diversifying grey literature… • Full analysis still in progress – more coming soon! • What’s next: • Create a single information service for the entire community • Include new features • Survey your users! Gentil, GL9 – User Needs

18. Evidence Based Practice5 Essential Steps in EBP • Convert information needs into answerable questions • Track down best evidence to answer questions • Critically appraise evidence performance for its validity and usefulness • Apply results of the appraisal in clinical practice • Evaluate performance and outcomes Gelfand, GL7 – Evidence based Practice

19. Hierarchy of Evidence • Meta-Analysis • Systemic Review • Randomized Controlled Trial • Cohort Studies • Case Control Studies • Case Series / Reports • Basic Laboratory & Animal Research Gelfand, GL7 – Evidence based Practice

20. Table 8. Expressions used by respondents to describe their experience in GL, ranking by frequency • Adjective Frequency • Interesting 16 • Stimulating 10 • Tiring 8 • Useful 7 • Demanding 4 • Formative 3 • Frustrating 3 • Complex 2 • Comprehensive 2 Di Cesare et al., GL8 - Users

21. (13) The Average Net User should at least recognize the term Grey Literature Leading Questions Boekhorst et al., GL6 – Users

22. Comparative Data Survey Data 2004 with a control population from the Author Survey 2005 Farace et al., GL7 – Access

23. Getting to the R&D Results • The Surface Web is accessible to popular search engines such as Google. • But less than 1% of government R&D results are currently accessible to crawlers. Slide compliments of Walt Warnick Carroll, GL8 - Access

24. What is missing? • Governmental reports • Funding body reports • Subject population materials Rudasill, GL10 – Access (Development Studies)

25. Possible reasons for lack of disclosure • Competition for funding dollars • Confidentiality of client records • Fear of reprisal • Lack of expertise in web programming Rudasill, GL10 – Access (Development Studies)

26. Web access to grey literature Schopfel, GL9 – Access

27. Access to full text Schopfel et al., GL10 – Access (full-text)

28. Freedom of Information • Mandatory FOI publication schemes provide an inventory of grey content • Police work requires well-organised records and this should permit retrieval in response to enquiries from the public • FOI enquiries received are in addition to high volumes of press enquiries • FOI enquiries have produced some ‘unwilling’ revelations from police, but also some information that is less revealing. Sturges et al., GL9 – Access and Retrieval

29. Some preliminary results CISTI • Decrease of supply requests • Open access: CNRC serials • Open access: institutional repository • Integrated, partnership-based approach • Change of acquisition policy? Boukacem et al., GL7 – Document Delivery

30. Some preliminary results INIST • Decrease of supply requests • Open access: CNRS strategic policy • Open access: institutional repository, portals • Open access: LARA, e-publishing, conferences • Change of acquisition policy? Boukacem et al., GL7 – Document Delivery

31. Conclusions • Search engines for grey literature are being widely deployed and represent a central information service in scholarly communication • Institutional repositories gain momentum and become dominant over disciplinary repositories • Standardized frameworks for distributed and federated document processing have been established Vesely et al., (CERN) GL5 - Repository

32. (14) GL Collections are beter managed by centralized repositories… The Document Level Boekhorst et al., GL6 – Repositories

33. More Definitions Institutional Repositories Set of services that a university offers to the members of its community for the management and dissemination of digital materials created by the institutions and its community members.” Gelfand, GL6 - Repositories

34. Faculty Benefits • Diminished Isolation • Shared Purpose & Cooperation • Increased Curricular Circulation • Fresh Approach to One's Discipline • Increased Satisfaction with Students' Learning Gelfand, GL6 - Repositories

35. Selective Repository Benefits • Free to contribute for all UC affiliates • Promising alternative • Increased visibility • Usage reports • eMail notification • Permanence • Global accessibility • Ability to upload associated content Gelfand, GL6 - Repositories

36. Long Standing and Ongoing Issues • Transcience of grey literature • Maturation of the repository • Timely publishing • Access • Standards • Multiple formats Gelfand, GL6 - Repositories

37. Institutional Repositories: Challenges • Difficult to convince researchers of benefits of institutional repositories • Consequent lack of interest in depositing materials, even if they will be managed by others • Will take further evaluation to determine if institutional repositories reach full potential • For now, coverage of grey literature in institutional repositories is encouraging Banks, GL7 - Repositories

38. Importance of Grey Lit from Public Health NGOs • NGOs and their information products are key to public health in developing countries • Grey lit from NGOs is often difficult to find Pakistan Healthcare Facilities Crowe et al., GL7 – Repository (NGO Publications)

39. Possible Solution: Repository • Digital collection that captures and preserves the intellectual output of an institution, agency or organization • Benefits • Data collection and coordination • Could support the building of health capacity in developing countries • Knowledge management tool for NGOs Crowe et al., GL7 - Repository

40. Conclusion • Grey lit from NGOs is a vital component of public health and medical care information, particularly in developing countries • Public health NGOs would benefit by having one or more repositories of grey literature • Relevant repository models exist • There are significant barriers and challenges • There are significant benefits to public health research community and to people world-wide Crowe et al., GL7 - Repository

41. Role of the institutional repository • The IR is a support for integration • A central point within the organisation for literature and data • Stores multiple manifestations for the different parts of the process • Brings clarity to work and version relationships • GL can be recorded and retrieved according to accurate and up to date personnel, project and business unit structure • IR integrated as part of the institutional memory Lambert et al., GL7 - Repository

42. A Little Less Grey? 1990: 67% of health sciences journals cite grey literature Alberani V, De Castro Pietrangeli P, Mazza AM. The use of grey literature in health sciences: a preliminary survey. Bull Med Libr Assoc. 1990 Oct;78(4):358-63. 2004: 85% of health policy journal articles cite grey literature 2004: 61% of think tank reports cite journal articles Dunn, GL6 - Citations

43. Triangle of Stakeholders Schopfel et al., GL6 - Citations

44. Grey versus Commercial A Comparison of Standard and Hyperlinked Citations Schopfel et al., GL6 - Citations

45. Language of Cited Documents and Citing Authors Schopfel et al., GL6 - Citations

46. Top Five “Types of Publications” Schopfel et al., GL6 - Citations

47. Guidelines for GL Citations • General • All conference papers should contain references • Standardization should be maintained among the citations provided • The more complete and accurate a citation, the more guarantee of a paper´s content and subsequent review Specific • Endnotes are preferred and should be numbered • Hyperlink need the accompanying name of resource and date;a simple URL is not acceptable • If the citation is to a corporate author, the acronym takes precedence • If the document type is known, it should be stated at the close of a citation • If a citation is revised and refers to an edited and/or abridged work,the original (re)source should also be mentioned. Schopfel et al., GL6 - Citations

48. Citations to GOMC PublicationsGulf of Maine Council on the Marine Environment MacDonald et al., GL8 - Citations

49. MacDonald et al., GL8 - Citations

50. MacDonald et al., GL8 - Citations