The index is dead Long live the index

1. 1 The index is dead � Long live the index UNAK Annual Conference Izmir, 9-11 October 2008 Maureen MacGlashan Editor, THE INDEXER A year or so ago, I attended an International Law Librarians� conference in St Petersburg. At one of the sessions, one of the participants mentioned casually that, of course, these days, with the possibility of on-line searches, indexes were no longer necessary or relevant. �Oh no�, came back the cry. The only problem was that students these days seemed to have no clue how to use an index. But, first things first �A year or so ago, I attended an International Law Librarians� conference in St Petersburg. At one of the sessions, one of the participants mentioned casually that, of course, these days, with the possibility of on-line searches, indexes were no longer necessary or relevant. �Oh no�, came back the cry. The only problem was that students these days seemed to have no clue how to use an index. But, first things first �

2. 2 What�s an index? A list of words or phrases, usually in alphabetical order, pointing the user in the right direction. In other words a signpost. And it�s not just books.And it�s not just books.

3. 3 Not just books but also A magazine or journal An audio or video tape A film A computer file Artefacts in a museum Records in an archive A collection of cartoons or photographs etc., etc..

4. 4 Why bother (1)? �I think � the order is not so good, that a man may think a place is missing, when it is only put in another place. Samuel Pepys on Newman�s Concordance (Diary, 8 June 1663) So, why bother? Here�s what a famous English diarist had to say.So, why bother? Here�s what a famous English diarist had to say.

5. 5 Why bother (2)? �Information that is left unorganized and hidden in inaccessible archives, in files, boxes, or on closed shelves, is as good as lost.� Meral Alakus, The Indexer, March 2008 And here�s Meral Alakus making the same point.And here�s Meral Alakus making the same point.

6. 6 Why bother? (3) �An electronic text of the whole work obviates manual indexing, since any word or phrase can be found by the software.� Gabriel Egan, Times Literary Supplement, November 2006 But say some � here�s an English literature professor writing on the subject � these days you don�t need an index. Just Google, or search your text. But let�s see what Googling or a free text search might produce � But say some � here�s an English literature professor writing on the subject � these days you don�t need an index. Just Google, or search your text. But let�s see what Googling or a free text search might produce �

7. 7 Long strings Churchill, Winston, 38, 207, 225, 306, 319, 327, 329-330, 332-337, 340-342, 344, 347, 350-352, 358, 364-71, 376, 379, 384, 399-402, 404, 433, 436-443, 447-448, 452-455, 457-458, 462, 465, 468, 471-472, 476-477, 480, 482, 484-488, 491, 495-496, 498, 500-501, 507-508, 514, 516, 518, 520, 523, 535, 541, 548, 550-552, 555-556, 558-561, 563, 565-568, 570, 576, 578, 592, 597, 602, 607-609, 614-16, 619, 626, 628-635, 637-673, 681, 689, 705-706, 712-714, 726, 737-738, 740-745, 754, 762-776, 782, 799, 814-815, 830 From the index to De Gaulle (Jean Lacouture, 1984) This happens to be an index, but it�s actually very much what a free text search would produce if you were looking for �Churchill�. In this case, 78 undistinguished �hits�. Frankly, what use is that? (Actually, a free text search would be even worse because it would stop every time it came to �Churchill�, regardless of whether it was Winston, (senior), Winston (junior), Randolph, Clementine etc.)This happens to be an index, but it�s actually very much what a free text search would produce if you were looking for �Churchill�. In this case, 78 undistinguished �hits�. Frankly, what use is that? (Actually, a free text search would be even worse because it would stop every time it came to �Churchill�, regardless of whether it was Winston, (senior), Winston (junior), Randolph, Clementine etc.)

8. 8 A more professional approach Churchill, Winston, Malakand Field Force, 234; in the Sudan, 283; and idea of imperial federation, 314; and tariff reform, 322; Gallipoli campaign, 358, 489; and Ireland, 380, 382; and Egypt, 391; and the Kurds, 398; and the Middle East, 400; air control of Iraq and Jordan, 401, 402; support for Zionism, 406; damns Amritsar massacre, 418; and Indian independence, 419-20, 429, 450, 483 etc. Here�s a more typical professional-type index entry for Churchill showing just the first few entries. 38 hits in all but most of them with enough extra information to show whether it�s worth the reader going to the page in question. From the index to The Rise and Fall of the British Empire (Lawrence James, 1994)Here�s a more typical professional-type index entry for Churchill showing just the first few entries. 38 hits in all but most of them with enough extra information to show whether it�s worth the reader going to the page in question. From the index to The Rise and Fall of the British Empire (Lawrence James, 1994)

9. 9 Variations on a name � (Erskine) Aersken Araskin Areskin Aerskine Aesking Araskine Areskin Areskine Arskeyne Arskin Arskine Auermesquin Auernesquin Aversequin Ershin Erschine Ersken Erskye Erskyn Erskyne Erskynn Ersskyne Herchine Herchyn Herskyne Hirskyne Ireskin Irskin Irskine Irskyn Irskyne Here we have a straightforward Scottish name, �Erskine�. But over the years it has had 32 acceptable variants. A nightmare for the family historian, but the indexer would find some way of easing the passage of a modern Erskine to his Auermesquin or Herchine grandfather.Here we have a straightforward Scottish name, �Erskine�. But over the years it has had 32 acceptable variants. A nightmare for the family historian, but the indexer would find some way of easing the passage of a modern Erskine to his Auermesquin or Herchine grandfather.

10. 10 A Turkish example Ahmet bin Ibrahim bin Ilyas Yesevi see Ahmet Yesevi Ahmet Yesevi (1093�1156) Haci Bektas-i Veli (1280�1337) Mevlana Celaleddin Rumi (1207�1273) Piri Sultan see Ahmet Yesevi Rumi see Mevlana Celaleddin Rumi (1207�1273) S�leyman �elebi (1408?�1421) Seyyid Muhammed bin Ibrahim Ata see Haci Bektas-i Veli This is a Turkish example, showing the indexer�s efforts in action. It comes from Meral Alakus� article on names in The Indexer. The entries in orange are see references. As Meral explains it � and all very familiar to all of you, of course, -- �Ahmet Yesevi� is the preferred name for Ahmet bin Ibrahim bin Ilyas Yesevi, who was also known as Piri Sultan. So see references are needed from Ahmet bin Ibrahim bin Ilyas Yesevi and Piri Sultan to Ahmet Yesevi. The usual brief reference to Mevlana Celaleddin Rumi is simply Rumi. So a cross-reference is made from Rumi to his full name.This is a Turkish example, showing the indexer�s efforts in action. It comes from Meral Alakus� article on names in The Indexer. The entries in orange are see references. As Meral explains it � and all very familiar to all of you, of course, -- �Ahmet Yesevi� is the preferred name for Ahmet bin Ibrahim bin Ilyas Yesevi, who was also known as Piri Sultan. So see references are needed from Ahmet bin Ibrahim bin Ilyas Yesevi and Piri Sultan to Ahmet Yesevi. The usual brief reference to Mevlana Celaleddin Rumi is simply Rumi. So a cross-reference is made from Rumi to his full name.

11. 11 Another Turkish example

12. 12 A reminder� (from Wikipedia) Internet search engines, such as Google, and full text searching help provide access to information but: They are not as selective as an index, as they provide non-relevant links: And they may miss relevant information if it is not phrased in exactly the way they expect. Just to remind ourselves why searches aren�t as good as indexes.Just to remind ourselves why searches aren�t as good as indexes.

13. 13 Search vs Index (Barnum, 2005) (Success) In 2004, Carol Barnum, professor of technical communication at Southern Polytechnic State University, Georgia, did a study on user preferences and performance. Let�s look first at the success. Search results are on the left, index results on the right. Of the 6 participants, 3 were much more successful using the index, 2 had the same 100% result with both, and one did much worse. The bottom line: searching produced an average success rate of 76% as against the index rate of 84%. In 2004, Carol Barnum, professor of technical communication at Southern Polytechnic State University, Georgia, did a study on user preferences and performance. Let�s look first at the success. Search results are on the left, index results on the right. Of the 6 participants, 3 were much more successful using the index, 2 had the same 100% result with both, and one did much worse. The bottom line: searching produced an average success rate of 76% as against the index rate of 84%.

14. 14 Search vs Index (Barnum, 2005) (Time) As far as the time taken goes, two people took longer using the index, the other four took significantly less time. The average time taken searching was 46:36 minutes, using the index 33:52. As far as the time taken goes, two people took longer using the index, the other four took significantly less time. The average time taken searching was 46:36 minutes, using the index 33:52.

15. 15 Search vs Index (Kingsley, 2005) (Time) Average time searching: 40 seconds. Average time with an index: 26 seconds. Here are the results of another study, by Ilana Kingsley (2005), looking at how long it took people to get a result on a website, using a simple search and the alternative of an index. The same striking results as in the Barnum study.Here are the results of another study, by Ilana Kingsley (2005), looking at how long it took people to get a result on a website, using a simple search and the alternative of an index. The same striking results as in the Barnum study.

16. 16 Search vs Index (AALL, 2005) (Success) Finally some results from some lawyer studies. presented at the 2005 Annual Meeting of the American Association of Law Librarians. Here are a set of success results. Finally some results from some lawyer studies. presented at the 2005 Annual Meeting of the American Association of Law Librarians. Here are a set of success results.

17. 17 Search vs Index (AALL) (2005) (Time) And here are the time taken results. Same story. A search took an average of 4 times as long to achieve success as did an idex (3.9 minutes as against 57 seconds). The Boston College Law Librarian commented that in her experience, using indexes improved accuracy, eliminated false positive results, and led to completion in ways that full-text searching could not. And here are the time taken results. Same story. A search took an average of 4 times as long to achieve success as did an idex (3.9 minutes as against 57 seconds). The Boston College Law Librarian commented that in her experience, using indexes improved accuracy, eliminated false positive results, and led to completion in ways that full-text searching could not.

18. 18 The index is clearly best, but will they use it? (Barnum) The index was obviously much better for finding things but � Only five of the 12 people we tested preferred the index. Five still preferred text-searching. Two didn�t care which method they used.

19. 19 The index is clearly best, but will they use it? (Kingsley) No. Nobody who took part in our survey chose to use the index. They all preferred searching.

20. 20 The index is clearly best, but will they use it? (AALL) �The survey asked the attorneys what kind of legal research they preferred to use. The number one choice was Google.�

21. 21 Why not? (1) Don�t know what an index is! Don�t know how to find it! Don�t know how to use it! It doesn�t give me what I want! Why don�t people use indexes? Here are some of the reasons they give (adapted from Browne and Jermey).Why don�t people use indexes? Here are some of the reasons they give (adapted from Browne and Jermey).

22. 22 Why not? (2) I don�t trust them� They are really, really long� They don�t use the same words as I do� In the latest manual on indexing to arrive in the bookshops, Glenda Brown and Jon Jermey put the problem neatly: [slide] In the latest manual on indexing to arrive in the bookshops, Glenda Brown and Jon Jermey put the problem neatly: [slide]

23. 23 Why not? (3) A librarian says: �Most undergraduates do not know what a journal article is, what a journal is, what an index is, or what is scholarly and reputable.� And a librarian says (Barker 1999) � just what the librarians in St Petersburg said 8 years later. And a librarian says (Barker 1999) � just what the librarians in St Petersburg said 8 years later.

24. 24 Why not? (4) �Library users had no idea what to call things. What librarians considered to be the most basic of terms were not known to them.� University at Buffalo Libraries The University at Buffalo Libraries found: [SLIDE]The University at Buffalo Libraries found: [SLIDE]

25. 25 Why not? (5) �To make things worse, most library web sites have been built by and for librarians, who do things differently than users do.� Carver (2002) But that was in 2000 and he did say his website was going through a major new design so perhaps things have got better. But that was in 2000 and he did say his website was going through a major new design so perhaps things have got better.

26. 26 So what do users want? �Basically, a straight �right� answer to a straight question, without the distraction of context or any ifs and buts. That�s what Googling and sound bites have brought us to.� Director of the Archives du Nord, Lille 2005 Isn�t that what so many of these usability studies are telling us? �I know what I want to know, and I want to know it fast, with no messing about.� The search habit has encouraged people to think questions which they think simple (but often aren�t) have simple answers (which they rarely do). So, what do you librarians, and what do I, an indexer, do about it?Isn�t that what so many of these usability studies are telling us? �I know what I want to know, and I want to know it fast, with no messing about.� The search habit has encouraged people to think questions which they think simple (but often aren�t) have simple answers (which they rarely do). So, what do you librarians, and what do I, an indexer, do about it?

27. 27 So what do we do about it? Sort out in our own minds what an index can do that a search can�t Provide good indexes wherever they will serve a useful purpose Educate, educate, educate �In fact, such is the illusive power of electronic searching that I suspect that many readers have no conception that they are searching indexes at all � However clever software may be � this prosaic fact remains: the hit relies on the index: a result is only a result because it has been indexed.� (Rupert Mann) �In fact, such is the illusive power of electronic searching that I suspect that many readers have no conception that they are searching indexes at all � However clever software may be � this prosaic fact remains: the hit relies on the index: a result is only a result because it has been indexed.� (Rupert Mann)

28. 28 How does Turkey measure up? �Whether we are talking about archive indexing, the serial indexing of ongoing publications, or back of book indexing, Turkey still has a long way to go.� Meral Alakus, The Indexer, March 2008 How does Turkey measure up to this test?How does Turkey measure up to this test?

29. 29 The Turkish story: archive indexing �A major problem is the inadequacy of systems in use in Turkish libraries � with major gaps in terminology and structure, inconsistencies, and duplication of expensive translation work.�

30. 30 The Turkish story: book indexes �To the extent that back-of-the-book indexes exist in Turkey, these tend to be just names indexes � �The concept of professional indexing does not exist � no professional training � no professional standards.�

31. 31 The Turkish story: the way forward �Turkey needs to form a national information policy and define knowledge management activities.� �Consider membership of one of the national indexing societies, distance learning, indexer discussion groups.�

32. 32 �A challenge well worth tackling� (and not just Turkey!) �The losers in the fight for understanding will be those who fail to organize information in a way that is honest, user-friendly and accessible.� I�ve spoken about Turkey so much simply because I am here, talking to a Turkish audience. But the rest of us are just as bad. OK, back of book indexes are still quite popular in the English-speaking world, and some archive and catalogue indexing has improved with the availability of new technology. But that illusion that nowadays a human index is an expensive and unnecessary luxury is fast taking hold. It�s not, for the likes of me, just a question of protecting our jobs, but much more importantly of doing all we can, in the fight for understanding, to ensure that we develop and maintain the very best tools for the purpose. I�ve spoken about Turkey so much simply because I am here, talking to a Turkish audience. But the rest of us are just as bad. OK, back of book indexes are still quite popular in the English-speaking world, and some archive and catalogue indexing has improved with the availability of new technology. But that illusion that nowadays a human index is an expensive and unnecessary luxury is fast taking hold. It�s not, for the likes of me, just a question of protecting our jobs, but much more importantly of doing all we can, in the fight for understanding, to ensure that we develop and maintain the very best tools for the purpose.

33. 33 And that means good indexing! Just a few pointers �Just a few pointers �

34. 34 In my view, the best manual presently available. As I said at the time of publication: �It has a real lightness of touch but also a profundity of knowledge and experience�. You should all have a copy on your shelves. Best way to get it? Through Amazon. In my view, the best manual presently available. As I said at the time of publication: �It has a real lightness of touch but also a profundity of knowledge and experience�. You should all have a copy on your shelves. Best way to get it? Through Amazon.

35. 35 Indexing societies around the world American Society for Indexing (ASI) www.asindexing.org Association of Southern African Indexers and Bibliographers (ASAIB) www.asaib.org.za Australian and New Zealand Society of Indexers (ANZSI) www.anzsi.org China Society of Indexers (CSI) www.chnindex.fudan.edu.cn Deutsches Netzwerk der Indexer (DNI) www.d-indexer.org Indexing Society of Canada/Soci�t� d�Indexation (ISC/SCI) www.indexers.ca Nederlands Indexers Netwerk (NIN) www.indexers.nl Society of Indexers (SI) (UK and Ireland) www.indexers.org.uk And if you fancy joining an indexing society, here are the details. If you join the British Society of Indexers or the Canadian Society you will get The Indexer free. With all the others, you are entitled to a generous discount.And if you fancy joining an indexing society, here are the details. If you join the British Society of Indexers or the Canadian Society you will get The Indexer free. With all the others, you are entitled to a generous discount.

36. 36 And here�s The Indexer itself (www.theindexer.org). You can read more about it on the postcards I have distributed. And here�s The Indexer itself (www.theindexer.org). You can read more about it on the postcards I have distributed.

37. 37 References Barnum, Carol, E Henderson, A Hood, R Jordan (Barnum 2004), �A Usability Study of User Preference and Performance�, TechnicalCOMMUNICATION, vol. 51, Number 2, May 2004, 185-206 BNA Law School (BNA 2005), �Using Online Indexes� http://subscript.bna.com/pic2/lsll.nsf/id/DTRS-5L3RPC?OpenDocument Carver, Blake (Carver 2000), Library and Information Science News, http://www.info-arch.org/lists/sigia-l/0205/0319.html Kingsely, Ilana M (Kingsley 2005), �Usability Study of Web Indexes at an Academic Institution� (http://www.web-indexing.org/asi2005-usability.pdf) Kupersmith, John (Kupersmith 2007), �Library Terms that Users Understand�, www.jkup.net/terms.html Mann, Rupert (Mann), �Searching the Oxford Dictionary of National Biography�, The Indexer 25, 16-18 Williams, Mary Elizabeth (Williams 2005), AALL Spectrum, Sept/Oct 2005, 20-1, http://www.aallnet.org/products/pub_sp0509_DR.pdf Wilson, T D (2004), �Information-seeking behaviour and the digital information world�, European Science Editing

38. 38 Other reading Browne, Glenda and Jon Jermey (Browne and Jermey 2007), The Indexing Companion, Melbourne and New York, CUP, 2007 Ciravegna, Fabio and Daniela Petrelli (Ciravegna 2006), �Annotating document content: a knowledge-management perspective�, The Indexer, 25, 23-7 Hiley, Nicholas, �Indexing cartoons�, The Indexer, 25, 100-4 Northedge, Richard (Northedge 2007), �Google and beyond: information retrieval on the World Wide Web�, The Indexer, 25, 192-4 Olason, Susan C (Olason 2000), �Let�s get usable!: Usability studies for indexes�, The Indexer 22, 91-5 Ryan, Christine and Sandra Henselmeier (2000), �Usability testing at Macmillan USA� Key words 8, 198-202 Scott, Mark (Scott 2005), �Indexing the law: a controlled vocabulary�, The Indexer, 24, 123-6 Wittman, Cecilia (Wittman 1990), �Subheadings in award-winning book indexes: a quantitative evaluation�, The Indexer, 17, 3-6

39. 39 The index is dead � Long live the index UNAK Annual Conference Izmir, 9-11 October 2008 Maureen MacGlashan Editor, THE INDEXER Back to where we started.Back to where we started.