SWM43 Research in Computing: Introduction to Computing Research

1. SWM43 Research in Computing: Introduction to Computing Research Anja Belz Natural Language Technology Group CMIS A.S.Belz@brighton.ac.uk

2. Purpose of this lecture • To touch on all aspects of research at least briefly, giving pointers to further reading (see list of references on last two slides) • To look in more detail at the two areas that are of most practical relevance to you at the moment: • Planning research • Reviewing the literature • To give practical guidance (including an exercise) on how to get started with research

3. Overview Part I – Introduction • Four golden lessons • What field are we in anyway? • Examples of research projects in informatics Part II – Preparing the ground • Planning research • Reviewing the literature • Exercise: first steps in planning research

4. Part I – Introduction • Four golden lessons • What field are we in anyway? • Examples of research projects in informatics

5. Four golden lessons (Weinberg, 2003) • No one knows everything, and you don't have to. • Go for the messes — that's where the action is. • Forgive yourself for wasting time. • Learn something about the history of science, or at a minimum the history of your own branch of science.

6. What field are we in anyway? • Computing science, computer science, information processing, information technology, software engineering … ? • Now increasingly known as: Informatics • Old subdivisions: • Computer science: study/build artificial systems • Artificial intelligence: emulate intelligence using artificial systems • Cognitive science: study mind from computational perspective … aren’t really separate anymore!

7. Informatics techniques (Bundy) Types of techniques: • Theories • Architectures • Information representation • Algorithms • Software engineering processes Research aims: • Extend knowledge about properties of techniques • Improve existing techniques • Create new techniques • Combine techniques to create systems

8. Informatics techniques (Bundy) Implications of technique type and research aim: • What broad phases your research will go through • What kind of evaluation is appropriate • What kind of use will be made of your research Specific research field also important: • Fine-grained divisions: evolutionary robotics, machine translation, computer vision, etc. • Standard tools and methodologies to apply • Terminology, knowledge you can take as given • Dissemination media and style

9. Examples of informatics research • Academic: • Masters and PhD thesis research • Research internally funded by universities etc.; some incidental, some more formally in projects • Publicly funded research projects (UK research councils, European Commission, US National Science Foundation) • Industry: R&D departments, research labs, dedicated research companies; limited dissemination • Private: people working from their home computers, in their garages, attics, garden sheds • French girl who invented speech recognition technology in her parents’ garage • Chinese farmer Wu’s robots • Participants in DARPA Autonomous Vehicle Challenge

10. Examples of informatics research • Some Informatics MSc topics from University of Edinburgh (2003-07): • A P2P Network Visualiser • Automated Probability Assessment in Plausible Crime Diagnosis • EvoTanks II: Co-evolutionary Development of Game Playing Agents • Cooperative Multi Agent Systems in Automobiles • A Web Service Interface to Astronomical Databases • Clustering Tags of Social Bookmarking Sites • Hardware Evolution: Automatic design of electronic circuits in reconfigurable hardware by artificial evolution (PhD, University of Sussex)

11. Examples of informatics research • Probabilistic Deep Generation (3-year project, funded by EPSRC): “to develop, for the first time, a comprehensive, linguistically informed, probabilistic methodology for generating language that substantially improves development time, reusability and language variation in language generation systems, and thereby enhances their commercial viability”. Value (£):211,199, University of Brighton. • Research Consortium in Speckled Computing (4-year project, funded by EPSRC): “a radically new concept in information technology […] realised by minute autonomous specks, each of which encapsulates sensing, programmable computation and wireless networking. Computing with minute specks will enable linkages between the material and digital worlds […] will be fundamental to truly ubiquitous computing”. Value (£):3,721,432, Edinburgh.

12. Part I – take-home points • Take on board Weinberg’s 4 golden lessons • There is a huge variety of research in informatics • Be aware of the types of technique you’re working on, and of what your overall research aim is • It’s important to learn about the research methodologies, evaluation criteria and other conventions in your field of informatics research • You can make an important contribution to research wherever you are

13. Part II – Preparing the ground • Planning research • First steps • Background reading • Methodology • Aims, outcomes, deliverables • Requirements • Subdividing and estimating effort: work packages • Writing a research proposal • Reviewing the literature

14. Planning research – first steps • Aim: Come up with a rough sketch of the research you want to do • Identify an area of informatics that interests you • Do some superficial background reading: • Wikipedia – but bear in mind it’s not 100% reliable, and don’t cite it! • Course web pages from leading universities • Websites of professional organisations • Research project websites • Individual researchers’ webpages • Decide on a smaller area in which to locate your research (but still larger than your project), and identify several ways in which you could make an original contribution

15. Planning research – first steps • Start identifying key characteristics of your chosen area: • Important conferences • Leading journals • Internationally leading researchers • Current research projects • Mailing lists • History: when did it begin? What are key advances, when did they happen? • How good is current technology? • What are the issues research is currently grappling with?

16. Planning research – background reading • Aim: familiarise yourself with chosen field of research well enough to decide on your project and write a short outline of it • Read survey papers and look up textbooks • Start reading (abstracts of) academic papers in conference proceedings and journals • Start compiling a bibliography (with star ratings) • Use tools like Google Scholar to check status of publications • Continue to collect key characteristics of area • Write project outline (a few sentences)

17. Planning research – methodology • Aim: to decide on the technical details of how you’re going to carry out your research • Won’t be able to specify all of this in advance – some of it is necessarily part of doing the research • E.g. if building a system: outline of architecture and functionality of modules; general approach (e.g. symbolic or statistical), even algorithms • Good idea to include fall-back options (if A doesn’t work I’ll do B)

18. Planning research – aims, outcomes, deliverables • Aim: clarify the purpose of your research to the point where you can write it down in detail • Aims: overall goals you hope to achieve with your research • Outcomes: specific results you plan to achieve with your research • Will knowledge be increased? How? • Will new resources be produced? Which ones? • Will new techniques be created? Existing ones improved? • Deliverables: the specific documents, software and other resources you commit to producing, with deadlines • Technical reports, manuals, webpages, etc. • Software specifications (modules, interfaces), tools, systems, etc. • Data collections (database of images, corpora of texts, etc.)

19. Planning research – requirements • Aim: to determine everything you will need to carry out your project, apart from your own time and effort • Are you going to carry out experiments involving people? How many subjects? Will the university’s research ethics allow it? • Programming environments, tools, your skills. • Equipment, data, licenses, etc. • Will any of it cost anything? Where will the money come from? • If you’re not sure, find out now!

20. Planning research – work plan • Aim: to create a detailed research plan which lists work packages, specifies the amount of time required for each and assigns a time slot to each • Divide tasks into related groups (work packages, WPs); write short descriptive summary for each WP • Estimate time/effort each WP will take (person days or weeks) – always add contingency! • Establish partial order of WPs – which WP requires other WPs to have been completed? • Create a calendar diagram where each WP is assigned a slot – don’t forget to allow for other commitments

21. Planning research – writing a research proposal • Aim: to put the results of your planning work into prose that will convince people that your planned research is of quality and worthwhile • Actually very time-consuming! • Sections: • Synopsis – “executive summary” • Introduction – motivate your research, why is it needed? • Aims, outcomes, and objectives • Related research – compare and contrast with existing work • Methodology – describe what you’re going to do, clarifying what is new, and where you’re going to use existing resources and ideas • Research in wider context – beneficiaries, dissemination, marketability, etc. • Work plan • Bibliography

22. Planning research – writing a research proposal • Give it to different people to read – for: • Grammar/style: does it read well? • Clarity: are your aims and plans clear? • Quality: is this a good idea? • Look at general advice on academic writing

23. Writing a literature review • Literature review can build on, but goes beyond, research planning • More in-depth reading/understanding than you need for project proposal • Typically part of reports of completed research (Masters and PhD theses, project reports etc.) • Or publications in their own right: survey articles in journals or as book chapters

24. Writing a literature review • Aim: to thoroughly review a given area of research, mentioning all important relevant research • Two basic forms: • Survey/state-of-the-art: balanced overview of given area of research; inclusion and space reflect importance of work in field; keep opinion to minimum • Project-specific review: inclusion and space reflect relevance to project; lead up to justification and motivation for project; opinion is important part of review • Important difference: for surveys, you don’t need to understand in detail how techniques work, but if it’s relevant to your project you do need to

25. Examples of survey-type literature reviews • Emotional language generation: http://www.itri.brighton.ac.uk/~Anja.Belz/Publications/ITRI-03-21.pdf • Speech technology (1994): http://doi.acm.org/10.1145/175247.175252

26. Writing a literature review – reading • Identify relevant publications • Determine status/importance (e.g. use Google Scholar to check on citations) – you can’t read all relevant publications • Use star-ratings to reflect importance • Read at different depths ***: read article in depth **: skim article *: read abstract and conclusions • Build up annotated bibliography (title, publication details, summary of contents, your comments)

27. Writing a literature review – writing • Don’t just list names and contents of papers – that’s an annotated bibliography, not a literature review • Turn it into a story – tell the story of the field • How does it all fit together? • What are the subfields, developments, controversies? • What is the state of the art? • What are the hot topics at the moment (reflected in special themes and special sessions at conferences, one-off workshops, and special issues of journals)

28. Writing a literature review – writing • Give your review structure: • Introduce the field • Bring out commonalities and differences between approaches • Can key results be summarised in a table or a graph? • Comment on more/less successful approaches • Conclusion: • Survey: summarise the state of the art of the field • Project-specific: identify area(s) where more research is needed • Bibliography • Appendix: research groups, data resources, web links

29. Part II – take-home points • Zoom in on your chosen area of research gradually (don’t start with academic papers): • Superficial reading of online material • Background reading of survey articles, text-books, etc. • Literature review of research area of appropriate size: academic papers in conference proceedings and journals, book chapters, etc. • Read at different depths: • Read most important papers carefully • Skim less important papers • Read abstracts and conclusions of least important papers

30. Part II – take-home points • Plan every aspect of your research thoroughly and in detail • For a literature review, do not just list publications and contents – instead, tell a story!

31. Exercise: first steps in research planning Aim: write a short review of the area of Machine Translation and prepare a brief presentation of it Steps: • Look at p. 14, do online research, and fill in as many of the categories on p. 15 as you can; • Write your findings up as a 1-page report; • Prepare a presentation of your findings, about 5 minutes in length; • Deliver the presentation on Friday morning. Work in lab (W622) this afternoon, tomorrow morning and on Thursday; finish report and presentation for Friday.

32. References • Stephen Weinberg’s Four Golden Lessons: http://www.nature.com/nature/journal/v426/n6965/full/426389a.html • Types of Research in Computing Science, Software Engineering and Artificial Intelligence by Aaron Sloman: http://www.cs.bham.ac.uk/research/projects/cogaff/misc/cs-research.html • Alan Bundy’s Researcher’s Bible: http://homepages.inf.ed.ac.uk/bundy/how-tos/resbible.html • CMU’s Advice on Research and Writing: http://www.cs.cmu.edu/afs/cs.cmu.edu/user/mleone/web/how-to.html

33. References • A Computer Scientist's Guide to Writing and Publishing Technical Articles by Paul Martin: http://www.mcs.vuw.ac.nz/comp/Publications/archive/CS-TR-95/CS-TR-95-4.pdf • Why you can’t cite Wikipedia in my class by Neil Waters (2007): http://doi.acm.org/10.1145/1284621.1284635 • Robert Dale’s tips for presentations: http://www.nltg.bton.ac.uk/teaching/SWM43/dale-presentations.pdf • Robert Dale’s advice on time management: http://www.nltg.bton.ac.uk/teaching/SWM43/dale-time-management.pdf

34. References • Cooper, H. (1998). Synthesizing Research: A Guide for Literature Reviews. Main points summarised here:http://library.ucsc.edu/ref/howto/literaturereview.html