Exploring Computer-Assisted Reporting: The Future of Data-Driven Journalism

1. tcij.org Twitter.com/cijournalism

2. Computer Assisted Reporting • What is CAR? • Examples • Tools and methodologies • Mistakes and misconceptions

3. Computer Assisted Reporting New approaches to Journalism born from:1. New technology 2. A cultural shift towards openness 3. Application of social and computer science techniques to news gathering and reporting.

4. Computer Assisted Reporting • “CAR is the use of computers and social science methods to acquire and analyze information to do stories that otherwise would be difficult or impossible” – Steve Doig • Pioneered by Philip Meyer and others in late 1960s (Detroit Riots, Pulitzer prize) • Increasingly used across newsrooms in Europe and America

5. Government Increasingly open

6. Open Data Campaigners opening the Vault Political, corporate, economic data Free, accessible and reusable

7. Wikileaks: a watershed moment • 250,000 US Embassy Diplomatic Cables • 391,832 Iraq War Logs Scale of data releases demands new approaches and techniques.

8. Within CAR, 2 Key Strands Data Driven Journalism • Requires trained journalists to analyze data, and uses traditional publication model. Database Journalism • Produces databases of ‘facts’ that are continually maintained and improved

9. Data Driven Journalism • Traditional reporting – but using Data as your ‘source’ • Data is the primary source material that is analyzed by journalists who report the findings to their readers. • Requires trained journalists to analyse data, and uses traditional publication model.

10. Data Driven Journalism Examples: • MP’s Expenses • Iraq War logs • Baby P Investigation • Election and campaign finance investigations • Anything produced by Guardian, NYT, LATimes Data Desks.

11. Database Journalism • Produces databases of ‘facts’ that are continually maintained and improved • Instead of stories, products are interactive, offering ordinary people instant access to information that previously required curation and publication by journalists or public officials. • Requires programming expertise to build platforms for sharing and accessing data, such as websites and APIs.

12. Database Journalism Examples: • TheyWorkforyou.com Database of MPS, their registered interests and Voting habits. • OpenCorporates.com/CompaniesHouse.gov.ukDatabase of all registered companies. • Police.ukMaps crime data. See the crimes reported in your area. • FixMyTransport.comDatabase of local transport issues – Allows direct reporting of problems to authorities. • FarmSubsidies.orgDatabase of EU farm subsidies and their recipients.

14. Methodology • Analysis – Process ‘raw numbers’ into narratives and stories your readers can understand. • Visualization – Present abstract statistics to readers in a way that makes them accessible and impactful to your audience. • Mapping – Allow readers to understand and check information as it relates to their neighborhood/region. Check/Compare service provision in your area

15. Mapping Data

19. Visualisation

22. Invest in Data projects – Ongoing returns • 12 member multi-lingual team • 1 database • Nearly 600 data sheets • 21 languages • More than 640,000 records • More than 100 agencies • 2 dozen stories … and counting

23. Where Can I find Data? • Data is everywhere • Get in a “data state of mind”: forms, databases, statistics, press releases with numbers. Don’t take numbers given to you at face value • Online databases (publicly available) • Freedom of Information requests (requesting whole databases) • “Scraping data” • Databases constructed ‘from scratch’

24. Tools of the Trade • Plain and simple Excel • Data cleaning tools (Google Refine…) • Database managers (SQL, Access) • Scraping software (or code) and tools (imacros, OutwitHub, ScraperWiki…) • Visualisation tools (Tableau, Omniscope)

25. Mistakes and Misconceptions • Giving up because you are told no • It’s too expensive • We can’t download it • It’s private information • We don’t hold it • Keeping your data to yourself: publish and be DAMNED! • Pretty visualisations are not an investigation • Being geeky and knowing the latest tools does not an investigative journalist make

26. Conclusion • “Analysing data is the future for journalists”- Tim Berner’s Lee • Transparency in Data is accountability – Demand data releases from authorities and governments • Data journalism demands new technical skills from journalists – Reach out to those with programming and technical skills. Will augment your existing digital efforts • Don’t be left behind – Cannot ignore the added value to your publication that a data desk will bring. Not an optional ‘extra’. ‘I don’t do numbers’ no longer acceptable.

27. Train or Fail • Russell Square, Algate, Edgware Road, Reported as most dangerous (crime) stations in London – Stations were attacked by terrorists. -Context • Telegraph claimed public sector pensions cost each citizen £4000 a year. More than most people pay in income tax. -Common Sense • 100% Increase in risk. From 1 in a million to 2 in a million. 13% increase in chance of death. To 113%? – Basic Numeracy. • Correlation and causation – Do mobile phone masts increase fertility?

29. Some suggestions for finding UK Data Buzz Data – Collaborative data sharing platformhttp://buzzdata.comData.Gov – Main hub for UK Gov Datahttp://data.gov.uk/London Datastorehttp://data.london.gov.uk/datastore UK Public Spendinghttp://www.ukpublicspending.co.uk/Scotlandhttp://www.esds.ac.uk/government/docs/ScotlandGuide.pdfUK Policehttp://www.police.uk/UK National Statisticshttp://www.statistics.gov.uk/hub/index.html