Going Down the Data Mine

Timeline created by datamineruk
  • CIJ Summer School

    CIJ Summer School
    I sharpened my Excel and Access skills and was introduced to SQL.
    Here is the overview
  • ScraperWiki Hacks and Hackers Day Birmingham

    ScraperWiki Hacks and Hackers Day Birmingham
    I went all the way to Birmingham for thsi Hacks/Hackers day. I met ScraperWiki for the first time. Found it through chatting with a random guy at a talk.
  • Hacks/Hackers London

    Hacks/Hackers London
  • Data Miner UK

    Data Miner UK
    I began my blog
  • Twitter account

    Twitter account
    I became DataMinerUK on Twitter and began collating a list of dataminers
  • Hacks/Hackers Meetup

    Hacks/Hackers Meetup
    I attended this meetup
  • Hacks/Hackers Meetup

    Hacks/Hackers Meetup
    I attended this meetup
  • Frontline Club

    Frontline Club
    I attended a talk on data skills and techniques for jounralists.
  • Hacks/Hackers Meetup

    Hacks/Hackers Meetup
    I attended this meetup
  • Internet Week - The Web Data Revolution

    Internet Week - The Web Data Revolution
    I went to The Guardian for this talk
  • Horizon digital Economy - Open Data Masterclass

    Horizon digital Economy - Open Data Masterclass
    I attened this geocoding-based masterclass
  • Hacks/Hackers Meetup

    Hacks/Hackers Meetup
    I attended this meetup, sponsored by ScraperWiki.
  • News Rewired

    News Rewired
    I spent all day looking for data related information at this conference
  • An Introduction to Data Jounralism

    An Introduction to Data Jounralism
    I attended this one-day course and actually found it too basic. I must be doing something right.
  • Programming for Non-Geeks

    Programming for Non-Geeks
    I virtually attended this online webinar from Poynter, the online news university.
  • ScraperWiki

    ScraperWiki
    I became a data journalist at ScraperWiki
  • Channel 4 News article

    Channel 4 News article
    Managed to find data interesting enough for a Channel 4 News article
  • Hacks/Hackers Meetup

    Hacks/Hackers Meetup
    I attended this meetup
  • Data and News Sourcing

    Data and News Sourcing
    I attended these workshops run by the Media Standards Trust, BCC College of Journalism and the Royal Statistical Society.
  • Channel 4 News Article

    Channel 4 News Article
    This visualization was built by ScraperWiki. I was product manager (so to speak) and I gathered the data.
  • Channel 4 News Article

    Channel 4 News Article
    This visual was made from the first data set I managed to scrape and I was the data journalist on the amazing ScraperWiki team that built this in a matter of days.
  • Hacks and Hackers Hack Day Cardiff

    Hacks and Hackers Hack Day Cardiff
    REad about the projects from the day on the Scraperwiki blog
  • Hacks and Hackers Hack Day, Glasgow

    Hacks and Hackers Hack Day, Glasgow
    Read about what went on here. We had 8 teams of hacks and hackers digging around the Scottish data beat. Data on fire incidents, planning applications, public-owned property and gifts councillors’ received were mined.
  • My First Successful Piece of Code!

    My First Successful Piece of Code!
    Read about how I did it here. Or watch the screencasts: part 1 here and part 2 here
  • Scraper: Special Advisers' gifts and hospitality

    Scraper: Special Advisers' gifts and hospitality
    Get the data here
  • Story: Special Treatment for Special Advisers

    Story: Special Treatment for Special Advisers
    I wrote this story using the data I scraped on gifts and hospitality for Special Advisers in No.10. It shows how the media companies are the major winers and diners.
  • Data: Scrape it, Save it, Get it

    Data: Scrape it, Save it, Get it
    I made these screencasts showing you the bascis of how to use ScrpaerWiki to scrape data.
  • Story: The Best Things in Life are Free

    Story: The Best Things in Life are Free
    I wrote this story after cleaning up some data I found scraped from the Work4MP jobs listing site. It was in regards to Nick Cleggs call to get Westminister interns paid.
  • Scraper: Ministerial meetings with external organisations

    Scraper: Ministerial meetings with external organisations
    Get the data here.
  • Story: Small Society

    Story: Small Society
    The story I wrote regarding what I found from the Ministerial Meetings scraper.
  • Maker Night

    Maker Night
    Embarking on a new venture to hack 'things' using Arduino with these guys. Stay tuned for the results.
  • Scraper: Ministerial Gifts from external organisations

    Scraper: Ministerial Gifts from external organisations
    Get the data here.
  • Scraper: Ministerial Hospitality from external organisations

    Scraper: Ministerial Hospitality from external organisations
    Get the data here.
  • Scraper: Permanent Secretaries’ meetings with external organisations

    Scraper: Permanent Secretaries’ meetings with external organisations
    Get the data here.
  • Scraper: Cabinet Office spend data

    Scraper: Cabinet Office spend data
    Get the data here.
  • Hacks/Hackers Meetup

    Hacks/Hackers Meetup
    Read about the meetup here.
  • Data Journalism: The Story So Far

    Data Journalism: The Story So Far
    A piece I wrote covering what I can about the state of data journalism (past, present and future) here.
  • Post: Opening Data, Opening Minds

    Post: Opening Data, Opening Minds
    Read the post.
  • Story: Cabinet Office Spending Data

    Story: Cabinet Office Spending Data
    Read the post.
  • #djcamp 2011

    #djcamp 2011
    Data Journalism Camp organised by The Digital Editor's Network. Check out the flyer. This blog was mentioned in the manual written by Tom Johnson of the Centre for Analytical Journalism.
  • Scraper: Technology Strategy Board Spending

    Scraper: Technology Strategy Board Spending
    You can download th data here.
  • Post: Data Harveting

    Post: Data Harveting
    Read the post.
  • News Rewired

    News Rewired
    I was a speaker at this conference whose theme was data journalism. I was honored to be included in a very impressive line up.
  • Knight-Mozilla Challenge

    Knight-Mozilla Challenge
    Had a great time at this jam for developers and journalists. Martin Belam was on my team and wrote up our team work on this blog post. I think The Guardian might do something with one of my ideas, the panopticon.
  • Post: Sorting the Social Media Chaos

    Post: Sorting the Social Media Chaos
    A write up of my talk at News Rewired for the tweeters that asked for it. For anyone interested in using social media for news gathering or investigation read the post
  • News Rewired Blog

    News Rewired Blog
    Featured and pictured on this blog post. Was honored to be in a line up of speakers that included Heather Brookes, James Ball, Chris Taggart, Kevin Anderson and Anna Doble.
  • Post: Visualization - A Cloud of Spending

    Post: Visualization - A Cloud of Spending
    Build my first visualization from scraped data. You can read the post here. ScraperWiki user quickly copied the code and made it better.
  • Power & Media Conference - POLIS

    Power & Media Conference - POLIS
    Attended conference. You can read about it here
  • Scraped government salaries and wrote a post on non disclosure

    Scraped government salaries and wrote a post on non disclosure
    You can read the post here
  • Scraped Berlin data for open democracy project

    Scraped Berlin data for open democracy project
    During a workshop I helped make some small scrapers for an open democracy project for Germany called meine-demokratie
  • Period: to

    Open Knowledge Conference 2011

    Attended and helped run ScraperWiki workshop. You can find teh lineup here
  • Post: To Scrape or not to Scrape

    Post: To Scrape or not to Scrape
    Read about my reflections on programming to find news here
  • Post: WhyData Needs Journalism and Journalism Needs Data

    Post: WhyData Needs Journalism and Journalism Needs Data
    My reflections on training journalists for the new data driven world. You can read them here
  • Inerview for BuzzData

    Inerview for BuzzData
    You can read it here
  • Scraped Arms Transfers Database for Hacks/Hackers Toronto

    Scraped Arms Transfers Database for Hacks/Hackers Toronto
    You can get the exports here and you'll find a link to the imports.
  • Post: Judges under the Hammer

    Post: Judges under the Hammer
    Publsihed a post here about a scrpaed twitter account I made for the judicial complaints. Follow @OJCstatements on twitter.
  • MozNewsLab Webinar: Aza Raskin

    MozNewsLab Webinar: Aza Raskin
    He gave this presentation about prototyping
  • Scraper: Civilian Casualties in Afghanistan ISAF

    Scraper: Civilian Casualties in Afghanistan ISAF
    Download the data here. First excel scraper
  • Scraper: Civilian Casualties in Afghanistan ARM

    Scraper: Civilian Casualties in Afghanistan ARM
    Download the data here.
  • MozNewsLab Webinar: Burt Herman

    MozNewsLab Webinar: Burt Herman
    The former AP foreign correspondent and founder of Storify gave this presentation about entrepreneurial journalism.
  • Email Alert: For US nuclear reactors

    Email Alert: For US nuclear reactors
    An experiment but you can check out the code here.
  • Post: Build a business for your newsroom or make building a newsroom your business?

    Post: Build a business for your newsroom or make building a newsroom your business?
    Read it here. My piece on how a data journalist shoulld be a newsroom microcosm for the entrepreneurial journalist.
  • Tech talk at the Guardian

    Tech talk at the Guardian
    They invited me of all people! Great to see @gudaithi and @currybet again. And good pal @newsmary is now there. Seems all of Hacks/Hackers are migrating to the Guardian.
  • Period: to

    CIJ Summer School Workshop

    I was one of the many great speakers this year!
  • MozNewsLab Webinar: Chris Heilmann

    MozNewsLab Webinar: Chris Heilmann
    He gave this lecture about web development and new capabilities in the browser and the Open Web.
  • MozNewsLab Webinar: John Resig

    MozNewsLab Webinar: John Resig
    John talked about building community around opens-source software ideas, and projects, and also how to successfully build software 'lego' -- that is, the building blocks that enable other people to do amazing things on the open Web.
  • Scraper: Civilian Casualties in Afghanistan UNAMA

    Scraper: Civilian Casualties in Afghanistan UNAMA
    Download the data here.
  • Post: Getting to grips with ScraperWiki for those who don't code

    Post: Getting to grips with ScraperWiki for those who don't code
    Go through the exercises outlined here. This is a copy of what I set up for the CIJ summer school workshop
  • MozNewsLab Webinar: Jesse James Garrett

    MozNewsLab Webinar: Jesse James Garrett
    In 2002, Jesse wrote the book The Elements of User Experience. Watch his webinar to the lab here
  • Post: Hack the Open Web, Not People's Private Phones

    Post: Hack the Open Web, Not People's Private Phones
    Written for OpenDemocracy and longer version on my blog.
  • MozNewsLab Webinar: Shazna Nessa

    MozNewsLab Webinar:  Shazna Nessa
    Shazna Nessa is Director of Interactive at the Associated Press in New York. Nessa runs a global department that provides visual and interactive news content to customers in all formats. Watch the webinar she gave to the lab here
  • Post: Online News - The Big Picture

    Post: Online News - The Big Picture
    My idea spark for the MozNewsLab. You can read about it here
  • Post: Telling a Story through Code - A Tale of Two Hamsters

    Post: Telling a Story through Code -  A Tale of Two Hamsters
    Tutorial to learn classes turned into a blog post which you can read here
  • MozNewsLab Webinar: Mohamed Nanabhay

    MozNewsLab Webinar: Mohamed Nanabhay
    Mohamed Nanabhay, is an internet entrepreneur and Head of Online at Al Jazeera English. Watch the webinar he gave to the lab here
  • Post: Online News - Geeky Guidance

    Post: Online News - Geeky Guidance
    Getting some advice from my geeky friends on my MozNewsLab idea. Read about it here
  • MozNewsLab Webinar: Oliver Reichenstein

    MozNewsLab Webinar: Oliver Reichenstein
    Oliver Reichenstein is CEO of iA (Information Architects, Inc.), known as “one of the best-known design agencies in the world.” Watch the webinar he gave to the lab here
  • MozNewsLab Webinar: Evan Hansen

    MozNewsLab Webinar: Evan Hansen
    Evan Hansen is the Editor In Chief of Wired.com. Under his stewardship, Wired.com’s traffic has grown fourfold, reaching more than 10 million unique visitors a month. Hansen has won numerous awards for technology reporting and writing. Watch the webinar he gave the lab here
  • Post: Journalism - To Learn, to Teach, to Know

    Post: Journalism - To Learn, to Teach, to Know
    Gave an interview to DataJournalismBlog which you can watch here
  • MozNewsLab Webinar: Jeff Jarvis

    MozNewsLab Webinar: Jeff Jarvis
    Jeff Jarvis is the author of What Would Google Do? He is associate professor and director of the interactive journalism program and the new business models for news project at the City University of New York’s Graduate School of Journalism. Listen to the webinar he gave the lab here
  • Post: The Big Picture

    Post: The Big Picture
    Read about my proposal for the MozNewsLab here
  • Post: 10 Things for Getting Started in Data Journalism

    Post: 10 Things for Getting Started in Data Journalism
    Read the post [here](Top 10 Things for getting Started in Data Journalism)
  • Post: ScraperWiki for those who don't code - the answers

    Post: ScraperWiki for those who don't code - the answers
    Answers to a previous tutorial which you can go through here
  • Post: Sink/Source Journalism - A News Model for the Digital Age

    Post: Sink/Source Journalism - A News Model for the Digital Age
    Read the blog post here