Fresh Pickings

Welcome to The Variable Tree, a blog all about Programming and Software Engineering, with a leaning towards articles about data mining and analytics.

This blog covers range of topics, but I (that is, me, James Siddle - see Bio below) have a particular interest in data mining and related topics. That includes things like data extraction pipelines, Natural Language Processing, classification and prediction using Machine Learning, data storage techniques, and more. That said, you might also find topics about programming in general, perhaps something about new programming languages, domain modelling, observations on development processes, or the odd article about interesting applications.

First up, I'll be writing a few articles about how to retrieve, process, and analyze data from the Steam community API, showing how to generate a few interesting statistics from online gaming communities.

Hopefully you'll find something of interest, enjoy reading :)

Image courtesy of anankkml / FreeDigitalPhotos.net

James Siddle
I'm an all round software engineer, and I've been programming since an early age - I started with BASIC on an Acorn Electron, and have touched every more or less every platform and language you can imagine since then. I work for Digital Science, an arm of Macmillan publishing, we build information systems and services to help scientists. That includes plenty of data extraction, Natural Language Processing, analysis, storage, and ultimately the creation of services to allow customers access to the data we gather and the information we extract. This blog is not affiliated with Digital Science, and all views expressed here are my own.


Comments

Post a Comment

Popular posts from this blog

I Know Where You Were Last Summer: London's public bike data is telling everyone where you've been

Image
This article is about a publicly available dataset of bicycle journey data that contains enough information to track the movements of individual cyclists across London, for a six month period just over a year ago. I'll also explore how this dataset could be linked with other datasets to identify the actual people who made each of these journeys, and the privacy concerns this kind of linking raises. -- It probably won't surprise you to learn that there is a publicly available Transport For London dataset that contains records of bike journeys for London's bicycle hire scheme. What may surprise you is that this record includes unique customer identifiers, as well as the location and date/time for the start and end of each journey. The public dataset currently covers a period of six months between 2012 and 2013. What are the consequences of this? It means that someone who has access to the data can extract and analyse the journeys made by individual cyclists within
Read more

Personal Data Hacks: Visualizing Data from OpenFlights.org

Image
A friend recently told me about OpenFlights.org , a website that allows you to record, analyze, and share personal flight data. He showed me his dataset, which contained a record of every flight he'd taken over the past 10+ years. I was keen to investigate the dataset further, and my friend was happy to provide me with a copy so I could have a play (thank you Luigi!). The end result is the following collection of visualizations created in Gephi , with a little help from R . They show key transport hubs and routes for airports, countries, and continents that my friend has visited, and demonstrate some of the fun, insightful ways you can use such personal data. If you're interested in how the visualizations were created, check out the section at the end of this blog posting where I briefly describe the technologies required and steps involved. Note: OpenFlights.org is free to use, and supported by advertising and donations. You can join me in supporting OpenFlights.org v
Read more

Visualizing The Wheel of Time: Reader Sentiment for an Epic Fantasy Series

Image
In the following blog post, I explore reader sentiment for the Epic Fantasy book series The Wheel of Time , as expressed in user-submitted ratings on Amazon and GoodReads. If you're a data scientist (or similar), you'll probably be interested in the data analysis which includes some interesting observations about the usefulness - or otherwise - of Amazon reviews. If you're a Epic Fantasy Series reader, you'll be interested in the outcome of my analysis: I've decided to go ahead and read all fourteen books. Note -   This is a spoiler free zone .  Updated 17th Sept: charts now more accurate, not quite as pretty * Recently, I was looking for a good book to read, and a friend recommended The Wheel of Time  series by Robert Jordan . But I'd heard from a few sources that the later volumes were harder going than the earlier volumes. Having struggled with later volumes of the Game of Thrones  series (sorry George ) I was wary of starting a mammoth fourteen vo
Read more