Personal Data Hacks: Visualizing Data from OpenFlights.org

A friend recently told me about OpenFlights.org, a website that allows you to record, analyze, and share personal flight data. He showed me his dataset, which contained a record of every flight he'd taken over the past 10+ years. I was keen to investigate the dataset further, and my friend was happy to provide me with a copy so I could have a play (thank you Luigi!).

The end result is the following collection of visualizations created in Gephi, with a little help from R. They show key transport hubs and routes for airports, countries, and continents that my friend has visited, and demonstrate some of the fun, insightful ways you can use such personal data.

If you're interested in how the visualizations were created, check out the section at the end of this blog posting where I briefly describe the technologies required and steps involved.

Note: OpenFlights.org is free to use, and supported by advertising and donations. You can join me in supporting OpenFlights.org via this link.

Hub Airports and Key Routes

The first visualization below shows the primary airports and routes used by Luigi. Each airport has been ranked in size and colour according to the number of other connected airports, while each connection has been weighted according to the number of times that route was flown. The layout here was generated in Gephi, ensuring (simply put) that related nodes are co-located:


As you can see, PSA (Pisa) and STN (London Stansted) are far and away the most used airports. Not only that, but the return journey between the two airports has been taken many times. These two facts make perfect sense given that Luigi is from Pisa, but moved to the UK a few years ago. Other significant hubs are London Heathrow, London Gatwick, and Rome - not too surprising.

Key Countries and Connections 

Given that many airports are within the same country, is it possible to reflect that in the visualization? One way to achieve this is to partition airports by colours corresponding to different countries, as follows:


So that's kind of OK - the predominance of Italy (yellowish-green) and the UK (blue) - is starting to show, but it's quite confusing.

A better approach is to group the airports and connections by country, and to layout the nodes according to (approximate) geographical positions. The following graph also has a few graphical tweaks for readability:


We're now getting to something approximating a worldwide travel heatmap for my friend. The key travel hubs of the UK and Italy are obvious, also key routes are also jumping out more: between Italy and the UK, Italy and Germany / France, and the UK and Spain. The significance of the other routes also becomes a bit more apparent - further afield countries corresponding to occasional holiday travel (for instance).

Continental Travel

What about different continents? If we return to the original graph and partition the airports by continent, a European bias becomes very clear:


It's also nice to see the groupings of continental airports jumping out - in particular the Green nodes in the bottom right corresponding to African airports. Note that I avoided grouping by continent here because the resulting node for Europe dwarfed all the other nodes, which didn't make for a good visualization.

Creating the Visualizations

The flight data is downloadable from OpenFlights.org as Comma Separated Values. I used a little command-line manipulation (awk, sort, and uniq) to compress the data into a list of unique flights, with a count corresponding to the number of times that flight was taken.

Next, I loaded the data into R, then converted it into a graph which could be easily exported to GML (Graph Modelling Language), then loaded into Gephi and visualized.

The downloaded dataset didn't contain city, country, or continent data. Adding this required an export of nodes from Gephi, followed by a merge with the OpenFlights.org Airport dataset (spreadsheet magic), and a re-import into Gephi.

Comments

  1. Hi,
    Nice job !

    I would like to share a similar job I've done several months ago with the OpenFlights.org global dataset.

    http://matthieu-totet.fr/Koumin/iatamaps/

    Eager to see other gephi works from you ;-)

    ReplyDelete
    Replies
    1. Are you still out there suffering from HERPES VIRUS, when You can get cured, i was happy to get my cure from Dr Ojoka i did not believe the first time until i took the herbal remedy, i can tell the whole world that am finally free from Herpes Virus, i can not hide my joy that is the reason why i am here to share my testimony, if you are still out there suffering from HERPES i we want you to contact Dr Ojoka and you we see the wonder power of his herbal medicine. Why wait and suffer when there is someone like Dr Ojoka contact him Email: drojokarootandherbal@gmail com or what-app number +2348144172934



















      Are you still out there suffering from HERPES VIRUS, when You can get cured, i was happy to get my cure from Dr Ojoka i did not believe the first time until i took the herbal remedy, i can tell the whole world that am finally free from Herpes Virus, i can not hide my joy that is the reason why i am here to share my testimony, if you are still out there suffering from HERPES i we want you to contact Dr Ojoka and you we see the wonder power of his herbal medicine. Why wait and suffer when there is someone like Dr Ojoka contact him Email: drojokarootandherbal@gmail com or what-app number +2348144172934

      Delete
  2. That's interesting to see the different visualisations, but why not simply overlay on a world map?

    ReplyDelete
    Replies
    1. Would certainly make sense, though it may make it a little harder to make out the connections.

      Delete

    2. windows 10 serial key crack , windows 7 ultimate key product key free , windows 10 activation error 0xc004f050 , cheap office 2016 series key , windows 7 keys sale , windows 7 profesiona key , win7 sp1 activation key buy , windows 10 activation error code 0xc004e016 , lr9FVb

      buy office 2016 product key

      windows server 2012 r2 free

      rosetta stone french key sale

      windows 10 product serial number offer

      Delete
  3. Nice work Jim! My favourite map is the worldwide travel heatmap. With the tools you used would it be possible to add an interactive 'time slider window' to the graph to see what routes were travelled between certain dates? The slider could run horizontally beneath the graph. The lefthand edge of the slider sets the start date and the righthand side the end date. Only data between those dates would be displayed.

    ReplyDelete
  4. Nice ideas. Unfortunately Gephi is fairly static, so it would be a lot of effort to create any kind of dynamic content :/

    ReplyDelete
  5. It's also nice to see the groupings of continental airports jumping out - in particular the Green nodes in the bottom right corresponding to African airports.

    Glyn Willmoth

    ReplyDelete
  6. Excellent visualizations, by the way, are there any corresponding software for that? On the other hand, I have some some personal data protection resource to share - (877)-871-1295

    ReplyDelete
  7. Very Good !
    May I ask how to see aircraft registration in the statistics or top 10 ?

    ReplyDelete
  8. how can I download the flightdata from the website?

    ReplyDelete
    Replies
    1. The flight data is private, because it represents individual flights. I believe you can get hold of flight data if people choose to share it with you, but there's no general open dataset of flights.

      Delete
    2. would you mind to send me the flight data used in your Gephi? Thank you.

      Delete
    3. Unfortunately the data isn't mine to share - I used a friend's personal flight records, on the condition that I wouldn't share it. Apologies.

      Delete
  9. very informative blog and useful article thank you for sharing with us , keep

    posting learn more about BI Tools Tableau Online Training

    ReplyDelete
  10. Thankful to you for this amazing information sharing with us. Get website designing and development services by Ogen Infosystem.
    Website Designing Company in Delhi

    ReplyDelete
  11. Your content is really awesome and understandable, thanks for the efforts in this blog. Visit Mutual Fund Wala for Mutual Fund Schemes.
    Mutual Fund Companies

    ReplyDelete
  12. Decent, Get Service for Night out page 3 parties and this magnificent service provided by Lifestyle Magazine.
    Lifestyle Magazine India

    ReplyDelete
  13. Great, I think this is one of the best blog in past some time I have seen. Visit Kalakutir for Fleet Painting, Godown Line Marking Painting and Caution & Indication Signages.
    Fleet Painting

    ReplyDelete
  14. INSTEAD OF GETTING A LOAN,, I GOT SOMETHING NEW
    Get $5,500 USD every day, for six months!

    See how it works
    Do you know you can hack into any ATM machine with a hacked ATM card??
    Make up you mind before applying, straight deal...

    Order for a blank ATM card now and get millions within a week!: contact us
    via email address::{Automatictellers@gmail.com}

    We have specially programmed ATM cards that can be use to hack ATM
    machines, the ATM cards can be used to withdraw at the ATM or swipe, at
    stores and POS. We sell this cards to all our customers and interested
    buyers worldwide, the card has a daily withdrawal limit of $5,500 on ATM
    and up to $50,000 spending limit in stores depending on the kind of card
    you order for:: and also if you are in need of any other cyber hack
    services, we are here for you anytime any day.

    Here is our price lists for the ATM CARDS:

    Cards that withdraw $5,500 per day costs $200 USD
    Cards that withdraw $10,000 per day costs $850 USD
    Cards that withdraw $35,000 per day costs $2,200 USD
    Cards that withdraw $50,000 per day costs $5,500 USD
    Cards that withdraw $100,000 per day costs $8,500 USD

    make up your mind before applying, straight deal!!!

    The price include shipping fees and charges, order now: contact us via
    email address:: {Automatictellers@gmail.com}

    Visit our Website for more Info: https://automatictellers.webs.com
    ®

    ReplyDelete
  15. POWERFUL SPELL CASTER THAT BROUGHT MY EX HUSBAND BACK INTO MY LIFE AFTER HE LEFT ME EMAIL HIM VIA EMAIL INFINITYLOVESPELL@GMAIL.COM OR INFINITYLOVESPELL@YAHOO.COM ADD HIM ON WHATSAPP +2348118829899
     
    Hello my fellow friends online I'm Diana Gilbert from USA, I want to use this medium to thank Dr Great because he brought back my Ex husband. my husband left me for his ex girlfriend he had before he married me it's a difficult situation for me I called and beg him to come back he refused he said he don't love me anymore I tried every possible way to get him back all was in vain I told my friend about it and she gave me Dr Great email and I emailed him and told him my problem and he told me what to do and I did it and he cast a love spell which brought back my husband within 24 hours. If you need help in getting back together with your Ex, email Dr Great at  infinitylovespell@gmail.com or infinitylovespell@yahoo.com WhatsApp him+2348118829899 view his website http://infinitylovespell.website2.me you can also visit my blog http://dianagilbert05.over-blog.com/

    ReplyDelete
  16. POWERFUL SPELL CASTER THAT BROUGHT MY EX HUSBAND BACK INTO MY LIFE AFTER HE LEFT ME EMAIL HIM VIA EMAIL INFINITYLOVESPELL@GMAIL.COM OR INFINITYLOVESPELL@YAHOO.COM ADD HIM ON WHATSAPP +2348118829899
     
    Hello my fellow friends online I'm Diana Gilbert from USA, I want to use this medium to thank Dr Great because he brought back my Ex husband. my husband left me for his ex girlfriend he had before he married me it's a difficult situation for me I called and beg him to come back he refused he said he don't love me anymore I tried every possible way to get him back all was in vain I told my friend about it and she gave me Dr Great email and I emailed him and told him my problem and he told me what to do and I did it and he cast a love spell which brought back my husband within 24 hours. If you need help in getting back together with your Ex, email Dr Great at  infinitylovespell@gmail.com or infinitylovespell@yahoo.com WhatsApp him+2348118829899 view his website http://infinitylovespell.website2.me you can also visit my blog http://dianagilbert05.over-blog.com/

    ReplyDelete
  17. Are you still out there suffering from HERPES VIRUS, when You can get cured, i was happy to get my cure from Dr Ojoka i did not believe the first time until i took the herbal remedy, i can tell the whole world that am finally free from Herpes Virus, i can not hide my joy that is the reason why i am here to share my testimony, if you are still out there suffering from HERPES i we want you to contact Dr Ojoka and you we see the wonder power of his herbal medicine. Why wait and suffer when there is someone like Dr Ojoka contact him Email: drojokarootandherbal@gmail com or what-app number +2348144172934

    ReplyDelete
  18. HOW I GOT CURE TO MY DIABETES WITH THE HELP OF DR ORIANE NATURAL HERBAL CURE 
    I can’t believe this. A great testimony that i must share to all diabetes patient in the world i never believed that their could be any complete cure for diabetes or any cure for diabetes infection,i saw people’s testimony on sites of how DR oriane prepare herbal cure and brought them back to life again. i had to try it too and you can,t believe that in just few weeks i started using it all my pains stop gradually and i was totally cured from diabetes and had to leave without the diabetes the doctor gave to me. Right now i can tell you that few months now i have not had any pain,delay in treatment leads to death. Here is his email: droriane6@gmail.com or via his WHATSAPP number is +2349031652461    

    ReplyDelete

Post a Comment

Popular posts from this blog

Harvesting Data from the Steam Community API

I Know Where You Were Last Summer: London's public bike data is telling everyone where you've been