IG&H goes big data IG&H Consulting & Interim has resolved to assist their clients in answering these questions. We are therefore setting up an expert group to help large corporates gain insights from data, but primarily to develop the capacity to do this within their company. I was asked to set up the group.
I learnt about big data in America where, from 2011, I worked on the re-election campaign of Barack Obama. This campaign was the first to be based on data in a big way. What we did there was so innovative that the core of the team continued as an independent company after his successful re-election. This was on the initiative of Eric Schmidt, Executive Chairman of Alphabet, and with his support, we became Civis Analytics. After nearly three years with Civis I returned to the Netherlands, where I have worked with IG&H since February 2016.
Big data is nothing new, but it will change everything
The first step we take with clients is to create some understanding. It’s difficult to build without a good foundation. The foundation is laid by realising that big data is actually a fashionable word for data analysis. It isn’t something that has been discovered in recent years, but has been around since time immemorial. But something has indeed changed so that you can no longer ignore big data, it is data analysis on steroids. In fact, three things have changed.
The first is the increase in data creation. IBM figured that daily we create 2.5 trillion bytes of data worldwide.
That’s not only unimaginable, but also the speed at which it is growing is enormous: it’s estimated that 90% of existing data was created in just the past two years. For example, 500 million tweets are sent every day, while every minute some 300 hours of video is uploaded on to YouTube. And that’s just the top of the iceberg.
All that data would have long been lost if the storage facilities had not also expanded greatly. This is the second important change. Floppy disks became CDs, and CDs became USB sticks. Hard disks are becoming increasingly larger and for those who need more, there are now subscriptions to cloud services: endless storage space in gigantic data centres.
Finally, the third change is the growth in calculation capacity. In 1965 Gordon Moore, who set up Intel, predicted that the number of transistors on a chip would double roughly every two years. This has since been called Moore’s law, and it is still applicable.
How did big data contribute to Obama’s re-election To illustrate the power of data analysis, I often use an example from my period on the Obama campaign. Just as with every American campaign, the grassroots organization was an important part of it. This is the army of volunteers who are sent out to convince people to vote for their candidate. These personal discussions have a great impact on the final result; they are far removed from the slick advertising messages that do not appeal to many people.
For the 2012 election we had more than 2.2 million volunteers available. And a budget of a billion dollars. A gigantic potential. Our task, as analytics team, was to ensure that all that money and the man-hours were spent as efficiently as possible in maximizing the chances of Obama winning. We did that by analysing our own and public data and using the results to make predictions at an individual level. For example, on the chance that someone would actually go and vote, and which candidate they preferred.
Subsequently – and this was crucial – these predictions were used to allot the available time and means without compromise. Our 2.2 million volunteers knew exactly who it was worth approaching and which topics they could best discuss with them. In this way they reached far more potential Obama voters, and the chance was also greater that they could convince those people to actually vote for him. And, in this way, in the deepest secrecy and from a room nicknamed ‘The Cave’, the analytics team contributed to the re-election of Barack Obama.
Big data for your company IG&H is not in the business of political campaigns, certainly not for an American presidency. But thanks to this campaign, and numerous other case studies reported since, we know just how valuable well-tailored data analysis products can be. I will be writing about this topic regularly to give you more insight into it, and to make the abstract concept of big data more concrete for you. If you are interested, keep an eye on my blog or get in touch with me directly via the links on this page.