Here’s a fascinating description of the 50,000 foot view of “big data,” and its potentials. MIT professor Alex Pentland is one of the most prominent experts in the field, and in this interview he outlines what big data is: evidence of our behaviour (bank transactions and location logs), as opposed to records of our consciousness (as in facebook content). Deeply embedded in social realities and institutions, our behaviour reveals a lot about what works and what doesn’t.
Thus, big data invites, and even requires (by virtue of its granularity) new approaches to the construction, maintenance, and revision of markets, transportation, healthcare, energy networks (and potentially even systems of justice). If this generates a kick of fear in you, as you imagine the Minority Report police at your door, Mr. Pentland has a rosier view:
The service-oriented government, as it were, or the service-oriented organization will tend to have better offerings for a lower price, as opposed to the ones that try to own the customer or control the citizen. As a consequence I expect to see that organizations with hard information boundaries will tend to dissolve, because there will be competition from things that are better that don’t have the hard boundaries and don’t try to own your data.
The Obama administration last Feb. produced “Consumer Data Privacy in a Networked World,” which includes the draft Consumer Privacy Bill of Rights. This might be a step in the right direction, should it get through Congress.