Data Visualization and Its Future in Consumer Web
Data visualization will likely be central to the future of the consumer web experience. Much design today seems to begin with the idea of a literal page on which we scroll up and down to view its length. It is as if we are looking at a long piece of paper which just doesn’t fit within the size of our monitor. Gaming has led the way in creating awesome experiences for interacting with objects. My hope is that we will see the world of data viz brought to bear on other types of content: data, news, and video.
For anyone new to the field, it is worth buying a book by Edward Tufte, considered the grandfather of data visualization. Trained as a political scientist, Tufte began exploring data representation on his own as a side project, 
self-published a “little book” (The Visual Display of Quantitative Information) on the subject, and over a few years essentially became the father of a new discipline. Next time you are around hackers, check out their bookshelf. Positioned next to MySQL, PHP, and JSON books, you’ll often seen the oversized Tufte hardback. A surprising number of serious hackers have been inspired by Tufte’s work. I have also heard that Tufte’s one-day in person course is truly superb—and a great way to get a good price on the books.
The applications of Tufte are really exciting. Consider Palantir, an amazing Silicon Valley company that is known to have recruited several hundred of the smartest engineers in the country. Their primary product is a data analytics/visualization tool that empowers human analysts to analyze multiple, large sets of real time data. Think about a military intelligence analyst trying to mash up phone call logs, bank records, and location. Added onto this is a sophisticated way of controlling access to specific parts of data. Essentially, it is the ultimate tool for large organizations with human analysts and lots of data… which is, well, all of the intelligence world, plus finance, government, and even industry (think about Walmart, Amazon, or Harrah’s). To top it off, TechCrunch has speculated that Palantir will be the first billion dollar company (its current valuation is $735mm) that never had a sales force. As founder Alex Karp said, “We are long on dealing with the most important problems we can find. We are short on the near term. So, we’re not hiring a sales or marketing team… and don’t plan to get any of them.” Any project with Alex Karp, Peter Thiel, several hundred first class engineers, and no sales force or “MBA-types” is one to keep an eye on.
- Alex Karp on Charlie Rose
On the consumer side, fewer companies have experimented with creative visualization. Digg and Twitter have experimented in-house and have seen neat experimentation using their APIs. Digg Labs has created a few very interesting visualizations of dug articles. Imagine one interaction display that shows the news content most important to you—colors, shapes, sizes, and motion all convey information about the content beneath. A more conservative integration of this might be the New York Times “TimesSkimmer” – which offers a flash-based interaction with squares of content. It is a beautiful way of surveying all the news quickly.
On the video front, Cooliris is a neat tool that improves the web experience for video and photos. I’ve installed it and enjoy using it to browse albums on facebook and for their news TV channel. Coincident.TV is a neat shop that is making the video experience social and interactive – imagine clicking on a cool car in a TV show you are watching to take a virtual test drive. Apture is slightly less sexy but equally handy – it enables quick popups of relevant content (news/video) on top of a site, reducing the need to navigate off site. Finally, Juice is a handy Firefox plugin that allows sidebar video watching; so, no more sitting on the same page just to keep the YouTube playing. Apture and Juice are a far cry from Palantir, but are little steps in the right direction.
An exciting added layer to all of this visualization would be integrating Facebook’s open graph. 
Imagine amazing video and news content displayedvisually but mapped onto your social graph. So, you might see an article node slowly growing and then the faces of friends right around it who are discussing it. Or as you see 30 live tv broadcasts you see which of your friends are also watching the same program and able to interact live. The big picture is that the social graph that facebook is building can dramatically improve how we filter content—from news and TV to restaurants and travel. The social graph and visualization together will make the web a much friendlier and exciting place.