Data Visualization and Its Future in Consumer Web
Data visualization will likely be central to the future of the consumer web experience. Much design today seems to begin with the idea of a literal page on which we scroll up and down to view its length. It is as if we are looking at a long piece of paper which just doesn’t fit within the size of our monitor. Gaming has led the way in creating awesome experiences for interacting with objects. My hope is that we will see the world of data viz brought to bear on other types of content: data, news, and video.
For anyone new to the field, it is worth buying a book by Edward Tufte, considered the grandfather of data visualization. Trained as a political scientist, Tufte began exploring data representation on his own as a side project, 
self-published a “little book” (The Visual Display of Quantitative Information) on the subject, and over a few years essentially became the father of a new discipline. Next time you are around hackers, check out their bookshelf. Positioned next to MySQL, PHP, and JSON books, you’ll often seen the oversized Tufte hardback. A surprising number of serious hackers have been inspired by Tufte’s work. I have also heard that Tufte’s one-day in person course is truly superb—and a great way to get a good price on the books.
The applications of Tufte are really exciting. Consider Palantir, an amazing Silicon Valley company that is known to have recruited several hundred of the smartest engineers in the country. Their primary product is a data analytics/visualization tool that empowers human analysts to analyze multiple, large sets of real time data. Think about a military intelligence analyst trying to mash up phone call logs, bank records, and location. Added onto this is a sophisticated way of controlling access to specific parts of data. Essentially, it is the ultimate tool for large organizations with human analysts and lots of data… which is, well, all of the intelligence world, plus finance, government, and even industry (think about Walmart, Amazon, or Harrah’s). To top it off, TechCrunch has speculated that Palantir will be the first billion dollar company (its current valuation is $735mm) that never had a sales force. As founder Alex Karp said, “We are long on dealing with the most important problems we can find. We are short on the near term. So, we’re not hiring a sales or marketing team… and don’t plan to get any of them.” Any project with Alex Karp, Peter Thiel, several hundred first class engineers, and no sales force or “MBA-types” is one to keep an eye on.
- Alex Karp on Charlie Rose
On the consumer side, fewer companies have experimented with creative visualization. Digg and Twitter have experimented in-house and have seen neat experimentation using their APIs. Digg Labs has created a few very interesting visualizations of dug articles. Imagine one interaction display that shows the news content most important to you—colors, shapes, sizes, and motion all convey information about the content beneath. A more conservative integration of this might be the New York Times “TimesSkimmer” – which offers a flash-based interaction with squares of content. It is a beautiful way of surveying all the news quickly.
On the video front, Cooliris is a neat tool that improves the web experience for video and photos. I’ve installed it and enjoy using it to browse albums on facebook and for their news TV channel. Coincident.TV is a neat shop that is making the video experience social and interactive – imagine clicking on a cool car in a TV show you are watching to take a virtual test drive. Apture is slightly less sexy but equally handy – it enables quick popups of relevant content (news/video) on top of a site, reducing the need to navigate off site. Finally, Juice is a handy Firefox plugin that allows sidebar video watching; so, no more sitting on the same page just to keep the YouTube playing. Apture and Juice are a far cry from Palantir, but are little steps in the right direction.
An exciting added layer to all of this visualization would be integrating Facebook’s open graph. 
Imagine amazing video and news content displayedvisually but mapped onto your social graph. So, you might see an article node slowly growing and then the faces of friends right around it who are discussing it. Or as you see 30 live tv broadcasts you see which of your friends are also watching the same program and able to interact live. The big picture is that the social graph that facebook is building can dramatically improve how we filter content—from news and TV to restaurants and travel. The social graph and visualization together will make the web a much friendlier and exciting place.
The Underappreciated Competitive Advantage: The Ability to Get Things Done
The more time I spend observing men and women who are high achievers, the more I realize that while many have visionary leadership, keen intellect, and a passion for change, perhaps their most unappreciated yet impressive skill is their ability to get things done. This sounds outrageously oversimplified. To many people “getting things done” seems like a commodity skill set—sure there are tricks but at the end of the day all that matters is that people check off their task list. From my study of a few outrageously impressive people, efficient task management is one of their key competitive advantages. So if you are serious about making a difference in whatever sphere you work, spend a few hours exploring project management. You’ll find that productive management results in more successful projects (e.g. that new non-profit finally gets off the ground), more free time (for your friends or family), more sleep (yielding more focus), and/or more time to explore your passions (to figure out how you actually want to spend your time). Below are a few resources I have found helpful in my journey. Please send your tricks and tips my way!
Knowledge:
Getting Things Done (buy here): the classic book on the subject. I really believe this book can change your life. Allen has consulted with hundreds of top executives. What he finds is that after these executives implement a trusted system for task/project management, they unleash an inner-creativity. They literally start coming up with all sorts of new ideas for their business or personal interests. Allen argues that since the human mind is so inefficient in determining what to store in its short term “RAM,” once you clear that memory of lots of tasks and reminders (by writing them down in an organized fashion), you free up intellectual computing ability to devote to other processes (including brainstorming). You can find a summary of some of the book’s highlights here.
The Art of Project Management: kudos to my friend Joe Barillari for sending this great piece my way. This piece is a chapter out of a book by the same title by Scott Berkun, a longtime project manager at Microsoft.
The summary points:
- Everything can be represented in an ordered list. Most of the work of project management is correctly prioritizing things and leading the team in carrying them out.
- The three most basic ordered lists are: project goals (vision), list of features, and list of work items. They should always be in sync with each other. Each work item contributes to a feature, and each feature contributes to a goal.
- There is a bright yellow line between priority 1 work and everything else.
- Things happen when you say no. If you can't say no, you effectively have no priorities.
- The PM has to keep the team honest and keep them close to reality.
- Knowing the critical path in engineering and team processes enables efficiency.
- You must be both relentless and savvy to make things happen.