Data Visualization and Its Future in Consumer Web
Data visualization will likely be central to the future of the consumer web experience. Much design today seems to begin with the idea of a literal page on which we scroll up and down to view its length. It is as if we are looking at a long piece of paper which just doesn’t fit within the size of our monitor. Gaming has led the way in creating awesome experiences for interacting with objects. My hope is that we will see the world of data viz brought to bear on other types of content: data, news, and video.
For anyone new to the field, it is worth buying a book by Edward Tufte, considered the grandfather of data visualization. Trained as a political scientist, Tufte began exploring data representation on his own as a side project, 
self-published a “little book” (The Visual Display of Quantitative Information) on the subject, and over a few years essentially became the father of a new discipline. Next time you are around hackers, check out their bookshelf. Positioned next to MySQL, PHP, and JSON books, you’ll often seen the oversized Tufte hardback. A surprising number of serious hackers have been inspired by Tufte’s work. I have also heard that Tufte’s one-day in person course is truly superb—and a great way to get a good price on the books.
The applications of Tufte are really exciting. Consider Palantir, an amazing Silicon Valley company that is known to have recruited several hundred of the smartest engineers in the country. Their primary product is a data analytics/visualization tool that empowers human analysts to analyze multiple, large sets of real time data. Think about a military intelligence analyst trying to mash up phone call logs, bank records, and location. Added onto this is a sophisticated way of controlling access to specific parts of data. Essentially, it is the ultimate tool for large organizations with human analysts and lots of data… which is, well, all of the intelligence world, plus finance, government, and even industry (think about Walmart, Amazon, or Harrah’s). To top it off, TechCrunch has speculated that Palantir will be the first billion dollar company (its current valuation is $735mm) that never had a sales force. As founder Alex Karp said, “We are long on dealing with the most important problems we can find. We are short on the near term. So, we’re not hiring a sales or marketing team… and don’t plan to get any of them.” Any project with Alex Karp, Peter Thiel, several hundred first class engineers, and no sales force or “MBA-types” is one to keep an eye on.
- Alex Karp on Charlie Rose
On the consumer side, fewer companies have experimented with creative visualization. Digg and Twitter have experimented in-house and have seen neat experimentation using their APIs. Digg Labs has created a few very interesting visualizations of dug articles. Imagine one interaction display that shows the news content most important to you—colors, shapes, sizes, and motion all convey information about the content beneath. A more conservative integration of this might be the New York Times “TimesSkimmer” – which offers a flash-based interaction with squares of content. It is a beautiful way of surveying all the news quickly.
On the video front, Cooliris is a neat tool that improves the web experience for video and photos. I’ve installed it and enjoy using it to browse albums on facebook and for their news TV channel. Coincident.TV is a neat shop that is making the video experience social and interactive – imagine clicking on a cool car in a TV show you are watching to take a virtual test drive. Apture is slightly less sexy but equally handy – it enables quick popups of relevant content (news/video) on top of a site, reducing the need to navigate off site. Finally, Juice is a handy Firefox plugin that allows sidebar video watching; so, no more sitting on the same page just to keep the YouTube playing. Apture and Juice are a far cry from Palantir, but are little steps in the right direction.
An exciting added layer to all of this visualization would be integrating Facebook’s open graph. 
Imagine amazing video and news content displayedvisually but mapped onto your social graph. So, you might see an article node slowly growing and then the faces of friends right around it who are discussing it. Or as you see 30 live tv broadcasts you see which of your friends are also watching the same program and able to interact live. The big picture is that the social graph that facebook is building can dramatically improve how we filter content—from news and TV to restaurants and travel. The social graph and visualization together will make the web a much friendlier and exciting place.
The Underappreciated Competitive Advantage: The Ability to Get Things Done
The more time I spend observing men and women who are high achievers, the more I realize that while many have visionary leadership, keen intellect, and a passion for change, perhaps their most unappreciated yet impressive skill is their ability to get things done. This sounds outrageously oversimplified. To many people “getting things done” seems like a commodity skill set—sure there are tricks but at the end of the day all that matters is that people check off their task list. From my study of a few outrageously impressive people, efficient task management is one of their key competitive advantages. So if you are serious about making a difference in whatever sphere you work, spend a few hours exploring project management. You’ll find that productive management results in more successful projects (e.g. that new non-profit finally gets off the ground), more free time (for your friends or family), more sleep (yielding more focus), and/or more time to explore your passions (to figure out how you actually want to spend your time). Below are a few resources I have found helpful in my journey. Please send your tricks and tips my way!
Knowledge:
Getting Things Done (buy here): the classic book on the subject. I really believe this book can change your life. Allen has consulted with hundreds of top executives. What he finds is that after these executives implement a trusted system for task/project management, they unleash an inner-creativity. They literally start coming up with all sorts of new ideas for their business or personal interests. Allen argues that since the human mind is so inefficient in determining what to store in its short term “RAM,” once you clear that memory of lots of tasks and reminders (by writing them down in an organized fashion), you free up intellectual computing ability to devote to other processes (including brainstorming). You can find a summary of some of the book’s highlights here.
The Art of Project Management: kudos to my friend Joe Barillari for sending this great piece my way. This piece is a chapter out of a book by the same title by Scott Berkun, a longtime project manager at Microsoft.
The summary points:
- Everything can be represented in an ordered list. Most of the work of project management is correctly prioritizing things and leading the team in carrying them out.
- The three most basic ordered lists are: project goals (vision), list of features, and list of work items. They should always be in sync with each other. Each work item contributes to a feature, and each feature contributes to a goal.
- There is a bright yellow line between priority 1 work and everything else.
- Things happen when you say no. If you can't say no, you effectively have no priorities.
- The PM has to keep the team honest and keep them close to reality.
- Knowing the critical path in engineering and team processes enables efficiency.
- You must be both relentless and savvy to make things happen.
Compassionate Conservatism: Its Brief History, Demise, and Future Prospects
Many allege—or, arguably, observe—that conservatives are without message. “Conservatives are not for anything—only against things,” they suggest. It is a good question to ponder: what are conservatives for? What is our agenda or platform? About a decade ago American conservatives developed and rolled out an agenda they called “compassionate conservatism,” spearheaded by several brilliant minds deeply schooled in the writings of Alexis de Tocqueville and Edmund Burke. The agenda was rooted in civil society, public/private partnerships, and an empiricism that examined what social policies actually delivered results. Despite the policy’s roots in
sound political theory and sociology and its exemplary advocates, the effort failed so miserably that even the phrase “compassionate conservatism” is unusable. In this reflection I detail what Bush’s compassionate conservatism was, why it failed, and how we might revive its sound foundations for future use. Especially given our nation’s current state—with the federal government growing in unprecedented ways (e.g. bailouts, healthcare, and energy), conservatives still having no credibility in fighting for limited government, and conservatives/Republicans struggling to find their message—the time is ripe for thoughtful conservatives to recall the best of Tocqueville to develop a uniquely American picture of human flourishing where families, neighborhoods, and communities thrive.
The Early Days
Marvin Olasky, John DiIulio, and Don Eberly were at the center of the brain trust of the “compassionate conservative” agenda introduced by President George W. Bush in the 2000 campaign. It provided conservatives with a way to respond to accusations that they had a cold heart and were indifferent to suffering. The concept—part political theory rooted in Tocqueville and Burke, part public branding effort—was promising. After all, conservatives had been a party of “no” since the ’94 Gingrich revolution (cf. their intense and successful effort in the ‘96 welfare reform act). Compassionate conservatism attempted to (rightly) make the case that conservatism is compassionate in that the domestic public policies it advocates result in Americans faring better—be better cared for, educated, and even paid.
In Defense of Management Consulting: A Critique of Matthew Stewart’s “The Management Myth”
I recently tweeted this Wall Street Journal book review titled, “Bogus Theories, Bad for Business,” a review of Matthew Stewart’s The Management Myth. I’ve not read the book nor do I know much about consulting, but found the article rather provocative. Sure enough, a friend who is much more knowledgeable about this space than I am penned the response below, which I found too insightful to not share. Adam B. Hopkins, Princeton ’05, is a former Mercer Oliver Wyman consultant and currently a PhD student in physics at Princeton.
Evan: Coming from a former consultant - and one who left consulting because he didn't believe that in the long run he wanted his life to revolve around increasing the "shareholder value" or profits of businesses that already make plenty of money - the article/book you tweeted is really off-base, and you might not want to support such a clearly biased "piece of work.”
Perhaps Matthew Stewart has some interesting things to say about management philosophy (or management science, as it is sometimes called) and its moral (or amoral) implications. I don't know, I haven't read his book. But his bombastic statements about the uselessness of consultants are quite overblown. He writes that management consultants are "intelligent nut-jobs devoted to corporate in-fighting, client-gouging, psychological humiliation and sexual harassment", according to the article.
My new Kindle DX
I received my Kindle DX (available here at Amazon for $480-eek!) about three weeks ago. In a short amount of time, my interaction with books and the written word has radically changed. And for the better.
I read more. Have you seen anyone on the subway staring at their kindle totally oblivious to their surroundings? That’s me. I find that I am reading more often for a few reasons: 1. its easier to carry my kindle with me than a book... so I have good material always hand; 2. I can carry dozens of great books with me all the time and therefore always have access to something I'd like to read; 3. it’s easier to hold than a real book; 4. I can read on the treadmill or elliptical (you can increase the font size so it’s no problem to read while jogging); and 5. I read faster on my Kindle; by altering the spacing and font size, you can optimize your reading speed. And, not turning pages is also a big help.
I read better books. I pickup book suggestions all the time; sometimes from book reviews, from friends, or via Amazon’s suggestion engine. I used to search for the book on Amazon and add it to my “wish list” or, even worse, my shopping cart. Then I “splurge” and purchase a few books per month. However, since I rarely do the due diligence on the book, I often find myself frustrated after the first chapter or two of reading it. Then I place it on my shelf, only to be packed up by movers every 12 months or so. The Kindle has totally changed this experience. The Kindle lets you download (for free and in 10 seconds) the first chapter of any book. Often these serve as a wonderful essay overview of the topic at hand and are worth reading in their own right; moreover, they always help me determine if I want to read the rest of the book. So now, I get a book recommendation, immediately go to Amazon, search for it, and click "send a preview to my Kindle." Then I read the first chapter of that book on my commute home. Brilliant.