The Innovator’s Dilemma

After reading Disrupting Class and several articles about disruptive technology on the asymco blog, I decided I should go to the source and read The Innovator’s Dilemma by Clayton M. Christensen, published in 2000. It’s one of those books that seems fairly obvious in retrospect — now that ten years have passed and its lessons have largely been absorbed into business practice and culture.

The book is based on Christensen’s PhD thesis, which originally looked at technology and business trends in the hard disk drive industry. He found that some technologies (such as improved read-write heads) served to “sustain” existing product lines and cement the dominance of existing companies, while other technologies (such as smaller form factors) ended up “disrupting” existing products to the extent that once-dominant companies sometimes went out of business in just a few years.

The reason these companies failed was not that they were poorly managed, but because the disruptive products were in completely separate markets (and accompanying “value networks”). The existing companies were simply not designed to compete in those new markets. For example, 5-inch drives were sold to minicomputer makers, while 3.5-inch drives were sold to personal computer makers (with shorter design cycles, higher volumes, and lower profit margins). The existing minicomputer customers had no need for 3.5-inch drives, so the 5-inch manufacturers saw no market and no need to produce them until it was too late and other startup companies were already dominating the emerging market for personal computer hard drives (3.5-inch).

In other words, the businesses of making and selling 5-inch versus 3.5-inch drives were so different that being the dominant expert in hard drive technology was not actually much of an advantage. In fact, it was a disadvantage because the whole organization was designed to compete in the old business and naturally fought attempts to undercut that business.

But how do you know if a given product idea is going to be disruptive?

One clue: disruptive products are usually simpler, less powerful, and have smaller profit margins than existing products. So they need to find markets that value product attributes like convenience, reliability, and ease of use over sheer power. For example, business accounting software in the nineties was driven by the needs of large enterprise customers and so was quite complex and powerful. Quicken disrupted this market by creating a simpler, cheaper product based on its personal finance software. This was so much easier to use that it quickly gained an 80% market share among small business owners who did not need all those extra features.

What makes technologies “disruptive” rather than just “niche” is when they progress far enough to compete up-market with existing product lines. For example, Quicken continued to add features so that larger and larger businesses were able to use its software, pushing out the old software companies to only serve the largest enterprise customers. Potential disruptive technologies should have a plausible development plan that will eventually displace existing products up-market.

The big take-aways are:

1. If you want to start a new company, do it with a product idea that is likely to be disruptive. Otherwise, you have very little chance of making any headway against existing players.

2. Generally the only way to manage disruptive technologies from within an existing company is to create a totally separate organization with the sole purpose of going after that disruptive technology. If you don’t keep it separate enough, resources will inevitably be borrowed to take care of existing business and the new products will languish.

Apple has a better record than most for its ability to disrupt its own products before competitors get the chance. Horace Dediu makes a good argument that the iPhone should be seen not as “a better phone” but as a disruptive technology for personal computers: a simpler and more convenient way to accomplish computing tasks such as email and web surfing. The inclusion of a phone capability just makes it all the more convenient. I know at least one person who decided to get an iPhone instead of a new laptop; and Apple’s iPad is even more competitive with laptop computers. iPhones and iPads will continue to “move up-market” by adding the ability to conveniently handle ever more computing tasks. As this happens, Macs and other desktop PCs will increasingly be seen as high-end tools for power users.

2001: Space Art

I just watched 2001: A Space Odyssey, mostly with the goal of better understanding nerd cultural references. I hadn’t realized until I looked at the DVD jacket that it was released way back in 1968, shortly before the first real-life moon landing in 1969.

I assume (and skim from wikipedia) that 2001 is legendary for its pioneering special effects (such as simulated zero-gravity environments and spaceship fly-bys) and the philosophical and scientific questions it raises. I’m not going to try to dispute its status as a work of genius. I remember enjoying the book version when I read it many years ago.

But of course, by this point in history, artificial intelligence has been thoroughly discussed, and the astronomical cost of space travel makes the lavish and enormous spacecraft in the movie seem absurd (for example, the jupiter-bound ship is way bigger than necessary for supporting a mere six crew members).

And it seemed to me that the parts of the film which actually moved the plot forward could have been condensed down to about 15 minutes. The rest is better interpreted as space art, to be enjoyed at leisure in a gallery while pondering the nature of humanity.

All of this is to say that I found the movie to be extraordinarily boring.

But at least I’m one step closer to understanding what the heck my co-workers are talking about…

Dramatic photo


I took this photo from the Queen Anne neighborhood in Seattle (walking distance from my office), looking southwest towards Elliot Bay.

Camera: iPhone 4.

Post-processing: Digitally removed power lines via Photoshop.

Collective intelligence depends on social skills

Researchers from MIT and elsewhere recently published a study where groups of two to five people had to solve various problems such as “visual puzzles… negotiations, brainstorming, games and complex rule-based design assignments.”

They found that “the average and maximum intelligence of individual group members did not significantly predict the performance of their groups overall.” However:

Groups whose members had higher levels of “social sensitivity” were more collectively intelligent [i.e. those groups had better scores on the problems they solved together]. “Social sensitivity has to do with how well group members perceive each other’s emotions,” says Christopher Chabris, a co-author.

The study was billed as a way for managers to form better teams. But the more important point to me is: social intelligence is critical in business. When students enter the workforce without well-honed social skills, the teams they’re a part of are less effective and make worse decisions.

As another of the study’s co-authors said, “What individuals can do all by themselves is becoming less important; what matters more is what they can do with others and by using technology.” If this is true, effective schools will need to prioritize social intelligence in the curriculum.

BASIC was designed for students

I’m not sure when I put this article about CS education reform into my Instapaper, but I learned something new:

In the early 1960s, the professors John Kemeny and Thomas Kurtz developed BASIC (Beginner’s All-purpose Symbolic Instruction Code) at Dartmouth College because they thought educated people, and future leaders of America, should have some first-hand experience with computing.

I like this precedent of developing tools for students first, and business markets later. Focusing on students (and particularly “100-level” classes) is a great motivation to keep things simple and easy to learn. When/if the technology eventually gets powerful enough to compete with other tools, it will win because real people might actually want to use it.

Bill Buxton strikes again

A few weeks ago, I saw Bill Buxton give a talk at UW for the Puget Sound SIGCHI meeting.

The main takeaway for me was Buxton’s call to study and learn from the history of design. “Know the good parts of history and what to cherry pick from it.” For example, the original set of five-color iPods took many design cues from an early consumer camera product line. “[Apple design chief] Jonathan Ive knows the history.”

Buxton showed photos of what he called the first educational technology device: the PLATO IV from 1972. It included graphical output and a touch screen, and was apparently put in schools all over Illinois. The similarities to the iPad are striking. He demoed a watch from 1984 that includes a touch surface capable of doing character recognition on decimal digits. It sold for just $145 (in today’s dollars). Buxton also took a look at the first real smartphone: the “Simon” from 1993. It is shockingly similar to the iPhone, complete with a row of app icons on the home screen. The only app “missing” is a web browser (the html web was still a research novelty in 1993).

There were many other examples which I didn’t note specifically, many of them MIT Media Lab prototypes published in SIGGRAPH. Buxton also pointed the audience to archive.org for more, such as a video on input devices in 1988.

The second takeaway was Buxton’s theory of the “long nose”: it takes about 20 years to go from an idea to a $1 billion industry. In other words, “Any technology that is going to have significant impact over the next 10 years is already at least 10 years old.” So the important act of innovation is not the “lightbulb in the head” but rather picking out the correct older ideas that haven’t yet hit the elbow of the exponential curve. When change is slow, humans don’t tend to notice; but you can counteract that by explicitly measuring the change as technology progresses. What are the technologies invented 20 years ago that are about to become huge?

Still Magical

As part of testing our upcoming iOS 4.2 release of OmniGraphSketcher for iPad, I just threw together this graph — a more or less exact replica of a textbook economics diagram.  All on the iPad, without any fuss.

Economics diagram f on OmniGraphSketcher for iPad

I could email the PDF directly to a textbook publisher.

Despite the fact that I’ve been working on this app ever since the iPad was announced, the whole thing still kind of boggles my mind. Even though I know in detail how the app works, Apple’s term “magical” is the best way I know of to describe the experience of using it.

Chaos: Making a new science

If there was any doubt that science is driven by people politics as much as anything else, look no further than James Gleick’s 1987 book, Chaos: Making a new science.

The book chronicles the history of chaos theory; but “chaos” is also a good word to describe the scientific community’s embarassingly slow acceptance of the findings and tools of this new mathematical subfield.

The book held particular interest for me because of an unsolved mystery in a research paper I wrote in college, Weather forecasting by computer. Edward Lorenz, who published the first research on what would become chaos theory, calculated in the late 1960s that “even with perfect models and perfect observations, the chaotic nature of the atmosphere would impose a finite limit of about two weeks to the predictability of the weather.” Despite this, I was reading brash predictions in books published in the early 1980s that we would soon be able to forecast the weather months or years into the future. Why did it take more than a decade for this fundamental mathematical result to make its way even to experts writing about weather forecasting?

Gleick wondered the same thing. The fact that his conclusions took the form of an entire book is testament to the many factors at play. (I was a bit relieved to confirm that I wasn’t just missing something obvious.)

Part of the answer is that chaos theory was outside the scope of existing academic disciplines, almost by definition. It tried to make sense of problems that couldn’t be solved using traditional mathematics — the very problems that most researchers (and entire science departments) stayed away from because the chances of progress seemed slim. Over time, disciplinary boundaries developed such that most of these problems were not considered valid topics in physics, biology,… and even weather forecasting.

A second part of the answer is that many of the important results of chaos theory themselves defined limitations on what is possible to know or achieve, especially when seen through the lens of traditional approaches. Scientists and other leaders didn’t want to believe these pessimistic claims, and they were easy to ignore when coming from a suspicious fringe group of career-insensitive mathematicians.

A third part of the answer is that even when the essential properties of chaos theory had been well established by mathematicians, the theory was not useful to mainstream scientists until practical mathematical tools were developed. Several important mathematical results eventually helped to show how disparate data sets all displayed chaotic “bifurcations” and “period doublings,” for example. As scientists were given more concrete patterns to look for, evidence of chaotic behavior became increasingly visible to them.

And yet a fourth part of the answer is that the main tools used to investigate chaos theory — computers — were new and unfamiliar to mainstream scientists. Lorenz was one of very few theorists in the 1960’s who had access to expensive computer time (and the knowledge to use it). And although rigorous mathematical proofs were eventually found for many components of chaos theory, for many years the most important results were simply the outputs of clever computer programs. Running experiments like this via numerical simulation was a totally new approach. Scientists and mathematicians had every reason to be skeptical.

At the time Gleick’s book was published, chaos had finally become broadly accepted in science and had led to a few high-profile applications such as heart pacemakers. Yet even now, 20 years later, chaos theory is not part of the standard curriculum at any level of school. I studied it for a few weeks in high school as part of a special end-of-year diversion; and in college as an elective math course that was only offered one semester every other year. And I went to very progressive schools. When Steven Wolfram unveiled his “new kind of science”, non-experts missed the fact that he was talking about this same line of research. The new science is still in its infancy.

Automatic color temperature

Bill and I had an interesting conversation today about how computer displays (particularly on mobile devices) should automatically adjust not only the brightness of the backlight, but the color temperature of the pixels. For example, in a room with warm lighting, the “white” on the display should look reddish, the same color as the reflection of that ambient light off white paper.

See Bill’s blog post for the full story.

Sculley on Apple

At the risk of being like all the other bloggers, I feel the need to write down what I didn’t know or found most interesting about the recent interview with John Sculley, who was the CEO at Apple for about ten years.

First, Sculley made it clear that Apple has always been a digital consumer electronics company, and in many ways was the first such company. As digital components continue to become cheaper and more powerful, thus enabling better consumer electronics, it makes sense that Apple will continue to thrive. Another way of saying this is that Apple was so ahead of its time that even after 25 years the market conditions are only beginning to really align with their strategy.

Second, I often explain to people how Apple’s control of both hardware and software is key to their ability to innovate. There is a lot they simply couldn’t do if they didn’t control the whole pipeline. Sculley recounts an excellent example of this, dating back to the first Mac.

Sculley: The original Mac really had no operating system. People keep saying, “Well why didn’t we license the operating system?” The simple answer is that there wasn’t one. It was all done with lots of tricks with hardware and software. Microprocessors in those days were so weak compared to what we had today. In order to do graphics on a screen you had to consume all of the power of the processor. Then you had to glue chips all around it to enable you to offload other functions. Then you had to put what are called “calls to ROM.” There were 400 calls to ROM, which were all the little subroutines that had to be offloaded into the ROM because there was no way you could run these in real time. All these things were neatly held together. It was totally remarkable that you could deliver a machine when you think the first processor on the Mac was less than three MIPs (Million Instructions Per Second). (NOTE. For comparison, today’s entry-level iMac uses an Intel Core i3 chip, rated at over 40,000 MIPS!)

This approach continues to be important in every category of device Apple produces. For example, today’s iPhones and iPads have special-purpose hardware for video decoding (that’s why they can only play movies encoded in certain formats). Microsoft could not really enter the graphical operating system market until standard processors and graphics cards became powerful enough to do most of the graphics routines themselves. If Microsoft produces an innovative operating system that requires hardware that is too specialized or not readily available, the hardware manufacturers will say, “sorry, we can’t support it.” At Apple, Steve Jobs says, “We will find a way.”

There’s a great little quote about a meeting at Microsoft. “All the technical people are sitting there trying to add their ideas of what ought to be in the design. That’s a recipe for disaster.” Sculley thinks that’s part of the silicon valley culture started by HP, where engineers are most respected. At Apple, designers are on top. “It is only at Apple where design reports directly to the CEO.”

Sculley repeats over and over how good a designer Steve Jobs is. The following story is about a visit to the inventor of the Polaroid camera.

Dr Land had been kicked out of Polaroid. He had his own lab on the Charles River in Cambridge. It was a fascinating afternoon because we were sitting in this big conference room with an empty table. Dr Land and Steve were both looking at the center of the table the whole time they were talking. Dr Land was saying: “I could see what the  Polaroid camera should be. It was just as real to me as if it was sitting in front of me before I had ever built one.”
And Steve said: “Yeah, that’s exactly the way I saw the Macintosh.”

Dr Land had been kicked out of Polaroid. He had his own lab on the Charles River in Cambridge. […] We were sitting in this big conference room with an empty table. Dr Land and Steve were both looking at the center of the table the whole time they were talking. Dr Land was saying: “I could see what the  Polaroid camera should be. It was just as real to me as if it was sitting in front of me before I had ever built one.”

And Steve said: “Yeah, that’s exactly the way I saw the Macintosh.”

The item which I did not know (or maybe had forgotten) is that Apple’s Newton product played a central role in the creation of the ARM processor design, which is now used across the industry in most smartphones and other consumer electronics (including the iPhone, iPad, and Apple TV). Moreover:

The Newton actually saved Apple from going bankrupt. Most people don’t realize in order to build Newton, we had to build a new generation microprocessor. We joined together with  Olivetti and a man named Herman Hauser, who had started Acorn computer over in the U.K. out of Cambridge university. And Herman designed the ARM processor, and Apple and Olivetti funded it. Apple and Olivetti owned 47 percent of the company and Herman owned the rest. It was designed around Newton, around a world where small miniaturized devices with lots of graphics, intensive subroutines and all of that sort of stuff… when Apple got into desperate financial situation, it sold its interest in ARM for $800 million. […] That’s what gave Apple the cash to stay alive.

So while Newton failed as a product, and probably burnt through $100 million, it more than made it up with the ARM processor.

So the product that has become a “celebrated failure” of business school lore was actually one of the most successful products in the industry, when taking the long-term technological view. Not only did it save Apple from bankruptcy, it enabled an entirely new category of power-efficient, mobile computing devices. True, the marketing and business strategy used for the Newton failed. But the underlying technologies and vision not only saved Apple in the 90’s, but also formed the basis of its remarkable growth today.

Everything Apple does fails the first time because it is out on the bleeding edge. Lisa failed before the Mac. The Macintosh laptop failed before the PowerBook. It was not unusual for products to fail. The mistake we made with the Newton was we over-hyped the advertising. We hyped the expectation of what the product could actually [do], so it became a celebrated failure.

Apple has gotten much better at this over time. They are often accused of over-hyping their products, but if you look carefully, Steve Jobs and Apple advertising never claims that their products can do anything they can’t actually do. They say things like “1000 songs in your pocket” and show people dancing to those songs. Some people react with “oh my gosh, that’s what I actually want.” Other people say, “no thanks.”

Technologists complained when Apple described the iPad as “magical.” Obviously the device is not actually magical. But I think that word pretty well describes the actual reaction of many customers to the product.