Paul Graham is an inspiring essayist. I agree with most of what he says. In August, he wrote an essay entitled Holding a Program in One’s Head. This essay gave me a lot to think about. He captured the essence of much of what upsets me about the corporate environment in which I try to practice my profession. He overlooks one important aspect of an environment that will encourage creativity and innovation. That is the freedom to fail without repercussions.
I sometimes listen to Terry Gross’ radio program Fresh Air on NPR . Recently she interviewed Steve Carell and Terence Blanchard on two separate programs. The thing that struck me is that they both had similar stories to tell about the environment that nurtured their respective careers.
In Steve Carell’s case, he spoke of his experience as a cast member of Second City, the famous sketch comedy troupe where many of Saturday Night Live‘s cast received their training. He said that the great thing about that environment was the opportunity to fail. He went on to explain that they were expected to go out and try new characters and ideas. If they worked, great. If they failed, there was no penalty. They were just expected to go out the next performance and try something different.
In a surprisingly similar fashion, Terence Blanchard described what it was like when he joined Art Blakey’s legendary Jazz Messengers band. Terry Gross asked him if Blakey was a disciplinarian. Terence responded that on the contrary, all the members of the band held Blakey in such high regard that he had to spend a lot of effort putting them at ease. When they asked him to let the band play the classics that made the Jazz Messengers famous, he told them that they should compose their own music and make the band their own. Terence went on to say that it was great to have the freedom to fail. To try new things without worrying about whether they would work or not.
I think that programming is the same kind of creative activity that acting and creating music is. By the very same reasoning, I think that the freedom to try things and fail without repercussions is essential to writing innovative software. Furthermore, I think that most software falls into the category of innovative software. We make the mistake of treating software development like the other engineering disciplines and ignoring the central difference between it and them.
If we are building a building or a circuit board or any other physical product, we have a clear idea of the functionality we are after. We might build a model or a prototype to clarify our thinking but we wouldn’t dream of trying to sell a model or a prototype as the final project. Typically, a lot of effort is expended to rigidly define the product so that we can effectively produce it exactly to plan. The reason we do this is that the process of production is costly to change. We get cost breaks from producing many items exactly the same.
Software is different. In the first place, the per unit cost of producing software is negligible. Secondly, you rarely know exactly what you want the product to look like until you are well along the road to producing it. Unlike physical production lines, changes to software are easy and inexpensive to make. In all but the most extreme circumstances, software requirements are discovered as you implement the program.
You can define requirements in very broad terms, e.g. “this is a point of sale terminal for a video rental store”, but the details of how that should be implemented should be derived from interactively creating software that automates the repetitive tasks of running a video rental store with someone that understands those tasks intimately. And, who knows what those tasks are better than someone that runs a video rental store.
An analogy that was created to explain the rationale behind eXtreme Programming (XP) captures the essence of this principle. You don’t drive a car on a long trip by carefully aiming it in the direction of the destination and tying the steering wheel in place. You would be in the ditch before you went a mile. Instead, you constantly adjust the course, you drive the car. In much the same way, you must drive toward the creation of an innovative piece of software.
That’s why I love being a programmer. That’s why I hate programming in a corporate environment where management doesn’t understand these basic facts about software development and wants to maintain rigid control over the process. But then again, the money is good.