Survival of the fittest

This is follow up commentary to this morning's share, Evolutionary Engineering, Linus Torvalds' Preference.

If we agree with Linus' sentiment (and I do) in regards to evolutionary engineerings advantages over carefully controlled methods, there are some interesting implications for the early design of projects. Long term direction should be left purposely vague and open, with only a nod of the head in a general direction (in Linus' case, unix). It's reasonable to accept that we are all essentially blind to the long term future. Instead of trying to predict it accurately a few years in advance, we are best to make several independent short term guesses in parallel. Life has been successful thus far by not putting all it's eggs in one basket. There are somewhere between 1.4 - 100 million different species on Earth all evolving along different paths. Many species die off in a few generations while some may last for hundreds of millions of years. Perhaps this is related to why there are so many flavors of Linux (and Unix, and operating systems in general). Kevin Kelly makes a compelling analogy from biological to technological evolution, and I suspect he and Linus Torvalds may share a vision of technological change.

Given that we are limited by whatever resources we can muster we need to make some tough decisions on how to proceed with a project. Evolutionary forces on technology doesn't benefit a total lack of planning. Here's a suggested path to begin project design:

1. gather as much information as we can
2. identify the immediate problems and guess at later barriers
3. find some brilliant folks to help you build something (or muddle your way through it)
4. foster open and flexible design choices, that best lend themselves to iteration
5. gather feedback from people who actually use what you helped build, then go back to 1 or pull the plug and move on to other projects. The earlier you recognize a doomed project, the better for all involved.

Hyper Specialized versus Flexible

In the 4th design suggestion I leaned towards flexible over functional but there are many times when the reverse is true (although these are short term decisions). A coarse design that functions poorly but allows for open ended improvement is usually a better decision than a single point design that's highly functional but not adaptable. Compare this to nature's analogy, when a species becomes hyper specialized to a specific environment, sudden changes in the environment can cause the extinction of the species. But sometimes if you're not fast or strong enough, you starve or get eaten.

Startups

For folks hoping to fire up startup businesses or initiate open source projects, the design decisions are daunting. There are so many different directions you can go with initial design. You can take another route, and look at novel combinations of existing technologies to create something radically new. The key is to not get too hung up on any single early decision as you're making progress. Be willing to scrap some dead end work to pursue a more efficient, productive, and spectacular route. It takes some of the pressure off when we realize: we're all agents of evolutionary change, but the world will move on with or without us.