Occam’s Razor and the Art of Software Design

The Simplest Solution is Almost Always The Best Solution

Art Gallery of Ontario Staircase (Flickr – paccpass)

“Everything should be made as simple as possible, but not simpler.” – Albert Einstein

Who the heck is Occam and what do he and his razor have to do with software design– oh… and creating software is a science, not an art – right?

There is much written about the varying levels of productivity of programmers. Research has shown that productivity differences between two programmers of nearly identical experience and education can easily vary by an order of magnitude (10x) and differences in debugging productivity can vary by a factor of 25x. The very productive ones write simple, elegant, easily understood code that accomplishes a lot with very little. These “Star” programmers “get it” in a way that most people can’t even comprehend. That being said, there are things that Mere Mortals can do to elevate their own productivity levels to approach that of the Star. In part, the Star programmers accomplish more by doing less. This is at first counter-intuitive, but if you consider that if a programmer spends time writing unnecessary code, or has to continually refactor code because the design doesn’t easily handle changing requirements they will have less time to produce the “needed” code and thus be less productive. The effectiveness of the top produces can be achieved in part by embodying the principles of Occam’s Razor and doing less.

William of Occam was a 14^th century Franciscan philosopher and monk who knew nothing of software. He did, however, understand the issues of complexity and the corresponding effectiveness of simplicity in all things. The principle behind Occam’s Razor admonishes complexity in favour of simplicity and accomplishes it by shaving off anything that does not directly support the conclusion. Occam’s Razor was originally stated as:

“Plurality should not be posited without necessity”

or

“Entities must not be multiplied beyond necessity”

Modern interpretation is often quoted as “The simplest solution is almost always the best”. The modern interpretation, however, misses the important point that “simplest” as meant by Occam means the “fewest assumptions”. The principle is also often referred to as the Principle of Parsimony: “The use of the simplest or most frugal route of explanation available”.

The statement “The lady down the street is a witch – she did it” is a very “simple”, unambiguous statement that introduces very few entities, and provides a simple explanation – or so it first seems. Examined closely, the statement, is built on a few very large assumptions and Occam would not have approved. The statement illustrates Occam’s notion of plurality and the unnecessary introduction of entities to support a conclusion. It assumes: 1) Witches exist 2) The lady is a witch 3) She did it 4) Her supposedly being a witch has something to do with whether or not she did it (whatever “it” might be). In truth, none of these elements are connected in any way to her purported guilt, but accepting them as support without questioning their veracity will likely lead to an incorrect conclusion – her guilt. History has shown us that many innocent people have been sentenced to horrible deaths based on such specious reasoning. When it comes to software design, are you sure your reasoning is not similarly flawed?  Are you certain that everything you are doing is truly being done in support of the project objectives? Have you unnecessarily introduced elements thinking that they support your planned outcome, but when closely examined they are not?

Fortunately one of the advantages of Scrum and other Agile Methods used in software development is that we focus on the most important things first. Specifically, we concentrate less on the technology and more on delivering value to the end users. Estimates vary, but in general it is thought that 80% of the functionality built into software application is either never, or rarely used. Using Scrum and Agile methods ensure that we are constantly assessing and re-prioritizing features and functionality as we go. Often, as we approach the end of the project, we realize that much of what was originally on the list of requirements is no longer relevant, but because we have delivered the elements of highest value first, stakeholders are often satisfied earlier in the development. Often they will decide that many of the capabilities originally requested are no longer relevant. Good development methodology that focuses effort on delivering value goes a long way towards the principles espoused by Occam, but it’s not enough.

The business schools reward difficult complex behaviour more than simple behaviour, but simple behaviour is more effective. –Warren Buffett

Software solutions are ever increasing in size and complexity as the requirements of application solutions also expand. Complexity introduces uncertainty, and uncertainty means risk; the greater the risk, the higher the probability of failure. Plurality (adding Entities) in support of a conclusion does not simply add complexity, it multiplies the complexity. This is illustrated by the following exercise: Take a blank piece of paper and a pencil and on the paper draw two dots with a straight line connecting the dots to each other. Now add a third dot and connect it to each of the other two dots. You should now have three dots and three lines. Now add a fourth dot and connect it to the previous three dots. You should now have four dots and six lines. Jump to a 100 dots and you will have 4,950 lines or interconnections. The number of lines connecting the dots to each other increases dramatically with each new dot added. There is a formula to calculate the number of lines connecting the dots. If you really want the formula to calculate the number of connections it is:

 nCr = n!/r!(n-r)!

where n is the number of items in the subset – 2 in our case because we are looking for the number of connections between any two dots, and r, which is the number of dots.

The complexity here is not in the dots, but the fact that each additional dot multiplies the number of lines that connect the dots. If you think of the entities in your software (classes, object instances, modules, components, databases, tables, etc…) you can easily see that the complexity of the system increases as the number of entities increases. But wait a minute… in software these things don’t all connect to each other. You are probably right but the real question is “Are you sure?” Can you say with absolute certainty that a change in any one entity will not have an impact on any other entity? What happens if you have hundreds or even thousands of these entities – far more than any one person can realistically understand – are you still sure that you understand all of the interactions? What about the collective effect of all of these entities? Now what if the system is not well designed with an appropriate separation of concerns and is not self-documenting with well chosen class names thus causing the function of each of the entities to be less clear? The “what if’s” could be added endlessly and they increase with the number of entities you add. These “What if’’s” represent assumptions in Occam’s view, and the more assumptions you have, the more complex your system and thus the higher the risk and the greater probability of failure.

My assertion is that “Assumptions are unknowns masquerading as knowns” and as such represent a hidden, and very real danger to the integrity of your system and the probability of success of your project. Assumptions are a necessary part of life. I assume that the sun will rise tomorrow; I assume that my Internet connection will work; I assume that milk will be available in the grocery store the next time I go shopping. Assumptions help us to bundle up the complexity of life into nice neat little packages that help us to manage the massive amounts of information we must deal with every minute of every day. Assumptions are very useful, but incorrect assumptions can have catastrophic effects. Clearly we can’t test every single assumptions and this is where experience has no peer. Knowing which assumptions to test comes from a heuristic “been there done that” body of knowledge and there is no substitute for that experience.

So it’s clear that complexity must be reduced and assumptions challenged – but how? This is where the art is involved. If you look at the field of architecture, you can find many strong parallels with the world of creating software. Both fields are very technical. In both fields simplicity and elegance and the facility with which humans interact with the result are key elements of success, and in both fields there are many ways and forms in which to deliver a solution – all of them satisfying the requirements. Some solutions, however, are clearly superior to others, and some are simply brilliant. In architecture, the results of the “art” are usually more evident and very clearly visible because they are on the exterior – and well – the seemingly gravity defying cantilevered design doesn’t fall down. There is no less inspiration, intuition or vision in great software than there is in great architecture, but it is not so clearly evident. A great software solution is simple, elegant, enjoyable to use and it doesn’t fall down. I believe it is no coincidence that the people who mastermind the designs in both fields are called architects, and in both fields, brilliant results require uniquely talented people. In some ways this notion is at odds with the Agile Approach as Agile focuses heavily on the team.

Most of the fundamental ideas of science are essentially simple, and may, as a rule, be expressed in a language comprehensible to everyone. – Albert Einstein

I have worked with several terrific architects over the years. They’ve had amazingly diverse education and work backgrounds. Perhaps the best I’ve worked with is a guy who did not have a computer science degree; in fact he didn’t finish grade 11. One of the traits I noticed that was common to him and the rest of the talented architects I’ve worked is that they ruthlessly asked the question “Why”. They questioned the key assumptions of the design, looking for conclusive justification for both its existence and its functions. A great architect and a great software architect will both have a solid reason for everything in the design. The Why for each element will be well reasoned, and can be simply explained. If the existence and function of an entity cannot be simply explained, then it is likely that much like the woman at the end of the street who is reasoned guilty based on the assumption that she is a witch, your design likely suffers from the “Plurality” admonished by Occam. Something I’ve noticed by constantly asking Why is that many requirements are expressed in terms of a solution, not the actual requirement. Asking Why can remove multiple layers of assumptions and take you to the real requirement which may be very different from what was originally stated. Entities with clear purposes for their existence tend to fit together well in large designs because the absence of duplication (plurality) contributes to a simpler and more elegant design – especially on the grand scale.

Clearly there is far more to good software design than simply using Occam’s Razor, and not all teams can have a great architect. For that matter, not all teams can even have any architect, but everyone on the team should strive to think like a great architect and of everything in the design ask “Why”. Like Occam, ruthlessly shave off the unnecessary entities and produce designs that are simple, elegant, easily explained and contain no assumptions.

A word of caution: Occam used the Razor to posit that “God’s existence could not be deduced by reason alone.” This was too much for Pope John XXII who excommunicated poor William. So temper your questioning with sound judgment so that you do not get excommunicated from the team.

They say the world has become too complex for simple answers. They are wrong. – Ronald Reagan

As always, I look forward to your comments.

Michael

Michael

I am an independent consultant who has been leading software teams, designing, building and delivering software for nearly three decades. It’s still as exciting and enjoyable for me today as at was when I wrote my very first Hello World program and saw it spring to life in front of me.