Michael Lant

Software Archictecture, Development, Agile Methods and the Intersection of People Process and Technology

Agile, Project Management, Software Development, Teams, Technology

Software Development and Creativity

What is Creativity

Light bulb
Lightbulb (flickr: Julius Narkunas)

Don’t think. Thinking is the enemy of creativity. It’s self-conscious, and anything self-conscious is lousy. You can’t try to do things. You simply must do things. – Ray Bradbury

A hunch is creativity trying to tell you something.Frank Capra

What does creativity have to do with software development – actually a lot. Software development is the process of creating software solutions that have never before been created. If your solution has already been created, then you shouldn’t be doing it because you are either plagiarizing, or you are needlessly reproducing work that you should be reusing.  Not only should your work be original, it should also be useful. Clearly if you are creating something that has never before been created, you are well… doing something creative, and thus knowing how to be creative is clearly a good thing. But before discussing creativity as it applied to software we must first set a foundation and define what creativity is.

Creativity is more than just being different. Anybody can be plain weird; that’s easy. What’s hard is to be as simple as Bach. Making the simple, awesomely simple, that’s creativity.Charles Mingus

There is nothing new in the world. Actually there is very little that is new, and the things that many of us consider to be new are not in fact new. The iPhone at its introduction contained little, if anything that was truly new.  There is no denying that the iPhone was a revolutionary product when it was introduced, but its genesis was that of evolution, not revolution. Wireless phones, digital cameras, touch screens, GUIs, MP3 players, SMS clients, and installable applications have all been around for years. What the iPhone brought us that was new, was the combination of all of the aforementioned technologies (and more) in a way that had previously never been seen. There were countless other things that could have been included in the iPhone, and many devices already included most of what the iPhone offered. In fact as a phone, the iPhone wasn’t even a particularly good phone. What made it unique was there was so much capability in one package, and it was so elegant and simple to use. Examine innovation in any field be it art, science, theatre, and even software, and you will find that most great innovations were mostly evolutionary, not revolutionary.

So what is creativity? Creativity is a difficult thing to define. You can’t see it, hear it or measure it in any way, yet we know it exists and we often (not always) recognize and appreciate its results. It is that bright light bulb that pierces the darkness, illuminating a solution that is often so simple that you wonder how it could have ever been overlooked. But these solutions are overlooked every day, and it is because we all view the world with our own set of filters that prevents us from seeing what other see.

Many people believe that so called “Creative People” dream up new ideas and new products out of the blue. While that may be true in a few rare cases, it is remarkably uncommon. Creativity is the process of breaking out of established patterns. It is an accretive process that builds upon the many innovations that have already occurred over the millennia. It is the reorganization, recombination or reinterpretation of any number of entities, concepts or ideas [elements] either by altering, removing, or adding new elements to the existing set in new and different ways. The result is something unique and previously unknown to the person or people creating it. It has nothing to do with aesthetics, practicality, usefulness, whether or not you or someone else likes or dislikes the results or even whether or not it has already been done by someone else. All that matters is that it is different than what was previously known to the creator. What is created does not even need to be manifested in the physical world. Physical manifestation is only evidence to the rest of the world of the uniqueness of the creative idea.

Picasso did not invent paint, canvas or even the techniques for laying paint onto canvas. Instead, he simply asked “What if on a single two dimensional canvas I could represent more than one view of a person or object?” Monet’s failing eyesight caused him to use larger blocks of colour so that he could actually see what he was painting. The principles of flight were well known long before the Wright Brothers first took to the air. What they did was find the right combination of bicycle parts, weight, wing surface, engine power, wood and canvas and the right balance point to make the first flight possible. Edison’s introduction of the light bulb to the world was no different. There were actually many people working on the creation of artificial light starting with Joseph Swan who obtained a UK patent for the first light bulb in 1860. Canadian Henry Woodward created his own version and obtained a US patent in 1874. Woodward’s invention was considered crazy in Canada and he sold the patent to Edison who saw the commercial potential and refined the design so that it became a viable product. There is no question that each of these individuals were highly creative and innovative people who were able arrive at solutions that had eluded so many. None of these creations rose out of nothing. It was their specific and unique organization of elements in a way that allowed them to arrive at the solution that had eluded so many that made them successful.

Productive Creativity

Creativity has more to do with the elimination of the inessential than with inventing something new.Helmut Jahn

It is one thing to be creative, but is another thing to create something that is unique and also useful or wanted. We live in a hyper-competitive world where the race to have the next creative innovation has become extremely intense. Hundreds of billions of dollars are spent worldwide every year on R&D. Even with so much effort being expended by so many organizations, and with creativity itself being nothing more than discovering new combinations of existing elements, it is clear that creativity – the ability to find that magic combination that is both unique and wanted or needed is very difficult, and something that few of us do well.

Vincent Van Gogh was unable to sell a single painting while he was alive, but seven of his paintings are now among the 40 most expensive paintings ever sold at auction. Pablo Picasso with ten of the top 40 is the only artist to have more. Together, these two masters hold 17 of the top 40 spots – nearly half of the total. Two people are creators of nearly half of the 40 most valued pieces of art in history (at least the ones that have come to auction). So why was Picasso’s work revered during his lifetime, and Van Gogh’s work considered near worthless when created, yet now command prices in excess of $100 million. Why do some creative software ideas succeed, while most never see the light of day? In Van Gogh’s case, his timing was just lousy; the world was not ready for his kind of creativity. No doubt timing can also be lousy for technology innovators; Apple failed with the Newton and succeeded with the iPhone. Business factors are certainly relevant, but putting business factors aside, most creative efforts that do not catch on are because the idea is just not sufficiently useful to enough people. Sometimes timing is the issue, but more often than not, the idea is just not of value. The difference between the two is Productive Creativity.

People who are productively creative focus their effort on bringing something of value to the market. Van Gogh and Picasso were both brilliant artists, and each was passionate about his own work. While Van Gogh did not understand how to convert his brilliant paintings into a commercial success (spending time in an asylum and cutting off part of his own ear didn’t help); Picasso did. Certainly there are also artists with little or no artistic talent who understand how to commercialize their work, and some with neither talent or business acumen. Parallels can be drawn with software developers because in many respects, software development is at the intersection of Art and Commerce. Sadly, this intersection is not understood or appreciated as such. I have also encountered more than a few brilliant developers whose contributions are marginalized because they didn’t know how to frame their ideas in a business context or because the business itself regards that software development as purely a technology effort with no art involved. To some degree, I think this attitude is because software development entered the world in large corporations to run payrolls, manage accounts receivable and many other important, but nonetheless mundane tasks. Software development now has very little to do with the data centres, servers and networking that enables it and has more to do with things like human perception, human cognition, language, ontology and taxonomy. Software development is the art of abstraction of things, processes and interactions in the real world into structures in code resembling sentences, paragraphs and chapters of a book. There is an infinite number of ways to develop any software application and none of them are either right or wrong unless the numbers don’t add up. Product Creativity is the art of figuring out ways to create software that are better than what anyone else creates. Anyone who doubts the importance of creativity and inspiration and its intersection with commerce need only look at the mobile space where the Android and iPhone platform each have more than 200,000 unique and innovative applications available.

Fostering Creativity in Your Team

Creativity Killed

Creativity Killed (flickr: Anant Nath Sharma

But the person who scored well on an SAT will not necessarily be the best doctor or the best lawyer or the best businessman. These tests do not measure character, leadership, creativity, perseverance.William J. Wilson

The truly creative people in this world are the outliers – they are rare. They are not identified by résumés that say they have ten years of Java experience, or perhaps the equivalent in .Net. These are simply measures indicating that they continued to show up for work each day. Sadly, these metrics are the primary filters used when recruiters and HR specialists begin the search process. The creative people you want on your team are unique in their ability to see the world differently from the rest of us. That unique vision is essential, but it is not enough. What is also needed is the ability to practically apply that unique vision within the structure and processes that are necessary to support the act of developing complex software projects. To be certain, an important part of that is experience. Very experienced people have a body of knowledge that gives them a distinct advantage over less experienced individuals, but the very filters we tend to use end up filtering out the people with the more important broad experience in favour of people with less useful narrow experience.

And it has become a kind of a truism in the study of creativity that you can’t be creating anything with less than 10 years of technical knowledge immersion in a particular field.Mihaly Csikszentmihalyi

I don’t entirely agree with the previous quote because there are people who have decades of experience who will never be creative, and there are those who simply “get it” far sooner than most. That being said, a sound technical background is important – not measured in years, but measured in how they’ve used their experience to innovate. Sometimes lack of experience can even be an advantage because the lack of preconceived notions can make obvious solutions that are missed by people who are used to the status quo. Experience inherently builds patterns of efficient behaviour; creativity necessitates the breaking of patterns. Understanding “why” leads to new solutions; just knowing “how” simply leads to more of the same.

Not only is it important to find and hire people who develop productive creative solutions, you need to provide them with an environment where they can be productively creative. The onus for this is on your company and on you as the person leading the team. Finding the unique combination of elements that produces the innovative new solutions requires experimentation. Experimentation means trial and error, and thus some of what is attempted will not work. Many corporations discourage such things, but if you and your organization really do want to innovate, there is no alternative path. Innovation does not come from process, protocol, structure, long status meetings or administrative trivia. While structure and process are important (without them anarchy reigns), do your best to free your team from such things and instead, spend that time experimenting and innovating. The benefits will far outweigh any any that could come out of the meetings and administrative noise. Help your team by giving them some structure within which to succeed. Set some objectives for them such as each team member must present their innovation work once per quarter to the whole team. It’s amazing what this kind of peer pressure can produce. It can create a competitive environment where each team member tries to produce the best they can dream up. You will be amazed at what comes out of this.

If creative people are rare, the person who can be creative and function well within a structured process is even rarer. To their great loss, many organizations also have a difficult time working with creative people. Some companies fear creativity, and some simply don’t want anything to do with it. I’ve had conversations with several recruiters who say that employers have come to them and said “Please don’t send me anyone with ideas. I just want someone who is going to sit in his/her seat and program what I tell them to.” Without a doubt, there are other companies that are innovating, while companies like these stagnate and die. If you’ve gotten this far in the article, you are clearly not of that mind, so what do you do?

Here are some things you can do:

  1. Try to find and hire those creative geniuses. I would rather have a small team of inspired out of the box thinkers that may be hard to manage than a team of drones. My team of creative geniuses will be more work for me to manage, but we will always finish first, and we will always have the best solution.
  2. Make your team as diverse as possible. Try to find the best group of people possible that includes the broadest range of backgrounds as possible. Mix Java programmers with .Net programmers; seasoned pros with the kid just out of school; web developers with main framers; and financial application developers with smart phone developers. The greater the diversity, the more ideas people will be able to bring to the table.
  3. Look for people to bring into your team with backgrounds in something completely different like art, literature, philosophy, music or languages. The more things they have done (particularly creative things) the more ideas, unique approaches and experience they have to offer. They still need to be excellent programmers, but excellent programmers with a broader range of experience have more to bring to the creativity table.
  4. When hiring your team, pay little attention to things like the number of years a person has spent with a particular technology or worked in a particular industry. Instead, ask them about their innovations and inventions. Find out what they’ve dreamed and created, and why it was successful and what they’ve learned from the things that were not successful. When asking questions like this during an interview, be sure that you explain why. People are trained to filter their responses in interviews to provide what they think the corporation is looking for. Rarely is creativity one of those things, so help your candidates help you to uncover their creative talents.
  5. Give creativity a home and a place to thrive. Creativity is a fragile thing that must be nurtured. Give your team one afternoon per week where they get to work on their own idea. Even better, give them an entire day each week. What they produce must be related to the business you are in, and they must present their idea to the rest of your team for possible inclusion in your project, or maybe something entirely new. Either way, it must be something its creator believes has value to the group and the organization as a whole. Celebrate the successes, and have some fun (not ridicule) with the ones that are odd or off the wall. Countless studies have shown that the companies that formally support creativity through such programs are consistently the ones with the best new idea, the highest productivity, and the best morale. Google is one of them. If you are running an Agile team – actually any development team, then schedule the creative time. The work that comes out of this time is important  (it could be the big break your company needs), so don’t let all of the other urgent things encroach on it.
  6. Encourage experimentation, and yes, failure. Most organizations tend to have little tolerance for failure of any sort, and this causes people to take the safe route, to not experiment, and to not be creative. Failure means that something new was tried, and not all new things succeed. If nothing new is tried, then all you are doing is repeating what you’ve already done, and you can bet that while you continue to do the same thing, the competition will be innovating while you are not. Minimize the downside potential of the failure by limiting the scope of what is being attempted to a series of smaller pieces that can each be assessed to determine if continuing down that path is worthwhile.
  7. Always ask Why, and then ask Why again. Keep peeling back the layers of onion until you find the real Why. You will often find that this makes the right solution evident, and this will provide your How.

Clearly the companies that innovate are the ones that win. This is especially true in the field of technology. For a company to be innovative requires both innovative people, and a company that encourages and even helps their people to be innovative and creative. This means you as a manager need to hire the best, most productively creative people you can find, and you need to create an environment in which they can be creative. Do this and your company has a decent chance of being a leader; not doing this means that you and your company will always be followers.

As always, I look forward to your comments.

Michael

11 Comments

  1. Tom Becke

    November 3, 2010 at 9:56 am

    I enjoyed your article very much Michael. Your historical and media references are especially noteworthy for me, being somewhat long in the tooth in IT.

    Summing up where I am at I remind myself that “If there is nothing new under the sun, then the freshness may come from the way old ideas are re-packaged. Having the skill to understand how things can be linked together comes from in-depth experience and many trials.”

    That you for the support in this vision that your article brings.

    Tom.

    1. Michael

      November 4, 2010 at 12:54 am

      Hi Tom,

      Thank you for your comments. I’m glad you enjoyed the article.

      Agreed… skill comes from experience, and experience comes from trying things and making mistakes. In retrospect, perhaps the best quote I could’ve used might have been from Miss Frizzle, the teacher on the Magic Schoolbus: “Take chances, make mistakes, get dirty.”

      Michael

  2. Abhinav

    October 27, 2010 at 7:06 am

    Hi Michael,

    A very nice article and I really liked the way you have tried to expressed the aspects and essence of creativity in software development. I especially liked point number 2 and 4 where you suggested that the team must consist of members from different technologies and of different experiences, also the recruiters should not take much consideration over the fact as how much experience a person has over a particular technology before recruiting but rather take care of how much creative and how a person approaches certain things and issues to solve certain problems. This really leads to bringing out creativity from a person.

    Usually in companies once a requirement comes and is assigned to a developer many dont even try to see whether there is a much better easier approach which can make life easier for the user if implemented in a different way and so do not suggest of any possible alternatives. This is a way of small creativity that people can bring up even in normal software development but do not do so and hence usually the quality of softwares is far less superior than what the development tools offer us today.

    The approach of google to bring up beta products and hence allowing to bring up new ideas is another great way of encouraging developers to come up with new brilliant ideas which may or may not work.

    1. Michael

      October 27, 2010 at 9:58 am

      Hi Abhinav,
      Thank you for your comments.
      Yes, sadly we tend to filter out very talented people based on a specious set of criteria. My father used to run a data processing centre and hired many programmers. He used to say “So the candidate has ten years of COBOL experience. Does that mean ten years of experience, or one year of experience ten times?” Ten years of experience with four or five different languages or industries – now that’s experience.
      I agree with your point about companies where once assigned they don’t even caring about whether something can be done better. It is simply “get it done”. This very short sighted approach leads to maintenance issues. You may save a bit up front through expedience, but you lose in the long run because no one has thought through the larger issues and you end up with a large refactoring exercise. Or you may have to rework things because the inspiration for innovation comes from customers telling you what is deficient in your solution. Apple doesn’t wait for customers to tell them what they want. In fact some might argue that Apple barely listens to their customers. Apple, however, is constantly innovating, and bringing surprising products to the market. The same can be said of Google and several other highly innovative companies. It would be hard to argue that creativity has not been a major part of the success of these two companies.

      Michael

  3. Nick Spanos

    October 26, 2010 at 2:06 pm

    Michael, good article. Here are a couple of items to complete your analysis.

    1. Most developers want to be assigned tasks that require creativity. Most will fail at these tasks because they do not know how to end the creative process and finish the task.

    2. Failure to manage the creative process is one of the biggest sources of IT project risk. The construction industry assigns the creative tasks to architects and everyone else’s job is to build the vision. In IT we spread the creative process across the entire life cycle so no one has a vision of the final result.

    1. Michael

      October 27, 2010 at 9:42 am

      Hi Nick,

      Great points!

      1. Agreed. I don’t think anyone’s career aspirations are to become a code drone. There’s not much job satisfaction in that.

      2. Unmanaged anything can become a real problem. Simply allowing developers to wander through the process of creating an application is clearly a bad idea. Encouraging them to think of alternative/better solutions and rec0gnizing them for their contribution is, however, a good thing. As you state, most of the creativity in a live project occurs with the architects, and for the most part, I believe that’s the way it should be – too many cooks spoil the broth. Assuming the architects are the only ones with great ideas, however, limits the organization to the creativity of only a few. Allowing everyone some scheduled time to innovate and create can bring new things to the project, or it can even be the genesis of something entirely new. This creative time is not “project time”; it is time independent of the project itself. Here the potential damage of unbridled creativity is isolated from the project. Only if it is proven outside of the project is it integrated into the project. This limits the potential negative impact of the explorations, and maximizes the possibility of new discoveries.

      Michael

  4. Offshore Outsourcing

    October 25, 2010 at 9:05 pm

    The idea of giving your team an afternoon to get creative is great, and while it’s a good breeding ground for brilliance, a lot of companies don’t believe in it. They only think profit, especially when the team is paid by the hour.

    1. Michael

      October 25, 2010 at 10:17 pm

      Yes, it ts often difficult for companies to accept that they are paying their employees for doing something over which they have no control and may not produce anything of value. It’s one of the reasons so little innovation comes from large organizations. This, however, is good new for the smaller, more innovative companies who get bought up by the larger companies who must acquire to innovate, rather than innovating internally.

      Michael

      1. Zdenek

        October 29, 2010 at 12:12 pm

        Yeah, but the problem is once the small innovative company is bought by large company it’s innovation potential is ‘killed’ as such company acts like a virus spreading its non-inovative processes and culture. True story, I’ve just been experiencing that as a developer.

        1. Michael

          October 29, 2010 at 12:26 pm

          Sad, but so often true. It seems that the larger the organization, the greater is its resistance to the introduction of new thinking. There are exceptions, however, but the innovative spirit seems to remain only when the founders of the original company remain and are allowed to continue leading the acquisition with the same entrepreneurial, innovative spirit.

          Michael

  5. Tweets that mention Software Development and Creativity -- Topsy.com

    October 25, 2010 at 3:05 pm

    […] This post was mentioned on Twitter by Juho Vepsäläinen: Software Development and Creativity: http://bit.ly/aJUnfh – […]

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.