Can you Measure the Success of your Designs or Ideas?

Edit 1: Originally published 08 June, 2006. Edited and updated.
Can you actually quantify something so subjective and intangible as a design or an Idea? Can it then be used as a foundation to tell you if its a good design/idea or not?
As someone who base most of my decisions on a “gut” feel developed through years of experience and critical insight, this quantifying concept really got me thinking…
It must be fate, as I had this topic broached to me twice in a day.
The first was an interview I conducted with a pleasant fellow who was extremely passionate about applying logic and quantifiable methodology in the selection process of designs and concepts. Apparently in very big companies like Nokia, Sony-Ericsson, and Dell, such methodology is common place. Makes sense as it helps justify the subsequent huge manufacturing and marketing investments.
The second time was this article at cph127. The gist of the article was:

How do you measure the effect of design from a business point of view? How do you know that design played a role in achieving business success?

Combining the two instances together we actually cover all aspects of this idea of “measuring” design. The first way is to use these methods to help designers select the right design during the concept phase within a corporate environment. The second method, as highlighted by the cph127 blog, checks to see if the designs have been successful outside of the corporate environment after it is launched.
Either way I think as designers we need to be careful.

Inside the corporate environment
There has not been a day where I have not faced this issue of justifying my designs in some form or another. I’m just intrigue that mankind has never stop finding explanations of the unknown or looked to quantify everything. Just look at the Fibonacci series or the Golden Ratio, not only were these blueprints to nature’s building blocks they also proved to be a basis for artists and designers in the renaissance era to configure their work.
However I digress. My issue with these tools covers the validity of the using of such methods to justify a concept in the first place. Some tools I have looked at include Value Analysis, Pareto Analysis, Six Sigma, and even simple ergonomic usage matrices. Such methodologies should be seen as a supporting tools and not a set of rules. Furthermore it should not be used too early in the concept phase as it could be a huge hindrance in creating a great concept. At the end of the day, a designer’s gut feel backed with critical insight should prevail. Imagine if ergonomic matrices were used at Apple, do you think we would have an iconic but unergonomic product like the Ipod’s click wheel?
Undoubtedly many organizations do use such tools as concept selection criteria during milestone. However there are so many ways you can do something, why use these criteria to limit your choices when it would mean possibly developing the same old boring products over and over again? Case in point, Dell has introduced a HDD Mp3 player to compete with Apple, but they have opted to use a more ergonomic scroll button which has a vertical orientation. Cleverly done, but do you see how many other similarly ergonomic Mp3 players are out there in the market?


I know there is more to this equation, but how many are as successful as the iPod or are still around today?

Outside the corporate environment
Measuring a success of a design in the market place is something very close to my heart. I have a constant struggle to validate my team’s usefulness within an organization. It’s particularly difficult because in organizations, departments do not operate in a billable hour format like a design consultancy.
I constantly argue that a design team cannot be judged on the sum total success or failure of a product in the market! This is mainly because many other factors are involved including sales and marketing strategies, purchasing of components, engineering safety of parts, logistical and supply chain management etc. Designers never design in vacuum and must consider all aspects, but unless the rest of the operation or business report to the design team, we can never be held completely responsible for a products failure in the market.
This is no laughing matter. I have read reports of companies who, not well versed in the value of design, constantly hold their designers responsible for the success of the products in the Marketplace. This is very common in China’s cut-throat mass consumer electronics manufacturing environment.
In other instances, I’ve seen companies ask design consultants to put a clause in their contract that says, should a product not sell well or fail in the Market, the consultancy must pay for any financial lost. How absurd was that?
What these companies don’t realize, is the designer’s role here is only to make a design recommendation. The ultimate responsibility and ownership of the design belongs to the client or decision maker. If a designer is somehow held responsible, then there better be some kind of royalty or profit sharing agreement system set up.
So then what can be used to measure the success of a product in the Marketplace?
Some that I can think of product include winning design awards, or good customer feedback on the aesthetic quality. It makes sense as these are elements designers are directly responsible for. Another way is the ability to command a premium above the selling price that is a result of the cost to apply the aesthetic on that part. Recently I’ve discovered the value of NPS or Net Promoter Scores. Which according to Wikipedia is “…a management tool that can be used to gauge the loyalty of a firm’s customer relationships. It serves as an alternative to traditional customer satisfaction research.”
I really don’t have all the answers as this is still a hotly debated topic, but I like to pass the baton over to you to share your thoughts on how we can better measure the success of our designs and ideas? I look forward to hearing from you!