You’ve probably heard someone say it, or you may even say it yourself:  “You design quality into a product; you don’t test quality into a product.”  The last time when I heard those words uttered, I had to restrain myself from reacting.  What did this person mean by that?   The product was in trouble.  It released late and with low quality.  Customers were rolled back to the former version, and now the product marketing team fears that if things aren’t corrected soon they will lose market share.

Did the person mean you can’t expect QA to find all the defects? Well of course they cannot.  Or perhaps it was meant that you cannot expect QA to certify that the release is ready for customer consumption?   I hope not.  In my opinion the primary purpose of the QA effort in any release is to certify that the release is ready for customers to begin using for their everyday purposes. 

Customers should expect when they purchase a software product for it to perform the functions as advertised flawlessly.  Isn’t that what we expect from our automobiles, our televisions, our iPods, our radios, our phones, our refrigerators, our toasters, and the myriad of products we purchase for our everyday uses?  I’ve seen people nearly lose it when they discover a small scratch on their new vehicle that they missed when they picked it up from the dealer.  While I agree that you design quality into a product, I don’t believe that assertion changes anything about the QA mission.  Let’s take a look at what are some of the sources of defects.

1. Requirements are not defined correctly or are misunderstood.
2. Design has flaws.
3. Code has flaws.

Of the three points where defects can be introduced (requirements, design, and code), the best design would only prevent 33% of the defects from being introduced as the design phase is only one third of the sources of defect introduction, assuming defects are introduced evenly between the sources.  Even the best design team would still introduce some number of defects at the design phase.  Let’s assume for a moment that the team was excellent at reviewing the requirements, design, and code, we would still expect some number of defects to slip through.  Further, let’s assume that the developers were very good at unit testing their code; we could still expect some number of defects to slip through.  Plus, unit testing isn’t the same testing as system test.  The QA team traditionally executes system test.

If we do an excellent job at requirements, design, code, unit testing, and reviews, does that reduce the need for a thorough and comprehensive QA test suite?  I think not.  One, if we can agree perfection is impossible to achieve, there is no telling where the defects — however small in number — will be found in the application, and so you have to test as if there are potential defects anywhere in the application; otherwise, why have the test phase?

The QA role in the lifecycle of the release has many folds, but the function of the QA lifecycle is to make certain that the project has satisfied its objectives, which includes the product being ready for customer consumption.  These objectives are shared by the developers as well, but it’s the responsibility of the QA team to certify that they have been achieved.  Consequently, whether the work products delivered to QA for test are great or poor, it doesn’t change the workload or the expectations of the QA team.  These are the objectives of the QA team:

1. To make certain that all the requirements have been delivered to specification in the release.  The QA team has many customers, but the business team customers rely upon the QA team to certify to them that they are getting what they asked for.
2. To make certain that changes to the product haven’t impaired any of the existing functionality.  Existing functionality works as before the changes were introduced.  This is normally covered in comprehensive regression test suites.
3. To make certain the new and existing functionality meets quality objectives.
4. To make certain the application performs to performance specification.

My expectation is that once the product exits QA after having delivered on these objectives, the QA cycle has confirmed that the software will meet the customer’s high expectations for quality - expectations like we have for everyday products.  Whether quality was designed into the product or not, I don’t see how any of these objectives can rightfully be skipped.  If the software engineers introduced few defects or many defects, it’s the purpose of the QA team to find them and to at least know whether the release decision is prudent or improvident.  Actually, it’s their information that should be the basis for the release decision.   So what is meant by “You design quality into a product” juxtaposed with “You don’t test quality into a product?”  T o me it only says QA cannot fix a poor design.  They also cannot fix poor requirements and poor code, but they better have an opinion on it, and they better have detected poor requirements, design, and code when it’s present. It’s up to someone else to fix it, of course.  What are your thoughts?