As we saw in the last essay, the 80:20 rule seemingly appears in many circumstances.   When I was attending college and working as a programmer during the 80’s, there were some commonly accepted tenets that guided our software development processes and behaviors. 

• 80% of development effort for a product is spent in maintenance, while only 20% is devoted towards delivering the first release.
• It is many times more costly to remedy a defect in the later phases of the SDLC than in the earlier phases.

We didn’t question these beliefs much because they were commonly accepted as true.  Our own experiences supported these beliefs as well.  I’ve spent 80+% of my career maintaining software while the amount of time devoted to delivering a first release has been small.  I bet most of you reading this can say the same thing.  Entirely new product development, where you start from a blank slate, is rare in this business.

80% Maintenance

Think about it: how often do you purchase an entirely new application?  Most of my software purchases are for the newest release of a product that I already own.  Maybe I’m a Troglodyte, but I haven’t purchased an entirely  new application  in many years. I have, though, upgraded to the latest version of many applications within the last year.

In the enterprise space, many projects are about upgrading to the latest version of the Windows OS, Oracle Database, Apache web server, or ERP application, to name a few.  Even in the software product development space, less effort is devoted to rolling out entirely new tools and libraries than effort devoted to upgrading to the latest version of tools and libraries that you already own and use.

Some may say yes, but being first in a space has many advantages.  Does it?  Who remembers VisiCalc?  Turbo Pascal?  Netware?  Netscape?  CPM? There are litanies of firsts that are distant memories.  There’s an advantage to being second in a space.  Google in Search, Microsoft in browsers and office applications, and Dell in Personal Computers are a few examples where being 2^nd, 3^rd, or later to market was not a disadvantage in their eventual success and market domination.

Realizing that 80+% of development effort is devoted to maintenance, the old school developers organized their processes and priorities accordingly.  Being first is rarely the most important objective.  While being first has it’s virtues, dominating your market is a higher priority, and too often the compromises made to succeed at being  first compromises your ability to dominate the market.

The Old School Manifesto

Here’s what the old school programmers and leaders emphasize in their application development efforts:

• Creating a lasting architecture over quick construction of the first build. When done successfully, this saves time, improves quality, and improves time to market not only for the first release but for each successive release.
• Understanding the customer value over doing what the customer tells you. Very often, real customers don’t know exactly what they want nor do they know what’s possible. Customers are sometimes too busy to tell you what it is that they want. It is the development team’s responsibility to understand the need and to use their knowledge and intelligence to create a superior solution. If you’re expecting your customers to tell you everything, you aren’t the team to be building the next thing.
• Emphasizing the up front work over constructing something quickly. Devote 60% of your effort to the upfront work, requirements and design; because when you do it successfully, the last 40% is a no brainer.
• Thinking before acting over acting before thinking. God gave us intelligence for a purpose. Understanding, prioritizing, strategizing, and planning support success.
• Analysis over just-jump-right-in-and-do-something. It always saves time, money, and heartache.
• Believing that if an error happens in the lab once your customers will see it many times, over rebooting the machine and hoping it never happens again. I know; many defects are difficult to analyze and fix. I’ve worked through many nights addressing difficult defects. Just do it. Eventually difficult defects become easy to analyze and fix, but first you must invest to get there.
• Doing superior work over doing just enough. Superior work inspires good people, and motivates not so good people to perform better. I know just enough is fashionable today, but fashionable is just a passing trend.
• Quality over hitting a date. If quality made Japan the top auto manufactures of the world while being 2^nd to market, it must be telling us something. Listen carefully!
• The future over the present. Developing a product is an investment in the future. Investments are rewarding when good decisions about the future are made in the present.
• Change management over change anytime. Change is a fact of life. We get old; we move; things change. Making good decisions about change is necessary for repeatable success. Most changes are not reactions to changing markets but reactions to the leadership’s change in thinking. This too often reflects inadequate analysis.
• Teamwork over individuals. Individuals come and go in an organization but teams remain. Teamwork is the hallmark of a successful product team. Process is a manifestation of good teamwork. Good teamwork is greater than the sum of its parts. It’s through teamwork that superior products are made and delivered. Value the team and the process that supports teamwork.
• Value over commitments. The ultimate goal of every project is to deliver value. Meet your commitments without delivering value, and all you’ll have is a failed project.

Think About It

It’s become fashionable today to live in the present: just look at the debt our society carries from corporations, to individuals, and to governments.   All are carrying unprecedented amounts of debt.  We act and live today like there will be no tomorrow or that tomorrow doesn’t matter.   Would the worldwide financial crisis have materialized if corporate leaders concerned themselves with tomorrow?  When you concern yourself with tomorrow, the present is normally better as tomorrow unfolds into the present.  At the root of old school values was the idea that we are all working to build a better tomorrow?  It’s time that our culture reaffirms that commitment.  Don’t you think?

What would you add or change in the old school manifesto ?

1. If you are following one of the Agile methodologies, are you satisified with your results?  If not, where is it failing you?
2. What do you find to be the best aspect of the methodology that you follow?
3. If you could change one thing about the way your team practices software development, what would you change?

• 80% of the sales are closed by 20%.
• 80% of the code on a project is written by 20% of the staff.  
• 80% of the processor execution time is spent in 20% of the processor’s instructions.
• 80% of your customers use only 20% of your application features. 
• 80% of your defects are in 20% of your code.

Erik Peterson presents on the 80:20 rule at a Google Talk, which you can view on YouTube.com.  He explains how to use the 80:20 rule to improve the quality of your testing with the benefit of delivering high quality software products.

History

The 80:20 rule was first demonstrated by Joseph Juran.  The 80:20 rule says that 80% of the effects come from 20% of the causes.  It’s an indicative trend supported by observation and experimentation. 

The principal was coined by Joseph Juran, a management consultant noted his focus on quality.  He named it after an Italian Economist, Vilfredo Pareto who observed that 80% of the wealth in Italy was owned 20% of the population.

IBM was able to use this insight with great effect when they found that 80% of the execution time was spent in 20% of the instructions.  With this information IBM optimized those 20% instructions giving them a competitive performance edge.  Apparently IBM treated this as a closely guarded secret.

What exactly is it?

Mr. Peterson explains that in Cricket the sweet spot on the bat is where the minimum effort maximizes return.  It’s that small area, the 20% area, on the bat that will catapult the ball the furthest with minimal effort from the batter. It’s what makes professional athletes as they are able to connect the ball with that small area of the bat more often then the amateur.  Identifying the sweet spot, that small area on the bat, is what Pareto Analysis is all about.

Pareto Analysis is the study of the main causes/classifications of something to link them to the properties in the results.  The objective is to establish the relationship between causes and results.

7 step process

A graph is used to perform Pareto Analysis.  There are 7 steps performed in the analysis to identify the most important causes. 

1. Build a table listing the causes and their frequency as a percentage.
2. Arrange the rows in the decreasing order of importance of the causes (i.e, the most important cause first).
3. Add a cumulative percentage column to the table
4. Plat with causes on the x-axis and cumulative percentage on y-axis.
5. Join the above points to form a curve.
6. Plot (on the same graph) a bar graph with causes on the x-axis and percent frequency on the y-axis.
7. Draw line at 80% on y-axis parallel to x-axis. Then drop the line at the point of intersection with the curve on x-axis. This point on the x-axis separates the important causes (on the left) and trivial causes (on the right)

Example Behavior

How many Peanut characters did Charles Schulz introduce into his Peanuts cartoons? A survey was conducted where people where asked to name their favorite Peanut character.  The results of the survey unsurprisingly followed the classic 80:20 rule where 80% of the total favorite votes were for 20% of the characters: Snoopy, Charlie Brown, and Linus.  The answer to the beginning question is that there were 54 named characters in the cartoons. 

Agile and the 80:20 Rule

Conceptually the Agile paradigm is based on the 80:20 where the objective is to deliver the most important features first. If those features are identified well, it is likely that only a small percentage of the universe of features will satisfy 80% of your customers.

I plan to discuss more about the Agile relationship in a future posting.

Testing and the 80:20 Rule

In 1976 Glenford Myers recommended writing test cases before you execute them, and he observed that defects cluster.   Based on this observation, he recommended pausing what you are doing and writing more tests for where you are finding bug clusters.  In addition he found the following:

• 80% of the defects come from 20% of the modules.
• 50% of the modules are defect free at system test.
• 10% of the modules are responsible for 90% of the system downtime.

Implications

• 1) Defects are where you are finding them. If you are finding defects in an area, there are more there.
• 2) Detailed tests are not required for all functionality.
• 3) Sweet spots are where a majority of the defects are. Find the sweet spots, develop more tests in those areas, and test them more.
• 4) 10% of the modules are responsible for 90% of system downtime.

As Beck would recommend, look for code smells.  Bugs are where you smell them.

How to Find Code Smells?

Unfortunately, you won’t find the Pareto relationships until you start testing.  It’s essentially a feedback loop, but there are ways to be proactive:

• Identify features/functionality by risk and develop more tests there.
• Concentrate your testing on new features or complicated areas.

As any QA professional knows, often developers aren’t very helpful with identifying these areas.  Erik recommends looking at the checkout records.  Where there are more checkouts the risk of defects is likely higher. 

Alternatively, focus on the areas the customers use most of the time and make them really usable.  Most defects will go unnoticed by customers where the user experience is superior. 

Prioritize on Feature Use

Test where the users spend most of their time.  Microsoft apparently uses approach to deliver their products, and in my experience with great effect (see Microsoft’s CEO: 80-20 Rule Applies to Bugs, Not Just Features).   I rarely experience a defect in Microsoft’s desktop applications; the operating system is another story, however.

I would sometimes wonder how Microsoft could deliver such high quality in their office products.   In hindsight it’s quite obvious how they achieve it.  I suspect that I’m like most Office users; I use 5% of the functionality 98% of the time. 

If they get that 5% correct, most customers rarely experience a defect.  Interestingly, when I think of the times where I did notice a defect in the Office products, it is when I use a tangential feature like mail merge in Word.

Summary

The 80-20 rule, when applied to testing, can have great effect on your quality and user experience.  In product delivery environments where there is constant pressure to deliver faster, applying the 80-20 rule helps to prioritize your efforts and makes it possible to deliver faster while satisfying a majority of your customers.  While our culture has embraced faster is always better, this is one tool to help identify your priorities with improved product success.

Watch the presentation.  It’s very good.

Similar to hardware fatigue, software exhibits fatigue under continuous usage as well.  For example, a memory leak in an application that runs for a very short time wouldn’t have any consequences.  However, after eight continuous hours of usage, a memory leak could render the application in operable.

Objectives of Soak Testing

Soak testing achieves a number of quality objectives by surfacing errors that would normally go undetected with simple usage.  By achieving these objectives, it is possible to release applications with unmatched quality in your market niche.

Memory

A memory leak of 20 bytes per minute isn’t easily detected after a few minutes.  It’s not easily detected after an hour with only a little over 1k of leakage, but after 24 hours, it’s becoming obvious that memory is allocated without being freed.  

For some applications a 24K memory leak over a 24 hour period may be okay.  However, it’s not just the heap that you may be concerned about.  In early versions of the Windows Operating System, the number of windows, brushes, and fonts that you could create was limited to 64K as the User and GDI data segments were limited to 64K each. 

If your application is leaking window handles and GDI handles, your application could exhaust the available memory to create a new window or font long before the heap is exhausted.  Not only does this adversely impact your application, it also adversely impacts any of the other applications running at the same time. 

As I recollect, exiting the application did not automatically free up these objects, so applications having these errors would eventually degrade performance of the operating system, and a reboot would be required to correct the condition.

While this is no longer a concern for the current Windows operating systems, it may be a problem for other third party libraries that you may be using in your application.

Race Conditions

Soak testing uncovers race conditions.  A race condition is an error in a calculation due to the sequence or timing of execution in multithreaded applications.  Continuous execution of an application raises the probability that the erroneous sequence is repeated many times.  Through repeated occurrences, it becomes possible for the software team to detect the condition.

Deadlocks

Soak testing uncovers deadlocks. In its simplest form, a deadlock is a symptom of a multithreaded application where two threads are waiting on each other to relinquish a resource before either one can continue to execute.  Deadlocks often require the right conditions to exist before they surface.   Through continuous execution over long periods of time, the probability of the deadlock condition being established increases.

Uninitialized Variables

Uninitialized variables are another source of unstable applications.  In particular, an uninitialized pointer can reveal itself in many ways.  If the uninitialized value points within the application data segment, using the pointer will corrupt the values of other global variables resulting in strange application behavior.

Sometimes this may manifest itself as a code exception when an unitialized pointer happens to point to the return address on the stack and the value is changed.  This happens to be a more difficult error to analyze.

Or in other instances, the value points outside the data segment and the application will terminate with a memory exception.  This form of a memory exception is actually the easiest manifestation of this problem to analyze and correct.

Performance Degradation

There are many manifestations of performance degradation.  Continuous execution flushes out many of them.  The most common sources of performance degradation are inefficient memory usage, throughput bottlenecks, and inefficient collection management.

• Inefficient Memory Usage - There are a number of ways this can manifest itself in an application: usually it involves a lot of unnecessary copying of data.  In some applications, the amount of memory copied to perform an operation may grow with repeated execution.
• Throughput bottlenecks - In real-time applications where data is constantly processed, data will back up when data arrives faster than it is processed.  If the application operates for only a short period of time, it can easily go unnoticed.
• Inefficient collection management - There’s nearly always some sort of data management in an application that makes the use of collections, whether it’s an array, a hash table, or linked list.  For some applications, these collections may build to appropriately large numbers of elements.  It’s then that you come to learn that the search algorithms or even the collection is the wrong solution for this problem.

Summary

Soak testing is the only reliable technique that I’m aware of for surfacing the class of application defects formerly described.  The objective of soak testing is not to assess the correctness of the application; it’s not to assess whether feature requirements have been correctly satisfied.  The objective of soak testing is surface defects that will impair application reliability.

Include soak testing as part of your test strategy along with quality release criteria to realize a dramatic improvement in the quality of your product releases.

You cannot control what you cannot measure.
- Tom DeMarco (Software Engineer)

How does the software community improve the control over managing our software development projects?  Ken Schwaber in “Agile Software Development with SCRUM” suggests that the only way to control software development with so much complexity and unpredictability is with an empirical process control model.

Tunde told me the empirical model of process control, on the other hand, expects the unexpected.  It provides and exercises control through frequent inspection and adaptation for processes that are imperfectly defined and generate unpredictable and unrepeatable outputs.  He recommended I study this model and consider its application to the process of building systems.

– Ken Schwaber, Agile Software Development with SCRUM (Series in Agile Software Development)

I happen to agree with Ken’s position on the need for an empirical approach to software process control.  While so much of what is offered by the Agile community is presented as new, the basis of CMM(I) is empirical.  Not only does CMM(I) require empirical control of the software project itself, it requires empirical analysis of the process as well.  Metrics are the basis for all process improvement and project control in the CMM(I) paradigm.

Background

It was during my first experience with CMM(I) process improvement in 1998 when the value of software metrics became apparent to me.  It happened quite accidentally.  I was neutral on the process that we developed based on CMM(I), but I was determined to be a good soldier and give it my support; after all, the company had invested a great deal of money in this initiative.  The least that I could do is to try my best to make it work.  And that’s when the light turned on for me to empirical process control.

CMM(I) identifies two requirements that helped me to arrive at this insight: the basis of estimating is to first estimate the size and then to transform size into duration with a productivity rate; the second requirement is to track completed size as well as completed time. 

Before this, I did not have the tools to accurately and reliably understand how well the project was progressing to schedule.

Why Measure Size?

When you contrast completed size against completed time, problems light up like a Christmas tree.   For example, if completed size on a project is 25% and completed time is 75%, the schedule problem is obvious.  When project management is done well, it’s all about frequent inspection and adaptation, and as Tunde explained to Schwaber, this is the basis for the exercise of control over software projects.  

Tracking with metrics identifies incorrect estimating assumptions early, and consequently, it gives you, the project manager, the control to deliver on the release with no changes in commitments even when the project estimates are incorrect.  That’s the power of an empirical process control model. 

My reservation with Scrum is that the fidelity of the empirical model is low.  You’ve probably experienced it on your software projects. One of the developers on an assignment informs the team that they are done with one of the deliverables.  Two weeks later, the developer continues working on the assignment.  The right metrics will confirm when done is done.

Scrum’s low fidelity is a consequence of only measuring effort remaining.  Measuring both effort and size is analogous to measuring distance to a distant object using the parallax method of measurement.  Measuring the base angles of the triangle is only an interesting observation, but when you add the distance between the base angles, you have the missing information to precisely measure the distance to that distant object.  Similarly, when you contrast completed effort against completed size, you have the necessary information to make accurate inferences about your project’s status with respect to schedule.

In this essay, I’ll explain one aspect of an empirical model that demonstrates the 6-point process described in my previous essay “Software Metrics: An Example Approach.”

Project Conditions 

• You’ve estimated 600 defects to find and fix for a six-month project delivery. 
• The project is 8 weeks away from delivery with 600 defects having been found and with only 200 defects left to fix. 400 defects have already been fixed.
• To reserve time for the unexpected, the manager sets a target to complete fixing the 200 remaining defects in 6 weeks.
• There are 10 developers on the project assigned to defect fixing.
• Project history calibrates average defect fix rate at 1 defect per 1.5 staff days.  Just in case this concerns some readers, this average includes defects that take 10 or more staff days to fix as well as ones that take only hours to fix.  If there are enough defects, we only need to concern ourselves with the mean as the Law of Large numbers tells us that the team will perform at the expected value, which is the mean.
• To keep the example simple for illustrative purposes, the total required defects to fix remains constant during the 6 week period, and developers do not introduce new defects with their fixes, and they fix defects correctly the first time.  While this never happens in practice, introducing these variables into the example unnecessarily complicates the presentation.

Apply the Six Step Process

1. Identify what done looks like with an objective measure.
   1. Inform the team that they have six weeks to fix 200 defects.
2. Break the big goal into smaller goals. 
   1. To achieve the release goal, the team needs to fix 34 defects a week. 
3. Give the team physical productivity goals. 
   1. I like to give stretch goals, so I’ll ask the team to fix 40 defects for the week.
   2. It’s also important to give individual goals because team success is based on individual success.  To fix 40 defects in a week, each developer needs to work towards fixing 4 defects for the week.
   3. Here’s an example of what I’ll say to the team:  “I’d like each developer to target fixing 4 or more defects a week. I realize some of you may have a difficult defect and fix only 1 or even none, and others will have very easy defects and fix 10 or more.  We aren’t being measured on our productivity as individuals, but on our productivity as a team.  If you exceed your individual target but we fail our team target, we failed as a team and individuals.  If we all shoot to exceed our target, the team will easily have achieved the required target of fixing 34 defects a week to deliver on time.”
4. Measure physical progress towards done.
   1. At the end of the week, count how many defects were fixed.  For this example, the team was successful in fixing 38 defects in the past week.
5. Communicate to the team the physical progress towards done and what production is remaining to achieve done.
   1. Last week we were successful in fixing 38 defects.  We are ahead of our goal.  We have 5 weeks left to fix 162 defects. 
6. Repeat starting at step 2 until the overall objective has been achieved.

As is often the case in a real project, additional defects are discovered during this 6 week period, and defects are reopened as they are not fixed correctly.   As a consequence, analysis of the metrics will likely indicate that the original goal cannot be met. In this case at step five, the team needs to make some choices: 

• Defer defects to a future release.
• Add more engineers to the project for defect fixing.
• Extend the release date.

Once the team has decided on a course of action, a new goal is established, and the process repeats beginning at step 1. 

Benefits of the Process

• Everyone on the team knows exactly what is expected of them to achieve success.  Consequently, team members focus more acutely on their deliverables for the week and guard themselves from needless distractions during the day.  People are more productive when they know what’s expected of them.  Having measurable targets enhances communication of expectations.
• Team members are more satisfied in their work.  Having clear and specific goals is motivating to individuals.  Having clear and objective project status is satisfying to the team and to management.
• There’s more collaboration since the only goal that matters is the team goal.  It’s in the interest of team members to help each other to achieve the team goal.
• Status reporting is more accurate, and you can give status with more confidence.  It’s blatantly obvious when productivity is not tracking to schedule.  For example, if only 60 defects are fixed in total after the first two weeks, it’s obvious that team productivity will not complete the work on time unless something changes.
• Once a schedule problem is determined, analysis of the metrics gives you more certainty of what’s required to put the project back on schedule.  Following from the previous bullet, mean productivity for fixing a defect is established at 1.67 staff days.   With that input, the team can accurately decide on extending the target date, deferring defects, adding additional staff, or any combination of the former.

It’s valuable to note when a data trend emerges, it’s rare for it to change - at least not significantly - so don’t expect the mean effort to fix a defect to improve significantly.  Expecting things to change is a sure way to miss the target because when you discover that nothing changed, you will have less time to apply corrective actions.

Nothing Goes to Plan

It is my experience for software projects to always estimate optimistically.  There always seems to be more work than planned, there always seems to be more defects than planned, and there always seems to be more time required to complete assignments than planned.

The strength of an empirical model of process control is when things don’t go as planned.  When more defects are found than planned, analyzing the metrics tells management there is a problem early, the extent of the problem, and what precise options are available to put the project back on schedule.

Summary

To apply the 6-step process successfully, the following data requirements must be met. 

• The size of a deliverable must be estimated and measured.  The more granular the unit of size, the more control you will have over your schedule.  While I have my preference, any unit of size that is measurable works.
• Effort must be calculated from size by dividing it by a productivity rate.

Software metrics are often a controversial subject in the software industry.  I suspect because of the fear metrics will be misused to measure individuals: opposition to software metrics often revolves around the problem of individual productivity.  Software metrics, when used properly, is a tool for measuring the project not individuals; it’s a tool for controlling project deliverables, assessing and communicating status, and making better decisions.

The application had processing throughput of approximately 1,000 400-byte messages per second. Our goal was to double it to approximately 2,000 messages per second.  When he completed the assignment, it was processing over 3,000 messages a second.

Approach

How did he do it?  Well…it didn’t start so smooth.  He dug right in, and that was the problem.  He knew the code base well, and he believed with a hunch, some experimentation, some feedback, and some refactoring that he could improve the throughput and reach the goal.

There was only one thing: it wasn’t working. After a few days, I asked him about his approach and where he believed the bottleneck to be.  He said that he tried a couple of ideas, but he didn’t get the improvements that he expected, and he had a few additional ideas to try.

I asked, “Ravi, why don’t you profile the code?  While I have no doubt that these areas need improving, you might be optimizing code that’s consuming only 5% of the overall processing, and optimizing those areas are not going to lead to the required solution.”  He understood, but he felt confident in his understanding of the code and his approach.

I allowed him to continue.  Ravi was a top gun.  Very few engineers could debug a UNIX stack dump and reverse engineer the Solaris OS like Ravi could.   His designs were solid, and his deliveries were always high quality.   He earned the opportunity to continue with his approach even though I wasn’t supportive of it, but there were limits.

Two weeks later, he was still working on it, and the performance improvement was only marginal.  I said, “Ravi, it’s time to profile the code.”  Reluctantly, he agreed.

The Application

Some technical details of the application would be helpful at this point.

The application was a network performance monitoring application. It performed passive surveillance of the edge network nodes.  It was a multi-tiered, distributed application using CORBA as the middleware technology.  The language was C++, and it was deployed on Sun/Solaris equipment.

Upfront Analysis

It took about a day to prepare the environment and the application for performance monitoring.  After a couple of runs, it was obvious what processing required optimization.

Once the message was received from the edge devices at the data receiver, the application parsed and transformed the network message into an application message object and forwarded it to the upstream processes.  The performance bottleneck was with the marshalling of this object.  As the message object moved from each distributed process, this expensive marshalling was repeated multiple times, and it consumed a significant percentage of the overall processing of the message.

Solution

The solution was quite simple: reformat the message into a byte array: essentially, “char Message[500];” was the only member of the message object.   Prior to the change, each field in the message had its own member variable.  The network message was reformatted  into the byte array to avoid the overhead of re-parsing the message as it made its way through the processing path, which was the original intent of the message object.

At the beginning of the byte array a field identifier, offset pair was inserted for each field in the message with the offset being an index from the beginning of the byte array to the location of the field data.  With a lookup and a simple cast, the data was easily extracted.

The overhead for marshalling the byte array was significantly less with CORBA.   With this approach, the performance more than tripled to processing over 3000 messages per second.

Finally, the duration required to profile, to analyze, to design, to code and to release the change into the next build took about 3 to 5 days. Had Ravi started with upfront analysis, he would have saved 2 weeks of effort, and upfront analysis arrived at a solution faster than the approach with no formal analysis.  None of the prior work provided any insight to this bottleneck and solution.  In fact, it was a surprise.

Analysis is Hard

Some of you may be saying well, this is quite obvious, but is it?  I’ve had similar scenarios repeated with other developers throughout the years.  Developers want to write code, but writing code is really the easy part of the programmer’s job. 

The hard part is creating the model for the solution, identifying the objects, and their methods.  It’s the analysis and the design that’s the difficult part of software development. 

Sure, not all problems are difficult to model, but the modeling is more difficult than the coding.  Any programmer can transform a good design into working code, but any programmer cannot transform a problem/requirement into a good model.

For example, making the code changes to fix a defect is the easy part.  It takes days - sometimes even months - of analysis to understand the root cause of a difficult defect.  Once the root cause is identified, the solution often only requires changes to a couple of lines of code or sometimes to add, subtract, or change a single character. 

Writing code is easy.  It’s the understanding that’s the hard part;  it’s the transformation of a problem into a model that’s the difficult part. The code is simply the realization of that model and understanding.

Simplicity

One of the twelve Principles behind the Agile Manifesto is “Simplicity - the art of maximizing the amount of work not done - is essential.”  Many interpret this to mean skipping documentation and/or skipping upfront analysis and design - essentially skipping any manifestations of formalism.  

“The art of maximizing the amount of work not done” is best achieved by working smarter.  Achieving simplicity requires an investment in upfront analysis, and like all investments, they are rewarding when invested well.  

Having poor results with upfront analysis is not anymore evidence that upfront analysis should skipped than having poor code is evidence that coding should be skipped.  Analysis is essential to writing code.  Poor results with upfront analysis is simply evidence that you need to do analysis better.

Summary

Work smarter, not harder is common wisdom for solving difficult problems.  Analysis is the process of intelligently applying our knowledge to solve difficult problems. Upfront analysis saves time.  It’s about working smarter, not harder.

In the case of the opening example, the savings from upfront analysis is 66% of the duration and effort.  To look at it another way, the required duration and effort without upfront analysis is indeterminate. For all I know, if Ravi had continued with his original approach, he may have continued to work on it for another two weeks or even more.

Skip upfront analysis if you must, but as Glen B. Alleman, author of Herding Cats, is fond of saying, “Don’t do stupid things on purpose.“

As a kid, I recall watching the Jerry Lewis Telethons.  To my surprise, I understand that the Telethon continues to this day, but I haven’t watched one since I was a kid. The Telethons are always successful in reaching their goal, yet they only have a fixed amount of time to achieve it: 24 hours, if I recall correctly. 

The Telethon’s success in achieving its seemingly unreachable goal left an impression on me:  I guess because our family was always cheering for Jerry to reach his goal towards this worthy cause, and he always managed to exceed it within the waning minutes of the event.  There’s nothing like wresting victory from the jaws of defeat to leave an impression on someone.

Jerry’s challenge with raising the target donations is similar to the challenges we face in estimating and delivering software projects on time.   He has no way to precisely estimate how much money he can raise in the 24-hour event, yet he has a target, and he always exceeds it.   For all we know, he may be given the target amount to raise, yet he is still successful in achieving the goal.  Similarly on software projects, teams are asked to deliver a fixed set of requirements on a fixed date, but unlike Jerry, we are rarely successful in delivering everything on time.   

How Did Jerry Do It?

Whether Jerry was given a target or estimated the target, he was successful in achieving the goal.  How did he do it?  He did it by rigorous use of metrics.  While I haven’t read anywhere about the technique that Jerry used, I’m certain from repeatedly watching the event that the technique I’m about to describe was instrumental in achieving and exceeding the goal.

• He clearly defined what done looked like.  It was to raise X amount of money in 24 hours.   How many software projects clearly and objectively define what done looks like?
• He broke the larger goal into many smaller goals, repeatedly communicated the amount of money he needed to raise to achieve the smaller goal, and worked to secure the goal.  How many software projects similarly break the project into many smaller goals, regularly communicate measurable physical productivity required to deliver the smaller goal, and have the team work towards delivering the physical goal?
• He repeatedly asked the donors for exact amounts of money.  He’d often say something like, “I need 10 viewers to call in with $100 donations each in the next 5 minutes.”  How many software projects request measurable productivity goals?
• He constantly measured how much he was accumulating towards his goal.  How many software projects closely track the progress of their projects by measuring physical percent complete?
• He regularly communicated to his viewers the progress he was making towards his goal.  How many projects regularly communicate their status using metrics to the team?  Often status is never fed back to the team members; instead, status is often only delivered to upper management.  Status on software projects is rarely delivered using metrics.

Summary

The process is simple:

1.     Identify what done looks like with an objective measure.

2.     Break the big goal into many smaller goals.

3.     Give the team physical productivity goals.

4.     Measure the physical progress towards done.

5.     Communicate to the team the physical progress towards done and what production is remaining to achieve done.

6.     Repeat beginning at step 2 until the goal is achieved.

Communication is paramount, and the communication is clearer with the use of metrics.  With more precise communication, aided by metrics, goals are more easily achieved and even exceeded.

Some of you are likely saying, “software is different.”  But is it really different?  All software development projects produce lines of code, all projects find defects, and all projects fix defects.  These variables are all easily estimable and measurable.  I’ve seen and managed teams who have used this process to deliver on time, with high quality, and exceed customer expectations, and they do it with less stress, less overtime, and more team and individual satisfaction.  To realize the benefits of software metrics, first the software community has to free themselves from their irrational aversion to metrics — especially lines of code.  

I enjoy playing FreeCell.  It’s a form of stress release for me, sort of like smoking is to some.  Whenever I need a quick break or I’m looking to take my mind off of something, I’ll often play a game of FreeCell.  

FreeCell is a puzzle, requiring analysis and strategy to win.  They say every game is winnable, and I believe that it’s likely to be true.   Whenever I lose a game and I’m determined to win, I eventually win after repeated retries of the same game.

Recently, when playing FreeCell to take my mind off an essay I was writing, a thought occurred to me: FreeCell illustrates the value of upfront analysis and design.  On a number of essays on yuwantitwhen.com, I’ve advocated that doing proper upfront analysis and design on our software projects is the best way to secure success for our projects and to shorten schedules.  Similarly, in FreeCell I’ve won more games on the first try when I devoted more time to upfront analysis and developing a strategy.

How to Play FreeCell 

For readers unfamiliar with the game, the objective is to discard all the cards in order starting with Ace to King for all suits.  The game is setup by distributing all the cards face up along eight columns.  Play begins by moving cards to the other columns, the temporary pile, or to one of the four discard piles.  To move a card to another column, the card being moved must be the next sequential card in descending order and must have the opposite color: for example, red must follow black and visa versa, so an 8 of diamonds (red) can only be placed on a column where the top card on the column is a 9 of spades (black) or 9 of clubs (black).   Only 4 cards can move to the temporary pile; any card can move there.  Cards can be removed from the temporary pile to any of the columns or to the discard piles.

The strategy is to move cards to other columns or the temporary pile to build the 4 discard piles in order beginning with Ace.  The game is won when you have moved all the cards to the discard piles, and the game is lost when you cannot move any cards to the temporary pile, to any column, or to any of the four discard piles.

Approaches to Playing FreeCell

There are two approaches to playing the game: one, evaluate only the moves that are currently available to you by only evaluating the top cards of each column or the cards in the temporary pile. Let’s refer to this approach as shallow analysis, or two, evaluate successive moves before deciding on a move.  Let’s refer to this approach as deep analysis.  While it is possible to win with shallow analysis throughout the game, the probability of winning is low.  I’ll often start a game with the first approach then after a few moves switch to the second approach to make the game more challenging.  I can’t recall ever winning a game without doing some amount of deep analysis.   Early deep analysis improves the odds of winning on the first attempt of a game.

What FreeCell Teaches Us about Software Projects

Here’s what the game of FreeCell teaches us about upfront analysis and design.

1. Reworking (refactoring) is expensive and it lengthens schedules.  When I play using shallow analysis, it takes many more restarts of the game before I finally win it.  To win a game with less restarts - even zero restarts - the likelihood increases significantly with quality deep analysis before moving.  Similarly in software, schedules are shorter when the solution is well analyzed and designed before writing code, resulting in less rework or reimplementation.
2. Anticipating the future results in less rework and reimplementation.  In FreeCell, evaluating only the first level moves before moving makes it more probable that you leave yourself with no possible moves later in the game.  A player can avoid being locked in a corner with deep analysis before moving.  Similarly a software team can avoid rework and reimplementation by anticipating future requirements or preferably by creating a design that is easily extensible.
3. Winning every game of FreeCell on the first attempt is possible with deep analysis if it is true that every playing combination is winnable.  The most difficult upfront investment is on the first move.  It is the most difficult to analyze and to get correct, but if you desire the highest probability of winning — even certainty of winning — there is no shortcut to this upfront investment.   Similarly, in software the first release is the most difficult release to analyze.  It’s a blank sheet of paper.  The possibilities are seemingly infinite.  It’s not difficult because it’s a wrong approach; it’s difficult because it’s hard.  As in FreeCell, a project team improves their likelihood of success with enough upfront analysis and design.
4. Starting a game with deep analysis results in faster analysis later. As a consequence of analyzing a game well, it becomes easier to make successive moves as the game evolves.  Similarly in software when an application is well designed, successive releases of that application become easier and faster.

Summary

Modern approaches to software development advocate an evolving approach to software construction with shallow analysis as its principle.  They trade deep analysis in favor of faster delivery for today’s requirements.  While that approach may succeed in creating a quick first release (only if shallow analysis does not lock you in a corner in the first iteration), it is sure to require rework and reimplementation on future releases.

As in the game of FreeCell, upfront analysis and design improves the likelihood of delivering a successful software release.   Deep upfront analysis and design avoids rework and reimplementation, and consequently, it shortens project schedules.   If you are looking to improve the project team’s chance of success, shorten schedules, and control the budget, deep upfront analysis and design as demonstrated in the FreeCell analogy is a method for securing those objectives.

How much upfront analysis is enough you may ask?  It depends.  How important is it to hit your release date?  How important is it to deliver on budget?  How important is it to secure future releases?   To the degree that these objectives are important to your organization, you invest in enough upfront analysis and design to give yourself confidence commensurate with the importance of hitting your date, meeting your budget, and securing future releases. 

This investment is not measured in time; it’s measured in the quality of analysis and design.   If the investment is too expensive, consider the alternative:  you are more likely to be late, over budget, and hamper future deliveries if you don’t invest.  Of course, not every project requires high certainty of hitting the date and the budget, and some products never have more than the first release.  For those projects, other approaches may be better.  But, know the tradeoff before you embark on your journey.

Mary Poppendieck speaks at a Google TechTalk on the role of leadership in software development. She presents an interesting history on the philosophy of management and how it has morphed over the years to its present day practices and theory.   Mary Poppendieck is the author of Lean Software Development: An Agile Toolkit (The Agile Software Development Series) and Implementing Lean Software Development: From Concept to Cash (The Addison-Wesley Signature Series).  She describes her Lean manufacturing experiences at 3M and how they relate to Agile Software development.