You cannot control what you cannot measure.
- Tom DeMarco (Software Engineer)

How does the software community improve the control over managing our software development projects?  Ken Schwaber in “Agile Software Development with SCRUM” suggests that the only way to control software development with so much complexity and unpredictability is with an empirical process control model.

Tunde told me the empirical model of process control, on the other hand, expects the unexpected.  It provides and exercises control through frequent inspection and adaptation for processes that are imperfectly defined and generate unpredictable and unrepeatable outputs.  He recommended I study this model and consider its application to the process of building systems.

– Ken Schwaber, Agile Software Development with SCRUM (Series in Agile Software Development)

I happen to agree with Ken’s position on the need for an empirical approach to software process control.  While so much of what is offered by the Agile community is presented as new, the basis of CMM(I) is empirical.  Not only does CMM(I) require empirical control of the software project itself, it requires empirical analysis of the process as well.  Metrics are the basis for all process improvement and project control in the CMM(I) paradigm.

Background

It was during my first experience with CMM(I) process improvement in 1998 when the value of software metrics became apparent to me.  It happened quite accidentally.  I was neutral on the process that we developed based on CMM(I), but I was determined to be a good soldier and give it my support; after all, the company had invested a great deal of money in this initiative.  The least that I could do is to try my best to make it work.  And that’s when the light turned on for me to empirical process control.

CMM(I) identifies two requirements that helped me to arrive at this insight: the basis of estimating is to first estimate the size and then to transform size into duration with a productivity rate; the second requirement is to track completed size as well as completed time. 

Before this, I did not have the tools to accurately and reliably understand how well the project was progressing to schedule.

Why Measure Size?

When you contrast completed size against completed time, problems light up like a Christmas tree.   For example, if completed size on a project is 25% and completed time is 75%, the schedule problem is obvious.  When project management is done well, it’s all about frequent inspection and adaptation, and as Tunde explained to Schwaber, this is the basis for the exercise of control over software projects.  

Tracking with metrics identifies incorrect estimating assumptions early, and consequently, it gives you, the project manager, the control to deliver on the release with no changes in commitments even when the project estimates are incorrect.  That’s the power of an empirical process control model. 

My reservation with Scrum is that the fidelity of the empirical model is low.  You’ve probably experienced it on your software projects. One of the developers on an assignment informs the team that they are done with one of the deliverables.  Two weeks later, the developer continues working on the assignment.  The right metrics will confirm when done is done.

Scrum’s low fidelity is a consequence of only measuring effort remaining.  Measuring both effort and size is analogous to measuring distance to a distant object using the parallax method of measurement.  Measuring the base angles of the triangle is only an interesting observation, but when you add the distance between the base angles, you have the missing information to precisely measure the distance to that distant object.  Similarly, when you contrast completed effort against completed size, you have the necessary information to make accurate inferences about your project’s status with respect to schedule.

In this essay, I’ll explain one aspect of an empirical model that demonstrates the 6-point process described in my previous essay “Software Metrics: An Example Approach.”

Project Conditions 

• You’ve estimated 600 defects to find and fix for a six-month project delivery. 
• The project is 8 weeks away from delivery with 600 defects having been found and with only 200 defects left to fix. 400 defects have already been fixed.
• To reserve time for the unexpected, the manager sets a target to complete fixing the 200 remaining defects in 6 weeks.
• There are 10 developers on the project assigned to defect fixing.
• Project history calibrates average defect fix rate at 1 defect per 1.5 staff days.  Just in case this concerns some readers, this average includes defects that take 10 or more staff days to fix as well as ones that take only hours to fix.  If there are enough defects, we only need to concern ourselves with the mean as the Law of Large numbers tells us that the team will perform at the expected value, which is the mean.
• To keep the example simple for illustrative purposes, the total required defects to fix remains constant during the 6 week period, and developers do not introduce new defects with their fixes, and they fix defects correctly the first time.  While this never happens in practice, introducing these variables into the example unnecessarily complicates the presentation.

Apply the Six Step Process

1. Identify what done looks like with an objective measure.
   1. Inform the team that they have six weeks to fix 200 defects.
2. Break the big goal into smaller goals. 
   1. To achieve the release goal, the team needs to fix 34 defects a week. 
3. Give the team physical productivity goals. 
   1. I like to give stretch goals, so I’ll ask the team to fix 40 defects for the week.
   2. It’s also important to give individual goals because team success is based on individual success.  To fix 40 defects in a week, each developer needs to work towards fixing 4 defects for the week.
   3. Here’s an example of what I’ll say to the team:  “I’d like each developer to target fixing 4 or more defects a week. I realize some of you may have a difficult defect and fix only 1 or even none, and others will have very easy defects and fix 10 or more.  We aren’t being measured on our productivity as individuals, but on our productivity as a team.  If you exceed your individual target but we fail our team target, we failed as a team and individuals.  If we all shoot to exceed our target, the team will easily have achieved the required target of fixing 34 defects a week to deliver on time.”
4. Measure physical progress towards done.
   1. At the end of the week, count how many defects were fixed.  For this example, the team was successful in fixing 38 defects in the past week.
5. Communicate to the team the physical progress towards done and what production is remaining to achieve done.
   1. Last week we were successful in fixing 38 defects.  We are ahead of our goal.  We have 5 weeks left to fix 162 defects. 
6. Repeat starting at step 2 until the overall objective has been achieved.

As is often the case in a real project, additional defects are discovered during this 6 week period, and defects are reopened as they are not fixed correctly.   As a consequence, analysis of the metrics will likely indicate that the original goal cannot be met. In this case at step five, the team needs to make some choices: 

• Defer defects to a future release.
• Add more engineers to the project for defect fixing.
• Extend the release date.

Once the team has decided on a course of action, a new goal is established, and the process repeats beginning at step 1. 

Benefits of the Process

• Everyone on the team knows exactly what is expected of them to achieve success.  Consequently, team members focus more acutely on their deliverables for the week and guard themselves from needless distractions during the day.  People are more productive when they know what’s expected of them.  Having measurable targets enhances communication of expectations.
• Team members are more satisfied in their work.  Having clear and specific goals is motivating to individuals.  Having clear and objective project status is satisfying to the team and to management.
• There’s more collaboration since the only goal that matters is the team goal.  It’s in the interest of team members to help each other to achieve the team goal.
• Status reporting is more accurate, and you can give status with more confidence.  It’s blatantly obvious when productivity is not tracking to schedule.  For example, if only 60 defects are fixed in total after the first two weeks, it’s obvious that team productivity will not complete the work on time unless something changes.
• Once a schedule problem is determined, analysis of the metrics gives you more certainty of what’s required to put the project back on schedule.  Following from the previous bullet, mean productivity for fixing a defect is established at 1.67 staff days.   With that input, the team can accurately decide on extending the target date, deferring defects, adding additional staff, or any combination of the former.

It’s valuable to note when a data trend emerges, it’s rare for it to change – at least not significantly – so don’t expect the mean effort to fix a defect to improve significantly.  Expecting things to change is a sure way to miss the target because when you discover that nothing changed, you will have less time to apply corrective actions.

Nothing Goes to Plan

It is my experience for software projects to always estimate optimistically.  There always seems to be more work than planned, there always seems to be more defects than planned, and there always seems to be more time required to complete assignments than planned.

The strength of an empirical model of process control is when things don’t go as planned.  When more defects are found than planned, analyzing the metrics tells management there is a problem early, the extent of the problem, and what precise options are available to put the project back on schedule.

Summary

To apply the 6-step process successfully, the following data requirements must be met. 

• The size of a deliverable must be estimated and measured.  The more granular the unit of size, the more control you will have over your schedule.  While I have my preference, any unit of size that is measurable works.
• Effort must be calculated from size by dividing it by a productivity rate.

Software metrics are often a controversial subject in the software industry.  I suspect because of the fear metrics will be misused to measure individuals: opposition to software metrics often revolves around the problem of individual productivity.  Software metrics, when used properly, is a tool for measuring the project not individuals; it’s a tool for controlling project deliverables, assessing and communicating status, and making better decisions.

As a kid, I recall watching the Jerry Lewis Telethons.  To my surprise, I understand that the Telethon continues to this day, but I haven’t watched one since I was a kid. The Telethons are always successful in reaching their goal, yet they only have a fixed amount of time to achieve it: 24 hours, if I recall correctly. 

The Telethon’s success in achieving its seemingly unreachable goal left an impression on me:  I guess because our family was always cheering for Jerry to reach his goal towards this worthy cause, and he always managed to exceed it within the waning minutes of the event.  There’s nothing like wresting victory from the jaws of defeat to leave an impression on someone.

Jerry’s challenge with raising the target donations is similar to the challenges we face in estimating and delivering software projects on time.   He has no way to precisely estimate how much money he can raise in the 24-hour event, yet he has a target, and he always exceeds it.   For all we know, he may be given the target amount to raise, yet he is still successful in achieving the goal.  Similarly on software projects, teams are asked to deliver a fixed set of requirements on a fixed date, but unlike Jerry, we are rarely successful in delivering everything on time.   

How Did Jerry Do It?

Whether Jerry was given a target or estimated the target, he was successful in achieving the goal.  How did he do it?  He did it by rigorous use of metrics.  While I haven’t read anywhere about the technique that Jerry used, I’m certain from repeatedly watching the event that the technique I’m about to describe was instrumental in achieving and exceeding the goal.

• He clearly defined what done looked like.  It was to raise X amount of money in 24 hours.   How many software projects clearly and objectively define what done looks like?
• He broke the larger goal into many smaller goals, repeatedly communicated the amount of money he needed to raise to achieve the smaller goal, and worked to secure the goal.  How many software projects similarly break the project into many smaller goals, regularly communicate measurable physical productivity required to deliver the smaller goal, and have the team work towards delivering the physical goal?
• He repeatedly asked the donors for exact amounts of money.  He’d often say something like, “I need 10 viewers to call in with $100 donations each in the next 5 minutes.”  How many software projects request measurable productivity goals?
• He constantly measured how much he was accumulating towards his goal.  How many software projects closely track the progress of their projects by measuring physical percent complete?
• He regularly communicated to his viewers the progress he was making towards his goal.  How many projects regularly communicate their status using metrics to the team?  Often status is never fed back to the team members; instead, status is often only delivered to upper management.  Status on software projects is rarely delivered using metrics.

Summary

The process is simple:

1.     Identify what done looks like with an objective measure.

2.     Break the big goal into many smaller goals.

3.     Give the team physical productivity goals.

4.     Measure the physical progress towards done.

5.     Communicate to the team the physical progress towards done and what production is remaining to achieve done.

6.     Repeat beginning at step 2 until the goal is achieved.

Communication is paramount, and the communication is clearer with the use of metrics.  With more precise communication, aided by metrics, goals are more easily achieved and even exceeded.

Some of you are likely saying, “software is different.”  But is it really different?  All software development projects produce lines of code, all projects find defects, and all projects fix defects.  These variables are all easily estimable and measurable.  I’ve seen and managed teams who have used this process to deliver on time, with high quality, and exceed customer expectations, and they do it with less stress, less overtime, and more team and individual satisfaction.  To realize the benefits of software metrics, first the software community has to free themselves from their irrational aversion to metrics — especially lines of code.  

When I was a child, I had a strong aversion to pineapples.  Just the look of them made me ill.  It didn’t matter whether it was cut or uncut; there was just something about the look that made me believe I would not like the taste.  Maybe it was the color yellow, but there was something terribly unappealing about the fruit, and no matter how much my mom would entice me with declarations of how sweet it taste, I would not try it.

I felt the same way about cranberry sauce too.  It was an emotional aversion; there was nothing logical about it even though I was convinced my reasons were all logical.  Cranberry sauce looks slimy; slimy is disgusting; therefore, it must taste like it looks: disgusting.  It’s a logical inference; though, it has little relevance to how cranberry sauce actually tastes.

As I got older, I became more open to giving foods a try (and other things, of course) that were unappealing to me.  When I finally gave pineapples and cranberry sauce a try, I discovered how I’d been missing out for so long on enjoying a food that was so pleasurable to me.

Much of the software community has a similar aversion to LOC. Many of their arguments against LOC are logical, but they aren’t relevant to the science and practice of LOC as advocated by its adherents.  Sure one can write a line of code with more defects than ten lines of code, but the Law of Big Numbers says the density observed in practice will be the expected value.

Scenarios to Consider

Let’s put aside defect densities and individual productivity rates.  Instead, I’d like for you to consider some hypothetical but possible relationships to evaluate whether LOC would be a valuable piece of information when managing your project.  For each of the scenarios described, all references to lines of code are for lines of code that the team writes themselves: they are not from a library, a wizard, or copied from another source.  Also assume, the team has integrity, and they aren’t looking to sabotage the measurements.

1. If you are in the middle of your software development cycle where you are releasing daily builds to the test labs and there is still significant functionality to complete, would it be interesting and useful to know that the size in LOC is flat lining or growing insignificantly for the past 3 weeks?
2. If you are managing a project and the growth of the code base is growing at an averaging of 100 LOC/week with 20 software developers writing code for this project, would you feel comfortable that the team is producing well at 1 LOC per day per developer of  “C” code?  If this team is off shore, would it be even more interesting to know?
3. If you are interviewing for a job and the team was supporting applications totaling  50 million LOC of custom “C” code with 10 developers, would that tell you something interesting about their capacity to effectively support their customer base?  Would it tell you something interesting about your learning curve?
4. If you manage a sustaining engineering team and the size of the applications that you are supporting is growing by 1 million LOC per year, would that tell you something about your staffing requirements to support your customers effectively?
5. If you are managing the release of a product, would you feel comfortable about the quality if your code base grew by 20,000 new LOC in the week that you plan to release?
6. If you are managing a release of a product to be delivered in five days, would you feel comfortable about the quality that the software will release with if your code size was flat lined for the last 3 weeks, no defects were uncovered in the last 3 weeks, and all the functionality was verified to be completed?
7. If you are comparing two completed applications with application A having a size of 50,000 LOC and application B having a size of 500,000, would you expect the number of engineers working on team B to be larger than the number working on team A?  What if application B is 2 million LOC in size?
8. If you are making decisions on source code to review because you don’t have time to review everything, would you choose to review the procedures with more lines a code over the procedures with less lines of code if that is the only input available to you?
9. If you are comparing two applications with application A having a size of 20,000 LOC and application B having a size of 200,000 LOC, would you expect the complexity of application B to be more complex than application A?
10. If you estimated that you would grow your code base by 50,000 LOC (assuming you had a way to estimate this accurately) for a release and you are 20 weeks into a 24 week duration schedule, would you feel comfortable that you are on schedule if the code base only grew by 10,000 LOC?  Would it tell you something interesting about the schedule if you had measured 60,000 LOC at week 20?

Insight Gained

I’m sure it’s unapparent to some, maybe even to many, that knowing something about the size of an application has value. LOC gives insight to the following aspects of the product:

• The complexity of the application – Size is one measure of software complexity. Software complexity increases as size increases. One person can easily understand the implementation details of a 10,000 LOC application. One person would be challenged to understand the implementation details of a 2 million LOC application. At a minimum, it would require more time to learn.
• The pace of development – All completed software projects deliver LOC. After all, requirements are finally manifested in completed LOC. If zero LOC is being produced on a weekly basis then zero requirements are being implemented. If ten LOC is being produced on a weekly basis then progress is being made towards the completion of the project. Further if 100 LOC is being produced on a weekly basis then more progress is being made towards the completion of the project.
• The support requirements – The number of features in an application correlates with the number of LOC in the application. The number of defects in an application also correlates well with the number of LOC in application. Every line of code written is one more opportunity for a defect to be introduced. It requires a larger support team to support more features and more defects.
• The staffing requirements – Larger software deliveries require larger teams to deliver them timely.
• The quality of the application – As noted earlier, every line of code represents one more opportunity to introduce a defect. More lines of code have more defects. The Law of Big Numbers tells us that defect densities into test will be found having a density of the expected value.
• The likelihood the team will deliver to schedule – If there is more quantifiable work to deliver than there is time left in the schedule to deliver it, the chances that you can deliver to schedule is unlikely unless something changes. Now some will naively believe that the initial output estimate needs to be accurate to provide value, but that would be incorrect. Of course, accuracy is better, but project tracking is about testing the estimating assumptions and taking corrective action when the estimating assumptions are wrong.
• The readiness for release - High quality releases are made when the code size stabilizes a number of weeks before the release date. If one of your objectives is to release with high quality, you will want to observe that the growth in LOC has attenuated and flattened for a number of weeks before the release date. It takes time to find and fix defects. New code always introduces defects into the application.

Summary 

Having insight to the measure of LOC permits the manager and the team to ask good questions, questions that you wouldn’t know to ask without the information.  You can ask questions like, how can we expect to release this product on Friday when we just added 20,000 LOC in the past five days? Where did they come from?  Why are they there?  Are they needed?  Something similar to this happened to me once on a project.  There was a good reason for it, although unexpected, and finding this out when I did allowed corrective actions to be taken and still deliver to schedule.

LOC isn’t a perfect measure, but it is a key measure as software development is essentially the process of producing LOC to achieve an objective, but it’s not the only software measure to use.  For some relationships, I’ve found the measure to be extremely precise, and for others less precise, but nevertheless still valuable.  It’s the starting point for asking good questions and drawing good inferences about quality and progress to schedule.  

It’s likely that most, if not all of you reading this, are still skeptical of the benefits with measuring LOC.   I was skeptical too when I first gave this a try by putting this into practice for a CMM process improvement initiative at one of my employers.  The unexpected happened when I began measuring.  Reliable patterns began to emerge.  Patterns that helped me gain insight to the pace of development, the level of quality, and the status of the schedule.  If more practitioners in the industry begin to use this measure, I believe we can come to understand the patterns and relationships better and develop even better techniques for measuring our projects, but this promise can only happen if we start somewhere.  From my experience I believe that measuring LOC is the best place to start.

During a presentation on software metrics, I asked the audience to estimate the distance from New York, NY to Sante Fe, New Mexico in Kilometers (they were asked not to participate if they already knew the answer), and I asked them not to do any conversions in their head before giving their answer.  For those of you reading in other countries, we use miles to measure long distances in the USA, so kilometers is a unit that we don’t have much experience using.   The answers to the question were all over the place with most of them being overestimates of signification magnitude, double or more.  Then I asked the audience to estimate the distance again, but this time, they were to estimate the distance in miles – a unit they were all very familiar with.  These estimates were much better with all being less than 20% away from the actual distance.   Some were so close to the actual distance that they were essentially accurate.

Why were the estimates in miles significantly better than the estimates in Kilometers?  The reason is that the attendees all had significant experience in the unit of measure.   From the time they were little kids, they’ve been measuring distance in miles when they travel.  When they were in school, they used maps with a legend in miles.  When they drive to distant locations that they’ve never been to before, they use roadmaps with a legend in miles.  The odometers in their cars are denominated in miles.  Over the years they’ve built an historical database of locations, distance, and time that they can use to infer good approximations from incomplete information, like what’s the distance from New York, NY to Sante Fe, New Mexico.  This is possible when you have a familiar unit of measure and when you have knowledge of the distance to other locations that you can use to infer the distance to the unknown destination.

This example demonstrates how units and measures begin to have more meaning when we are familiar with them and use them regularly.  Most here in the US would be confused if they hopped in a car with their friend, and their friend said, “We’re going to take a 150 kilometer drive to my college friend’s house for a short visit.”  We would be clueless about how long that would take.  We’d be wondering if this would be a trip requiring an overnight stay.  Should we cancel evening plans?  It’d be disorienting to us.

Benefits

Metrics can help in the following ways:

• They give us a better, more accurate way to estimate our projects.  When the project completes, the actual values can be used to improve future estimates.
• They connote meaning when we all use the same unit for size: how many people are required to staff the project? How many defects can I expect to find? How long will it take?
• They improve monitoring and control of the schedule when actual values are compared to the estimating assumptions.  Consequently, status is more objectively reported.
• They give us objective criteria for making release decisions based on quality.  They tell us when done is done.

Summary

These benefits are all possible if we take a disciplined approach to using software size metrics.  The challenge to being successful with metrics is the ability to stick with a disciplined approach.  Too often discipline is a casualty of intense pressure to hit the date on software projects, but if you give this an earnest try, you may find it easy to be disciplined as the benefits are tangible, powerful, and immediate. 

In the next installment in this series I plan to develop the benefits more through example.

What if you could be more accurate in your software estimates?  What if you could gain more control over your project schedule? What if you could improve your product quality and know that you did before your customer received the product?  Okay, now that I have you’re attention, if I told you that metrics was the key to realizing these goals, will you continue reading?  If a software team is ever going to improve successfully and repeatedly in these areas, they will need to embrace metrics.

Measurement of appropriate variables is an integral tool for many serious disciplines.  For many disciplines metrics are the difference between success and failure; they are the difference between mediocrity and eminence.  Metrics are essentially the difference between the practice of art and science.

In the software profession metrics have a long history of controversy.  Some argue that software is an art and can never be measured.  While others argue software is an engineering discipline, and a failure to measure is malpractice.  To me, all science is art.  Without creativity and imagination progress in science is not possible, and science is also a rigorous disciple: for without measures and mathematical models, the practice of science has not matured beyond art.

Unfortunately, the software profession continues to debate whether measurement is even appropriate.  Is it applicable? Is it useful? Is it possible? These are questions that the profession continues to debate, and sadly, often those who have great successes in using software metrics are derided as weird by those who often have little in the way of tangible accomplishments to their credits.

For some metrics are dismissed because of the level of imprecision is too great. However, the history of science has been the evolution of less precise measures to greater precision. The level of precision for many fields was never reason not to measure, but it was a motivation to improve the measurement tools and techniques.  The software community needs to move the debate from whether to measure or not to how to measure better.

I did some research on what renowned figures have had to say about the subject of measurement, and I’d like to offer their position to support the need for the software community to give more attention to software metrics.

When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind: it may be the beginnings of knowledge but you have scarcely in your thoughts advanced to the stage of Science.

- Lord Kelvin (Physicist)

I believe this is the stage we are at in software project management.  We have the beginnings of knowledge, but it is of the unsatisfactory kind as evidenced by how often projects overrun their schedule, overrun their budgets, and release with low quality.   Do you think Lord Kelvin would recommend better measurement to remedy the situation?

You cannot control what you cannot measure.

- Tom DeMarco (Software Engineer)

Here we have one of the more prolific researchers in the software profession emphatically state that measurement is the path to control.   Why do we not listen?  Why do so many projects make feeble attempts at measurement?

I know no way of judging the future but by the past.

- Patrick Henry

Well this one really is an indirect case for metrics.  If we have no way of quantifying the past, how can we use it to judge the future?  Assuming the past is the best way to judge the future then metrics is the tool required to make that judgment objectively and reliably.

What is not measurable, make measurable.

- Galileo

Can you be any more emphatic than that about the importance of measurement?

All the scientists formally quoted, have made significant contributions to science.  In the case of Galileo, he made revolutionary contributions to human’s understanding of the universe.  These great scientists are telling us something very important about measurement.  If you think about it, measurement was vital to their contributions to science. Maybe that’s why they are so emphatic in their support of measurement.  I think it’s about time we listen.  Don’t you?

It happened when I participated for the first time on a SEI/CMM process improvement initiative at my employer.  That’s when I realized the power of software metrics.  Instead of handing us a ready made process to follow, each of the different product organizations were tasked with the objective of developing their own processes with the goal of achieving Level 2 certification.   Each group formed teams around the key process areas (KPA) to develop the practices to be followed.  It was without a doubt the best process improvement experience of my career changing the way I manage and view software management permanently.  

When the process areas were complete, it was then my responsibility as a Technical Project Leader to use the processes developed on the implementation of my next assignment.  While there was skepticism with the initiative, the organization I worked in decided that they were going to support it and give it a good college try, and we did.  Some organizations simply wrote the procedures and loosely followed their processes, which is more often the case.

That’s when the epiphany happened.  One of the key principles for estimating is to first size the project and permute the size into a duration using historical data or industry standard data when no historical data exists.   Since we never recorded this type of data before, we used industry standard data, or what we believed to be.   Once the estimates were developed and work began, the power of that approach revealed itself when I began to track the project.  The tracking KPA required that all estimating assumptions be tracked, both time and size.

Benefits

This had many important benefits, but the most important benefit was that I could be wrong in my estimating assumptions and I could detect the error early enough when it was still possible to do something to correct it.   If I had only tracked one of the two dimensions, time or size, the fidelity is not there to draw meaningful inferences about how well the project is tracking to the schedule until it is too late.  

If we only track time, we really do not know whether the work is on track until we get close to completing the assignment – typically too late to correct things.  If we only track size, well we only know that widgets are being produced or no widgets are being produced, but we can’t draw good inferences about whether it’s on track to completing on time.  

If I had to pick one or the other to track, I’d pick size because time only tells you that time has been consumed, you have no clue as to whether time was consumed towards reaching your delivery goals.  For example, completing 50% of the time budgeted does not tell you that 50% of the work is completed.  Projects deliver completed work, not completed time.

Better Communication

Interesting things began to happen when I drew inferences from the data.  I was able to see more clearly which deliverables were progressing well and which were not, and most importantly I was able to quantify it.   As a result, the conversations with the developers become more interesting and beneficial for both parties.  The questions were better, and consequently the answers were better.  Some of the conversations we could have were as follows:

1. It looks like this assignment is bigger than we estimated it to be, and here is why I’m drawing that conclusion.  What do you think?  
2. Now that you’ve been working on the assignment and your knowledge is much better, how much bigger do you think it’s going to be?
3. It looks like progress has really slowed this week, and here’s why I’m drawing that conclusion.  Is the delivery smaller than we thought it would be?  If no, is there something slowing you up?   You might find that when you pulled the developer to do some other necessary task, it had a bigger impact than you thought it would, but in the past you wouldn’t ask the question because you couldn’t see the change.

This also permitted some better discussions with management and the product manager. I could now go to management with hard data and request an additional resource.  For example, I could say to management:  we underestimated feature A by eight weeks, and we had Joe assigned to deliver feature B once A was complete.  All the other team members are fully assigned for the duration of the project.  If I can get one developer for twelve weeks, I can keep with the schedule.

On the other hand, you can have a similar discussion with the Product Manager as follows:  if we deliver feature B and don’t get another developer on the project, we’re going to need to slip the schedule.  We scheduled feature B latest because it was a low priority feature.  We would need to slip the schedule by six weeks to deliver it, or since it was the lowest priority feature, we can remove it from the release and deliver it in the next release.  What do you prefer?

Interesting Results

It’s not that these conversations couldn’t or didn’t happen before.  The difference is that they happen earlier.   Typically you wouldn’t have this conversation until after you slipped the completion of feature A by a number of weeks.  Now you are having the conversation before it was even scheduled to be delivered.   Also, the trust is greater because you have data backing up your argument.  Presenting that something was wrongly scheduled because it is quantifiably bigger than estimated is more trustworthy than the frequent argument that the assignment just took longer than we thought.  Bigger implies that it would take anyone doing it longer than planned, but longer than thought may simply mean it was mismanaged, or the person assigned to it was not very productive.   It’s a subtle nuance, but one that makes all the difference.

Other interesting things happened when I began managing this way.   Developers were more productive.  They were working significantly less overtime and delivering more, which I will discuss more deeply later in this series.  On one project where I assumed the management after the team had just completed a death march delivery, one of the experienced team members commented, “it doesn’t feel like we’re working very hard on this project, yet we are on schedule, quality is good, and we are roducing more.” 

What’s Next

In this installment my goal was to give you a little history with my experience in metrics and how I came to appreciate the benefits of using software metrics to manage projects. In the next few installments, I plan to give some further support for using metrics with some obvious examples in other industries to illuminate the benefits of metrics without complicating it with how we might use metrics in software projects.