Reflection: Unrealistic Schedules
I often wonder why software teams always seem to be committing to unrealistic schedules. You know when the sales team signed a contract with a customer to deliver functionality on a date without ever asking the engineering team whether it were possible. Never mind the roadmaps identify an entirely different set of functionality than what was committed. And guess what? The product roadmaps can’t change either; the sales team has signed contracts on that functionality too.
It’s not just the sales organizations though; the software organizations are all too happy to over commit without any help from outsiders. You probably have one of those programmers on your team that when you ask him for an estimate, he says a week and three months later he’s still working on it. When you ask him what happened, his answer is well it was a bit more complicated than I thought. Does he learn from it? No way. Ask for an estimate on the next project, and he’ll still quote a week.
I often wonder if these same people would commit an organization to building the Empire State building in a month. It sounds ludicrous, doesn’t it? But, software teams are asked regularly to deliver on similarly impossible feats — well maybe not on the scale of building an Empire State building in a month, but you get the point. Really, even these same salesmen wouldn’t make that kind of commitment for a building because even to them, it’s an impossible effort.
What is it about software that makes it so likely that the sales teams and engineering teams will over commit? I believe a large contribution - not the only — is that we haven’t been successful in using metrics to size our projects. The entire construction industry doesn’t make a single commitment to a project without knowing the size of the project. Before you’ll even get a ball park estimate from a contractor for time and money, they want to know how many square feet the project is. Even a salesman, without even knowing the exact square footage of a similar proposal to constructing the Empire State Building, would know just by looking at the plans that a month is an impossible commitment. While he doesn’t know exactly how big it is, he knows it’s huge, and a month won’t be enough time. In software, they just have no clue, and it’s the fault of the software profession.
It is my belief that we can begin correcting this problem if we can begin to size our software projects using a consistent unit for size. The measure needs to be pushed all the way out to the team members who are making commitments for the organization. Initially, the metrics won’t have much meaning to those people hearing and using it, but after they see the time and effort required to deliver projects in those units, they will develop a feel for what it takes to deliver functionality in those units, and they’ll be able to roughly size projects themselves after gaining experience. They’ll at least have a feel for whether they are committing the organization to constructing the Empire State Building in a month or less.
At a minimum, when sales teams over commit, they’ll know it as soon as the software organization returns the estimates, and possibly more productive conversations about how do we fix it can be had, instead of work harder when it’s obvious that working harder won’t solve the problem. If every project that was 500 units in size or greater took 6 months or greater to deliver, it’s obvious to all but the dullest minds that something needs to change to deliver 500 units in 3 months as was committed. Only then can we begin to have productive conversations. What are your thoughts?
Periodically, I plan to write a reflections essay on a peculiarity in the way we practice software development in industry — at least in my experience. I hope to bring a new perspective on the topic, and I hope you comment with your thoughts and experiences.
October 4th, 2007 at9:36 am
Well said Bill. One reason I commonly see for committing to unrealistic schedules is short-sighted career planning. People over-commit to make things look good in the short term. And in big teams, schedule pushback is bottoms-up is difficult to do.
Very educational blog. Keep up the good work.
October 4th, 2007 at3:10 pm
I believe part of the problem is that Developers are never trained in the art of estimating. Instead they learn to first write code, then write more complex code, then design and if they are real good, then architecture. But you rarely see where organizations train people on the art of estimation.
Taking your example of construction, I’ve acted as a GC on a few homes being built. What I found is that many professionals lack the same skills. They know how to weld a pipe or bang a nail and can even tell you how many 2×4s they need to put that wall up. But most all blow just how long it will take them to put the wall up.
The best estimators in the construction field end up retiring their hammers and welding irons and instead become the owners of successful businesses with teams under them. The art of estimating is far different than the art of doing the work. Why is this?
Because you need people who are bigger thinkers to estimate and not people who are great coders, great designers or even great architects. These big thinkers have to see all the pitfalls of the projects, really understand Risk Analysis and what really goes on during the course of a day. The great “do’ers” are great at doing because they can focus on a problem and solve it properly. But these type of people’s strongest asset is their ability to focus. What is the flip side of this? They’re so focused that they don’t see the distractions of the day. They don’t see all those risks that are going to hit them and throw them off schedule.
Estimating is a skill that demands a certain personality. Sometimes this can’t even be tought, but is part of your make up. I think we all know those people out there who are so damned focused you often say “that’s the man I want on this job”, yet these are the same people you tend to steer far away from looking at bigger issues, solving issues that involve lots of people. Or you even tell others “please stay away from Bob as he’s focused on this job and I can’t have him distracted”. Because we all know these very focused minds when distracted often go deep dive into their distraction forgetting what they were originally working on.
Something to think about.
October 4th, 2007 at8:42 pm
Furthermore I find that sales people–and many executives–think that talking is working and that they can just talk about it until it comes true. But talk is cheap while work is dear. At some point, someone actually has to do some work. My technique: turn it around. Whatever a sales person says about development, say about sales people. Sales: “Why can’t you ever deliver on schedule without descoping?” Dev: “Why can’t you ever sell without discounting?” They can’t stand their own ridiculous statements used against them. How is it that we expect near-perfect estimation from a developer when no one believes a sales person’s estimate until December 32?
October 4th, 2007 at9:50 pm
Steve, thanks for commenting. I like your point about discounting and hitting sales targets. It’s certainly a case of the pot calling the kettle black. Pointing out the other guys failings, while it feels good, isn’t very productive. Sometimes in life you just have to suck it up, and not let the environment/people distract you from your interests.
Developing software and selling in a competitive marketplace is a tough business. I don’t believe anyone has ill intent; I believe we all lack the tools to manage the circumstances better: sales and engineering.
There are no perfect answers, but there are things we can do to manage the circumstances better, and maybe when we manage them well because we have the right tools, it feels like perfection, though it can never be perfect. My goal is to share with you some tools and techniques that have worked very well for me: I’m not just writing about things I think would work. Not everyone is going to like them or agree with them, but I can tell you they work extremely well when used effectively. I think our quest for the perfect answer prevents us from embracing the best solution.
Please keep reading and keep sharing.
October 4th, 2007 at10:06 pm
RaviKant, thanks for commenting and the for the kind words. You’re right it’s really difficult to pushback on the schedule from the bottom-up. The one thing you have control over is the quality of what you deliver. You need to make sure it’s the best, even if it means you have to suffer the pain of delivering it late. I’ve found, as long as deliveries aren’t too late, everyone forgets the small slip if what’s delivered is great. Develop a reputation for delivering high quality, and most good managers will go to bat for you.
Please keep reading and keep sharing.
October 4th, 2007 at10:38 pm
Ray, you raise some valuable points. The quality of the estimate is dependent on the skill of the estimator, but that doesn’t imply the estimator shouldn’t use the best tool for the job, or that estimating is some black art only possible by the right person with divine insight. I curse my guitar everyday because I can’t play like Jimmy Hendrix. It’s not the guitar’s fault the music I make with it sounds like noise.
I believe estimating is a skill that anyone can learn to do if they choose to learn and do it. Not everyone will be as good at it who learns how to do it. We can never take the skill/talent element out of the job. That’s what makes olympic champions. It’s the individual that’s perfected the skills to the highest level that comes out on top. It’s the same for writing code and estimating. Some people are more skilled/talented at it than others. Those teams that follow the best practices and are the best skilled at them will perform the best. Using the the right tool and having a bad outcome is not evidence of using the wrong tool. We need to be careful of not throwing the baby out with the bath water.
Are you suggesting that sizing a project isn’t the right approach? Or that in constuction they don’t size the jobs first?
October 5th, 2007 at10:42 am
I believe you’ve misread me. I’m not at all suggesting that we don’t size projects, rather my point is about doing proper scheduling.
Some people are better at seeing the big picture and are able flush out the risks, their likelihood and cost. While others are very good just estimating their 20 lines of code. But knowing that it will take X days to cut so many lines of code does not make a schedule. And at the end of the day our customers don’t care how accurate we were with our estimates on effort to code a module, but scream at our accuracy on the overall schedule. Some people can’t see the forest through the trees and are better suited for telling me the size of a tree and not how long it will take them to walk thru the forest. Make sense?
October 5th, 2007 at12:57 pm
I was hoping you were saying that, but I wasn’t really sure. I agree there’s more to planning a project than just getting good estimates. All of the points you raise are relevant and necessary aspects of creating and delivering on a successful plan. For me, though, it all starts and ends with having the best possible estimates, and estimating in such a way that the deliveries are trackable as they are being developed. Otherwise, we are back to taking the developers word for it when your 50% into the time budgeted and he says he’s 75% complete, and then the deadline passes, and he’s 98% complete for the next 8 weeks. Obviously something is wrong. Metrics give you a way to see something is wrong early and to quantify it, so that the developer and the manager can make intelligent decisions without being in denial.