Refactoring Isn’t A Design Methodology
One of the difficulties I have with the Agile software methodologies and its proponents is that they go to far. They are often susceptible to hyperbole while hawking their methodology like used car salesmen. Their embracing of the practice of refactoring is one example of this.
Of course, we should refactor, but refactoring isn’t a design methodology, it isn’t a way to be agile, and it doesn’t help deliver software faster. Refactoring as advocated by the Agile processes is essentially a design methodology. A team following Agile arrives at an optimal design and implementation via a series of iterations and refactoring efforts. Refactoring as a design methodology is incompatible with the Agile goals of delivering faster and accommodating change anytime in the project life-cycle.
Refactoring is necessary, the Agile proponents say, because it is impossible to know enough about the system up front to arrive at a good enough up front design. In fact, Agile is in part a reaction to the Big Up Front Design approach embraced by Waterfall adherents. The Agile camp has essentially given up on the goal of arriving at an optimal up front design and instead relies on refactoring at every iteration until an optimal design is achieved.
Let’s think about this approach for a moment. There is a cost — maybe even significant cost — associated with refactoring. It squanders time away from the schedule that can better be spent developing new features.
While refactoring is change, it isn’t the agile change that was intended: it’s change that involves redoing work that essentially delivers no new features to the customer. This rework is encouraged to be repeated multiple times throughout the product life-cycle.
It costs time and money to constantly rework code while stealing time and resources from delivering additional value to the customer. Further, with each refactoring effort the code base is destabilized adding new defects and temporarily lowering quality of the software that was already tested and delivered to customers.
One would believe that an approach with the goal of being agile would attempt to minimize the amount of refactoring, but they don’t: they encourage it. It is a mistaken belief of the Agile proponents that it is impossible to arrive at an elegant, orthogonal, extensible design up front. My experience is that it is possible to arrive at an optimal design up front, and when you do, the benefits to the product are immense. Benefits include accelerated time to market, high levels of quality, and shorter learning curves for new employees and lower costs over the product lifetime.
Orthogonality is a characteristic of a good design and even a good process. The assembly line process had its birth in the auto industry, yet it has been applied successfully to a myriad of industries, demonstrating its orthogonality. When a design or process is orthogonal it isn’t confined to a small domain of applicability. If you’ve ever worked as a cook in McDonalds, you would have witnessed assembly line practices successfully at work. The Waterfall software life-cycle is another example of the assembly line process successfully at work.
The Agile practices, on the other hand, are not likely to be adopted by many other industries. Arriving at an optimal implementation over many iterations and rework is too costly in time and resources to be adopted successfully by other industries. Imagine building a house, or any structure, with an Agile approach. The time and resources and consequently costs would be too great with building a part of the structure, tearing it down, and reassembling it differently to arrive at the desired architecture when it could have been discovered up front with more investment in requirements and pencil and paper design.
The Agile practices give the illusion of faster, more, and cheaper, but it’s only that: an illusion. It achieves this legerdemain by delivering more releases of shorter duration with less functionality. In a time when society is looking for easy answers to difficult problems while embracing the appearance of success, the Agile practices are the right practices for our troubled times.
(+1 rating, 1 votes)
Loading ...
What strikes me odd about the Agilistas is that if they possess a great deal of expertise, why would they have to make some many passes at a solution, especially since there are a number of existing patterns and techniques to choose from and apply to problems. There are only so many ways to read data from a database. If you the ActiveRecord pattern then you have made some design decisions that have charted a course for you. If the data model is sound and completed, why all the finagling?
For me the questions boils down to the problem domain: How different are your set of problems you need to solve from other domains and what solutions already exist for those domains. Most business apps interact with a database, so there is no reason for a constant refactoring if you let your problem domain solution be driven by databased driven architecture, e.g. ActiveRecord, Business Layer Logic, MVC, etc. You still have to perform CRUD, and therefore you will only have so many permutations of screen development. This makes me believe that you should, for the most part, have an upfront design. Have to do a workflow? Guess you have to have a process model completed first. Again, up front design.
You raise a good point. If you have experienced software engineers on the project, they will arrive at a good up front design without much effort, but if you have inexperienced software engineers on the project, you will need some trial and error to arrive at a good solution, thus a greater need for refactoring over many iterations.
Additionally, TDD stuff should only commence once you have the domain settled, and at that stage your tests are verifying what you established up front. The series of unit tests can serve documentation and a training tool.
There may be the odd occasion where you need to prototype ideas quickly, and TDD can be a very good tool towards the goal of a fast model, but in that case it is a throw away solution.
An up-front-design and architecture is a good thing for well-understood problems. But when we are doing something new, a detailed up-front design can be wrong - and therefore dangerous! In many cases, we can more easily discover the best solution via TDD and refactoring.
Regarding safety, Agile proponents know very well that refactoring without extensive supportive testing is a bad idea. They have never suggested or encouraged it.
Steve, welcome and thanks for sharing your thoughts. I have difficulty comprehending how doing something new makes it different. For me I find new requires even more rigorous up front design and analysis not less. I can easily write code with little up front design and analysis for something that I understand very well; I struggle to find solutions for things I don’t know very well.
The best way for me to solve problems I don’t understand very well is to understand them better by deep analysis and design. Pen and paper work best for me, but that may have something to do with the technology that I grew up with.
Let me give a concrete example. I recently designed a reporting system, and I said that there should be the following roles, and I specified how they should interact using a sequence diagram: Report Provider, Data Adapter, Options Provider, Renderer. Now in the final code, there were classes representing each of these, but there was also inheritance, rendering strategies and other classes I had not thought to specify.
If we can agree that the code is never the same as the design, then we can also agree that we can over-design something, i.e. spend too much time specifying intricate details that may be unrealistic.
So why not let the developers figure out those details themselves? They will have immediate feedback when they get it wrong, and if they have the tools of TDD and refactoring, then they can fix their mistakes “on the run”.
In the environments that I’ve developed in, we never had time to do it twice, so it was always in our interest to spend the time to do it right the first time for the sake of the schedule, quality, and money, so if these are the goals for your projects, then you specify all the details that prevent you from doing it twice.
But what’s wrong with having the developer also do the design? This way he is figuring it out for himself. When I first started in this business, the developers wrote the functional specifications, did the design, wrote the unit and acceptance test plans, executed the tests and fixed the bugs. We lost something when we moved to having job titles for each role on the development team.
“Separation of concerns” and dividing the duties and assigning the roles of BA / Designer / Tester to different people has not been good for development. I have worked with some extremely talented coders in the past who were very one dimensional, and could not write a spec to save their lives. This begs the question: if you can’t express it, how do you know it’s right? Same thing with testing - the developer should be able to create a series of tests BEFORE they cut a majority of their code, as you should be able to express what the code conforms to as expressed by the spec.
A lot of developers these days will refuse to be BA’s as it takes “too much time away from the work”. Not good.
I agree “separation of concerns” hasn’t been good for development. It’s been particularly problematic with the project manager role. As I was working my way up through the ranks, a project leader wrote and managed the schedule, wrote the functional spec, lead the engineers, helped develop the designs, reviewed the designs and code, reviewed the test designs, helped debug, and managed the test effort. It was demanding but the amount of control over the schedule and insight it gave you to the pulse of the project was unparalleled. Though on really big projects, you will need to separate the concerns some. Now the resource managers leave managing the schedule to another individual, the project manager, but it’s not as effective because to manage a team well that is on a schedule, you also need to manage the schedule. It can be done, but resource managers also need to see managing to the schedule as their responsibility too, and too often they do not, or not to the degree that they should, in this arrangement.
You’ve got me thinking about a Y2K remediation gig we did for a national bank. The project lead, whose mantra was “do what is necessary and sufficient”, focused the entire team with a project context diagram and Ganes Sarson process model he did for the 19 systems that passed transactions around. These 2 documents, where we could trace all transactions, became the starting points for all of our tests and validation and test data creation apps we developed. “Read the spec” was the answer when we had questions, as he detailed everything there.
We in turn were responsible for crafting our test plans and specs from that point, and although he had the framework in place we still had analysis to complete. Our four member team built test environments and tested the 19 apps in 6 months. By the end of engagement we had completed over 400 pages of documentation and each of us could pick up each other’s systems. Although we had to specialize, there were no separation of concerns as we had to communicate effectively with each other and the client.
I compare that team to the others I have been involved and I see a vast difference in quality of communication. Sad and scary at the same time.
Leave a comment!
EMAIL
Subscribe
Visitors
Polls
Categories
Blogroll
Tag Cloud
Agile Analysis BDUF Certifications CMMI Critique Debate Defect Free Design Estimating Ethics excellence Future good enough Hiring Leadership LOC Management Methodology Metrics Outsourcing Popular Process Product Management Project Project Management Project Tracking Quality Recruiting Refactoring Requirements Schedule Scrum Skills Stable Test Strategy Tracking Traditional Upfront Video Waterfall wrap-up XPRandom Posts
Latest Video Post
Recent Comments
Most Commented
Most Viewed