I agree that there is something to be gained by having a working application and getting there as soon as possible, but traditional practitioners have been doing that since the 80′s when I first started in this field. It’s not new. What is new is to artificially timebox that first iteration. Whenever I plan a project, I look to identify when I can have that first build, but I let the work dictate the iteration, not an artificial time frame.

Where I’m confused is this idea that you cannot know if a design will work or not until it is implemented. I was fortunate enough to be mentored by really smart guys when I first started out who could rip apart a design, any design. They knew if it would work or not before it was coded. This saved time. If I had coded the work that I designed without the redesign that they recommended from reviews, all the design mistakes would have been caught at unit testing where it would have taken much longer to refactor and get working. I’ve never reviewed a design where I didn’t know whether it would work or not at review time. Reviews are valuable, and good reviews are necessary to be agile.

But really, what does seeing something work early tell us that’s valuable about the design? If an Agile team had implemented my contrasting example, they would have seen working results in the first sprint, but it still would have been a lousy design and implementation. Working is NOT evidence of a design being qualitatively good. The Tacoma Narrows Bridge worked, but it was a lousy design, and it finally collapsed. Sometimes knowing something is a lousy design takes time when you rely on only observational based evidence of it working to conclude it is good.

In the case of the contrasting example, a good review by good engineers would have provided more benefit than a working first sprint. Time well spent in design and analysis would have provided more benefit than a working first sprint. I’ve managed large software projects, and whenever I’ve given into the pressure to deliver something sooner and sacrafice sufficient design work, it’s always been a mistake.

You raise several good points that all deserve discussion. Let me respond here only to the main point.

I agree that no software methodology (Agile or BDUF) can rescue a bad solution. And I also agree with you that, unlike BDUF, the Agile methods do not emphasize a process for designing a good solution. But perhaps, as we say, that’s not a bug but a feature.

The advantage of using Agile methods is that they identify bad solutions more quickly and more definitively than BDUF methods. They also embody a particular promising approach. This allows better risk and project management.

Maybe it would help to know where I’m coming from. I have a science and engineering background whereas most of my fellow software engineers I believe have a math background. I’ve noticed many mathematicians live in a Platonic world. I note many scientists and engineers do not. Most of my cohorts tend to think that if a software engineering process is logical and consistent in handling a problem’s requirements, the process will work. But I don’t. (I also don’t think a sense of conviction has anything to do with reality. Just because my cohorts think they have a great design doesn’t mean they actually do.)

The attraction of Agile methods is that at the end of a sprint, a relatively short time, you will know (not think, but know) if a particular approach won’t work or needs major modification. Logic and analysis can’t do this, no matter how formalized a process is used, for the same reason scientific theory has to rely on experiment.

I’m not at all offended. I value good honest exchange of views – even if they are rough. I do realize you didn’t use the word ludicrous. I was taking your naïve view to an extreme to make a further point.

My problem with Agile is many folds, but mostly that the focus is on the wrong things, and I believe at a minimum this essay makes the point: agility with a small “a” is best achieved via good solutions, solutions that are extensible and make it easy to change the working application. No process methodology can rescue that contrasting solution in the essay: not a waterfall based methodology nor an Agile based methodology. Small iterations don’t rescue a bad solution. The quantity of documentation (little or a lot) won’t rescue a bad solution. Agility is foremost achieved through good designs, implementations that rarely require re-factoring.

The Agile Manifesto doesn’t make a single point about the need to create good solid designs and architectures. In the Principles Behind the Agile Manifest, the word “simplicity” is used not in reference to architecture and designs but to do less work. Only one small point in the Principles Behind the Manifesto about good design: “Continuous attention to technical excellence and good design enhances agility.” Now the Agile guys can caricature the BDUF approach favored by the so called waterfall proponents, but make no mistake the emphasis of the traditional practitioners is on design and architecture and the Agile camp is ridiculing them for emphasizing the right thing. Agile has the wrong emphasis.

I’m never going to suggest that complicated designs won’t be developed by traditional practitioners. Of course, they will. Complicated designs are a function of the caliber of people. It’s not a methodology issue unless you have a methodology like Agile that encourages going to implementation too quickly. I’ve read blogs of Agile proponents who promote that analysis, design and construction happens in parallel in Agile. How? How do you write code for a problem where you’re doing analysis and design concurrently? It’s silly.

You are mistaken about CMMI. It’s a process guide. It identifies KPAs (key process areas) and the objectives for satisfying compliance to the goals of the KPAs. It’s up to the implementers to define the process. It’s a much more rigorous and objective form of the Agile Manifesto with a lot of Greenspan like obfuscation. The obfuscation is both its blessing and its curse. It’s a maturation model in that you build your process like building a multi-story building: one floor at a time. The challenge is to develop a lean process while meeting the objectives. Too many teams that use the model go overboard. They miss the essence.

I’ve been involved in 3 CMM(I) process initiatives and heavily involved in developing one of them. I would say only one of the 3 processes that I was involved in would I consider good. Not sure what you refer to with the empirical foundations, but there is no doubt that a company has level 2 practices. There are very objective criteria for establishing that conclusion. This is missing in Agile, and it’s why even the Agile proponents seem to be at odds as to what Agile is.

I hope you weren’t offended. That certainly was not my intent. I would like to point out that I did not say that the described Agile design was ludicrous — rather I used the word naive. I agree with you that many teams have used even worse approaches to similar problems.

Do Agilistas really know what they are talking about? I would think so. The Agile method is DEFINED by the Agile Manifesto (http://en.wikipedia.org/wiki/Agile_Manifesto) which is a pretty succinct document . But I am not surprised you don’t think I understand it too well. I often don’t exactly know what I am talking about. I have to change my mind a lot to keep on the right track.

The Nokia Test (http://djellison.blogspot.com/2007/12/nokia-test-are-you-really-agile.html) has recently been put forward by several people as objective criteria for determining if an Agile method is actually being performed. It too is pretty short.

I agree with you that Cowboy Coding (http://en.wikipedia.org/wiki/Cowboy_coding) is less likely using BDUF methodologies since they discourage immediate coding efforts.

I was hoping you would allow me an exception or two on my observation about long-time-interval designs being complicated. But I think I can go more than one-to-one on examples vs. counter-examples: the FBI’s Virtual Case File project being an “outstanding” one. (http://en.wikipedia.org/wiki/Virtual_Case_File)

Finally, I would point out that CMMI is not a process guide, but a process improvement guide. It contains no software engineering methodologies. Also, its empirical foundations are just as flimsy as Agile’s.

That solution may be naive to you, but I bet many teams have solved that problem with that approach, and my experience is that human nature being what it is, most teams will go straight for BFM when they feel the pressure to deliver — I’ve seen it repeatedly. To the extent that Agile adds to that pressure with short iterations, you will see teams solve problems in a BFM way. Not all of course, but the process encourages the behavior. That’s all.

But I’m really starting to get confused as to what Agile is because the Agile proponents don’t have a consistent message, and maybe that’s the problem, the Agile proponents aren’t really sure either of what it means to be Agile.

But the real message in the essay is that agile is best achieved through proper designs and architecture. Even if you believe the contrasting solution is ludicrous, there is nothing a development process can do to rescue it.

Now if any Agile proponent observed our practices developing that problem, they would call us troglodytes. This was BDUF, it was waterfall, not in the traditional sense that Agile proponents like to caricature it. Yet we were very agile with a small “a” because our solutions permitted a high degree of scalability in implementation and because you can deliver smart solutions faster than naive solutions. We always aimed to deliver smart.

Now we came up with a simple, elegant architecture with an approach that you said would only lead to complicated solutions. How do you explain that?

I’m not sure why a methodology would pick the name Agile if it weren’t chosen primarily to represent the definition of the word. As dictionary.com defines it, Agile is defined as “quick and well-coordinated in movement.” The principles behind the agile manifesto has as one of the principles to “Deliver working software frequently, from a couple of weeks to a couple of months, with a preference for the shorter timescale. So I have to believe being quick is a very important goal of Agile.

Some solutions don’t lend themselves to deliver working code in a couple of weeks to a couple of months. We didn’t have a working build for at least 2 months. Now how does a methodology such as Agile permit a team to deliver a solution that requires that kind of up front investment? Don’t tell me that all problems can deliver working code in an iteration of 2 months or less because that’s just nonsense.

Now here’s the problem with Agile: you can read many blogs with supporters of the Agile processes, and they will all have a somewhat contradictory definition of what Agile is. It reminds me of the line in “The Incredibles, if everyone is special, then no one is special.” Likewise, if every practice is Agile, then nothing is Agile. The Agile definition is too vague. There are no objective criteria for deciding whether a practice is agile or not. So we live with this constant controversy and confusion. If the Agile community wants to gain some respect, they need to be clear about what Agile is and isn’t with objective criteria. It will help the Agile community and help it gain support with those who haven’t bought in yet.

I also agree with your statement, “My impression is that a lot of people claim to be “doing Agile” as an excuse for allowing themselves to do things in a “cowboy” fashion.” This is a consequence of a vague definition. You don’t have this problem with CMMI. It has very objective criteria for determining whether a team meets the standard. Agile needs a similar definition.

Who would try to apply an Agile approach as naively as described? I have some I18N experience and can imagine taking an Agile approach to successfully implementing a solution. Perhaps my imagination is faulty, but I find this particular example unconvincing.

But I agree with you that, as put by Fred Brooks in “No Silver Bullet”: “The hardest single part of building a software system is deciding precisely what to build.”

Rather, the point I would like to make is that something being “hard” and something “taking a long time” is not, in this case, the same thing. This is the key to our difference of opinion. In my opinion, a good software design is less like running a marathon and more like high jumping. Either you clear the bar or you don’t. Either way, success or failure doesn’t take that much time, regardless of the height that needs to be cleared.

And you need the people to do it. Three high jumpers that can clear 2 feet can’t take the place of one high jumper that can clear 6 feet if that’s the height needing to be cleared.

(I will concede that developing the skill, knowledge, and inspiration necessary to conceive of a good design can take some time. So can the documentation, validation and implementation of the design.)

However, I think you have set up Agile as a straw-man in your argument. Specifically your assertion…

“An Agile team has one primary goal: deliver working code in a short iteration”

…is not true. This is taking a common Agile *practice* and presenting it as representative of Agile *values*. (It is not even a principle of Agile, although it is similar to the principle of “Deliver working software frequently”).

It is a common misunderstanding that Agile suggests up-front architecture is unnecessary. It is always a good idea to plan your architecture – Agile just advises that you do *no more* than you need. This is a direct consequence of valuing working software over comprehensive documentation and responding to change over following a plan.