I enjoy playing FreeCell.  It’s a form of stress release for me, sort of like smoking is to some.  Whenever I need a quick break or I’m looking to take my mind off of something, I’ll often play a game of FreeCell.  

FreeCell is a puzzle, requiring analysis and strategy to win.  They say every game is winnable, and I believe that it’s likely to be true.   Whenever I lose a game and I’m determined to win, I eventually win after repeated retries of the same game.

Recently, when playing FreeCell to take my mind off an essay I was writing, a thought occurred to me: FreeCell illustrates the value of upfront analysis and design.  On a number of essays on yuwantitwhen.com, I’ve advocated that doing proper upfront analysis and design on our software projects is the best way to secure success for our projects and to shorten schedules.  Similarly, in FreeCell I’ve won more games on the first try when I devoted more time to upfront analysis and developing a strategy.

How to Play FreeCell 

For readers unfamiliar with the game, the objective is to discard all the cards in order starting with Ace to King for all suits.  The game is setup by distributing all the cards face up along eight columns.  Play begins by moving cards to the other columns, the temporary pile, or to one of the four discard piles.  To move a card to another column, the card being moved must be the next sequential card in descending order and must have the opposite color: for example, red must follow black and visa versa, so an 8 of diamonds (red) can only be placed on a column where the top card on the column is a 9 of spades (black) or 9 of clubs (black).   Only 4 cards can move to the temporary pile; any card can move there.  Cards can be removed from the temporary pile to any of the columns or to the discard piles.

The strategy is to move cards to other columns or the temporary pile to build the 4 discard piles in order beginning with Ace.  The game is won when you have moved all the cards to the discard piles, and the game is lost when you cannot move any cards to the temporary pile, to any column, or to any of the four discard piles.

Approaches to Playing FreeCell

There are two approaches to playing the game: one, evaluate only the moves that are currently available to you by only evaluating the top cards of each column or the cards in the temporary pile. Let’s refer to this approach as shallow analysis, or two, evaluate successive moves before deciding on a move.  Let’s refer to this approach as deep analysis.  While it is possible to win with shallow analysis throughout the game, the probability of winning is low.  I’ll often start a game with the first approach then after a few moves switch to the second approach to make the game more challenging.  I can’t recall ever winning a game without doing some amount of deep analysis.   Early deep analysis improves the odds of winning on the first attempt of a game.

What FreeCell Teaches Us about Software Projects

Here’s what the game of FreeCell teaches us about upfront analysis and design.

1. Reworking (refactoring) is expensive and it lengthens schedules.  When I play using shallow analysis, it takes many more restarts of the game before I finally win it.  To win a game with less restarts – even zero restarts – the likelihood increases significantly with quality deep analysis before moving.  Similarly in software, schedules are shorter when the solution is well analyzed and designed before writing code, resulting in less rework or reimplementation.
2. Anticipating the future results in less rework and reimplementation.  In FreeCell, evaluating only the first level moves before moving makes it more probable that you leave yourself with no possible moves later in the game.  A player can avoid being locked in a corner with deep analysis before moving.  Similarly a software team can avoid rework and reimplementation by anticipating future requirements or preferably by creating a design that is easily extensible.
3. Winning every game of FreeCell on the first attempt is possible with deep analysis if it is true that every playing combination is winnable.  The most difficult upfront investment is on the first move.  It is the most difficult to analyze and to get correct, but if you desire the highest probability of winning — even certainty of winning — there is no shortcut to this upfront investment.   Similarly, in software the first release is the most difficult release to analyze.  It’s a blank sheet of paper.  The possibilities are seemingly infinite.  It’s not difficult because it’s a wrong approach; it’s difficult because it’s hard.  As in FreeCell, a project team improves their likelihood of success with enough upfront analysis and design.
4. Starting a game with deep analysis results in faster analysis later. As a consequence of analyzing a game well, it becomes easier to make successive moves as the game evolves.  Similarly in software when an application is well designed, successive releases of that application become easier and faster.

Summary

Modern approaches to software development advocate an evolving approach to software construction with shallow analysis as its principle.  They trade deep analysis in favor of faster delivery for today’s requirements.  While that approach may succeed in creating a quick first release (only if shallow analysis does not lock you in a corner in the first iteration), it is sure to require rework and reimplementation on future releases.

As in the game of FreeCell, upfront analysis and design improves the likelihood of delivering a successful software release.   Deep upfront analysis and design avoids rework and reimplementation, and consequently, it shortens project schedules.   If you are looking to improve the project team’s chance of success, shorten schedules, and control the budget, deep upfront analysis and design as demonstrated in the FreeCell analogy is a method for securing those objectives.

How much upfront analysis is enough you may ask?  It depends.  How important is it to hit your release date?  How important is it to deliver on budget?  How important is it to secure future releases?   To the degree that these objectives are important to your organization, you invest in enough upfront analysis and design to give yourself confidence commensurate with the importance of hitting your date, meeting your budget, and securing future releases. 

This investment is not measured in time; it’s measured in the quality of analysis and design.   If the investment is too expensive, consider the alternative:  you are more likely to be late, over budget, and hamper future deliveries if you don’t invest.  Of course, not every project requires high certainty of hitting the date and the budget, and some products never have more than the first release.  For those projects, other approaches may be better.  But, know the tradeoff before you embark on your journey.

Achieving Agile goals of delivering faster and responding to change quickly is best achieved by designing an architecture that leverages those goals, I reasoned in an essay titled “Agile Isn’t a Process“.  Being agile is about leveraging the entire organization and business partners to deliver solutions to your customers. Agility is best achieved when solutions require less IT involvement, not more. 

While having a good development process is important and necessary, the Agile process is not well suited to developing agile architectures; an emergent, iterative approach does not offer much when the requirements for solving the problem require an investment in up-front analysis and design. Some would call the investment big, but I would call it proper.  In this essay, I’d like to offer a real world example to support that original thesis.

Criticisms

It’s interesting to read some of the criticisms of the article. One reader suggested that an investment in up-front analysis and design results in overly complicated solutions.  George Crews commented, “It has been my experience that taking a long time to architect software only guarantees complex software, not software that will actually work.” 

It’s a common opinion.  I actually agree with this comment if taking a long time is evidence of having an untalented team working on the problem.  I have found the weaker performers always seem to take longer and develop more complicated solutions. On the other hand, if the team is talented, taking a long time is evidence of a difficult problem, and difficult problems require an investment in time for even talented people to solve well and simply.

Another common criticism is that short iterations reduce risk. Well, it depends.  If the iteration is designed to deliver less, then sure, risk is reduced.  On the other hand, if the short iteration forces the developers to cut corners to deliver within the iteration, risk is increased as requirements are missed and error conditions aren’t considered in the interest of delivering something quickly.

It’s also suggested in some blog articles that investing time in a simple but thoughtful architecture compromises present requirements. Big Design Up Front (BDUF) steals time from delivering on the more pressing present requirements.  It’s said that present priorities require precedence over future needs, and therefore, it’s suggested to plan just enough for the future.  However, if you expect the product that you are building to have longevity, then the future is a present requirement that should not be ignored.

Of course, the future has to be balanced with the present; there is no formula for this.  Arriving at the right balance between present and future requirements is an art; it is situational (sometimes future requirements have higher priority and sometimes they don’t), and striking the right balance is usually a sign of a vibrant team, a team where contrasting ideas and opinions are plentiful

There’s another criticism in the present versus future argument. The position is that analyzing and implementing solutions that anticipate the future requires more time. My experience doesn’t support this position.  Yes, analyzing and designing a good solution often takes more time to deliver that first working build, but the time lost in getting to that first build is often compensated in delivering the overall solution faster.

Another suggested reason for delivering a working solution faster is that it reduces risk since working code proves out the design.  Risk is increased when the production of working code takes longer, since a flawed solution will be found late in the release cycle, where rework will impact the schedule.  This position is specious.  Up-front analysis and design is a risk mitigation strategy; it doesn’t increase risk.  I’ve never seen a design where I could not determine whether it would be successful or not.

BDUF Example

In 1991, I was assigned to develop the solution to realize the requirement to internationalize our product.  This product was known as Reuters Terminal.  It was essentially a financial browser for navigating and displaying Reuters’ real-time financial news and information.   Reuters financial network disseminates financial news and information from every market in the world to every financial data center in the world.  Consequently having a product that was in the native language of the local market would be a competitive advantage – especially at this time when internationalization support in the Windows operating system was just being conceived.

The culture of this team was to do what is pejoratively called today Big Design Up Front, but to us, it seemed like the sensible way to design a good solution and to deliver it to market quickly.  Project initiation to release was approximately six to nine months, and there were many requirements in addition to the internationalization requirements that were delivered in this product release. 

Design

The architecture was a 3-tier architecture:  The top tier is the application requiring the internationalization support.  The middle tier abstracts the details of each language instance from the application, and the bottom tier is a language driver that captures all the details and nuances of rendering an instance of any particular language for the application. The application resources are linked into the language drivers.  Essentially all the Windows APIs for rendering text, dialogs, and menus are replaced in the middle tier.  Below is a pictorial representation of the architecture.

Benefits

While the first build may have taken longer than a build if this was implemented with an Agile approach, this approach has considerable advantages over the impetuous need to have a quick first build.  The advantages were. 

• Increases implemenation parallelism.  After the interfaces for the language API layer and language driver layer are designed, the implementation can be accomplished in parallel with one or more developers working in the application layer, one developer working in the language layer, and one or more developers working in the language driver layer.  The implementation is highly scalable.
• Anticipates future requirements. The application is to be customized for many languages, and this approach permits support for many languages without obfuscating the code with many macros to selectively link the application for any particular language.  It permits new languages to be released without releasing new versions of the application.  And it permits internationalization to be outsourced without much involvement from the core application team.
• Provides the capability to switch between languages at run time.  With this architecture the application can switch between languages at run time.  This is in part possible because of the architecture.  The application works with the middle tier only, and switching between languages is simply having the middle tier dynamically link to one of the language drivers.  Plus, there’s essentially very little extra code to support that feature.  It requires the middle tier to activate a different language driver, and the application forces each of the windows on display to repaint.
• Makes software configuration management practices easier.  Since at the application layer the code behaves the same no matter the language, there is no need to merge code while additional languages are in the process of being supported, and since support for new languages are delivered independently of the application, there is no need to manage and control many different versions of the application.

Once the internationalization architecture was completed in that first release, the architecture was in place for the lifetime of the application.  Re-factoring is rare when problems are analyzed well.

Agile Example

An Agile team has one primary goal: deliver working code in a short iteration.  If these are my goals, I would immediately begin tackling one aspect of the user interface that I can readily begin to internationalize.  Perhaps, one would start reworking all the text writing code in the application to display multilingual.  To begin seeing results quickly, the easiest approach would be to statically link a different resource file.  Then, rewrite the application code wherever LoadString and TextOut are used.   To deliver the solution quicker, one or more developers would be assigned different application modules to internationalize.  While this approach would deliver working code fast, it has some disadvantages. 

• Code is obfuscated with macros.  To update the application code for each language, the logic for each language would be selectively compiled with macros to enable the compilation of the code for each specific language.  When enough languages are supported, the language code overwhelms the primary logic in those procedures.
• Each module would likely solve the problem a little differently.  By implementing the solution inline, there is more variability in the implementation when multiple programmers are working on each module.  Even if the same developer were to fully implement the entire solution, there is likely to be variability in the algorithm as the developer converts each module. This makes the product more difficult to maintain.
• Quality is lower. Since the internationalization is completed inline rather than via a one-for-one Windows API replacement, a defect in the inline code may need to be fixed in every place that it’s been re-engineered, but a defect in the API only needs to be analyzed and fixed once.  Additionally, there’s more of an opportunity to introduce a different defect in every place that the inline code is touched.
• The implementation is less scalable.  It’s difficult, if not impossible, to outsource the internationalization development as code merging would be a requirement.  There is even less ability to parallelize development when the same file needs to be re-engineered for each language to support.
• Adding new languages is harder.  As new languages are supported, there will be constant merging of files with work going on to support other features.
• Configuration management is more complicated. Each supported language requires a separate build of the application, and therefore, a separate executable is required for release.  Also, a bug in one language may not be present in another language, so the code base will be changing out of synch with the executables deployed.

This is certainly a viable solution with distinct and troubling disadvantages for moving the code base to the future and responding quickly to changing requirements.  Unnaturally forcing a short iteration has disadvantages.  We used to have or own pejorative name for this: BFM, Brute Force Method.  Shortchanging up front analysis and design forces a BFM solution.

Summary

When I was implementing this solution with up-front analysis and design, we didn’t use the words Big Design Up Front.  We called it good analysis and design.  Sure there were engineers who spent too long in analysis and design and arrived at overly complicated solutions, but this had nothing to do with the approach: it had everything to do with the personality of the individual solving the problem.  Placing the blame on BDUF is a case of mistaken identity.

Proper up front analysis and design is required, and yes, sometimes it takes longer than we’d all like it to take.  However, when it’s done well, architectures are less complex, not more; quality is higher; time to market is faster, and products and teams are more agile.  Proper architectures leverage the resources of the entire organization to deliver solutions to customers.