Reader's Comments

on Software Design Review
In my years as a software engineer, it has always been business people that resisted the formal software engineering process. The engineers are the ones who see and fear the complexity of major projects but business people often do not. Perhaps your thesis ought to be that software firms run by engineers succeed due to an appreciation of these risks and hence are willing to finance the risks of mitigating them? Secondly, note that I did not claim that business manager are 'lazy' or incompetent. If one has respect for the professiona software engineeringl regiment, one should at least feign respect for the professionals who also advocate it.

-- Bob Zi bub, October 20, 2009
"Programmers will not keep themselves honest."

I've seen senior, principal, and senior principal programmers who can write beautiful algorithms, construct advanced object hierarchies, and implement what many would consider to be elegantly architected solutions. Those senior programmers will give you a perfect solution; yet still you'll run into issues mentioned where pages will take five minutes to render. Confront these senior programmers with the issue, and the response will be something along the order of "what's the problem, the solution is perfect?". And because these programmers are considered experienced, management will often back them up.

There is a certain class of programmers who will keep themselves honest. This class of programmers has had to deal with customers directly. They have experienced the mistakes of not documenting, not testing, and writing cool, clever, elegant code instead of code that may be ugly but robust. They will provide some documentation, but not enough that it is easily outdated. They will test the solution, but not to the point where the tests become academic. They will write cheap hacks into their code because they realize the end goal of a happy customer is infinitely more valuable than another page in their portfolio of a textbook perfect solution.

-- Fred Moyer, October 20, 2009

Remembering my control systems classes in college, there are three primary faults here: bad control module, bad sensors, and poor feedback loops. The control module takes in data from the sensors (feedback) and adjusts the system's outputs to keep it on track. If any one of the three (control module, sensors, feedback loops) are faulty, the outcomes will not be as expected. If it is an electrical system and the electricity goes out, then the system stops.

Let's use this as an analogy for the ecommerce project in your article and map control system components to the project:

If there is no goal or vision document, then there is no 'why' to the project. I have central heating and cooling for a reason: so I can be comfortable in my own home during hot and cold weather. That's the purpose of my HVAC system, and if I didn't have that goal or purpose, then there would be no reason for me to expend the money and effort to buy and operate the system.

For the ecommerce site, the purpose may be more complex -- at its root the purpose is to make more money than is spent on developing and operating the system, what I'll call the 'generic' purpose; others might call it profit. But for that outcome to occur, the system must provide something the customers who use it actually want and are willing to pay for, what I'll call the 'specific' purpose. Some people call the explanation of this specific purpose a business plan. It's important for the programmer to understand the purpose or business plan so they can assess their own progress in developing the system. The better the programmer understands that purpose, the less tight the communication feedback loop between the programmer and the business person has to be. I'm quite willing to admit that in many cases there is no initial vision or goal; people often putter around and end up with things like twitter and so on kind of by accident. They start somewhere simple, and evolve based on feedback or interest without having some grand scheme or goal in mind at the beginning. That's OK. Serendipity is a fine business plan if it doesn't make you broke, and these days it costs very little to create online applications.

If there is no design document, spec or whatever you want to call it, for the system you're creating (or in the simplest case, a clearly understood goal the system must meet) then there is no basis of measurement by which to determine whether the system needs to adjust itself, and therefore no basis for knowing when your implementation is working properly or 'done'. Projects that have no defined goals can't be considered failures because they have nothing to shoot for, no basis for measuring success. They're simply wastes of money, or jobs programs.

The system you are creating has its own goal or purpose that is separate and distinct from your business plan: the purpose or goal of my HVAC system is to maintain the temperature inside my house as close to what I have it set to as is reasonably possible. My HVAC system does not care whether I'm comfortable or not; its goal is only to maintain the temperature within specified bounds. This is where you can more easily define the boundaries around the system you're building -- you must decouple your business purpose, goals or reasons from the technical goals or reasons that the system will embody. The system doesn't know you want to make money, and it can't tell that you're losing money and come up with ideas for how it can adjust itself to make you happy.

The control module (i.e. thermostat) analyses the data coming in from its feedback loop(s) (the temperature sensor or sensors), compares that to the 'goal' I have given it (maintain the ambient air temperature inside my house at 75 degrees Fahrenheit), and turns on, shuts off or leaves in its current state the heater or A/C depending on whether the temperature is within the bounds I set for it or, if not, whether the measured temperature is too high or too low against what I've set it for.

The control module in our mapping is the programmer: if the programmer doesn't understand the system's technical goals (and ideally the business plan so perhaps he or she may catch where the system's design spec may fail to provide your business outcomes), or ignores the data coming in (which is the same thing -- not knowing the goal or not seeing whether what you're doing is leading to the goal is really the same as not having a goal), then there is no way for the project to be successful from a business plan perspective. And the business person, unless they themselves are technical, will have no chance of figuring out what needs to be done or why things aren't working the way they want.

So, to recap:

If the control module is faulty (not appropriate for the task at hand, ignoring the feedback inputs, etc.), or the feedback loops are bad (the data is being corrupted between the sensors and the control module), or the sensors are faulty or not appropriate to the data collection task (they are collecting stock prices instead of temperature readings), then it's hard to understand how you can possibly have a successful business outcome. Of course, having your system work as you specified doesn't mean your business idea will actually make you money, but that's not the system's problem. And if you cannot establish the parameters the technical system itself must meet, then it is an open-ended project that cannot succeed technically except by sheer luck or because you have a programmer smart enough and self-constrained enough who will figure out what the system's technical goals need to be as they do the work.

Keep in mind that this is only a thought experiment, a way to model the dynamics of the project to provide insight into where the issues might be. If you lose sight of the fact that the programmer, bill payer and possibly other 'parts' of this system are human beings and not machines or parts of a machine then you will likely treat your programmers and other people as machines and the means to your ends instead of as human beings.

/s.



-- Scott Goodwin, October 22, 2009
Great article, couldn't agree more. As a software architect with about 30 years' experience in the field, I also have spent untold hours "nagging" developers and architects alike to document not only their design decisions, but their code! Software development should be easy, but this article is another of many showing how cutting corners leads to disaster. Cutting corners almost *always* lowers value and ROI for software projects.

In my own most recent experience, inside one of the largest and (formerly) most successful online advertising networks, I saw at close hand the truly epic failures of a company-wide initiative to refresh the entire adserving platform (legacy UNIX to .NET). This involved a technology division of at least some 150 folks. Management made all the *classic* software development mistakes: hiring outside consultants to do the job, for instance; having *way* too few architects, for another. They soldiered on through YEARS of poor designs, missed dates, browbeaten teams, fired consultants, and literally tens of millions of dollars burned into thin air. I personally expended most of my political capital trying to get them to do the right thing, in as many ways as I knew how. Now this company, recently worth billions of dollars, is having its workforce slashed and is reduced to a mere shadow of its former self.

How did this happen? A total disregard for creating and fostering any type of software engineering culture. REAMS of requirements were drawn up -- but no one really understood them. Designs were presented -- but they weren't conceptually coherent and didn't work. Project plans were constructed, time and time again -- but they almost never delivered on their commitments. Software development was started and re-started -- but it was almost always late, and much of it had to be re-written. And so on. Classic.

And *why* did this happen? There was a management culture of arrogance, especially on the technical side. One simply cannot labor on for so many years without asking the simple question: "why could we not deliver on our commitments?" These hard questions *are* asked, as Phil points out, in companies like the Microsofts, the Googles, the Amazons, where the leadership is steeped in technical knowledge and cannot have the wool pulled over their eyes.

Personally, I'm proud of my record of success on projects: I have learned a lot of lessons from a lot of great folks. I lead teams with relative ease to meet committed dates, with high-quality, well factored, scalable, maintainable, monitorable and deployable software solutions.

To add to the points made by this article, it's a simple matter of discipline: knowing WHAT to pay attention to, namely all the elements of a software lifecycle and the value it creates for the business. The entire team must be clear on requirements. The design must be simple and justified against its alternatives. An appropriate level of test suites must prove out that the requirements have been met. The software must be easy to deploy, and when running it should be easy to monitor that it is, indeed, running correctly.

All these elements of software discipline are borne out of a single, simple, focused practice: the virtuous cycle of continuous improvement. It's OK to fail, once; but then one must stop and ask a sincere "why," decide how to prevent it in the future, and incorporate this literally into the DNA of the technology teams via an updated software process. Likewise, we want to also ask why we succeed and incorporate that as well. The result is that the team or organization always continues to get better and better with time.

So, an emphatic "yes!" to this article: most software is horribly written, and, unlike hardware design (circuit boards, bridges, etc), the creation of software machines is largely practiced as more of a black art than a science.

The one point I would take issue with is that "programmers will not keep themselves honest." While I agree with the result -- poor quality software -- most software developers really do want to do the right thing. It's just that they are not given the proper guidance on *how* to do it. Most developers are happy if not eager to understand how to frame, design, develop, and deploy software solutions that deliver real value to the business.

In the end, if your software project is having trouble, it's likely you may need more and better architects. No one would attempt to build a house of any note without an architect - imagine telling master bricklayers, carpenters, and plumbers to "just do it" without first drawing up detailed plans. It's the same with software.

In the end, you have to measure yourself by your results.

- Keith

-- Keith Bluestone, December 21, 2009

Add a comment