Andrew Grumet and I have drafted an article on software design review and would be grateful for comments (persistent comments underneath the article itself; comments about what should be changed in the article or typos should be posted here on the blog). Software projects would seem to be extremely amenable to external design review and yet though this practice is common in business agreements (have a lawyer look over a letter agreement) and in construction (builders bring in an architect or an engineer; architect brings in an engineer to review the structural design), it has been almost unheard-of in software development. A team of programmers is selected and then is able to do more or less whatever it wants.
15 thoughts on “Software Design Review”
Comments are closed.
“Philip said he wanted the document where the programmer set forth what problem he was trying to solve, how large a data set was being handled, what each server did, what software had been selected and why, where the files and programs were on the server, and what the data model was.”
I may be missing this, but it might be nice if you included a sample document of this sort.
Trevis: What do you think of http://philip.greenspun.com/doc/chat as a simple example?
As an electrical engineer working on projects for large, commercial clients who demand preliminary and critical design reviews as a matter of course, I fully agree with the idea of design reviews for software; indeed, I hadn’t realized this wasn’t a universal practice.
You write, however, that “there are no published standards for how software development is to be done.” This is not true; there are many such standards, among them those issued by the IEEE and listed at:
http://standards.ieee.org/reading/ieee/std_public/description/se/.
It’s another matter altogether whether applying those standards and recommended practices actually increases software quality or reduces development time, cost, or risk.
Looks good! I am reminded also of: http://philip.greenspun.com/ancient-history/webmail/
In general, getting the data from the backend to the frontend is THE bottleneck, and that process needs to be kept simple and transparent. Because of the rise of APIs that abstract away the data-fetching part from the programmer (you’ve written about this elsewhere, I think?) it’s important to emphasise in your guide that such abstractions should be viewed with extreme suspicion in production systems. It’s almost always going to be better to fetch data in a big chunk with a simple “select * from … where …” and then pick it apart in the application, than to have multiple fetches at finer granularity.
And on a similar note, non-relational DBs (glorified key-value stores) are very hot these days, but people should remember that many man-years of optimisation efforts have gone into making RDBMSs fast. Not going with one should be a very carefully considered choice, not the default.
I must say this is a wonderful article!
Anonymous EE: I probably should have said “no published and agreed-upon standards”. Certainly there are plenty of textbooks on software engineering. Unfortunately there is a lot less agreement among those textbooks than you’d find in, say, physics or medical texts. I have worked with an excellent programmer who simply stated that comments were bad. They might get out of sync with the source code if the source were changed and therefore there shouldn’t be any comments. I came into another project where a Harvard graduate had been flailing away at the source code for a few years. He had perhaps changed 0.1% of the lines of code. Nonetheless, he was concerned that the docs were now out of sync with the code. So he deleted them. All of the docs that the original programmers had written. As programming is not a profession and there is no professional society enforcing standards, I don’t think I can find a document that would definitely prove either of these guys wrong (yet I believe them to be wrong, as a matter of personal opinion).
That original document really should have come from the ecommerce site owner. It should have said *what* was trying to be achieved. The programmer was remiss in not demanding it (presumably it was all verbally communicated) before doing any work. The programmer should then have documented *how* the requirements would be met.
The majority of your the rest of the document goes over what product managers do for a living (the good ones anyway). When a company sells products that is an obvious need. In this case where there is bespoke development neither party performed that function and it looks like neither party realised what was missing.
I wonder if it could be mostly automated? Imagine a web site where the client and programmer collaborate to produce these documents substantially aided by workflow based on templates (eg for a web site “which devices” and “which browsers” have to be answered). A small amount of expert human review finishes it up.
Roger: Now you’re reminding me of the commenters on the public school postings who say that the 15,000 hours of contact time that the government schools have with kids isn’t sufficient. It is the kids’ and parents’ responsibility to do all of the learning and teaching. We want to force the business people and the decision-makers to think and write precisely? If they have that level of skill and interest, why wouldn’t they send the whole project out to the low bidders on rentacoder.com?
@philg: I didn’t mean it that way. The difficulty in your example is that neither side knew what they were missing – ie they didn’t know what they didn’t know. (And in future cases how would others know about your software design review?)
Obviously hiring a person to perform the intermediary functions is expensive which is why I was wondering about a cheap but good way of doing it. And situations are never as unique as people believe, which is why it can be substantially automated. It is a hell of a lot easier for the decision makers to be guided through issues like scalability, compatibility, backups etc than to expect them to just come out with it.
The good thing about rentacoder is that they do force both parties to reconcile what it is that is trying to be achieved. Paying the cheapest bidder is a separate issue but your friend chose the programmer he did for some reason.
Hi Phil,
This is a great article, and manages to answer a comfortable mix of technical and business decisions, though I’d take exception with a single small point.
A small RDBMS probably does not need 10-20 disks, and indeed using a setup like this, while more performant, costs more operationally and is significantly more prone to failure (including by merely depending on a RAID controller).
While a new application is seeing low traffic, it would be much better to spend that extra cash on an entirely separate backup machine, rather than attempt to predict for capacity that may never be required (sod’s law dictates the backup will always be required).
David: Depends what you mean by “small”. If by “small” you mean something that can be handled comfortably by MySQL, then one spindle would be plenty. In the Oracle world, though, “small” could be 1 TB and 100 updates per second from 25 simultaneously connected clients. Keep in mind that the database hardware sold by Oracle holds 336 TB.. per rack (see http://www.oracle.com/us/corporate/press/033684 ). Anyway, if the data size is “tiny” and the number of transactions is just a few per second, the RDBMS could be run on a pizza box machine with software mirroring (eliminating the hardware RAID card altogether) and that decision documented…. The article is not trying to be a comprehensive DBA guide. It is trying to make sure that DBA decisions are written down.
Hi Philip:
You might have hand-waved past the most difficult problem a non-technical business person would face: how could he or she possibly decide whether a prospective reviewer was competent and compatible with the project?
I could see all manner of catastrophe occurring in the project if the reviewer was significantly less competent than the development team — or even if the reviewer simply subscribed to a different development religion. An ‘agile’ developer would be aghast at the level of documentation that you want up front. A ‘test-driven’ reviewer would drive a lisp programmer to distraction. And an unqualified reviewer who didn’t understand the language might not be satisfied until the code was filled with line by line comments describing every library call and common idiom. That last one would probably even sound like a great idea to the uninformed business-guy.
Of course, the non-technical business person has this same problem when trying to pick out a good developer. But now he or she would need to independently pick out two good developers.
Travis: The reviewer isn’t there to redo the work done by the developers. Nor is it his or her job to impose particular standards on the developers. The reviewer’s job is simply to make sure that the developers have done a complete engineering job.A COBOL programmer should be perfectly competent to review a project being done in Haskell.
The design review is not a religious discussion. It is going into the server room, seeing how many boxes are in the rack, and then making sure that there is a paragraph in the hosting document corresponding to each box.
Code review was used in our team in my last project. Meaning a Scrum-story was not accepted as done before it had been peer-reviewed by another team member. This also helped distribute information on different components inside the team.
Software design reviews are standard and required where regulations are involved such as software for devices regulated by the FDA or the FAA. The company had published procedures and the activity was required for the project to move along to production. Such procedures are necessary for ISO 9000 certification.
With that said, it was difficult to get the right set of developers who understood enough of the design being reviewed to find all of the problems. Certainly the management did not but there were roles in the review that they could handle like being the recorder. As these reviews were not an activity that was a lot of fun, we made it a tradition to have ice cream at the reviews.
There were occasions where outside consultants were used to review the software with regard to compliance to specific regulations such as 21 CFR Part 11 of the FDA that deals with the handling of electronic records.