Can we fix the software development process with innovative management?

One of the things that I did down in Silicon Valley was attend a lecture by Alan Cooper, notable for his authorship of the book About Face in 1995 and his criticism of exposing the hierarchical file system and the RAM/disk distinction to end-users.  Cooper’s latest project is “ending the death march”, figuring out how to fix the chronically broken process of software development.  Cooper starts from the proposition that software development projects are opaque to management, generally failures, and therefore management underfunds and understaffs them, hoping that, if failure is inevitable, at least it will be a cheaper failure.

Cooper wants to see interaction design, the software’s behavior, made more explicit in the design.  He notes that he has met two kinds of programmers in this world: (1) those who like to get stuff right, and (2) those who like to ship product and get it into users’ hands.  His proposal is “the Triad” in which three kinds of people are given equal status on the project:  Interaction Designers, Design Engineers, and Production Programmers.  The design engineers figure out how to get state of the art technology and algorithms into the product and they may write a lot of test code that will eventually be chucked.  The production programmers handle all of the edge cases and compatibility/installation/deinstallation concerns.  He presented a lot of graphs for CEO-types about measuring return on investment at various stages.  I didn’t understand these.

I personally enjoyed seeing Cooper’s elevation of interaction design.  In the course that we teach at MIT, we try to hammer into students the importance of planning the data model (what can be represented in between clicks and sessions) and page flow (the interaction design itself) before writing the pages.  For all too many systems, the interaction design is only specified implicitly in a huge pile of CGI scripts or whatever.

The question for commenters is “Do we believe that changing the management process can fix software development?”

Let’s look at some potential arguments against Cooper’s proposal.

First, it is unclear that the average software project is underfunded or understaffed.  The U.S. govenrment regularly throws $billions at seemingly fairly simple software projects that a handful of expert programmers ought to be able to do by themselves.  A typical project has lots of mediocre programmers on it whose contributions are negative.  In this sense, the project is clearly overstaffed because it would go faster if you fired 50 percent of the people.

Second, the fundamental tools and debuggers for software have changed but little in 50 years.  During that time, many tens of thousands of organizations have executed software development projects.  Among these tens of thousands of organizations, every conceivable management structure has already been tried.  If there were a silver bullet, it would already have been discovered by one of these companies, perhaps randomly, and then propagated to the world at large.

Third, when genuine progress has been made in software development productivity and software quality, it has come from improved tools such as SQL (a declarative language that replaced a lot of unreliable and unreadable imperative code).  Maybe the reason software development projects fail is that we’re using the same tools, the programmers have the same IQ as in decades past (or perhaps lower as the smart ones trot off to professional schools), and the requirements have become more complex.

Fourth, when you see a company whose products are better than competitors, it is usually because they have programmers with higher IQs, not a magic management technique.  Google has the smart people; the federal government (and its contractors) has the dullards.  Google gets innovative stuff out on time; the federal government ends up years behind schedule on a simple rewrite.

Fifth, over the life cycle of a software product, we already almost have the kind of structure and division of labor proposed by Cooper.  The programmers who like blazing new trails, albeit somewhat sloppily, show up on Day 1.  They solve the hardest algorithm problems and get a working v1.0 out the door.  By the time v3.0 is shipped, natural turnover within the development group has resulted in the code being worked on only by production programmers who like methodically sweating the details.

Some arguments in favor of Cooper’s proposal:

Managers of software development projects tend to be so incompetent and lacking in information that they can’t recognize and reward the strong contributors (this results in the best programmers getting paid only 20-30 percent more than the mediocre ones (compared to a 500 percent ratio in most areas of law, for example)).

Making the interaction design explicit makes it easy for decision-makers to evaluate whether programmers are accomplishing anything.

Full post, including comments

California quotes

A few random quotes from Californians…

“The state of software is such that we now have toilets that understand when a person is standing in front of them, but not computers.” [computer-human interaction expert]

“McAfee Coliseum” [sign on the enormous baseball stadium in Oakland; there are so many security holes in Microsoft Windows that you can earn enough money to sponsor a stadium by attempting to paper over the C programmers’ bugs]

“I picked the navigation system that I liked and then bought the car that came with it.” [computer-human interaction expert; car turns out to have been the Acura RL]

“The avocados are from the front yard; I just picked them up from the driveway.  We have another 100 in the garage if you want some.  The orange juice is from the back yard tree.” [Nobel Prize winning physicist and suburban farmer in Palo Alto]

“We could have airline service and even 747s on the 10,000′ runway at Moffett (old military field in Mountain View; very lightly used by the government now), but the City of San Jose spends a fortune lobbying against it so they can continue to collect fees at the San Jose airport.” [Flight instructor at the Palo Alto airport.]

“I’m going out to a company party.  Unfortunately most of the people there will be programmers.  They’re just the dullest people in the world.  It is even worse because I’m a woman and they are afraid to talk to women…. I hope that you’re not a programmer.” [Software marketing executive sitting next to me on American Airlines.]

Last night:  Good night and good luck with my friend Toby (aged 76, so she lived through the era).  We both enjoyed it.

Tonight:  Some sort of multi-media lecture by Stephen Hawking at Oakland’s Paramount Theater (my cousin got the tickets).

Tomorrow:  Hacker’s Conference in Santa Cruz.

Full post, including comments

A talk from BayCHI about SmartPhones plus dock

Here is a 51-minute audio clip of me talking about the SmartPhone + Dock >> PC idea.  (I think that the first portion of the MP3 has to do with BayCHI administration announcements.)  If you hear background noise, it is people walking out of the somewhat hot and stuffy room.  The speaker who preceded me was scheduled to talk for 30 minutes, but spent 1.5 hours going through a PowerPoint presentation.  Edward Tufte likes to remind people always to end early, but apparently not everyone heeds his advice regarding this or the evils of PowerPoint.

Normally I try to delete comments that fall into the “this sucked” or “this was great” category (on the grounds that other readers have already consumed the posting in question and aren’t interested in someone else’s opinion of its quality, but only in an alternative perspective on the same subject).  In this case, however, I’d appreciate some feedback on whether this kind of audio clip has value to readers/listeners.  If people love it, perhaps I’ll try to do more.

Full post, including comments