|
|
The Web Software Industrya look forward on October 23, 1997 by Philip Greenspun for Web Tools Review |
I wrote an entire book chapter on the future of Web services that will make money. I devoted various portions of an entire book to explaining what the state-of-the-1997-art software tools do. But that left one hole: What is the future of the Web server-side software industry per se?
My basic argument: the Web server-side software industry's development has and will continue to recapitulate the development of the business data processing software industry.
In the 1960s, people who needed to do business data processing bought
iron. They hired programmers to write what today we would call a
database management system. The same programmers would then build data
models and application programs to put data into and pull data out of
those data models.
In the 1970s, people would buy a commercial database management system plus some kind of iron to support it. They were tired of suffering with bugs in their programmer's ad hoc database management schemes and figured that their data storage needs weren't any different from hundreds of other computer users. Company programmers were used now only to write data models and application programs.
In the 1990s, people buy an enterprise software system such as SAP or Oracle Financials. They then buy an RDBMS to support it. They finally buy some iron to support the RDBMS. Company programmers are used only for some customization of the canned data models and apps.
Note that over these 40 years, there has been a huge transfer of power from iron vendors to data model/app vendors. Savvy companies realized this and adapted. IBM, for example, went heavily into the DBMS software business and then into business apps. Oracle has acquired its way into data models and apps. Thus not everyone noticed that what was being sold was radically different.
Given that business managers pride themselves on being innovative, why
the huge rush to have everyone using the same handful of enterprise
software systems? Possible Answer 1 is that computer science is
stagnating and programmers are consequently using primitive 1960s-style
tools to create complex systems. Possible Answer 2 is that managers have
figured out how to pay programmers so little that only stupid or lazy
people are willing to work these days as programmers. Possible Answer 3
is that business managers are in fact no more innovative than a herd of
sheep. They all read the same business literature. They all think
about purchasing and invoicing and payroll in exactly the same way. So
they'd no more write custom information systems for their company than
they would write a custom word processor.
Maybe it is a combination of all three. In any case, if your company handles payroll exactly the same way that Wombley's Widgets, Inc. does and they have a working program to do it, then you might as well use Wombley's software. If you hire programmers to build it from scratch, the best case is that you'll spend some money and get a working system. In the expected case, you'll spend a few years and $millions working through all the bugs that Wombley Widgets worked through 5 years before.
In the early days of the Web, publishers started with iron, usually a
Unix box. Then they'd hire a programmer who'd write a Perl/CGI script
that pulled data out of and stuffed data into a Unix file in a custom
format. By 1995, people had noticed that such programs tended to have a
lot of strange mutual exclusion bugs (I wrote about this in Chapter 11 of Database Backed Web
Sites). So they started building on top of a Unix box plus a
standard RDBMS like Oracle, moving to where business data processing
folks were in the 1970s. That's more or less where we are now in
October 1997 as I draft this document.
Could publishers all run off one data model? I've built about 50 RDBMS-backed Web sites now and have cut and pasted so much code among them that I'm convinced they can. Or if not one data model/app then a handful.
Take a look at photo.net, my personal on-line photography magazine. Every publisher of a magazine-like Web site will want at least the collaboration services you see there. photo.net has classified ads. It has a Q&A forum. It has a reader comment collection system that operates at the bottom of every article (look at the reader comments in the camera bags article for example). It has a database-backed contest system that lets the publisher collect demographic information on the readers. It lets readers add themselves to a mailing list. It has a magic-cookie based user tracking system that measures, among other things, clickthroughs.
Every publisher wants the same kind of back-end administration pages. They want to "find me readers who have entered at least one contest, commented on two articles, read all the articles about cars, looked at least four banner ads for cars, and clicked through on one of those ads." They want to "send email to all the people who said they were interested in new travel-related articles". They want to "show me the most effective and least effective banner ads in terms of percentage of clickthroughs".
Someone who wants an on-line magazine with at least these meager collaboration services and admin functions has to do the following:
Suppose that the new publisher makes all of these decisions correctly.
It will still take him six months and cost $500,000 to get to where
photo.net was in 1995. In fact, in terms of management attention,
hourly wages, and lost time-to-market, I think it would cost at least
$100,000 just to make the operating system/RDBMS/Web tool decision
(think about it: managers who don't know SQL will sit in meetings with
salesmen who don't know SQL, trying to figure out whether Oracle,
Informix, Sybase, or DB2 is best).
Most publishers will not make these decisions correctly. In fact, most won't even choose a scalable raw infrastructure of OS/RDBMS/HTTPD (see my diatribe against junkware in Chapter 10 of Database Backed Web Sites). If they somehow randomly succeed in picking a decent infrastructure then surely they will overlook some things at the data modeling stage. And they definitely will come to grief with their user interface and have to hire a bunch more people to support customers until they figure out how to make the site's forms 99% self-evident as I've done with photo.net (since the site is noncommercial I can't hire anyone to support users).
Bottom line: a fortune will be spent on programming; schedules will slip; the users will get reamed by all the bugs. Just like any other custom software development effort.
| Described by the engineers who built the software | Described by the marketing department |
|---|---|
| Here's a collection of hacks that we've assembled after building data processing systems for 15 companies. We're sorry that we never really finished it and that it doesn't do everything that you need and that it will take 50 programmer-years to fill in the cracks and make it work for your business. But when you're done you'll probably have fewer bugs than if you'd started from scratch. | This is a comprehensive turnkey business data processing system already in use by 15 large and sophisticated companies. It does absolutely everything you need and is so flexibly designed that it will only take you 50 programmer-years to customize for your unique business practices |
Whose description is accurate? They both are. Which description do you think results in $35 million software license revenue plus another $100 million for consulting?
In fact, some friends of mine and I are actually doing this ourselves. We're packaging up the software that we've used to build 60 RDBMS-backed Web sites and calling it "the ArsDigita Web Publishing System". Do I think it will dominate the industry and crush all other Web technology vendors flatter than the bugs that they are? No. This market isn't like the word processing or operating system market where only one product tends to prevail. Just as MIT chose SAP for its data processing software and Harvard University chose Oracle Financials, there will be people whose publishing philosophy is more in line with the photo.net data models and apps and other publishers who philosophy is closer to another packaged Web publishing system. Moreover, since each installation takes some time and effort to customize, nobody is big and resourceful enough to serve all the potential customers.
What about people whose sites don't look anything like an on-line magazine? Suppose you're a manufacturing company and want to present a collaborative face to the world. Well, then just take the system that I built for MIT Press. It serves a catalog of products, keyed with unique ids. Users can do full text searches against a variety of data. Users can browse by category. Users can record their experience with the products (comments in the case of MIT Press books). Users can record their desire to be placed on a mailing list to be apprised of updates. Users can order products. The MIT Press folks have a comprehensive set of backend pages for maintaining the catalog, figuring out which products are selling best, sending email notifications to groups of readers, etc. Your widgets might not look exactly like MIT Press's books but fundamentally isn't there a lot of commonality?
What about an on-line travel booking system? Uh... well I guess... um.. yeah... we could adapt it... I mean, it's just some Tcl and SQL scripting... anyway... Well, I guess we can't solve everyone's problem.
Sometimes there is no way to avoid from-scratch programming. A lot of Web sites are really moderately sophisticated computer programs with Web interfaces. Nobody has solved the automatic programming problem and I don't think it is going to be solved because people are tired of writing Web API scripts.
Here are the steps:
This article makes a great deal of sense. I notice that Netscape LiveWire is not mentioned as a possible solution. Based on my one-year attempt to get LiveWire to work, I would not argue for including them.In fact, I would think that Netscape should give away their server source code--while they still can. If enough people have access to the source code to fix it, they may be able to compete in the business that you describe.
-- Arnold Kling, December 2, 1997