Chapter 1: Envisioning a Site That Won't Be Featured In suck.comby Philip Greenspun, part of Database-backed Web Sites
Note: this chapter has been superseded by its equivalent in the new edition
This is a book about how to be a nerd. A book about how to make a Web server on a cheap computer serve a million hits every day. About how to write clever programs that collect data from Web users and stuff them into a relational database. About how to add thousands of images to your Web site in a few minutes. About how to get serious about user interface design for Web sites.
Before ascending into Nerd Heaven, though, I thought it was worth two chapters to step back and ask, "Why did the world buy 10 million expensive computers and connect them together?" Was it really so that a kid in Botswana could look at a flashing GIF 89a logo from a Honda repair shop in Sunnyvale, California? And, if we don't think that is true, how should we design sites?
An alternative formulation of this question is "How can I design a site that won't be featured at http://suck.com?"
When you put up a Web site, you are publishing. Virtually all of the important decisions that you must make are publishing decisions. Eventually you will have to select technology to support those decisions, but that is a detail.
Start by putting yourself in your users' shoes. Why are they coming to your site? If you look at most Web sites, you'd presume that the answer is "User is extremely bored and wishes to stare at a blank screen for several minutes while a flashing icon loads, then stare at the flashing icon for a few more minutes." Academic computer scientists refer to this process of fitting software systems to people as "user modeling."
Slightly more content-rich sites are based on the user model of "User wants to look at product brochures" or "User wants to look at fancy graphics." After pulling the server logs for the sites that reflect these user models, though, it is tough to have much faith in them.
Think about it for a minute: If a user wanted a flashing computer screen and confusing user interface, he could stuff a CD-ROM into the drive. He could get an even more enticing show without the crummy user interface by picking up his television remote control and flipping channels. If a user wanted product brochures, he could get them by calling manufacturers or visiting shops. If a user wanted fancy graphics, he could flip through dozens of pages' worth in a print magazine in the amount of time it would take to load a single corporate Web page.
Users come to Web sites because they have questions. They are not bored losers. You are not doing them a favor by putting a product brochure online or showing them some huge logo GIF. They are doing you a favor by visiting your Web site. They are paying to visit your Web site, if not exactly with money then at least with their time. You have to give them something of value or they will never come back.
If you can anticipate user questions and make sure that your site answers them, then you will be a successful Web publisher.
Suppose that you are building a humble personal home page. Why would a user come to your page? Someone might be a trying to contact you or send you a package. So you obviously need to include your phone number, mailing address, and perhaps fax number. Perhaps you've invited some people to your house for a party and they don't know how to get there; they can't call you because you're out shopping for bagels. So you should have a map to your house (I just made my home address a hyperlink to the Yahoo map server).
Imagine that a friend of yours is at a party talking to Dale, a beautiful member of whatever sex you happen to fancy. Your friend is describing your charity, great humanity, and kindness to animals. Dale, however, won't agree to meet you without seeing a photo first. So you'd better hope that there is a portrait of you somewhere on your home page before Dale and your friend stroll over to the WebTV.
Or imagine that you've given an interview to a reporter on deadline. It is 2 a.m. when the reporter realizes he forgot to ask you for some background biographical information. You'll be getting a wake-up call unless you remembered to put a copy of your resume online.
At this point, some people might object that this information is too, well, personal for a personal home page. The Internet frightens them. Sure, their phone number is listed, and the price they paid for their house is public information. And their credit record is open to almost anyone who cares. But they think that if their name is known to an Internet search engine, suddenly all of the privacy that they supposedly formerly had will evaporate. Well, my home phone number has been available via the Internet for 20 years, and my Web site gets about 500,000 hits a day. I include a picture of myself naked with my old dog George and maps to my house and office on my site. And just as I typed that last period, sitting in what I thought was the privacy of my own home, someone I didn't know called me up. At 8:20 p.m. on a Monday. Would I make a donation to the March of Dimes?
Probably about 1 percent of my unsolicited phone calls are from readers of my Web site. Surprisingly enough, people who find me on the Internet seem to send e-mail instead. Go figure.
Suppose instead that you make cameras for consumers. Why would a user come to your page? Possibly to look at brochures, so it might be nice to have some advertising literature on your site. But the most important users are existing customers, such as someone who is traveling with a fancy Nikon single-lens reflex and has forgotten how to work the flash. Caught without the owner's manual, this customer has surfed in from an Internet café hoping to find the full text online.
Someone with a broken camera will want to know how to get it fixed and how much it will cost. So you'll want to put up maps to your service centers, warranty details, and prices for out-of-warranty repairs. Your site is a natural focal point for customers to meet, interact, and share their experiences with each other. So you should have moderated Q&A forums, bulletin boards, and classified ads where your customers can contribute.
Everything that I said about the camera manufacturer goes for the software company as well. However, the camera maker is under no obligation to demonstrate an understanding of the Internet. But if you are in the computer business, you can't afford to have your customers say "My 10-year-old built a better Web site than that last week."
All of your documentation must be online and current. Demonstration programs, ideally running on the server or as Java applets, should be available. Customers ought to be able to purchase and download everything that you sell. If you can't use the Internet effectively, then why would anyone believe that you are capable of doing anything with computers effectively?
You can objectively measure how well your site meets user needs. Are you getting a lot of e-mail questions? As soon as two users ask the same question, you should beef up your site to answer it unattended: Build a page, fix the navigation model, or tune your search engine.
Since you are a competent webmaster, you are running a full-text search engine on your site. Why not hack the CGI scripts so that the search engine sends you e-mail when a query results in zero matches. Given information about which user queries are failing, you can add content or keywords as appropriate.
At this point you can relax. You aren't a loser with a big budget, a lot of ugly graphics, and no traffic. Users are not leaving your site in frustration, shaking their heads, and saying "They just don't get it." Now that you are safe from the wags at suck.com, what can you do to make your site fulfill its potential?
Look up "Bomarzo" in the Michelin Green Guide to Italy: "Extending below the town is the park of the 16th-century Villa Orsini (Parco dei Mostri) which is a Mannerist creation with a series of fantastically shaped sculptures." Compare that description to the photos in Plates 1.1 and 1.2 (in the color plate section of this book), showing just a tiny portion of the Parco dei Mostri ("Park of Monsters"). Do these not suggest a somewhat richer place than the sentence in the Michelin guide? Yet the sight of a tourist slavishly following the Michelin guide is a commonplace. Something really fascinating and unexpected is happening in front of him, but he has his nose buried in the guide, trying to figure out what the next official point of interest is. The tourist is literate. Not literate in the "I read Classics at Oxford" sense, but literate in the "knowledge is closed" sense. Everything about Italy can fit into a book. Perhaps the 350 pages of the Green Guide aren't enough, but some quantity of writers and pages would suffice to encapsulate everything worth knowing about Italy.
Plates 1.1 and 1.2
Parco dei Mostri, 1.5 hours north of Rome, described in the "Become Illiterate" section of Chapter 1 (see http://webtravel.org/italy/ for 500 more of my photographs of this country)
Oral cultures do not share this belief. Knowledge is open-ended. People may hold differing opinions without one person being wrong. There is not necessarily one truth; there may be many truths. Though he didn't grow up in an oral culture, Shakespeare knew this. Watch Troilus and Cressida and its five perspectives on the nature of a woman's love. Try to figure out which perspective Shakespeare thinks is correct.
Feminists, chauvinists, warmongers, pacifists, Jew-haters, inclusivists, cautious people, heedless people, misers, doctors, medical malpractice lawyers, atheists, and the pious are all able to quote Shakespeare in support of their beliefs. That's because Shakespeare uses the multiple characters in each of his plays to show his culture's multiple truths.
In the 400 years since Shakespeare we've become much more literate. There is usually one dominant truth. Sometimes this is because we've truly figured something out. It is kind of tough to argue that a physics textbook on Newtonian mechanics should be an open-ended discussion. Yet even in the natural sciences, one can find many examples where the culture of literacy distorts discourse.
If you were able to stay awake long enough to read through an academic journal for taxonomic botanists, you'd learn that not all botanists agree on whether Specimen 947 collected from a particular field in Montana is a member of species X or species Y. But you'd see quite clearly that everyone publishing in the journal agreed on the taxonomy, i.e., on how to build a categorization tree for the various species.
I learned all of this interesting stuff about taxonomic botanists from Peter Nürnberg; see his paper at ftp://bush.cs.tamu.edu/pub/publications/papers/96.webnet/96.webnet.html
However, if you were able to stay awake long enough to get through a cocktail party in a university's department of botany, you'd discover that even this agreement is illusory. There is widespread disagreement on what is the correct taxonomy. Hardly anyone believes that the taxonomy used in journals is correct but they have to stick with it for publication because otherwise older journal articles would be rendered incomprehensible. Taxonomic botany based on an oral culture and/or a computer system capable of showing multiple views would look completely different.
Open today's New York Times. A Republican politician is arguing for relaxed regulations on widgets. A Democrat is quoted arguing in favor of tightened regulations. There is a vote; widget regulations are tightened. On the surface, it looks like multiple perspectives. Yet at a cocktail party, your friend Sue might argue that the government shouldn't be regulating widgets at all. Joe would probably interrupt her to say that widget regulation is a canard; we really ought to talk about flag burning. Dana would bring up the Simpsons episode where Bart went on a school field trip to the widget factory. Alan would tell his grandfather's widget factory stories from World War II. Elizabeth would talk about how she was surprised to see that they had no widgets in New Zealand and apparently did not miss them. Compared to the texture of the cocktail party, the New York Times article sounds like two rich old white guys saying more or less the same thing.
Some people like a one-truth world. If you have a huge advertising and PR budget then you can control your public image very effectively in a literate world. Ford Motor Company has enough money to remind you 2,000 times a year that "Quality is Job One;" unless you lost a friend in a Pinto gas tank explosion, you probably will eventually come to agree. Microsoft via the genius of Bill Gates invented the mouse-windows user interface, reliable operating systems, affordable computing, and the Internet; if you don't think all that is true, ask someone who has never used a computer and whose only exposure to the industry is through mass media.
Perhaps it is because I'm a few billion dollars short of the necessary funds to create a one-truth world of my own, but I think the greatest artistic achievements hold the mirror up to a multiple-truth life.
The Internet and computers, used competently and creatively, make it much easier and cheaper to collect and present multiple truths than in the old world of print, telephone, and snail mail. Multiple-truth Web sites are much more interesting than single-truth Web sites and therefore will get a lot more traffic. For example, the car manufacturers' sites are mostly collections of product brochures tarted up with flashing graphics. They get minimal traffic compared to the plain text rec.auto.* newsgroups, which present the real experience of car owners from around the world. The newsgroups don't have pictures, animation, sound, or video clips. But they have multiple truths.
OK, enough philosophy and Shakespeare. This is the point in the infomercial where the guy wearing the CAT Diesel cap asks, "Do I need a college education to build one of these here multiple-truth Web sites?" The answer is no.
You need magnet content to attract readers and their comments. The magnet content for my Costa Rica site (http://webtravel.org/cr) is a 50-page personal story, 200 photos, and a long excerpt from a guidebook. Every .html file has a link to my comment server, but nobody would come to the site in the first place if I hadn't done a little work myself.
General Electric (http://www.ge.com) is a rare good example of a corporate site that takes the Web as Distribution Medium seriously.. They have brochures, yes, but also owner's manuals and installation guides for every GE product. If you moved into an apartment with a GE appliance but the previous tenants did not leave you the instructions, you can grab a PDF file from the GE site and print it.
You don't have to be a traditional publisher per se to benefit from collaboration tools. Manufacturers can collect and redistribute consumer comments about dealers who carry their products. Academic researchers can collect comments, ideas, and questions sparked by their writings.
Does this sound like too much work? If your site is commercial and you claim to be a smart businessperson, then you ought to be able to figure out a way to interact with users more cheaply over the Internet than via 800 numbers. If you are a writer or a photographer who has built a non-commercial site, then disintermediating the publisher means total artistic freedom. User feedback might be annoying at times, but if you want to be insulated from your readers, why publish on the Web at all?
Figure 1-1: the question and answer forum in http://photo.net/photo, which contains more than 1500 archived messages. Answers are e-mailed automatically to the user who posted the question, even months later. Users can choose to be notified by e-mail of new messages instantly, daily, or weekly. (Note: if you'd like to run a similar forum on your own site, just visit http://webtools.com/wtr/ and fill out a form. The software is free and you can even use one of my Web/RDBMS servers for free.)
Figure 1-2: the Loquacious comment server collecting reader responses to my Berlin/Prague story (http://webtravel.org/bp/). The original 30-page story and 80 photographs attracted a reader-contributed story about the Terezin concentration camp. That story in turn attracted the comments that you see here (http://db.photo.net/com/philg/bp/terezin.html). If you'd like to use the Loquacious system for your own Web site, just visit http://webtools.com/wtr/. It is free.
A site that exists primarily to tease people off the Internet into buying something will almost always cry out to be ignored. Web sites should stand alone.
Joe Greedy puts up a site showing the cover of his book and a headline: "Buy me for $17." You'd think that this would be the lamest possible Web site. But Mr. Greedy manages to earn extra suck points by making sure that the only ordering option is by telephone. That way the modem crowd will be forced to write down the number on a Post-It and then disconnect before they can order their pile of processed tree carcass.
Jane Clever puts the full text of her book online. With several hundred pages of text instead of one, her content will be several hundred times more likely to attract users of search engines. By enabling online ordering for those dead-tree huggers, Ms. Clever will sell far more copies of her book than Mr. Greedy. Also, with 100,000 hits per day, Ms. Clever can sell ads on her site and links to other Web publishers.
Another kind of site that almost always disappoints is the adjunct to the physical event. Pathfinder did a companion to a traveling museum show by well-known photographers such as Annie Liebowitz. The biggest available pictures on the Web site were tiny, occupying about 1/100th of my 20-inch screen. The editor of the site asked me to add a link from photo.net to the Pathfinder site. I refused, saying that my readers were accustomed to filling their screens with the 500x750 pixel and 1000x1500 pixel images on my site. They would just get frustrated squinting at the tiny Pathfinder images. He replied that the museum show photographers were concerned about copyright infringement. I said "Well, maybe they shouldn't be on the Net. They are getting plenty of promotion in bookstores, museums, magazines. Why don't you find some less known photographers for whom the Web is primary?"
In October 1995, the MIT Media Lab threw itself a tenth anniversary party. I skipped it so that I could take snapshots of colored trees in Vermont, but my friends said that the physical event was fabulous: little gifts for everyone, cleverly packaged; a Photomosaic poster by Rob Silvers; women in plastic pants. As an adjunct to the physical event, the Media Lab was going to create the best Web site ever. They got NYNEX to bring in a 45-Mbps T3 network connection; you wouldn't want millions of users to get slowed down working their way through MIT's 100-Mbps backbone. They got Hewlett-Packard to donate a huge pile of multiprocessor machines with disk arrays. They hired expert consultants to plug all the computers in and hook them up to the network. They hired professionals to do graphic design and site layout.
After getting back from foliage country, I visited http://www.1010.org with high hopes. There wasn't any magnet content. Nobody had bothered to write stories or take pictures. Every day a Media Lab editor posed a question and then sat back to watch a USENET-style discussion evolve. There were only a handful of postings in each area. One user had contributed a smiley face. Colon dash right-paren. That was his entire message. This didn't really shock me until I noticed that on a scale from 1 to 7, this post had been rated 4.3 by other users. Yes, several other users had taken the trouble to rate this three-character posting. When the 10-day Web event was over, the massive disk arrays held almost enough user-contributed data to fill two 3.5 inch floppies.
How could so much money and hardware have resulted in a site that an elementary school would have been embarrassed to make public? The Web site wasn't primary. The party was the important thing. The Media Lab, one of the last groups at MIT to put up a Web site, had no real Internet culture. They wanted to reach corporate managers and bring them into the physical space of the Lab. Since corporate managers didn't tend to have a TCP/IP connection on their desktop, it wasn't obvious how Internet would be useful in attaining the Media Lab's goals.
Don't try to visit http://www.1010.org. The Media Lab eventually got Web-savvy enough to realize the embarrassment value of such a site and pulled the plug on the server.
Money is nice. Bandwidth is nice. Graphic design is occasionally nice. But your Web site will suck if you see it as a pimple on the butt of something much larger.
The stuff in this chapter isn't meant to be true. I'm not a great writer like Shakespeare (yes, I know that I could write this observation up for Duh magazine; you don't have to send e-mail). I'm unable to transcend the medium of the printed page and present multiple truths. My hope is that I have provoked you into doing the following:
This chapter was about people talking to computers and people talking to each other with a computer mediator. In the next chapter, we'll talk about how computers talking to each other could finally deliver all of the benefits that the Computer Age was supposed to have conferred on us (and, if you insist, how you can make a buck off of it).
Note: If you like this chapter you can move on to Chapter 2.
God I loved your article! Here's a few words on why....
I teach a "Taming the Electronic Frontier" course via distance education (internet+TV). One of the first assignments is "Here's how to write HTML. Use it to build a digital portfolio that delivers quality to your readers".
And frankly, I got what I deserved. Crap. Even using Pirsig's "Zen and the Art of Motorcycle Maintenance" as the textbook to teach about quality, I still got crap.
Eventually I got (most of) them to pay attention to quality by assigning peer assessment exercises in which everyone reviewed and RATED everyone's work according to quality delivered to readers.
I loved this article because it teaches something that the world really doesn't know, judging from my students (and 99% of the web as a whole). The biggest complaint from my students is that I never TOLD them that quality was determined by their readers, not by them. Big secret I was expected to "teach" in class. Funny... I always thought it was common sense... until I taught this class.
Next time I hope to get them off on a better foot by assigning them your article at the beginning? May I? Should be about 200 students this time.
If you're interested the old (spring 1997) version of the course is at http://www.virtualschool.edu/97a. I'll be building the fall 1997 version later this summer at http://www.virtualschool.edu/97c. Drop me a line and say what you think. firstname.lastname@example.org
PS: Could you please add a "wrap=virtual" tag to these textarea boxes? I hate typing carriage returns at the end of each line; wrap=virtual fixes this.
-- Brad Cox, May 22, 1997
OK, so I'm inspired to build a "humble personal homepage" with no intention of making money. I use your model and include all the personal contact information, including a link to a map to my house. So far, this all makes sense to me. Next, I might add a page about my extensive music (yuppie scum CDs) collection, with my ratings and some favorite links. Then, I'd add a page about photography with some images and links. If I'm really feeling adventurous, I might add a page on competition handgunning.
Have I not just made my home address a tempting target for nefarious folk? Here, this is where I live, and this is all the cool stuff I have to steal? By the way, I included information about where I work, so you can deduce when I won't be home.
How do you get around that? While your Stearns and Foster is probably safe, I fear for your Linn Sondeck!
-- Todd Peach, January 20, 1998
I keep coming back to your pages for whatever reason. Today I was doing some research on Web page redirection and I got -- duh -- "redirected" to your "Envisioning a Site That Won't Be Featured In suck.com" write-up. (I hope you and Flanders are on speaking terms, but that's another site...) How is it possible that one guy can focus so much wisdom about the WWW, design, and content into one place? Conversely, how can site managers be so completely numb between the ears when it comes to site design?
I am a hardware/software consultant working in embedded electronics. I do a lot of computer-controlled machines like medical diagnostic and treatment devices, motion control, process automation, etc. The engineering managers I deal with are not always as Internet technically hip as I would like to see. Oh, well. As bad as that may be, they are geniuses compared to the project management staff at most of my clients' locations. I strive to make people aware of what a powerful tool an intranet server can be. Some of these guys out there are using things like document automation, version control, and distributed engineering to get actual work done through a server. This can be a very cool deal if done correctly. Just the simplest pages with links to file server locations for files can solve a raft of problems. How many exchanges like this do we need?
"Where did you put that document?"
"Oh, it's up on LAMBDA06 in the /test/doc directory."
"What was the file name?"
"TEST06BA.tmp, I think."
"What format is that in?"
"I don't remember."
How about if the file is linked into a page that has a bunch of other supporting documents listed on it? Then the poor fool trying to find the info just goes to one place to look for this stuff. The variations are obvious.
Well, all that is great, but some folks just insist on adding fluffy graphics -- even sound -- to pages that are supposed to be all business. I'm still trying to figure out how to convince the WWW-naive how to create content without creating the Wait portion of WWW. They are in love with glitz. I plan to use your book as a tool for changing that. If they won't read it then I can use it over the muzzle to keep the noise down as I take drastic measures.
-- Marty Schrader, October 26, 2000
Some Colour Phases of Black Bears at Riding Mountain
Very informative. I love your sense of humour!
-- Dr. Jim Irwin, February 27, 2001
Thanks for the informative site.
-- Mike McCracken, July 13, 2001