Massachusetts Institute of Technology
Department Electrical Engineering and Computer Science

6.916: Software Engineering of Innovative Web Services
Problem set 2

Reading for this week:

Online assistance: Problem Set 2 Q&A forum

Objectives: Students should learn

In the first problem set, you learned how to use lots of new software tools. Mostly you needed to learn commands and procedure calls. You didn't care about the internals of Unix, Oracle, AOLserver, or the Tcl interpreter. In this second problem set, you will learn how to extend a library of source code. Thus, you'll have to learn about the internal structure of the toolkit. If you thought looking at your own source code was unpleasant, here's your chance to look at 200,000 lines of someone else's :-)

The Big Picture

Most software development projects start with a dialog between the users and the application programmers (you guys). From this dialog, the application programmers form a big picture of what needs to be done.
Users: We're the world's leading experts on computers. We have a big building full of researchers, hundreds of advanced computer systems, and a handful of conference rooms. We'd like a Web-based system for scheduling conference rooms.
You: How are you doing it now?
Users: Secretaries, pencils, and paper appointment books.
You: Ah. So how do you envision the computer-based system working?
Users: Jane Nerd comes to the server and asks for a room from Time X to Time Y, capable of holding N people. The server offers Jane a choice of rooms, each one accompanied by a description of the AV facilities, and she can pick whichever she likes the best.
You: Anything else special?
Users: There is a set of Uber-Users. If Alice Uber can't find a room that she likes, we want her to be able to boot some grunts out of their reserved spot. Alice should not be able to preempt another Uber-User, though. The booted grunt should get an email alert from the server saying "Alice booted you; come back and pick another room."
You: So any user can reserve any room?
Users: No. Most of the conference rooms are reservable by anyone. However, some rooms require approval of reservations by a designated administrator (who can be any user in the community), unless the reservation was requested by an Uber-User. The designated administrator is notified by email any time that a reservation is requested for a particular room, and should be able to approve or deny the request by clicking on one of two encoded URLs in the email message.

The Big (Software) Picture

You're building a program that keeps track of users, groups of users, rooms, and room reservations. You could build this program from scratch, but we already have a documented debugged toolkit, the ArsDigita Community System, designed to support communities of users, subsets of which are collected into groups. You can probably get your program finished faster if you start with the toolkit, but more importantly you will achieve higher reliability (e.g., the user registration stuff has been debugged already) and higher maintainability (because the user and user groups portions of the software are documented in a book chapter residing at a permanent URL: http://photo.net/wtr/thebook/community.html).

Some of what you'll be doing in this problem set is creating RDBMS tables from scratch and writing Tcl procedures to perform transactions on those tables, just as you did in problem set 1. Much of what you're doing here, though, is figuring out clean ways of using and extending the toolkit. Sometimes this is as simple as visiting the toolkit admin pages (/admin) of your server and using Web forms to add user groups.

Exercise 0: Installing the Arsdigita Community System

The first thing you need to do is install the ACS. It is the base of code from which you will be building your service. Here are the steps for you to follow: Read the installation instructions. (Which should also now be available in your /web/studentXX/www/doc/ directory.) Your AOLServers have already been configured so you can start at "Feeding Oracle the Data Model," and skip the "Configuring AOLServer" section. Remember to run the "load-geo-tables" script before loading the ACS data model.

Exercise 0.5: Give your server a personality

The "parameters" subdirectory contains server personality constants in a file called student**.ini. This has been customized with your server's name (e.g., "yourservername" will be changed to "student51"). You will also find the original ad.ini in your parameters file. You can use this for reference but if you want to change the behavior of your server, you must edit server**.ini.

One horrible bug with AOLserver (at least in 2.3.2): If there are Emacs backup files in the /parameters directory that start with #, you might find that your AOLserver is unable to start. If all seems hopeless, do a m-x Dired on the /parameters directory and look for backup files to delete.

You should consider at least setting the system name, the publisher name, the email address of the system owner to your actual personal email address. For example, suppose you set the first parameter (SystemName) to "Joan's Scholarly Center" and then restart your server (with restart-aolserver student** from the shell). Subsequently, you'll see many pages of your site display "Joan's Scholarly Center", including the very top-level page.

Exercise 0.7: Become your own best friend

You now have an installed ArsDigita Community System but there aren't any users. Visit the top-level page on your site (e.g., http://lcsweb51.lcs.mit.edu/ ) and note that you're asked to log in. Type your standard personal email address and register as a user.

Exercise 0.8: Notice that your server is self-documenting

Use a Web browser to connect to the /doc subdirectory of your server and notice that you have a complete set of ArsDigita Community System documentation. You might find it useful to read the installation instructions and you will certainly want to read the developer's guide (developers.html). The latter starts off with an admonition to use proc_doc instead of proc when defining externally called procedures. Please follow this convention for the rest of the semester.

Exercise 0.9: Create some users

As noted in the installation instructions, you'll want to add yourself as a user, log in as the "system" user, add yourself to the site administrators user group, and change the passwords of system and anonymous users.

Repeatedly visit the login pages to create a bunch of users for your community, i.e., pretend you are "billg@microsoft.com", "president@whitehouse.gov", "wile_e_coyote@iq207.supergenius.com" and fill out the registration form. Remember that these will be the folks who are reserving conference rooms. Make sure that at least a couple of them have real email addresses because you want to be sure that you can test your email alerts. If you don't have any cooperative friends, you might wish to make yourself a hotmail or yahoo email account. Sadly, you can't make several users with the same email address because we constrain the email column in the users table to be unique.

Exercise 0.95: Create a user group

Read the documentation for the permissions package at /doc/permissions.html. You're going to use this package to figure out if someone is an uber user or not.

Using the Web forms in /admin/ug/ create a new user group of type "Administration". Call it "Reservation System Uber Users". The module field should be "reserve" and leave the submodule field blank.

This is the only user group that you will need to complete the pset. Using the admin forms, place one or two users in this group. They don't need to have any special role within the group, simple membership is fine.

Exercise 1: Build the data model

Using the file naming and placement conventions set forth in http://software.arsdigita.com/www/doc/custom.html, create a SQL data model file to hold new table definitions for your room reservation system. Since part of the naming convention is that your data model file must be called "module-name.sql", you have to come up with a name for your module. In order to make life easier for us in looking over your shoulder, please refrain from being creative and call your module "reserve".

A good thing to do now is to step away from your computer with hardcopies of this problem set and http://photo.net/sql/data-modeling.html. Read the problem set cover-to-cover before writing anything down. Otherwise, halfway through the pset, you might find yourself having to back up, alter tables, rewrite .tcl scripts, and rewrite documentation.

Here are some guidelines for your data model. Please note that this is not intended to be a complete listing of all the necessary steps.

Now that you're written all of your create table statements and put them in a file named "/doc/sql/reserve.sql", feed the definitions to Oracle. If you don't want to cut and paste into SQL*Plus, you can use Unix shell I/O redirection:
sqlplus / < reserve.sql
If you slept through 6.170 and didn't learn how to write bug-free code, you might find yourself running this command repeatedly. Although Oracle provides powerful transaction management, data definition language statements such as create table can not be rolled back. So if your file defines three tables and there is an error in the definition of the second table, you will usually end up succeeding in defining two tables with the preceding command. You can go into Emacs and edit reserve.sql and then rerun the shell command. Oracle will ignore the first and third table definitions since the tables already exist and define only your second table.

You can check your work by using the SQL*Plus "describe" command, e.g., "describe users" or "describe rooms".

Exercise 2: Set down a list of legal transactions

Write a design spec for your system in a file named in accordance with http://software.arsdigita.com/www/doc/custom.html, i.e., it should probably go in /doc/reserve.html.

Include in this file a list of the legal transactions in your system. For each transaction, sketch the SQL and note any external implications, e.g., "administrator is sent email alert".

As an example of a documented module, visit http://www.arsdigita.com/doc/glassroom.html (note that we neglected to carefully document the transactions to the tables; try to do a better job than we did).

Exercise 3a: Get the basic admin pages done

We're not going to worry about being able to edit room names or designated room administrators for the moment. Instead, let's build the simplest possible system that enables the site owner to create rooms: Connect to your admin directory, e.g. http://lcswebXX.lcs.mit.edu/admin/reserve. Create at least five rooms.

Note: By placing these pages under the /admin directory, you've restricted them to use by the site owner. This is acceptable because we presume adding a room to be an infrequent activity. Therefore, it isn't worth building user submission pages for new rooms and admin approval pages for those submissions.

Exercise 3b: Write the basic user pages

We're not going to worry about uber-users or room approval for the moment. Instead, let's build a basic system:

How could there be a time conflict given that the preceding page only showed free rooms? Here's the concurrency situation which we're worried about:

Olin Egghead says that he needs a room for 8 people at 3:00 pm tomorrow, for a 45-minute meeting.  
Your module serves Olin the new.tcl page, showing that the Boesky Room is available. Dorothy Alvelda says that she needs a room for 5 people tomorrow at 3:30 pm.
Olin is about to confirm the Boesky Room when the phone rings. A Wall Street firm is offering him $250,000 to become a C programmer in their commodities trading division. Olin immediately replies that he wouldn't consider abandoning his sacred academic principles for a minute.

An hour later, Olin is still on the phone.

Since Olin has not yet booked any rooms, new.tcl shows Dorothy that the Boesky Room is among the available rooms at 3:30 tomorrow.

Dorothy confirms the Boesky Room and new-2.tcl inserts a row into the database.

Olin hangs up and gets preoccupied with copying impressive-looking equations from the pizza delivery section of the Thessaloniki Yellow Pages into his latest research paper.  
Olin remembers that he needs that room for tomorrow and unburies his browser window, but does not reload new.tcl (i.e., he is looking at the page generated a couple of hours ago). Oh yes, the Boesky Room will be fine. Olin clicks to reserve the Boesky Room at 3:00 pm tomorrow.  
If you want to act like a professional Web programmer, you therefore have to run the same query in new-2.tcl that you ran in new.tcl to make sure that the room is in fact still free.

What do the users get from their professionally programmed systems? Double booked rooms. The average professional Web programmer probably won't handle the case where two users submit room requests at precisely the same second.

Hold yourself to a higher standard by making new-2.tcl lock the reservations table before querying to confirm room availability. Read the transactions chapter of SQL for Web Nerds (http://photo.net/sql/) to find out how to lock tables.

Making it Real

We've got the rudiments. The site owner can define rooms. Users can book rooms. Reservations won't conflict. If you were a West Coast software company, you'd declare victory at this point. Issue some press releases, ship the product in a nice-looking shrink-wrapped box (as Release 3.2), and sell some shares to the public.

One of the ugly facets of offering a Web service, though, is that users will hammer you with email until you make the thing real.

Exercise 4: Finish the admin pages

Here's a step-by-step plan:

Exercise 5: Finish the user pages

You now have a working system as envisioned by your clients. Congratulations.

Exercise 6: Making it Pay

If you're not doing this on an ArsDigita-maintained machine, you'll probably find it more trouble than it is worth to install the CyberCash software on your machine (painful), install our CyberCash interface from AOLserver (annoying), and get a merchant credit card account with CyberCash capability (takes 6 weeks and costs around $1000). You can still do the exercise, though, by using Eve Andersson's CyberCash emulator, available from http://www.arsdigita.com/free-tools/cybercash-emulator.txt.
Occasionally folks in Academia ask themselves "What is there that distinguishes us from McDonalds, Microsoft, and Oracle Corporation, aside from a lower growth rate and smaller revenues?" If the answer turns out to be "not much" then why not apply commercial logic to the conference room scheduler?

Assume that the admin reports show that the Boesky Room is in heavy demand. Let's make people pay to use the room. We'll bill their credit card numbers in real-time. Best of all: no refunds (even if the fee-based room has a room administrator and that person doesn't approve the request).

We're not kidding. Your code will be putting real transactions into the real banking system using CyberCash. If you set a $1000 price for a room and use your credit card to test your software, you will find a $1000 charge on your next statement. Ergo, you might prefer to set prices more in the range of 50 cents.

Here's the plan:

Congratulations. You're now an Official Web Entrepreneur (TM).

Exercise 7 (extra credit): Find a Venture Capitalist

Take the code that you wrote for this problem set. Give it a name that includes the neologism "middleware". Find a venture capitalist to give you $6 million.

The Wide World of Oracle

We're going to shift gears now into a portion of the problem set designed to teach you more about Oracle and SQL.

Oraexercise 1: Concurrency and Isolation

Connect to Oracle in SQL*Plus and type "describe my_stocks" to make sure that your my_stocks table is still defined from problem set 1.

Oraexercise 2: The Importance of Commitment

Repeat the above sequence but use ROLLBACK at the end in SQL*Plus instead of COMMIT.

The Narrow World of AOLserver

Imagine that you are a working Web developer and a client calls you to ask how much traffic the site you've built is getting. At the very least, you need the following skills: Then the client calls upset because a development site is available for public access. Read the AOLserver administrator's guide at www.aolserver.com and then restrict access to URLs beginning with /foobar. Actually make a directory "foobar" under your server root and place a file in it. See if you can grab the file from a browser. (Note that AOLserver 3.0 will have you editing the perms database files, and calling Tcl procs, while 2.3 has a fancy GUI interface for editing permissions while 3.0.)

Mostly relevant for AOLserver 2.3: suppose that the client wants access to server admin pages, e.g., so that they can create new users. You don't want to give the client access to the nsadmin password, which you might share with other programmers for use on many servers. Create a new user called "myclient" in the "system" group so that they can have the same privileges as nsadmin.

Who Wrote This and When

This problem set was written by Philip Greenspun and Hal Abelson in February 1999 for MIT Course 6.916. It is copyright 1999 by them but may be reused provided credit is given to the original authors with a hyperlink to this document. (Updated February 10, 2000.)

It is permanently housed at http://philip.greenspun.com/teaching/psets/ps2/ps2.adp.


Maintainer: teadams@mit.edu