Using the ArsDigita Community System
to build various kinds of Web sites, by Philip Greenspun for the Web Tools ReviewLast Update: April 21, 2000 (to coincide with being able to run ACS with the Apache Web server)
|
|
Using the ArsDigita Community Systemto build various kinds of Web sites, by Philip Greenspun for the Web Tools ReviewLast Update: April 21, 2000 (to coincide with being able to run ACS with the Apache Web server) |
Here's what's in this document:
If you're a lazy Wall Street analyst, the best way to think about ACS is "like Broadvision or Vignette but (a) much stronger on community, (b) free and open-source, (c) backed up by education worldwide, (d) a bit weaker on catalog shopping than Broadvision and on newspaper/magazine production workflow than Vignette."
If your laziness falls in the middle, here's a summary of the ArsDigita Community System:
This document exists to give publishers some broad ideas of how the
toolkit can be used and roughly how much work is required to adapt it.
example: online community
The software was originally created to support photo.net, a site with a large body of static,
carefully authored content, comments on static pages, a lively set of
discussion forums, classified advertising, and a recommendation service
where people can talk about their experiences with buyers or sellers of
photographic equipment. In order to better support photo.net, I
recently added some modules for dishing out news (e.g., "Canon releases
the EOS-3 body") and calendar items (e.g., "Ansel Adams show at National
Gallery, March 5-June 9").
If your idea of an online community differs from photo.net in that you want tighter moderation (e.g., content doesn't go live until a moderator has approved it), you can achieve that simply by setting some switches in the ad.ini file. If you want banner ads, just reference the ad server that is included with the standard ACS distribution.
If your community is large and contains an overwhelming number of documents, it might be good to use the curriculum module to guide new users through a set of important tutorial documents. This software puts a little progress bar into every page that they fetch from the server until they "graduate" by reading each item in the publisher-established curriculum.
If your community contains several different classes of user, you might consider using the portals module to give each class an easy-to-update portal into your service (and back out to the wider Internet). You can optionally configure the portal software to let each user customize his or her own personal portal.
Configure the customer relationship management module so that you can target email to people depending on how much they've contributed, how recently, and whether they were a really active participant at some point in the past.
The modules that support an online community include the following:
What if you're running a site for a group of people who work together in
one organization? You start with our intranet module, documented at
http://www.photo.net/doc/intranet.html.
This is one of the modules that we're most actively developing (since we
use it to run arsdigita.com, a 51-person company with $10 million in
sales).
A lot of the stuff that is useful for public online communities turns out to translate very nicely for intranets. Some modules that are more useful for intranets than public communities include the following:
The main support for ecommerce, naturally, is our /ecommerce module. It
does pretty much everything that you'd expect from an average ecommerce
site like amazon.com (see /doc/ecommerce.html).
You might not see it this way, but any interesting ecommerce site is actually an online community. Amazon.com, for example, derives most of its value from collecting reader comments on books and from using reader buying patterns and preferences to help other readers find relevant books.
The other weird thing about ecommerce is that there are a million trivial software packages to support online catalogs, shopping carts, and the rest of the stuff that most of us got bored with back in 1995. However, most products aren't priced or sold this way (the same price for everyone).
If you're a company like Cisco, for example, what you would do with the ACS is to set up an online community in which some of the registered users work for Cisco and registered users work at companies that have purchased Cisco routers. You use the ACS's user-groups module to define a user_group_type of "company". You use groups of type "company" to bundle users together according to where they work. This enables you to have private discussion forums for Cisco internal people. This bundling enables you to assign a standard discount to each company that is a customer. Then if joe@hp.com is logged in and wants to see prices, the price display code can check which user-group joe@hp.com belongs to and then grab the discount and adjust the prices accordingly.
We've rolled a nice ticket tracking system into the standard ACS distribution. This ties in again to the user-group tables. When joe@hp.com reports a problem with a Cisco router, the next available support person is notified. If the problem stays open longer than N minutes, a service coordinator for the HP account can be notified. If the problem stays open longer than that, the sales reps for HP can be notified via email. At any time, the system can display reports of customer problems and how they were resolved, summarizing by user-group (i.e., company). If the Cisco salesperson for HP is over on Page Mill Road trying to sell some more routers and the HP customer says "how good are you guys at resolving problems", the Cisco salesperson can show a report that the average response time was 6 minutes and the average time to resolution was 2 hours.
Configure the customer relationship management module so that you can market to people depending on how much they've bought, how recently, and whether they were a really great customer at some point in the past.
Examples of live sites using the ACS ecommerce module: http://www.eplay.com/ecommerce/; www.perfectionlearning.com.
Note 1: if your ecommerce vision is truly banal and extends only to hawking a list of products to customers with whom you're building no long-term relationship, then ArsDigita Shoppe might suit your needs better. It is also free and open-source.
Note 2: Cisco and HP aren't actually using the ACS this way; I just used them as a convenient example since these words are coming to you from an HP Unix server through a Cisco router.
Perfection Learning Corporation has implemented an ecommerce system that has processed hundreds of orders with real-time credit card processing using entirely open-source software, plus Oracle. The installation, integration, and customization took four weeks with a total cost of $30,000. The price was evenly split between the web/database server, the licensing and support for Oracle, and the developer time used. We were able to keep the costs low by using the ArsDigita Community System (ACS) and the AOLserver web server, both available under the GPL. Our web/database server is running RedHat Linux. Installation of the ACS consisted of feeding the data model to Oracle and putting the HTML and Tcl documents in the correct locations. Adding our products to the catalog was done with a browser-driven import routine. We have over 30,000 products in the database with no speed or scalability problems so far. The ACS ecommerce module was easy to integrate into our existing site. By modifying our pages to include links for adding the items to the shopping cart, we made all of our products orderable in a few days. Site maintenance and modification is easy due to the fact that all of the Tcl code and HTML is stored in flat files. Any errors can be fixed with a normal text editor. The functionality and presentation of the ecommerce section is separated by using templates. By combining an open-source operating system such as Linux with the ACS, AOLserver, and Oracle, you can deploy an ecommerce solution in a fraction of the time it would take you to write your own or integrate commercial software with your business.
If you have a bunch of people taking a high school or university course,
the ACS can serve as the hub for distributing course materials,
collecting questions, and fostering discussion. It has in fact been
used this way at MIT, Harvard, and lots of schools that aren't in
Cambridge!
Don't expect miracles, though. Internet and computers are great for connecting people who are separated in time and space. If you've been able to collect everyone in the same room at the same time with an expert, how can you expect plunking down 17" monitors and the limping products of the modern software industry to improve matters?
Anyway, the lessons from our experience are that the ACS works as well as any other collaboration software for supporting folks in a class. Just be sure that they have a reason for using a Web service in the first place.
Architecturally you could set up everyone at a university in one big
ACS. Then use the user-groups module to define various group types.
The most important structures in a modern university are committees. So
you'd have those sorts of groups to accrete administrators. Then you'd
probably want to think about a group of type "course". Students and
teachers in a particular course would be lumped together in a user group
and could have a private discussion group.
Another thing that would work is setting up a standalone ACS just for
one class. It takes us at least half a day to set up an ACS (download
.tar file, set up new AOLserver or Apache, create new Oracle user and
tablespace, edit the parameters file to give the service a name, etc.).
So it might be a little too much overhead for a class that is going to
be over in 13 weeks anyway.
The users are doctors and other folks who work at the hospital. User
groups are departments within a hospital, e.g., orthopedics or
cardiology. You have to write a custom module to collect patient data.
These are tied to users and user-groups. Note that patients do not
become registered users of the ACS.
(If you're thinking that this is the complete set of features that eBay has, you're right. We built it before they did but we forgot to raise money and hype it; so they are multi-billionaires and we remain ... computer programmers.)
Any AOLserver application can be requested as either HTTP or HTTPS.
The .tcl and .adp pages don't know or care that the request has come in
via SSL or not. So you can configure your ACS to only talk to the world
via HTTPS and therefore nobody can sniff your data in transit.
Does that mean you're home free? No. Aside from any security holes that there might be in the ACS, keeping sensitive data on any Internet server presents big security problems. Someone could attack your Oracle database or your Unix server itself and essentially get all of your data. So you want to have really good sysadmins and security folks on staff. These people are expensive but without them there is no way that you can run any kind of Web service and guarantee confidentiality.
If you want your service to be reliable, you probably need something like the ArsDigita Server Architecture.
Some short answers first. Because most people can't accept the idea
that others might be altruistic, we will note that there are some very
big, very rich companies using our software (see http://arsdigita.com/projects.html
for an up-to-date list). We are some of the highest paid programmers in the
world, with 100% ownership of a ArsDigita.com, a 60-developer profitable
company with $20 million in annual sales (February 2000; sales should
double every six months). We do not need to spend every second of every
day trying to make money.
Free software does not mean that programmers get paid less. In fact, programmers probably get paid more since users of free software don't have to feed a vendor's marketing, PR, advertising, HR, trade show, and management departments.
That said, we didn't go into this to make money. We wanted to solve some hard problems in computer-supported cooperative work. Commercial software is fine when what you're buying is a clone of something Xerox PARC did in the 1970s (e.g., Microsoft Word, Microsoft Windows). Free software is the only hope when you're attacking a brand new problem that nobody knows how to solve. To us, the dimensions of being a software development professional include
Longer answer: towards the end of Chapter 2 of "the book".
Yes! We love to train people in how to use our toolkit. We publish
a free book (you can also buy a hardcopy in
bookstores). We give one-day courses (see http://register.photo.net),
arrange
boot camps,
and have exported our one-semester course from MIT
to other universities (including Caltech).
In the summer of 1997 some guys at Hewlett-Packard were laughing at my
plans to build a free open-source software toolkit for building all
kinds of Web sites for collaboration, including business-to-consumer
ecommerce sites. "You'll never make it, Greenspun," they said.
"Everyone will be using iCat. That's what we're using and all of our
customers are using it and it is the future. Get used to a
closed-source commercial software world."
I didn't hear much more about iCat until I happened upon a cover story in the October 4, 1999 PC Week: "Risky Business, IT rolls dice when it picks tools from unproven firms" (by Jim Kerstetter and Matt Hicks). Here's how the story begins:
On May 13 [1999], Greg Streeter was jilted by his e-commerce software vendor."Dear Greg," the goodbye e-mail message said, "Intel is discontinuing the iCat Electronic Commerce Suite, effective immediately." The note went on to remind Streeter that his end-user licensing agreement, which bars copying the software, was still in effect, so there would be no tinkering with the defunct software.
"At least it was personal," sighed Streeter, IS manager at Herman Goelitz Inc., a candy maker based in Fairfield, Calif.
The e-mail was a painful reminder of a regrettable decision Streeter made two years earlier. Faced with an immediate need for e-commerce software that could support personalization and offer an attractive presentation, and unable to find such features in products from established vendors, he committed development of Goelitz's JellyBelly.com Web site to software from iCat Corp., then a small startup.
However, Seattle-based iCat fell prey to the myriad problems that face startups in any industry. It ran out of money, forcing iCat executives to shift strategy. Then, earlier this year, iCat was acquired by Intel Corp., a company that had no interest in iCat's customer base. In August, Intel, of Santa Clara, Calif., discontinued support of the iCat software product line. Customers, including Streeter, got burned.
I went to the iCat Web site and found that they had press releases
proudly announcing the 6500 stores built with their software. They
proudly announced their acquisition by Intel. They had no trouble
calling themselves "the leading provider of e-commerce software". No
mention of leaving behind some orphaned users when they shifted to a
strategy of simply copying Yahoo! Store (developed in the mid-90s by
some Harvard CS grad students, and authored in Common Lisp!).
Contrast Point 1: The source code and documentation for ACS have been scrutinized by thousands of programmers worldwide; the iCat source code has probably never gone beyond a handful of cubicles.
Contrast Point 2: iCat ran out of money and had to sell out to Intel to pay back their investors, then do whatever Intel told them to do. ArsDigita is a company with $15 million in annual revenue (as of November 1999), doubling every six months. Since we've funded our growth out of operating profit, we don't have any investors to pay back. It is tough for me to see how we could go bust, but if we did the thousands of publishers who are using our software could continue to operate and extend it in comfort. That's part of the beauty of open source. We have faith in ArsDigita but we aren't asking you to bet the future of your company on the future of our company.
For additional perspective on the state-of-the-art in the closed-source
world, read http://www.peterme.com/bvsucks/.
Getting the Software
The software is available in a TAR file from
http://software.arsdigita.com/dist/.
Pick the most recent version! We only keep the old ones around so that you
can humiliate us with how stupid we used to be.
The software.arsdigita.com server is worth some discussion. Basically it exists so that you can browse around our source code without having to download and untar the software. The AOLserver behind it is configured to serve .tcl and .adp pages (normally dynamic content) as plain text. The server is also rooted one directory lower than normal so that you can see the parameters file and the private Tcl scripts that are loaded at server startup.
Check out a random page such as http://software.arsdigita.com/www/gc/index.tcl (source code for http://www.photo.net/gc/). Even more instructive is the /doc/sql directory, which contains the data model for the ACS.
To do a really low cost collaborative web server, can the Solid database system be used instead of Oracle? Are drivers available for that software?
-- Peter Tower, February 9, 1999
You can use Solid and AOL distributes a driver for it. However, you'll have to go through our code and remove all the Oracle-specific stuff. It is probably cheaper and better to simply get an Oracle license.
-- Philip Greenspun, March 1, 1999
Also, Solid has changed its licensing fee structure and market focus; it's now $3,000 for a single-server license on any platform. Not such a bargain.
-- Nic Wolff, March 17, 1999
I'm curious about the design of you Software developement manager. Do you have any more detailed info on it ? Since it has its own page describing it this doesn't seem like the right place to go into more detail, but that page doesn't have a comment section ...
-- Lee Schumacher, April 6, 1999
Note that InterBase port of ACS is now available at http://www.lavsa.com/acs-interbase/. Although it has some limitations, I find it quite useful. Among other things, it runs InterBase Q&A discussion forum.
-- Sebastian Skracic, May 27, 1999
AOLServer is now open-source, I believe. I have not download it yet since its only beta version. Waiting for the regular version!Sanjeev
-- Sanjeev Mohindra, July 30, 1999
When we are talking about lowering the cost why not work on open source databases like mysql ? Whatever one says oracle is not a very cheap database to buy and moreover the connectivity cost for hosting it is prohibitively high.
-- Richy Marx, January 13, 2000
Consider an alternate view to the ACS (ArsDigita Community System) assertion of insulation by using open source components. It is January 2000 and the required release of AOL server is the closed source version from October, 1998. Also the 'open source' license is strongly implied to be the GPL, but no copy appears on the referenced pages.So, why trust your software to a system without a strong enough development community to get a full release out every year or so? The trade-offs would be to use the Apache webserver with its larger development and user community group, and to establish the ACS for a subset of Apache configurations.
It is wise to look past the open source label to see if it provides actual investment protection.
-- Charles Merriam, January 26, 2000
Although not specifically stated, the implication of "opensource" is that it also be popular. While I am very impressed with the technology and acomplishments with the ACS system, I fear students will be learning specfic technologies which are not in wide use. Rather, I suggest other more popular mainstream and rapidly advancing "TRUE" opensource technologies be considered. For example, the relatively new PHP scripting tool has recently topped the one million installion mark on worldwide Apache servers. Based on its relative youth, this milestone of achievement should be indication of its skyrocket growth in popularity. This tool interfaces with many database backends but the two most popular are MYSQL (for extreme performance)and PostgreSQL for transaction suppport. Visit http://www.php.net for more info.
-- Scott Perkins, January 26, 2000
Ummm. PHP is just a programming language, just like TCL (that ars-digita uses) is a programming language. You can't compare Ars Digita to PHP. You compare PHP to TCL, both of which are mega-popular.
-- Chris Bitmead, January 27, 2000
Actually there is much to be learned from using the ACS/AOlserver other than its solid proven technology. Vignette's StoryServer software is some of the hottest commercial software to be powering websites from dot.com's to Fortune 500 firms... and it uses... bingo... a TCL scripting engine. In fact, I just converted a commercial site from PHP to Storyserver and frankly it was much cleaner in TCL.
-- Jamie Ross, February 5, 2000
In response to Charles's comment above in which he refers to AOLserver and says, "why trust your software to a system without a strong enough development community to get a full release out every year". The only reason the last non-beta version of AOLserver was released (as of February 2000) in October 98 is because AOLserver is so stable and well designed it doesn't need to be released with bugfixes every two days like Apache. I'd be surprised if Apache could go for two weeks without a patch. The "must upgrade often" syndrome is the Microsoft way to hide bugs.And, the ACS no longer "requires" the closed-source AS 2.3 version (you could get it running on AS3 in January too).
-- No One, February 9, 2000
Scott Perkins wrote "While I am very impressed with the technology and acomplishments with the ACS system, I fear students will be learning specfic technologies which are not in wide use". I think the value of something such as ACS goes far beyond the specific tools used. Anyone who learns to build web systems with this kind of an integrated approach is likely to implement effective solutions to difficult problems no matter whether they use ASP, Servlets, mod_perl or whatever.On the other hand, a peek at the AOLserver source code will tell you that it is a superbly engineered piece of software with a very thoughtful implementation of the Tcl API. In combination with Oracle on Unix you have a very solid architecture.
-- V. Nair, February 16, 2000
Re:Databases - We're in the final stages of setting up our intranet using the ACS ported to Postgres. It's working great and we're really happy this far - a great system. The specs are (aolserver 3 beta6; postgreSQL 7 beta 3; ACS/pg 3.2.2). This circumvents the need to rely on either Oracle or Solid, and the online community support we've discoverd has been impressive.
-- Grant Schofield, May 3, 2000
Three years ago, Chris could compare PHP and TCL by saying that it acts of "an other" language...It was already a fast comparison and now it's even more obvious: PHP is a programming language (with whole share) whereas TCL (and well of others) are only languages of script.
-- Fabrice PELLEAU, January 9, 2003