Philip Greenspun’s Weblog

When universities created business schools in the 20th Century traditional academics decried the collapse of standards.  Instead of students studying Literature, Art, History, and Science they would be going through the motions of a scholar while occupying their minds with things that formerly had been learned at a desk as an apprentice in a dreary Victorian counting house.  Now in the 21st century the B-schools are degrading the term “computer hacking”.

Here are the facts:

• Harvard and a bunch of other B-schools with a collective IT budget of maybe $50 million decided that writing Perl scripts was too hard so they outsourced Web-based applications to a company called ApplyYourself.
  
• You’d think that the main advantage of a centralized service such as ApplyYourself would be that a prospective student could fill out one application and the information be sent simultaneously to many schools.  However, this is not how it works.  Each school has a totally separate area with ApplyYourself.
  
• All the smart young Americans have gone to law, business, and medical school.  Companies don’t like to hire old people (> 30 years) to write computer programs because it saddens them to see old folks doing something so degrading.  Thus ApplyYourself hired whoever was rejected by professional schools to write up some Visual Basic scripts to process HBS and other B-school applications.
  
• The ApplyYourself code had a bug such that editing the URL in the “Address” or “Location” field of a Web browser window would result in an applicant being able to find out his admissions status several weeks before the official notification date.  This would be equivalent to a 7-year-old being offered a URL of the form http://philip.greenspun.com/images/20030817-utah-air-to-air/ and editing it down to http://philip.greenspun.com/images/ to see what else of interest might be on the server.
  
• Someone figured this out and posted the URL editing idea on the BusinessWeek discussion forum, where all B-school hopefuls hang out and a bunch of curious applicants tried it out.
  
• Now all the curious applicants, having edited their URLs, are being denied admission to Harvard and, due to the fact that  universities form cartels to fix tuition prices and other policies, presumably to the other B-schools as well.

One interesting data point is that I once supervised a couple of MIT students building an online system for submission of essays to be graded.  MIT and a bunch of other schools have writing requirements.  Students submit essays.  These are held in confidence from other students.  A subset of users are authorized to grade essays and they are handed essays to evaluate.  One server with a single database is programmed to handle students and evaluators from many different schools and keep everything that should be separate separated.  The students building this system had never programmed in SQL before.  Nor had they ever written a Web script to glue their SQL code to an HTML template.  Nor had they ever written HTML before.  The entire project, which requires the same workflow and main features of the ApplyYourself service, took them three months at 20 hours per week.  Those kids are probably just graduating from med school now and preparing for their careers in radiology…

In the 1960s the term “hacking” meant smart people developing useful and innovative computer software.  In the 1990s the term meant smart evil people developing and running programs to break into computer systems and gain shell access to those systems.  Thanks to Harvard Business school the term now means “people of average IQ poking around curiously by editing URLs on public servers and seeing what comes back in the form of directory listings, etc.”

[Update:  People have been asking me whether I think the schools are justified in rejecting the applicants who mucked with ApplyYourself’s URLs.  Had I been an MBA applicant and heard about this security hole I probably would have tested it out.  Not so much out of curiosity as to whether I’d gotten in but mostly to see if a school with nearly $30 billion in assets really was so contemptuous of quality in IT and also to see just how far the Web development industry has slid from its apex (probably 1994, when 5 reformed Lisp hackers built Amazon.com out of C CGI scripts talking to Oracle).  I did something similar when writing Philip and Alex’s Guide to Web Publishing.  I needed examples of Microsoft Active Server Page source code.  There was at one time a bug in IIS/ASP that enabled anyone to view the source code by appending “::$DATA” to any .asp URL.  Months after Microsoft had released a patch for this bug, I surfed around and found scripts at lots of prominent public servers, some of which scripts contained database usernames and passwords.  I published the results in http://philip.greenspun.com/panda/server-programming#ASP, which was turned into a hardcopy textbook by Harcourt.  So it seems that my curiosity into just how incompetent an institution with $billions in assets could be would have led to me failing the ethics test, being convicted of hacking, and being denied admission to a top business school.

Where would I personally draw the line?  A grad student at MIT figured out that Fandango, the movie ticketing service, was passing the price of the movie ticket as a hidden form variable in the HTML instead of doing the pricing on the server at the final page.  He was able to edit the HTML form in Emacs and submit it to Fandango and buy tickets for any price that he felt was fair (being a grad student, his preferred price for tickets was $0.25).  He invited me to try it out but it but I thought that either Fandango or a movie theater would end up having to make up the difference and it didn’t feel right to take their money.  The HBS/ApplyYourself situation falls into the “poking around with a browser” category where you get to see stuff but the Web publisher hasn’t been injured because they still have the stuff on their server (one of the strange characteristics of the digital age).  As progressively dumber programmers build progressively more complex systems we will see more of this kind of attempt to paper over coding mistakes with lawyers, sanctions, policies, and laws.  Hollywood and the RIAA are usually the most successful at getting the government to do their bidding.  Thus I predict that one day Disney will have a Web site where you can buy access to any of their movies.  Because all of their profits are being used to pay executive salaries this will have to be built at extremely low cost.  Deficiencies in the softwrae will enable vast numbers of Americans to download Bambi for free, their ISPs will be forced to rat them out, and they will all get to see Martha’s Stewart’s cell in West Virginia first hand…]

Full post, including comments

Having finally gotten Photoshop scripting sort of under control, http://philip.greenspun.com/images/20030817-utah-air-to-air/ contains some photos of my old airplane flying over the spiral jetty artwork in the Great Salt Lake.  These were taken by Jenny Reinman from a physician-piloted Cessna [a plurality of private aircraft owners seem to be MDs] while her husband Paul Reinman and I flew in the Diamond Star.

Full post, including comments

Syria and Lebanon have been in the news lately.  Lebanon has a population of 3.7 million.  Syria is able to control these people with just 14,000 soldiers, according to the latest articles, and to judge by the tens of thousands of supporters who rallied today in Beirut, Syrian control of Lebanon is just fine with most of the Lebanese.  This is a remarkable achievement considering that the U.S.-led coalition has roughly 150,000 troops in Iraq for their 25 million people and Iraqis don’t seem to be turning out in any great numbers to show public support for the foreign soldiers to continuing hanging around.  One wonders if we should not have outsourced the Iraq occupation to Syria…

Full post, including comments

Boeing’s CEO seems to have gotten fired for “having it off” (as the Brits say) with a fellow employee.  The message this sends to me is that Fortune 500 company boards will tolerate and facilitate managers who indulge in looting but not those who indulge in fucking.  These are perilous times for investors…

Full post, including comments

A friend of mine who works at a database management system company asked for thoughts on how long a string a database table needs to be able to store, as a practical matter, to serve most Internet programming needs.  This prompted me to do some queries into the photo.net discussion forum.  Here’s my message to him, which I thought would be interesting to nerd readers….

Three basic issues for Web development relating to varchar/clob:

1) Strings are uploaded from a browser-rendered TEXTAREA are of a length that is impossible for the programmer to predict. In a sense, then, every text slot in the database must be prepared to accept a string of arbitrary length.

http://www.cs.tut.fi/~jkorpela/forms/textarea.html#browserlimits reveals that some browsers have limits of 32K or 64K but that as Microsoft and Mozilla get more sophisticated these seem to be disappearing.

2) Software developers of Internet applications are often first-time SQL programmers and sometimes first-time programmers altogether. Unless a DBMS can make CLOBs work with every SQL function and command these novice programmers must learn a whole new computer language, essentially, to deal with CLOBs.

3) Internet applications are often developed using feeble ad hoc tools, such as PHP (my students in 6.171, MIT’s Software Engineering for Internet Applications, mostly picked PHP to do their semester project even though I would have discouraged this, being a mistruster of thrown-together unnecessary new languages). Many of these tools don’t have facilities for dealing with anything beyond the basic SQL data types so they couldn’t use CLOBs if they wanted to.

I think for Web development it is reasonable to expect the average string length from the user to be 300 chars, despite wanting to be prepared for a maximum of 32K or even larger. Oops. Typing that prompted me to do the query (see below). Averaging 2 million messages on photo.net, the correct number is 425. The histogram query reveals that out of 2 million messages over a 10-year period a 32K limit would have resulted in 6 messages being rejected and a 16K limit something like 30 rejections.

If you wanted to implement something like Salon.com as a single RDBMS table for both articles and comments on articles I think a 64K limit might be required. If someone authors a 5000-word magazine article in Microsoft Word and then saves as HTML that will be 25-30k of content plus at least a factor of 2 in HTML tags and other Microsoft-added filler.

http://www.photo.net/bboard/q-and-a-fetch-msg?msg_id=002oFh is an example of one of the big postings on photo.net. It is 37962 chars long, the HTML is very clean (i.e., much less filler than if saved by Word), and yet the page doesn’t seem excessively long.

So… my conclusion from looking at the queries below is that 32K would do the job for a pure discussion forum system and that it would be marginal for storing articles unless a publisher decided that everything should be broken up into “part 1”, “part 2”, and “part 3”. Looking at the .html files on photo.net, the vast majority are in fact under 32K and most are under 64K.

However, if you look at

http://philip.greenspun.com/seia/ the largest chapters are 88.7k.

If you want to facilitate novice programmers building full-scale content management systems where all the content is uploaded from browsers it might be necessary to make the varchar datatype bloat up to 100k-ish. But 32K would be adequate to build something like eBay, amazon (user-uploaded content and much of the publisher content as well), or photo.net discussion forums.

———– some stats from photo.net

select avg(dbms_lob.getlength(MESSAGE)),count(*) from bboard;

AVG(DBMS_LOB.GETLENGTH(MESSAGE))   COUNT(*)
——————————– ———-
        424.672669    2052290

select round(dbms_lob.getlength(MESSAGE),-3), count(*)
from bboard
group by round(dbms_lob.getlength(MESSAGE),-3);

ROUND(DBMS_LOB.GETLENGTH(MESSAGE),-3) COUNT(*)
————————————- ———-
        0  1452035
     1000   510236
     2000    58330
     3000    11972
     4000     3399
     5000     1303
     6000      481
     7000      264
     8000      138
     9000       93
    10000       66

    11000       38
    12000       22
    13000       21
    14000       17
    15000        6
    16000       12
    17000        9
    18000       14
    19000        4
    20000        4
    21000        1
    22000        2
    23000        1
    24000        4
    25000        3
    26000        2
    27000        1
    30000        2
    32000        1
    33000        2
    34000        1
    37000        1
    38000        1

Note:  Oracle did a much better job formatting these in SQL*Plus; for some reason the tabs didn’t carry through after cutting and pasting.

——————————

Epilogue (not from my email to the friend):

Look how much fun it is to program SQL.  Three lines of code and you get an interesting answer (and those three lines would have been much cleaner and simpler if we hadn’t been forced to use the CLOB datatype, which has its own strange accessor functions).  Compare to Java and C where typing until your fingers fall off usually doesn’t result in much of anything.  SQL, Lisp, and Haskell are the only programming languages that I’ve seen where one spends more time thinking than typing.

Full post, including comments

CC Mixter, at http://ccmixter.org/, has been launched by Creative Commons.  This is a service for musicians to make their work available for sampling, remixes, mash-ups, and other purposes that a middle-aged Boston Symphony Orchestra subscriber wouldn’t understand.  What I do understand, however, is that this system was built as a project in MIT class 6.171 (Software Engineering for Internet Applications) by Ian Spivey and Matt Drake.  It is exciting to see the service apparently live and well.

[Many postings today due to 30-knot wind gusts blowing away my helicopter lessons.]

Full post, including comments

A 40ish friend he told me about life with his twentysomething girlfriend:

• “I plan the dinner, shop for all the ingredients, choose and buy the wine, cook and clean up.”
  
• “We were staying at a friend’s house.  When it came time to leave she was relaxing while I cleaned up and put the sheets and towels in the washing machine.”
  
• “We earn about the same amount of money and yet I pay for everything.”
  
• “She seems to think equality means doing less than half the work so she won’t ever have to feel mid-twentieth century housewifelike.”

At the same time we know a huge number of women who seem to be good at everything except holding onto a guy long enough to get married, something that they claim to want.  Could it be the case that in the old days mothers sat their daughters down and explained to them how selfish and spoiled most men are and what they needed to do to keep the guy happy?  Whereas now young women are exposed to wisdom from Jada Pinkett Smith, a popular actress:

“Women, you can have it all—a loving man, devoted husband, loving children, a fabulous career,” she said. “They say you gotta choose. Nah, nah, nah. We are a new generation of women. We got to set a new standard of rules around here. You can do whatever it is you want. All you have to do is want it.” (speaking at Harvard, a talk that got her into hot water for being too “heteronormative”)

The implication of Pinkett Smith’s remarks was that a Harvard girl, in virtue of being bright, well-educated, and ambitious, was entitled to these things without doing too much work except maybe on the career part.  She never added “if you’re willing to do the laundry, plan and pay for half the evenings out, straighten up the house in between visits from the cleaners.”

Some of our women friends do seem to have figured out what compromises and efforts are entailed but they suffer through many inexplicable (to them) dumpings and are into their late 30s by the time insight is acquired.  This wouldn’t be a problem except that by this age they are past their best reproductive years and are often rather embittered toward men.

A potential solution:  Find couples where the man is satisfied and not thinking about walking out.  Do time-and-motion studies of the female partner in these couples and figure out how much effort they are putting forth.  Report the results so that single women can decide if it is worth the bother.  Perhaps when they see the data they will come to agree with Isabel Archer in Portrait of a Lady, who, when all around her were trying to marry her off, thought

“she held that a woman ought to be able to make up her life in singleness, and that it was perfectly possible to be happy without the society of a more or less coarse-minded person of another sex.”

[Update:  Some (married) friends pointed out that there are quite a few books targeted at women who want to get married, offering advice.  However this advice is anecdotal and not based on hard numbers gleaned from surveys.  A friend in her 40s pointed out that perhaps by coddling our kids we’ve produced an entire generation too selfish to make a marriage succeed.  This afflicts both young men and women equally with the difference that men have the biological luxury to wait until they are 40 or 50 to figure it out.]

Full post, including comments

A 17-year-old polo champion is visiting us from Argentina and today was my day to give her the grand tour of Boston.  Naturally the MIT campus was on our agenda.  MIT’s new president, Susan Hockfield, rather than doing something interesting like starting a medical school, has made her first public action beating up on Larry Summers for his musings on why there aren’t an equal number of women and men in super nerdy academic jobs.  Hockfield says that “The question we must ask as a society is not ‘can women excel in math, science and engineering?’ but ‘how can we encourage more women with exceptional abilities to pursue careers in these fields?’”  I felt proud to be doing my share.  I had brought a 17-year-old girl who can do anything she wants to with her life onto the MIT campus to be inspired.  What happened?  Just downstairs from Hockfield’s office we ran into a woman who recently completed a Ph.D. in Aero/Astro, probably the most rigorous engineering department at MIT.  What did the woman engineer say to the 17-year-old?  “I’m not sure if I’ll be able to get any job at all.  There are only about 10 universities that hire people in my area and the last one to have a job opening had more than 800 applicants.”

[Spending the day with a young person is fraught with potential for humiliation.  She looked at my collection of 2000 LP records and asked “What are those?”  When I explained that they were records, she asked “What are records?”  It is too bad that the Supreme Court won’t let us execute 17-year-olds anymore…]

Full post, including comments

Just back from seeing Merchant of Venice on the silver screen.  It is amazing how badly behaved nearly all of the characters are.  Shylock, mostly referred to as “the Jew” and addressed as “Jew,…”, is bitter and unwilling to forgive all the times the Christians have spit on him.  Shylock’s daughter is ungrateful for all of his loving care and trust and happy to run off and never see the old man again for the rest of her life.  The young Christian gentleman is typified by Bassanio, who squandered his fortune on high living and who decides to find a rich chick to marry so that he can pay his debts.  The rich chick Portia impersonates a judge so that she can help the rest of the Christians cheat Shylock out of the 3000 ducats he lent for the fortune-hunting expedition plus the rest of his wealth.  The only person in the entire play who behaves creditably is Antonio, the actual Merchant of Venice in the title.

It is tough to argue with a cast that includes Al Pacino and Jeremy Irons.  Teenage boys will also want to feign an interest in classic theater in order to get into this film, which covers a period in history where all women displayed either beautiful cleavage or entirely bare breasts.

[Those who complain that Shakespeare painted the Jews in a negative light should be reminded that Shakespeare almost certainly never met a Jew.  The Jews were expelled from England in 1290, with their property confiscated by the king.  Shakespeare finished Merchant of Venice in 1597.  Jews were re-admitted to England in 1655.]

Full post, including comments

At the Ralph Lauren car exhibit at the Boston Museum of Fine Arts, which opens to non-members on March 6, a plaque next to a 1955 Porsche 550 Spyder contains the following:

“In September 1955 legendary actor James Dean … crashed his new 550 Spyder and was killed.  This tragic event immortalized the Porsche name and transformed a relatively small company into a very big business.”

So… if it worked for Porsche with James Dean, how come it didn’t work for Piper when JFK, Jr. crashed his Saratoga?  If anything you’d expect the truck-like family man’s 6-seater Saratoga to have fared better than the rear-engined Porsche, which was notorious for hard-to-handle oversteer.

[Don’t rush down to the MFA to see this exhibit.  There are much more interesting car collections at a lot of the U.S.’s car museums, including the one 30 miles west in Stow, Massachusetts at the Collings Foundation.]

Full post, including comments