Best Web server program for a lot of static files?

Dear Nerds:

We have a new computer to serve the photo.net photo database.  We thought we might do something intelligent, but we ended up instead buying a machine that will carry us forward another 6 months without us having to think.  Here are the specs on the new server, which arrived yesterday from Silicon Mechanics:

  • 2 Opteron 2212 2 GHz CPUs (each of which is dual core, so effectively I think that means that four threads can be running simultaneously)
  • 4 GB RAM
  • hardware RAID of 8 750 GB Seagate SATA drives
  • CentOS 64-bit operating system

We need to serve a continuous stream of photos from this machine.  The data are static and in the local file system.  The current load is 2.5 million JPEGs per day, of which 1.2 million are small thumbnail images.  We don’t need to query the relational database management system or do anything fancy, just serve the files via HTTP.  So maybe, after 12 years, it is time to look beyond AOLserver!  Should we consider lighttpd?  Apache 2.0?  Squid?  Should we run just one process of any of these and let threads handle the multiple clients from a single Unix process?  Or run multiple copies of the Web server program and tell our load balancer that two sources of these files are available?

[Oh yes, and what about the file system block size?  The thumbnails are around 10k bytes.]

Assistance via email or comments here would be appreciated!

Thanks,

Philip

Full post, including comments

National Yiddish Book Center

One of our local shiksas, apparently deciding that I lacked sufficient Jewish zeal and/or that I should read about an obsessed guy doing something that everyone else thought was stupid, gave me a book by the founder of the National Yiddish Book Center. Aaron Lansky writes about how he spent the past 25 years collecting books in Yiddish from all over the world and bringing them back to Amherst, Massachusetts. Everyone said that he was crazy because nearly all the people who could read Yiddish were dead, killed by their European neighbors in the 1930s and 40s or expired from old age in the U.S. Children of dead Yiddish readers were throwing the books out when Lansky’s organization alerted them to the possibility of donating them to his new library.

With money from Steven Spielberg, his center has scanned all of these donated books. So now all of these books that nobody wanted in hardcopy exist in digital form. I visited this great digital library and typed in “Singer”, the one Yiddish author whom I’ve read. It turns out that there aren’t any entries for Isaac Bashevis Singer, the Nobel Laureate, but there are some obscure titles from obscure Singers. I picked #4 at random, a 111-page book by Israel Joshua Singer, who died in 1944. The book could be mine for $48 plus $5 in shipping. Is there a way to view the text online? No. Could it be that the center didn’t have enough money to buy a few HTTP servers and deliver the bits to interested scholars worldwide? They raised $7 million to build themselves a fancy headquarters building. They got organized to set up an ecommerce and print-on-demand system for hardcopies. So here we have a MacArthur “genius” award winner trying to make Yiddish books more widely available who goes to all of this trouble to collect and scan them, who says that his mission is to make this treasure trove accessible to a new generation of folks worldwide (many of whom live in poor countries with unreliable postal services and/or are starving graduate students), and then prices the collection out of the reach of any of the people who might conceivably be interested. (The Yiddish book center earns a dismal two-star rating from Charity Navigator, spending 33 percent of their budget on fundraising and only 56 percent on program expenses.)

Full post, including comments

Water buffalo: Worst possible Christmas present?

A friend got a water buffalo for Christmas from her dad. She won’t actually take delivery of the animal. The Web page says that it will be given to a family in Asia. If you read the fine print on the page, however, it turns out that there is no actual buffalo and no actual family and you won’t get a photo of your family and your buffalo. The money simply gets dumped into the common fund at the charity. We are trying to decide if this is the crummiest possible Christmas present.

[Would it actually be effective to give every poor family in Asia a water buffalo?  Wouldn’t that simply result in overgrazing?  It shouldn’t be that hard to breed water buffalo, so you’d think after thousands of years they would have the optimum number without Western intervention.]

January 19 update:  Things got a little out of hand with this posting, including the purchase of a real water buffalo for a real family.  See this 57MB Quicktime movie for the full story.  All credit to Robert Thompson.

Full post, including comments

Unequal income distribution in the United States

While driving the clogged freeways of California, I listened to an NPR show in which the central complaint was the inequality of income distribution in the United States.  Walter Benn Michaels bashes universities for being obsessed with admitting rich kids with Hispanic last names or dark skin, instead of kids from poor families.  Nobody raised what to me seemed like obvious questions, e.g.,

1) Should we stop accepting poor immigrants if we are worried about income distribution?  A Somali immigrant might end up achieving a higher standard of living here than in Somalia, but he or she is going to swell the ranks of below-average earners.

2) Are the people whom we consider poor today better or worse off, materially, than America’s poor were in the 1950s or 1970s?

3) Is it inevitable that as an economy gets more complex, those who are clever and talented will find ways to get rich that weren’t available in a simpler economy?  (And people who aren’t clever or talented won’t get any boost.)

Question 3 seems like the big one for me.  I was driving from a photographer’s house in Napa to an animator’s house in Oakland.  My host in Napa would have been lucky to earn a middling salary on a newspaper or magazine staff in the 1950s.  He is moderately rich today because our more sophisticated economy (1) allows him and his wife to finance and publish their own books, cutting publishers out of much of the profit, (2) allows him to market his decades of photography via the Internet to stock photo customers, and (3) allows him to do assignment work for magazines worldwide, the phone and the jet airliner making him just about as accessible to a European magazine as a European photographer.  My host/cousin in Oakland has a great talent for art and loves doing animation.  50 or 100 years ago, he would have been a commercial artist selling illustrations for $5-25 apiece.  Maybe if he had been lucky, he would have gotten what would then have been a low-paid job at Disney (as actors were not well paid under the studio system, animators did even worse).  Today there are dozens of employers of animators in the U.S., including Disney, Dreamworks, and Pixar.  Licensing deals with toy companies and cable networks, and new technologies such as the DVD make animated movies vastly more profitable than they were 50 years ago and enough of those profits have trickled down to the animators that they can afford to live very comfortably indeed.

The folks on NPR are complaining about how the rich are getting richer and we need to change government and institutional policies accordingly.  However, both of the folks I visited owe most of their wealth to changes in the economy and world markets that have nothing to do with government or university policies (nobody even cares if they have a college degree).

Full post, including comments

Unsafe at any speed… Philip and a piston twin

It is almost impossible to insure or to give away a piston-powered twin-engine airplane these days.  The oft-stated rule is that, in the event of an engine failure, “the second engine takes you to the scene of the accident”.  Engines tend to fail shortly after takeoff when the pilot is busy, the airplane is slow, and the ground is going to come rushing up if swift action is not taken to feather the propeller (feather = turn the propeller blades so that the edges face the wind and it doesn’t create a huge amount of drag on one wing).  When an engine quits, the pilot is supposed to push up the two mixture controls, the two prop speed controls, the two throttles and then make sure that the gear and flaps are up.  After that it is identify and verify the dead engine by pulling back the throttle and seeing that there isn’t any yaw.  Finally one is supposed to pick the correct prop speed control from among the six power levers and pull it back to feather.  I thought I’d done just this and was a bit surprised by the fact that the airplane was yawing as I pulled the lever back.  I kept pulling.  My instructor, Jim Henry, is normally the soul of cool and calm.  He jumped out of his seat and pushed my hand out of the way.  “Maybe you shouldn’t pull back the mixture on the good engine.”

Ooops.  One lever too far to the right.  We were up at 3000′ above the ground, so there was no real hazard, but now I’ve learned why the average engine failure in a twin isn’t managed very well by the pilot.  It really may be beyond the capabilities of the typical pilot.  Turbine-powered airplanes, by and large, don’t ask pilots to be this good.  On a King Air, a dead engine’s prop will feather itself and some rudder boost will be applied as well.  On a turbojet, there isn’t a concept of feathering.

I’m ready for my multi-engine instructor checkride, but I might be kidding myself if I said I was ready to handle a real engine failure shortly after takeoff in a piston twin.

Full post, including comments

Alisa Howell (1976-2006)

I was saddened to receive a newspaper article on the death of Alisa Howell, who introduced me to the Bell 206 Jet Ranger and flying the TV news helicopter in Richmond, Virginia (old weblog story plus some photos).  I remember that Alisa had studied English at University of Washington before deciding to train towards a career as a helicopter pilot.  She was a good writer, took care of two dogs, and had a wry sense of humor.  She would have stood out in most crowds, but even more so at the airport where she was an attractive slender stylish young woman in a world of mostly beefy older slobbish guys.  The details of the crash aren’t known, but it sounds as though marginal weather was a factor.  I remember that Alisa was more cautious than the high-time guys at Helo Air with whom I flew later in the week.  She also was a very capable and thorough pilot.

It is a real shame to lose someone like this.

Full post, including comments

A trip to the MIT campus

I went to the MIT campus the other day to swim and practice a bit with swim fins in celebration of my new Nitrox SCUBA certification (see http://photo.net/travel/diving/decompression-illness for why Nitrox is such a good idea).  Before swimming, I had coffee with a former student, a girl from a Persian Gulf country.  She asked what I thought of her idea for solving the problem of Israel’s existence: all of the Jews from Israel could be deported to the U.S. and resettled here.  I pointed out that most of the Jews in Israel were the remnants of Jewish communities from Arab countries (e.g., from Baghdad (source)).  Her fellow Muslims had found the presence of these particular Jews intolerable in ghettos within their cities in 1950.  Her fellow Muslims found the presence of these particular Jews intolerable 500-1000 miles away in Tel Aviv.  Why did she imagine that Americans would welcome these people as intimate neighbors?

I picked up a copy of the December 8 issue of The Tech.  The biggest employment advertisement reads as follows…

Interested in Hedge Funds, Private Equity, and Real Estate?  MIT’s Investment Management Company is Looking for Investment Analysts to Help Steward MIT’s $12 Billion Investment Portfolio.

MIT has achieved a new level of self-sufficiency.  Profits from tuition stack up into a multi-$billion hoard.  The students who pay that tuition can be tapped into to manage the accumulated $billions.

The pool locker room is still without any supply of soap in the showers (soap supplies in public swimming pools are required by law in many states, but not in Massachusetts).  The scale that had been in the men’s locker room has been removed.  We may be getting fatter, but at least we won’t be depressed by knowledge of that fact…

Full post, including comments

How many seats does an airplane need to hold two modern-day Americans?

Our flight school is getting rid of its two-seat trainer airplanes.  At least half of the new students, with an instructor, will be over the payload limit.  The Katanas hold 400 lbs. with full fuel and the Piper Tomahawks around 350 lbs. with tab fuel.  At 200 lbs. I certainly am using up my full share of the payload.  How will beginners learn to fly?  In four-seat airplanes such as the Piper Warrior.  Two present-day Americans are about the same weight as four Americans were back in the 1950s when these airframes were certified.

Full post, including comments