Category: Uncategorized

We’ve bought a couple of very nice computers from Silicon Mechanics recently. They are cheap, compact, and hold lots of hard drives. We bought them mostly for scratchpad NFS servers and also figured that we’d use them somehow for our photo database. Trying to save some $$, we configured each with only 4 GB of RAM. As it happens, we were able to replace a very complex collection of multiple computers, each with multiple processes, with just a single Silicon Mechanics machine with eight hard drives in rat-simple hardware RAID 5 and two dual-core Opteron CPUs. The machine does pretty much nothing except run 8 lighttpd processes, each of which runs a single thread and serves multiple Web clients via async I/O. Jin set all of this up based mostly on comments here in this Weblog on an earlier posting.

Now that we’ve disabled the operating system writing “most recently read” times for every file served (“atime”), the server is acceptably fast. However, it is running disk-bound, with the RAID drives at 65 percent utilization, i.e., not much room for growth.

Our old SQUID server had 8 GB of RAM and was limited to serving thumbnail images. I suspect that if did obvious simple thing and filled this new computer up with 16 or 32 GB of RAM, we would have superb performance. But I’m wondering if we can’t save a few $$ and cheat a bit by controlling the Linux Virtual File System (VFS) cache. Right now we are not doing any caching in lighttpd, nor does the underlying ext3 system cache. This is by design. We want to keep only one copy of any given photo in RAM cluster-wide. I.e., we want the only copy to be in Linux VFS. We think we could get by with less RAM if we prevented Linux from pushing thumbnails out of the cache when pulling a medium or large JPEG from the RAID. In other words, we’d like to say “preserve 2 GB of the machine’s RAM for caching files of the form *-sm.jpg”, in effect having two caches for files, one for thumbnails and one for all other files.

Can this be done without massive amounts of C-code hacking and making it hard to upgrade the OS version?

Full post, including comments

As of today, I have an Airplane Multi-Engine rating on my Flight Instructor certificate. It seems like a good time to solicit comments on this multi-engine training page. What do the pilots here think the lesson plans need in terms of improvements?

Now we just need to find a rich fool to buy a DA42 and lease it to East Coast Aero Club…

Full post, including comments

From the January 15, 2007 New Yorker magazine, page 54, in an article on a poorly performing school in Denver, Colorado…. Norberto is a junior in high school. His “elder relations” advise him against college: “If you study too much, you forget to get married until you’re so old that nobody wants you.”

Full post, including comments

Apple introduces its first phone today. It is a bit tough to tell from looking at Apple’s Web site, but it appears that this is yet another smartphone that is not a flip-phone. In other words, if it brushes up against something in your pocket it will make or answer unwanted calls. Basically all Japanese phones are flip-phones and it baffles me as to how American consumers are denied the simple interface of “open to make or answer a call; flip closed to hang up”.

Apple gives us an MP3 player, which other brands of smart phones have had for several years. What I want is a phone that won’t make calls from inside my pocket.

[The Web site is http://www.apple.com/iphone/ ; be forewarned that this is unviewable in MSIE on XP and it crashed Firefox on XP at first.]

Full post, including comments

Dear Nerds:

We have a new computer to serve the photo.net photo database. We thought we might do something intelligent, but we ended up instead buying a machine that will carry us forward another 6 months without us having to think. Here are the specs on the new server, which arrived yesterday from Silicon Mechanics:

• 2 Opteron 2212 2 GHz CPUs (each of which is dual core, so effectively I think that means that four threads can be running simultaneously)
• 4 GB RAM
• hardware RAID of 8 750 GB Seagate SATA drives
• CentOS 64-bit operating system

We need to serve a continuous stream of photos from this machine. The data are static and in the local file system. The current load is 2.5 million JPEGs per day, of which 1.2 million are small thumbnail images. We don’t need to query the relational database management system or do anything fancy, just serve the files via HTTP. So maybe, after 12 years, it is time to look beyond AOLserver! Should we consider lighttpd? Apache 2.0? Squid? Should we run just one process of any of these and let threads handle the multiple clients from a single Unix process? Or run multiple copies of the Web server program and tell our load balancer that two sources of these files are available?

[Oh yes, and what about the file system block size? The thumbnails are around 10k bytes.]

Assistance via email or comments here would be appreciated!

Thanks,

Philip

Full post, including comments

One of our local shiksas, apparently deciding that I lacked sufficient Jewish zeal and/or that I should read about an obsessed guy doing something that everyone else thought was stupid, gave me a book by the founder of the National Yiddish Book Center. Aaron Lansky writes about how he spent the past 25 years collecting books in Yiddish from all over the world and bringing them back to Amherst, Massachusetts. Everyone said that he was crazy because nearly all the people who could read Yiddish were dead, killed by their European neighbors in the 1930s and 40s or expired from old age in the U.S. Children of dead Yiddish readers were throwing the books out when Lansky’s organization alerted them to the possibility of donating them to his new library.

With money from Steven Spielberg, his center has scanned all of these donated books. So now all of these books that nobody wanted in hardcopy exist in digital form. I visited this great digital library and typed in “Singer”, the one Yiddish author whom I’ve read. It turns out that there aren’t any entries for Isaac Bashevis Singer, the Nobel Laureate, but there are some obscure titles from obscure Singers. I picked #4 at random, a 111-page book by Israel Joshua Singer, who died in 1944. The book could be mine for $48 plus $5 in shipping. Is there a way to view the text online? No. Could it be that the center didn’t have enough money to buy a few HTTP servers and deliver the bits to interested scholars worldwide? They raised $7 million to build themselves a fancy headquarters building. They got organized to set up an ecommerce and print-on-demand system for hardcopies. So here we have a MacArthur “genius” award winner trying to make Yiddish books more widely available who goes to all of this trouble to collect and scan them, who says that his mission is to make this treasure trove accessible to a new generation of folks worldwide (many of whom live in poor countries with unreliable postal services and/or are starving graduate students), and then prices the collection out of the reach of any of the people who might conceivably be interested. (The Yiddish book center earns a dismal two-star rating from Charity Navigator, spending 33 percent of their budget on fundraising and only 56 percent on program expenses.)

Full post, including comments

I’ve drafted a Canon Digital Rebel XTi review at http://philip.greenspun.com/photography/canon/rebel-xti

I would appreciate comments/corrections either via email or as comments to this Weblog.

Thanks!

Full post, including comments

The main gym at MIT reopened this week after some maintenance. There is still no soap for the pool showers. The men’s locker room has a new scale, which looked curiously massive from a distance. I approached for a closer inspection. The new scale reads up to 500 lbs.

Full post, including comments

A friend got a water buffalo for Christmas from her dad. She won’t actually take delivery of the animal. The Web page says that it will be given to a family in Asia. If you read the fine print on the page, however, it turns out that there is no actual buffalo and no actual family and you won’t get a photo of your family and your buffalo. The money simply gets dumped into the common fund at the charity. We are trying to decide if this is the crummiest possible Christmas present.

[Would it actually be effective to give every poor family in Asia a water buffalo? Wouldn’t that simply result in overgrazing? It shouldn’t be that hard to breed water buffalo, so you’d think after thousands of years they would have the optimum number without Western intervention.]

January 19 update: Things got a little out of hand with this posting, including the purchase of a real water buffalo for a real family. See this 57MB Quicktime movie for the full story. All credit to Robert Thompson.

Full post, including comments

While driving the clogged freeways of California, I listened to an NPR show in which the central complaint was the inequality of income distribution in the United States. Walter Benn Michaels bashes universities for being obsessed with admitting rich kids with Hispanic last names or dark skin, instead of kids from poor families. Nobody raised what to me seemed like obvious questions, e.g.,

1) Should we stop accepting poor immigrants if we are worried about income distribution? A Somali immigrant might end up achieving a higher standard of living here than in Somalia, but he or she is going to swell the ranks of below-average earners.

2) Are the people whom we consider poor today better or worse off, materially, than America’s poor were in the 1950s or 1970s?

3) Is it inevitable that as an economy gets more complex, those who are clever and talented will find ways to get rich that weren’t available in a simpler economy? (And people who aren’t clever or talented won’t get any boost.)

Question 3 seems like the big one for me. I was driving from a photographer’s house in Napa to an animator’s house in Oakland. My host in Napa would have been lucky to earn a middling salary on a newspaper or magazine staff in the 1950s. He is moderately rich today because our more sophisticated economy (1) allows him and his wife to finance and publish their own books, cutting publishers out of much of the profit, (2) allows him to market his decades of photography via the Internet to stock photo customers, and (3) allows him to do assignment work for magazines worldwide, the phone and the jet airliner making him just about as accessible to a European magazine as a European photographer. My host/cousin in Oakland has a great talent for art and loves doing animation. 50 or 100 years ago, he would have been a commercial artist selling illustrations for $5-25 apiece. Maybe if he had been lucky, he would have gotten what would then have been a low-paid job at Disney (as actors were not well paid under the studio system, animators did even worse). Today there are dozens of employers of animators in the U.S., including Disney, Dreamworks, and Pixar. Licensing deals with toy companies and cable networks, and new technologies such as the DVD make animated movies vastly more profitable than they were 50 years ago and enough of those profits have trickled down to the animators that they can afford to live very comfortably indeed.

The folks on NPR are complaining about how the rich are getting richer and we need to change government and institutional policies accordingly. However, both of the folks I visited owe most of their wealth to changes in the economy and world markets that have nothing to do with government or university policies (nobody even cares if they have a college degree).

Full post, including comments