Philip Greenspun’s Weblog

Thanks to the many helpful responses to my posting about VMware, we have concluded that the combination of Linux software mirroring and VMware is never going to work. There doesn’t seem to be any way to get this server to work without wiping and reinstalling. Here is my proposed solution, to be mercilessly critiqued by the experts among the readers.

1. wipe Disk 1 and install ZFS and a fresh operating system (CentOS) on Disk 1, creating one group per development service
2. reboot the machine from Disk 1 and install most things by copying from Disk 2 (maybe a fresh install of Oracle from the tar file
3. wipe Disk 2 and tell ZFS that Disk 2 can now be used as a mirror for Disk 1 (I could be wrong, but I think this is something that ZFS is known to do, i.e., adding a mirrored disk dynamically)

We will not run VMware on the rebuilt machine, but rely on standard Unix user/group permissions. This eliminates a lot of moving parts (the one wizard to whom we have access has no experience with VMware, which by itself is probably a good reason to chuck it). Instead of whatever ad hoc bag-on-the-side mirroring has been kludged into Linux by volunteers we will run ZFS, a system designed from the start to include mirroring as a fundamental part of the file system.

Risks:

a) does Oracle run well over top of ZFS? (understand that the write performance of ZFS can be poor but we are barely doing any updates as this is primarily a development server; the production servers it will run are read-only)

b) can we truly add a mirror after ZFS has been up and running and in use?

What do you guys think of this idea?

Note that there are a few goals on which we cannot compromise: (a) the server must be able to survive the failure of a disk without any human intervention, (b) the server must run Oracle and AOLserver to support some legacy code, and (c) the server must support a couple of simple read-only production services.

Full post, including comments

From Patrick Giagnocavo: http://billburnham.blogs.com/burnhamsbeat/2008/07/fannie-maes-gol.html

This is a explanation of the Fannie Mae debacle that makes sense. It is very similar to the recent posting here in this blog: what happens when you pick a number for management bonuses and then let management work that number.

The Burnham article does not dwell on the role of management bonuses too much, only says that the managers were trying to increase book profit. Fannie Mae has had well-publicized problems in the past ten years stemming from accounting fraud. These led to substantial restatements of earnings. (reference) The incentive to engage in fraud was higher pay for managers (reference).

Full post, including comments

I’ve started The Forgotten Man, an economic history of the Great Depression. Much in the book was news to me. I’ll kick off my weblog coverage of this work with one quote: “Data from the 1930 census would show black unemployment nationally standing slightly below white unemployment.” (i.e., in 1930, among whose who wanted jobs, a greater percentage of black Americans held jobs than white Americans)

After FDR’s New Deal and Lyndon Johnson’s Great Society and all of the other Big Government efforts over the past 80 years to fight inner city poverty and discrimination against blacks… the black unemployment rate is roughly double the white unemployment rate. During the same period, the Federal Government share of the economy, as a percentage of GDP, has grown from roughly 2 percent to roughly 20 percent.

[http://www.bls.gov/opub/cwc/cm20030124ar03p1.htm offers some historical unemployment data. In 1930, the year under discussion, the rate was 8.9 percent. During the Calvin Coolidge years (1920s), the rate was 3.3 percent. Note that these numbers would be much lower given modern measurement techniques that exclude large numbers of potential workers from the labor force. The ballooning of the unemployed during the Great Depression was an anomaly that will be discussed in a future post about the rest of the Forgotten Man.]

Full post, including comments

The $650 13″ screen Toshiba laptop that I purchased a year ago was running slowly with multiple browser windows open. I poked around using the (excellent) performance tools included with Windows Vista and found that the machine was page-faulting like crazy. Windows Vista by itself took up 650 MB of RAM and browser windows running gmail, Acrobat, Flash, etc. were chewing up the rest of its 1 GB. I spent $45 at Amazon on two new DIMMs to bring the machine up to its maximum of 2 GB. and thought it would be a good time to reflect on one year of experience with Windows Vista.

The worst part of the machine is the keyboard, which works if you type with just the right touch but otherwise will drop characters. The second worst part is some software that Toshiba larded onto it. When you roll the mouse up to the top of the screen a bunch of pull-down menus appear that seem to be related to function keys. These are hard to get rid of and conflict with the Windows user interface. I think that the solution is to click right on the desktop and disable anything that says “hot keys”. An annoyance is that the fan kicks itself on and off loudly, making it seem that the computer is laboring mightily.

Hardware failures: none, despite cheap plastic lightweight construction.

Networking: Windows often puts up a dialog box saying “Windows needs your permission to continue”, e.g., when accepting a wireless connection at a hotel. This on a machine that has only one user account, which is configured with no password. On the other hand, the machine is often able to get a connection, e.g., from an 802.11N base station with WPA security, when expert Linux and Macintosh laptop owners are unable to connect.

I have installed the following software on the machine:

• an open-source ssh client
• Firefox
• Rhapsody streaming music
• Netflix streaming video
• AOL Instant Messenger
• Google Earth
• Java
• iTunes (I admit to owning an iPod)
• Real Player
• OpenVPN (virtual private network to get into a cluster)
• Picasa (free Google tool for converting camera RAW photos to JPEG)
• various Adobe products, including Acrobat and Photoshop

There have been no conflicts or incompatibilities with the operating system and no calls to tech support for the OS or any of the apps. I have not done any manual updates or system administration until today’s 5-minute RAM upgrade. I disabled all virus protection and firewalls when the machine was new and yet there have been no viruses of which I am aware.

Things that would add a lot of value to this product: better keyboard, fan-free cooling design.

Changing the operating system to something other than Vista would have saved no time and enabled no additional capabilities.

Vista hasn’t proved too scary or complex for someone like me who had a bit of Windows XP experience (though I’ve never programmed or administered XP), but for the average person a mobile phone that can be used as a home computer would make a lot more sense.

[If anyone wants the old RAM, two sticks of 512 MB DDR2, please send email to philg@mit.edu with your mailing address. This is a $20 value but can be yours absolutely free… ** UPDATE: these were claimed very quickly by a starving graduate student in Pittsburgh **]

Full post, including comments

In the primitive old days we would get a new server with a 400 MHz CPU, 512 MB of RAM, install Unix, Oracle, and AOLserver, and be up and running after one long miserable day of system administration. Fortunately we live in the modern age. Two months ago, I bought a server with 4 GB of RAM, a processor with multiple gerbils running at the speed of light, and handed it over to a couple of young whiz kids. They laughed at my idea of simply installing Linux, Oracle, and AOLserver. “You’re going to use this box for multiple development servers,” they said. I replied that Unix was all set up for this with users, groups, and file/directory permissions. We would just create one group per development server and add people to the group. This response resulted in peals of laughter. Didn’t I know about VMware? They would create a virtual machine for each development server and entirely separate user account bases for each virtual machine. The poor little pizza box would now be burdened with running four copies of Linux, one for the underlying machine and one for each of the three development servers. This seemed like a waste of the gift that the brilliant hardware engineers had given us of 4 GB of RAM, but isn’t it the job of programmers to render worthless the accomplishments of hardware engineers?

I let them do it their way. Two months later, the box still isn’t up and running. What are the issues? We have one minor issue with time keeping. VMware supposedly lets you have the underlying box look up the time from NTP servers and set the system clock and then the virtual machines are supposed to get their time from there. That isn’t working for some reason and the virtual machines always have the wrong time. (We can laugh at Windows Vista, but I have never seen a Vista machine that was off by more than a second or two.)

The more serious issue is that the machine simply hangs up and won’t respond to keystrokes for several seconds out of every minute or two. At first I figured that the problem was virtual machines being paged out to disk so I asked the whiz kids to disable swap for each and every one of the four Linux installations. They did that and the machine is still halting temporarily. Here’s a description of the problem from one of the whiz kids (they will remain nameless so that they don’t need to be more ashamed than they already should be)…

We're running VMware Server on Linux and have noticed that the virtual machines will hang for several seconds at a time. This is creating serious performance problems and making even the most basic console interactions painful. The disk access light in the status bar of the management console is lit for the duration of the freeze. When the light goes out the VM resumes running smoothly. At first we thought this was a paging problem but upon adjusting memory allocation neither the VMware host nor any of the virtual machines report any swap use whatsoever. The problem continues to occur even with swapping disabled. Checking the vmware.log file reveals hundreds of disk timeouts. Here's a five minute span from the log file:


Jul 07 14:57:30: vmx| DISK: DISK/CDROM timeout of 13.156 seconds on ide0:0 (ok)
Jul 07 14:58:00: vmx| DISK: DISK/CDROM timeout of 7.435 seconds on ide0:0 (ok)
Jul 07 14:58:35: vmx| DISK: DISK/CDROM timeout of 9.563 seconds on ide0:0 (ok)
Jul 07 14:59:13: vmx| DISK: DISK/CDROM timeout of 9.763 seconds on ide0:0 (ok)
Jul 07 14:59:45: vmx| DISK: DISK/CDROM timeout of 5.775 seconds on ide0:0 (ok)
Jul 07 15:00:21: vmx| DISK: DISK/CDROM timeout of 8.015 seconds on ide0:0 (ok)
Jul 07 15:00:57: vmx| DISK: DISK/CDROM timeout of 9.939 seconds on ide0:0 (ok)
Jul 07 15:01:31: vmx| DISK: DISK/CDROM timeout of 10.320 seconds on ide0:0 (ok)
Jul 07 15:02:00: vmx| DISK: DISK/CDROM timeout of 3.345 seconds on ide0:0 (ok)
Jul 07 15:02:37: vmx| DISK: DISK/CDROM timeout of 6.182 seconds on ide0:0 (ok)

Any advice from a VMware hero? When is it time to wipe the machine and run it like we would have in 1978?

Full post, including comments

Boston has some of the highest per-mile taxi rates in the U.S., higher than far wealthier cities such as New York. At the same time, our streets aren’t all that congested at most times of day, unlike, say, New York. The result is that people do a lot of extra driving in private cars in order to avoid using taxis.

Consider a guy who lives in the suburbs who needs to go to Logan Airport for a five-day trip. He could drive his gas-guzzling SUV and pay $100 to park at the airport. He could pay $120 for a round-trip cab ride in a gas-guzzling seven-year-old full-size American sedan, the mainstay of our taxi fleet. What is he likely to do? Have his wife drive the gas-guzzling SUV to and from Logan twice.

How could we have lower fares and brand-new hybrid vehicles at the same time? Current taxi fares go primarily to pay rent on the medallions. The City of Boston artificially restricts the number of taxis to roughly the same number that existed in the 1930s, when the city was much smaller and poorer. The consequence is that it costs roughly $400,000 to buy a medallion, 20 times the cost of a brand-new 2008 Toyota Prius (a medallion for New York City is closer to $600,000). How come your driver barely speaks English, doesn’t know how to navigate anywhere, doesn’t have a $200 dashboard-mounted GPS, looks poor, and is driving a wreck? As an economist would predict, with the supply of medallions limited, all profits from a taxi operation go to the medallion owners. The drivers earn a subsistence income regardless of the rates set by the city. They cannot be paid less because they would quit and take another job requiring no skills. They cannot be paid more because any higher salary for drivers would attract unskilled workers willing to work for less. When someone hands $40 to a taxi driver here in Boston, most of the money ends up in the hands of a millionaire or billionaire who owns the medallion.

In the old days nobody seemed to mind a system left over from the 1930s that made life in Boston more expensive and clogged our parking spaces with private cars that people used so that they wouldn’t have to pay for artificially inflated taxi fares. When gas is over $4 per gallon, though, and we’re choking ourselves and our planet, perhaps we can summon the political will to expand our taxi fleet with hybrids.

One advantage of hybrid taxis is that a taxi is operated more miles than a private vehicle, so replacing an old Ford Crown Victoria with a new Prius has a lot more impact on gas consumption if done for a cab than for a family car. Another advantage is that taxis tend to be operated mostly in stop-and-go city traffic, where hybrids perform best. Finally we have the opportunity to reduce air pollution to make Boston more attractive to people and employers who have been fleeing south and west.

Right now the politicians and bureaucrats are debating whether to approve a requested 50 percent fare increase, on the stated theory that it will help drivers pay for gas. In reality any fare increase must end up in medallion owners’ pockets. Perhaps it is time to allow anyone who is willing to meet safety and technical standards to operate a taxi here in Boston at rates that are 30 percent lower than current rates. To qualify, a driver would need to be in a vehicle that burns no fuel when stopped in traffic and that consumes, overall, no more fuel than a 2008 Toyota Prius. That should ensure a plentiful supply of efficient taxis on the road and at rates low enough to get people out of their SUVs.

Background: http://www.bostonmagazine.com/articles/fare_game/ (a 2004 article with some useful information)

Full post, including comments

I’ve drafted a new article on preventing runway incursions: http://philip.greenspun.com/flying/runway-incursions/. Please comment by clicking the “add a comment” link at the bottom. If you have a typo or correction or short-term comment, feel free to make that here.

Full post, including comments

If you liked my April 23 posting on cooking economic numbers, you’ll like this article by Bill Gross, which concentrates on the inflation numbers alone and this time from the perspective of an investment manager. Gross is a bond fund manager, whose Wikipedia page lists his personal fortune at $1.3 billion. It sounds as though he is going to short bonds.

Full post, including comments

Deval Patrick has been in the headlines for misuse of State Police helicopters for personal transportation. He has also been in the news lately for trying to impose a sales tax on aircraft used by his subjects. I learned today that Patrick has been shopping for a private jet to be paid for by the taxpayers. Patrick has his eye on an $11 million Lear 45. Deval Patrick was elected governor of a state that can be traversed north-south in about 1 hour by car and east-west in less than 2 hours. Why would he need a transcontinental private jet? To go to New York City to see the bankers from whom the state will have to borrow money to pay for his expanded spending programs? There are commercial flights every 30 minutes that leave from Logan Airport, just a 10-minute ride by limousine from the State House. To go to Washington, D.C. to hobnob with his fellow Democrats? There is a commercial flight every hour Monday through Friday.

[Folks: I had no idea that this posting would generate so much interest. I’m going back to my source for more info. I learned about this while flying with a jet pilot buddy. My friend is connected to all of the jet brokers and jet pilots at Hanscom Field. The plan that he heard about was for Governor Patrick to lease one of these jets, which costs more than buying but would not require an appropriation of all $11 million at once. Anyway, shopping for a jet can be a long and involved process due to the long backlog of orders for new jets (typically 3-5 years). Someone who wanted to lease or buy an existing airplane would need to contact a lot of different brokers.]

Full post, including comments

Eclipse Aviation, the company that brought Microsoft ambition to the very light jet market, is in the news again. Apparently the engines on a new Eclipse got stuck at full throttle and the pilots were lucky to get it back on the ground in one piece. Note that Eclipse, contrary to the New York Times article on the incident, uses the same engines as many other very light jets and they are not the cruise missile derivative that Eclipse attempted to use back in the late 1990s. One difference between the Eclipse implementation and the other VLJs is that Eclipse wrote its own engine control software. Everyone else relies on Pratt and Whitney’s old-school engine control system.

Full story: http://www.nytimes.com/2008/06/13/business/13air.html

[Note the implication of this story is that if you own one of these airplanes it is not useful for anything other than soaking up hangar rent and insurance premiums.]

[Saturday update: It seems that the New York Times, as they do with most aviation stories, got the significant facts wrong. The AD does not ground the Eclipse fleet, but only requires a paperwork addition to the checklists, requiring the pilot to check the thrust levers somehow. It is apparently pretty simple. In its early years Eclipse did great managing the press. The stories were always much rosier than the reality. Now as the Eclipse market implodes the newspaper articles are darker than reality.]

Full post, including comments