“Doomed Boeing Jets Lacked 2 Safety Features That Company Sold Only as Extras” (nytimes):
Boeing’s optional safety features, in part, could have helped the pilots detect any erroneous readings. One of the optional upgrades, the angle of attack indicator, displays the readings of the two [angle of attack] sensors. The other, called a disagree light, is activated if those sensors are at odds with one another.
Boeing declined to disclose the full menu of safety features it offers as options on the 737 Max, or how much they cost.
When it was rolled out, MCAS took readings from only one sensor on any given flight, leaving the system vulnerable to a single point of failure. One theory in the Lion Air crash is that MCAS was receiving faulty data from one of the sensors, prompting an unrecoverable nose dive.
[Watch the Aerodynamics lecture from our MIT FAA Ground School to learn more about angle of attack.]
As I noted in a previous posting, the Pilatus PC-12, a much cheaper and simpler airplane (1 engine and 9 passenger seats), doesn’t do any nose-down pushing unless two separate angle-of-attack sensors, and their respective computers, agree. Boeing’s ideas of
- a system that works silently (so pilots don’t realize it is operating)
- a system that works if just one sensor suggests a high angle of attack
- a system that has the authority to drive the airplane into a full nose-down trim situation
- a Band-Aid on the above in the form of a “disagree” warning light
are all terrible ones, as far as I can tell, and unconventional within the industry.
Does that mean we need much more stringent oversight by regulators? (as noted in this other previous posting, the “regulators” in the case of the above system were mostly Boeing employees) Maybe.
The prices of these optional items that would have made Boeing’s unsafe design a little less unsafe were too shocking for Boeing to admit or the NY Times to publish. But reasonably high-quality systems for homebuilt 2-seat and 4-seat airplanes are less than $2,000, including both the sensor and indicator. Examples:
- Garmin, probably as good as anything in the airliner world (and also supposedly available for small certified planes)
- A $389 LIFT system
- AOA Sport, Angle of Attack Instrument
- iFlyAOA (can be added to small certified airplanes as well)
- LevilBOM (my favorite! can be mounted under the wing of a certified airplane; gets its power from the moving air)
So it is tough to know whether regulation should have been relaxed so that Boeing’s costs of putting reasonably modern avionics into the airplane were reduced or toughened so that the crazy bad ideas were squashed. (Or, as my previous posting suggests, shifted so that an independent private engineering service would do the steps that Boeing’s employees were doing while nominally wearing FAA hats.)
Do airlines routinely make decisions on skipping “optional” safety equipment when purchasing airliners? It seems like the liability consequences of this would be huge.
When the airlines bought the B737 MAX it was not clear that the AOA display was in any way safety related (pre-MCAS). Most pilots use the flight management computers to set pitch, not seat-of-the-pants Chuck Yeager style.
It is unclear that more displays and warning lights in the cockpit would add safety. Not engineering runaway trim into the plane would have added a lot more!
Airspeed is the primary reference for avoiding a stall. I’m not typed in the B737 so I can’t be sure what they already had some boiled-down angle-of-attack information as we had on the CRJ-200 and as I know that they do in the Airbus. There is a reference speed put up on the airspeed indicator calculated to be about 1.3X the stalling airspeed. This is not advertised as being AOA information, but I think that it is derived from an AOA sensor. If you have that reference speed (“green dot” on an Airbus) it might not be helpful to have anything more.
@philg
There is quite a good video on YT where the instructor pilots demonstrate stall recovery after stick shaker on a B737 NG simulator: https://www.youtube.com/watch?v=TlinocVHpzk
The ‘barber pole’ display shows the range of safe speeds as the stall approaches. AOA display is a different way of achieving the same thing.
ZdNet is posting an article that challenges if the 737 MAX is even safe at all, sensors or no. MCAS was added because Boeing chose to put high efficiency engines with larger fans and cowlings than were designed for the airframe (more air through, more compression). The engine mounts had to be redone to accommodate the new engines. The mounts and engines changed the aerofoil so that the MAX was unstable on takeoff and required the angle of attack sensors and MCAS system. What Boeing claiming that “it just needs a software upgrade” ignores the fact that AirBus completely redesigned the airframe to create the A320 and A321, with an airframe designed to handle the larger engines. Boeing chose to “patch” the 737 to meet AirBus in the market in 2 years, rather than take 8 years to design a new airframe and give those sales to AirBus. So, this was a decision made by business people rather than scientists and engineers and it’s no surprise there have been two crashes.
There’s no question that MCAS was a kludge and a clean sheet design would have been better. But the existing 737NG is already a kludge (for example the nacelles are flattened on the bottom so they don’t hit the ground) but it is a very safe aircraft. Working off old technology can in many respects be safer – for example on the 787 problems were caused by the new lithium battery technology. It’s OK to have kludges as long as they are SAFE and effective kludges. You can’t do a clean sheet design in all cases.
MCAS was designed to prevent stalls under certain circumstances. AFAIK, it has worked in that there have been no stall accidents of the type it was supposed to prevent. I don’t know how rare those would been. I don’t know whether the MCAS has ever been (properly) triggered in service. But, were it not for the problems it has caused with false positives, it probably would have done the job and you never would have heard of it and the Max would have been a plane with a good record.
– were it not for the problems it has caused with false positives, it probably would have done the job and you never would have heard of it and the Max would have been a plane
And if pigs had wings, they would be able to fly.
Having two sensors and then acting upon readout from one of them doubles the chances
of an error from faulty readout. I hope this is not the case. Charging extra for a disagree light
would seem to be outrageous.
Just like with cars, plane manufacturers make a lot of profit on extras. If the disagree light was not legally required by the FAA then it is an optional extra and they have every right to charge whatever the market will bear for it. This enables them to price their base models lower for 3rd world airlines.
The real issue lies with FAA regulation – no one there (or the Boeing guys wearing FAA hats) seems to have thought thru the safety issues regarding relying on a single sensor. Apparently, the initial MCAS system as originally designed on paper only had authority up to 1/2 degree of trim, so it was thought to be a non-critical system, which does not need redundancy. Later, in flight testing, they realized that 1/2 degree of trim would not be enough to correct an imminent stall and they increased the authority of the MCAS software to 2.5 degrees of trim – just change 1 line of code, simple tweak. But they forgot to tell the FAA of this change or to reconsider whether this meant that the AOA system should now be treated as a critical item.
“an independent private engineering service would do the steps that Boeing’s employees were doing while nominally wearing FAA hats.”
A system similar to this exists in NJ for reviewing environmental contamination of former industrial sites. Formerly the state bureaucrats were supposed to do this job and of course they could not keep up and this was delaying reuse of these sites, etc. So the solution was to sub out the job to 3rd party environmental consultants hired by the land owner – they would be licensed by the state and could have their licenses pulled for giving false certifications that a site was clear. AFAIK, this works reasonably well (certainly better than the old system) but if you are being paid by a real estate developer and give him too hard of a time, then he might not hire you the next time and might shop for a consultant who is more liberal in giving out clean bills of health.
Whether you call the 3rd party an “employee” or an “independent contractor”, Boeing would still be signing his check so I don’t think that is a complete solution.
So what is said is that they increased the trim limits by 500% on a system that had
a reverse redundancy that doubled the chance of failure then they marketed a “fix” that would alert the pilots of a sensor problem and all this was within their “right” to charge extra as an option.
That is hard to agree with. I think charging extra for safety is a bad idea and they should be held
to account for this.
Agree! Charging extra for safety…..should be illegal.
“a reverse redundancy that doubled the chance of failure ”
What redundancy was that? IIRC, the MCAS alternated which sensor it used every time the plane was started, but since both sensors had an equal chance of failure switching between them wouldn’t change your odds.
“charging extra for safety is a bad idea and they should be held to account for this.”
Take cars for example. As of right now, automatic braking is an extra cost safety feature (but it is going to become standard in the next few years). If you get whiplash because I rear end you, should GM be held to account because they didn’t install automatic braking in every car from the day that it became available? The history of safety improvements is that they are first introduced as optional extras at the high end of the market and once they have proved themselves and come down in cost they can then be legally mandated on all vehicles. If you had to install them on every economy car from day 1 they might not be introduced at all, or an “economy” car would cost $30K because it was filled with cutting edge features.
Moving to planes, in Japan the regulators require TWO fire extinguishing systems in the cargo hold – I don’t know the history but usually this is because there was once an accident when 1 system was inadequate so you close the barn door after the horse is gone. In the US, the FAA only requires 1 system, but you can buy a 2nd one if you pay for it. This adds “safety” but the US regulators decided that it was not worth it. If an American plane crashes because a single system did not put out a cargo hold fire, should Boeing be “held to account”? Or the airline that failed to order the upgrade? Or the FAA? And if two are good, wouldn’t THREE be even better? Aren’t the Japanese compromising safety by having only two systems?
You are reacting emotionally and not rationally. There is no bright line thing called “safety”. Every aircraft, every manufactured product, needs to balance cost, safety, weight and other factors – you strike the balance where you think it is appropriate but you could always do more.
well I did not understand the two sensor logic but since you have two there is still
twice the chance of failure of the sensor itself (since it is still at least flopping in the breeze and not only that, since you cycled between the two on every other flight you were bound to pick the bad one. However, it also appears that you have all the answers so for that reason I will abandon the prepared text and pray you will continue to field questions from the floor.
The accuracy of the two AOA vanes is itself a complex topic, as this article discusses, amongst other issues: https://www.satcom.guru/2019/03/ethiopian-et302-similarities-to-lion.html
I am trying to understand the redundant logic in two units that are used independently without the optional disagree light. If an aircraft had 50 sensors the odds of one of the fifty being out of kilter would be 50 times having only one. Within 50 flights you would select that unit. if you compare that to the odds of 50 of them failing at the same time for 50 flights, you might feel considerably safer if agreement was required. This same logic would apply to two units. It seems to me that having two units and only using one of them at a time is no better than using one and in fact increases the chance of selecting a malfunctioning unit. Maybe this could be a kind of a reverse redundancy like having twin engines with not enough power to fly on one. If this were the case it would be crazy bad alright.
If you take your example of having 50 sensors, then yes, the odds that any one of them is bad is 50x greater than one being bad. I have some cheap LED lights from China that were designed with maybe 30 LEDs in series and they went bad in a very short amount of time because as soon as one failed the entire light failed.
But, in the Boeing implementation you are only using 1 out of the 50 sensors on any given flight, so your odds of choosing the bad one are 1/50th of a plane with only 1 sensor. So you are right back where you started from – 50x times 1/50th = 1. The odds are the same as if you only had one sensor.
But, on the third hand, as I understand it, the Boeing system rotated among the (two) sensors so if you had 1 bad sensor then you would have a 100% chance of choosing it eventually over the course of 2 (or in your case 50) flights. However, in the Indonesia crash, supposedly the immediate prior flight experienced the same problem which wouldn’t have happened if it was alternating sensors. Then again the plane may have been restarted twice with an in between movement to taxi to a storage area overnight or something.
BTW, someone is reporting that the disagree light was an $80,000 option, which is a pretty hefty markup on a $2 LED bulb.
Was there a reason why the MCAS had to be connected to the trim at all? Would it have been too slow if the MCAS instead shook the stick and shouted “Trim nose down” to pilot instead?
Used one at a time, it may be inherent that multiple sensors are a less reliable selection pool than one. Perhaps the safety lies in the selection by comparison and nothing else. It would sure be a weird safety directive requiring that the unit be energized twice if you did not have the optional comparison light.
But the plot thickens if a sequenced event in the lion air event is known to have used the other sensor and it also failed the next flight which would mean either both sensors had an issue at the same time or it was an issue downstream of sensor output. Curiouser and curiouser
Adding more hardware usually leads to less reliability. You have to be very careful about adding more stuff. In one of the comments above someone mentioned that if you have 50 sensors the probability that at least one sensor is bad is 50x the chance of a single one. But actually the probability that no sensor fails, when you have let’s say a 1 out of 1000 chance, is 1 – 0.999^50, is about 5%, which is 200 times greater. In spacecraft for critical systems they use a voting system, always with an odd number of sensors, and the system follows the majority. It was a multi-billion dollar mistake to only use one of the inputs, and to not give away the disagree light as clearly this system will engage easily given how powerful the engines are on the plane and how they are mounted. Bad engineering management, and really crappy programming. This bad programming example eclipses the previous record holder, the Toyota brake program bug, and will hopefully set a new standard for losses due to bad programming for many years to come.
Phil,
Could the data from Black Box and all systems be easily read from a another system (not intrusive) which does analysis and anomaly detection? This could be on the plane, use machine learning if applicable, and have a wireless display to an IPAD?
Could all of the data be released within hours/days/weeks of every flight in the US for anyone to analyze, an open-flight data sort of thing?
Besides the bureaucracy is there any reason this would not work?
My feeling is the sensors likely experience some minor short term failures perhaps which went unnoticed on many flights before the crashes. Also when there is a problem being able to say “what anomalies exist right now” can be helpful in spotting problems quickly.
Your thoughts?
Thanks
Your idea is good, but I think it would run afoul of certification bureaucracy (and therefore spectacular costs). That’s why I am a favor of something that can work all by itself just with a camera (see https://philip.greenspun.com/blog/2017/08/02/time-for-a-robot-assistant-up-in-the-dome-light-of-the-cockpit/ ).