In the past couple of weeks, we’ve put about 70 people through helicopter ground school (outline of topics), followed by a 25-question multiple choice exam. The goals of the ground school include developing a student’s understanding of qualitative physics such as the Bernoulli effect (through conservation of energy), of basic aerodynamics (no equations), and of the practical requirements that lead to the helicopter being constructed the way it is (e.g., Why is there a tail rotor?). The class is conducted as a discussion around a conference table with about one third of the time devoted to students answering questions from the teacher (me so far!). Some reading is assigned prior to the class, but mostly the oral questions can be answered based on material presented in the class and with commonsense physics reasoning.
In looking over the 70 exam scores, what has surprised me the most is the lack of predictive value of a bachelor’s degree in science or engineering from an American university. One customer showed up wearing a Boston University Engineering sweatshirt, confirmed that a bachelor’s degree in engineering ($150,000?) had been obtained two years ago, and proceeded to score 6/25 correct on the exam (the all-time low score and worse than picking answers at random).
People who’ve done the best in the class and on the exam:
- certificated airplane pilots
- foreigners with science or engineering degrees from universities in India, Germany, and Israel
- Americans with advanced degrees in science or engineering
Some Americans who held bachelor’s degrees in science/tech did reasonably well, but no better than those who’d majored in Art.
Is this really that surprising?
It squares 100% with my subjective professional experience in software consulting.
Ability to make sense of new information quickly and apply generic problem solving skills to that new information, I’ve found to have little or no correlation to education.
Rob: Is it surprising that someone who had taken at least one year of Physics at a $50,000/year university wouldn’t do any better at understanding and applying conservation of energy and conservation of angular momentum than someone who had never taken any Physics at all? It surprised me. This is not “generic problem solving”, but a machine that is subject to physical laws. About $400,000 of society’s money had been invested in the Boston University graduate’s 16 years of school. I would have expected some return on that investment.
A qualitative understanding of physics is not so easy to come by; just think of Feynman’s story about Einstein’s personal assistant (here).
As a second data point, the best living physicist (by h-index), who happens to be American, did his undergrad in history.
Can you give an example of the kinds of questions you were surprised that people could not answer? I’m curious.
pdwalker: Keep in mind that this comes after 1.5 hours of the 2-hour talk. And that they were supposed to have read about 150 pages of FAA tutorial material prior to showing up. An example would be “Suppose that, at high altitude on a hot day when the air is thin, when trying to land in a confined area, the pilot pulls up on the collective to increase the main rotor blade pitch, but the engine is already at full throttle, producing as much power as it can. What happens to the blades? Does the rotor system produce more lift?”
[the answer is “the blades slow down” and “no; lift varies with airspeed squared [as previously explained] and the drop in airspeed results in a reduction in lift, so the helicopter sinks faster, despite the increase in collective”]
Earlier in the lesson we ask simpler stuff, such as “If we have a helicopter parked on a frozen lake where nothing is rotating and we start up the engine to spin the main rotor system, what else happens to the helicopter?” We are looking for a student to say “The fuselage wants to spin the opposite direction.”
Doesn’t surprise me at all. Bachelor’s degrees from U.S. colleges and universities are virtually worthless. I’ve seen comparable incompetence time and again in IT. Consider this blog post and its links: http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html
Daniel: Thanks for the link. I didn’t mean to imply that these folks were incompetent. After all, they had not studied helicopter aerodynamics for four years at school, so the fact that weren’t good at helicopter nerdism does not strike me as “incompetence”. My point was that the educational system in the U.S. is wasting a remarkable amount of students’ time if 16 years of school with a science/technology emphasis does not give a graduate an edge over someone with fewer years of school or an art/art history emphasis. It isn’t fair to blame the graduates; they didn’t show up swaggering and saying how great they great they were going to be at physics/aerodynamics. I blame the people who pocketed their tuition dollars.
philg: Ah, thanks. I see your point. You are asking questions based on the principles of basic physics, or questions based on material they have been taught recently.
I’m not surprised that people would have problems with the first of those questions. A lot of people just don’t “get” physics and avoid the subject as soon as they are able to. Solving these kinds of problems requires a general problem solving ability that utilizes the knowledge they should have at hand. (For what it is worth, I found that the physics classes I took in university provided the best kind of training in solving these kinds of problems – my professors were devious sons o’guns)
The second kinds of problems (We just taught you that!) might be either a lack of experience in studying/note taking or an inability of that person to understand the material presented.
That you are seeing both problems doesn’t actually surprise me, given the increasing enstupidification (no, it’s not a word, but it should be!) of the public education system in many places. That it has reached the point where a university graduate with one year of physics is unable to handle those junior high school questions means that the destruction of the education system, as a tool for education, is basically complete.
pdwalker: On average, I think that our students have done well. Some did the reading carefully prior to showing up. Some were obviously bright and thoughtful during class. This isn’t a posting about how Americans are not as smart as foreigners or how our education system has collapsed (for all I know we would have gotten the same result 30 years ago). What surprised me is that in a discussion that is mostly about applied physics, the humanities and business majors from the U.S. have done just as well as those who majored in science or engineering.
Phil,
you mentioned that “foreigners with science or engineering degrees from universities in India, Germany, and Israel” did well. These are the most intelligent people of their engineering classes – they ended up in the Boston area because they were intelligent enough to get accepted into highly competitive grad schools in the Boston area, or they were so smart at what they did, just with a bachelor’s that somebody paid for an H-1B for them. The US by contrast, you’re going to have the entire engineering class sign up for your helicopter class, not just the smart ones. If you charge $150k as you mentioned for a degree, you have to have some idiots pay for all the administrators and diversity activities. So you’re comparing the over-achievers with the average american. I bet if you look at just the BS in Eng. students who did well in their studies, you will find that they performed at the same level as the advanced graduates, and their foreign peers.
ps I live in the piston general aviation flight path for John Wayne Airport. If a certificated airplane pilot did poorly in your class, I am really concerned!
Judging from my experiences in Israel (up to and including a Ph.D. in math) and the US (5 years of post-docs), it is hardly surprising. With few notable exceptions, US B.A./B.Sc. is mostly about four things (in no particular order):
1. being politically correct (everyone “deserves” a degree)
2. coping with the very diverse level of the K-12 education
3. learning how to learn
4. bringing up a “well rounded person”
Israeli higher education is about teaching you a profession. As far as I know the situation is very similar in all of Europe. The most striking example of this difference I’m aware of is the first year of Math/Phys/CS: In the US first year is usually rather easy; in Israel one has to take two semesters of real analysis and two semesters of linear algebra – by the end of this year half the students would have left the program.
While I think that political correctness should not interfere with quality, the diversity of the K-12 education is a real problem, some people would certainly benefit from being well rounded, and all people would benefit from learning how to learn (the last one is a huge drawback in european programs compared to good american ones).
I started in the computer business at an aerospace company in the mid 60s before there were degrees in computer science. The best programmers that I knew had degrees in diverse fields like music and library. I am not sure that even the degree helped but to earn twice the salary it was required. I started while I was an undergraduate and although my job or work did not change, my pay was doubled the day I got my degree.
I got my degree in electrical engineering and had worked a summer job at a electronics company where were doing things that were not being taught at the school.
Perhaps the engineering student didn’t enjoy physics any more than the business majors did, thus the retention level was low enough that after 4 to 6 years, the knowledge retained is as minimal as what a business student learned in high school. It wouldn’t suprise me that an engineer working in the field for 2 years doesn’t use a ton of physics. Ok, well it did at first, but then I saw how much Solidworks actually does for you, how compartmentalized your task as an engineer is, and how much trial and error there seems to be without comparison the theoretical values. Granted the engineers I know tend to make food plant conveyor systems, not space shuttles.
I think rapid task switching also contributes to a lack of retention/caring. If I had to do nothing but physics for 4 or 8 weeks instead of 4 classes for 16 weeks, I have a feeling that retention would be better after 4 years if I were successful at it. As it stands now though, given 4 classes competing for time and energy, if I like one, am moderately interested in 2 more, and dislike the 4th, I’m probably not going to remember the 4th.
The BU student probably slipped past the system — he didn’t know his stuff, and probably was not inclined to learn, but thanks to grade inflation, probably passed the relevant courses with either a B or a B-minus.When I attended grad school in the Boston area (twice), any number of incompetent classmates squeaked past, thanks to “group projects” and the like.
From my days as a student in India, I can tell you that your own professor cannot influence your grades. All examinations are corrected anonymously by a professor in a college different from yours. Therefore, chances for grade inflation by one’s own professor are virtually nonexistent.
That said, I would not imply that an Indian education is anywhere close to being perfect. There are issues other than grade inflation: most professors teach to the test. It would be a favorite pastime of ours to bring up the tests from the last six years for a given subject (say, Engineering Math), and create a probability rating for which chapters from the book show up in the test how often. We could game the system well enough to get by with decent grades learning just half the number of chapters in the syllabus.
Jagadeesh: Thanks for the insight into how things work in India. I’ve written a few times about how little sense it makes for professors to grade their own students (e.g., this 2007 posting). I like to ask college professors “How come more people fail the MCSE exam than fail your supposedly challenging class?”
As an aspiring fixed-wing CFI (who has been flying about 12 years now and doing computer hardware/software engineering for 10) I find your observations really interesting. I was assuming that once I started teaching flying, I’d need to be prepared with analogies and abstractions suitable for a range of students from ‘the housewife’ (yes, its sexist but I hope illustrates my point) to ‘the engineer’; someone whom an explanation of a concept or principle rooted in physics is appropriate. Now you’re making me second guess my bedtime reading of Aerodynamics for Naval Aviators to hone my skills in presenting to the latter.
jessek: You will need to teach all of your students to understand the physics of flight. There is no other way to develop the long-term understanding that is necessary for a person to be a safe aviator. The take-away from my experience with these 70 is that it is unwise to assume that a U.S. college degree in science or engineering is going to enable the student to catch on quicker.
Learning and thinking per se isn’t really taught anywhere. They probably should be.
I think there are people who have somehow developed the ability to easily learn, absorb, and apply information by themselves, and people who haven’t. It seems to be very rare that a person consciously overcomes the lack of this ability and actually learns that ability. I would be surprised to expect schooling and education to make any difference in that.
The ability is not directly measured in tests – those who have it do have the personal edge that they generally have to work less in order to come up with the right answers. However, in school there are many ways to the right answers and not much attention is paid to which one the student ended up using.
Since this ability to learn, absorb, and apply is quite invisible, both kinds of people end up both in scientific and arts curricula. Some don’t end up anywhere but are still down-to-earth smart, capable of learning fast, absorbing information, and applying it in practice.
While having this ability and deciding not to exercise it through schooling and studying leaves a person at a slight disadvantage there are, conversely, people who have managed to finish all the hoops and loops of education without ever having had this ability.
A bit OT, but people of all educational backgrounds have similar trouble with economic intuition. You example of slower blades from increased collective reminds me a bit of Krugman’s frustration with people forgetting the dynamical implications of the zero bound on interest rates.
Of course the bigger problem is that the economics profession *itself* has no consensus model of economic dynamics (at least in macro), so the ‘debate’ is a lot of ignorant ranting from all sides.
yason: You suggest that “thinking” be taught by schools. I believe that this is what our public schools claim that they are already doing. American students don’t test as well in knowledge and skills as students from many Asian and European countries, for example, but they supposedly have been given some unmeasurable advantage in having learned “critical thinking”.
My personal experience as a teacher of computer nerdism and airplane/helicopter nerdism is that students learn by actively solving problems and getting help when stuck. So it wouldn’t help to add a “how to think” lecture to a university curriculum. The only thing that helps is to throw out the lecture system entirely and replace it with active open-ended problem solving (essentially what Olin College of Engineering has done). If a student is sitting passively in a lecture he or she is probably not learning anything. If a student is sitting alone in a dorm room trying to do a homework assignment, he or she is probably not learning efficiently (due to the myriad distractions of modern life plus the consequences of getting stuck).
I sometimes wonder if we need more hands on work for our engineer in training, more field engineer vocational internships or something of the like. Learn better execution and application skills by doing.
In regards to aviation, I build free flight and RC airplanes, usually of 1920s vintage designs, and have amassed a working knowledge of flight dynamics probably more from my failed designs which are analyzed and improved upon.
I’ve had the chance to encounter, aft-CG induced stall-spins, power-pitch oscillations, accelerated stalls, flown in what would be hurricane scale winds, thermalling, wing icing, all of which has resulted in more than a little broken balsa, foam, and knowledge gained. Working with energy management is fantastic and doing dead stick landings is terribly addictive.
I’m not a full-scale aviator yet, but I’m now scouting out flight schools, with the intention of going the LSA route. Actually I’m sort of troubled with the suspicion that I like the aircraft, the hardware more than the actual aspect of flying. I wouldn’t mind being an A&P, but to give up my good paying web development job for aviation industry uncertainty?
At any rate, I’d like to think experience based study in any aspect of engineering can lead to better critical thinking for all aspects of engineering.
Peter: The folks who love to build airplanes are indeed a mostly disjoint set from the folks who love to fly airplanes. That’s one reason why homebuilt airplanes crash so often. The builder spends four years in a garage building rather than taking flying lessons. He then goes to the airport with the bare minimum license and currency and initiates a series of test flights. This is not a recipe for safety.
Built into this is a an assumption that your students wanted to pass the test. Maybe they were looking for a few hours of fun in a helicpopter for a great price, and didn’t care about their score.
Did the students need score at a certain level in order to take a ride?
Jerry: Do the students need to pass the test? Of course not! Self-esteem is the most important value in our society and flight training is no exception. We have modeled ourselves after the local public school system, which has a tremendous track record in producing young people who think very highly of themselves. The student gets credit for sitting through the class. If they fell asleep while doing the prereading we give them an A for effort. I already have tenure because I’ve been teaching for more than two years. I will be taking three months off this summer, with full pay. I work only half a day on Wednesdays because I need to talk to some other teachers and fill out some paperwork. I am looking forward to retiring at age 50 with a full pension.
We’re so enthusiastic about flying with people who aren’t prepared that we schedule folks with the lowest scores with priority. Then we get into one of our $350,000 helicopters and ask the student to lift the machine up into a hover. One of the great things about the helicopter is how stable it is so even if the student moves the cyclic an extra inch or two it really doesn’t matter.
When it comes time for them to get a license, the procedure is very similar to high school graduation. The FAA examiner flies with the applicant. If the applicant can’t fly, the examiner says “Based on testing it would appear that this person hasn’t learned anything and has no skills, but I know that he sat through 40 hours of instruction so he must have learned something, even if I can’t see or or measure it. Therefore I will give him a pilot’s certificate.”
So, if they didn’t need to pass to go for a ride, then you answered your own question. Wanting to do something once is a lot different than wanting to do it as a career or even multiple times. Given that you priced this quite low, and had people ask if it was just for a “ride,” I bet you had some people who weren’t motivated by getting a good score.
Jerry: I didn’t intend my preceding comment to be read literally. In aviation, a student does not advance to the next step until he or she has demonstrated proficiency. As explained in a Web page that students see before they show up, http://philip.greenspun.com/flying/pre-ground-school , passing the written exam is a prerequisite for being signed off by the instructor as ready to take the intro flight. We also explain the process at the beginning of the class. As is also common in aviation, students who fail the exam have the opportunity to go home, do some more reading, repeat the class, and take the exam again.
It would not be safe, legal, or ethical for me to sign off a poorly prepared student to go fly with one of the 5 other instructors at East Coast Aero Club.