Awasu » How do we know whom to choose?
Wednesday 17th December 2008 6:56 AM [General]

It's been a long time since my last non-Awasu post but my head's been in another place for a while. But now that the 2.4 release is baking, some thoughts on an article from Malcom Gladwell in the New Yorker that really caught my attention.

There are certain jobs where almost nothing you can learn about candidates before they start predicts how they'll do once they're hired. So how do we know whom to choose in cases like that? In recent years, a number of fields have begun to wrestle with this problem, but none with such profound social consequences as the profession of teaching.

I've written a few times before on my long interest in how people learn and our efforts to teach them [1] so this was definitely an article I was going to find interesting (despite the long ramblings about American football πŸ™„ ).

One of the most important tools in contemporary educational research is "value added" analysis. It uses standardized test scores to look at how much the academic performance of students in a given teacher's classroom changes between the beginning and the end of the school year.
It's only a crude measure, of course. A teacher is not solely responsible for how much is learned in a classroom, and not everything of value that a teacher imparts to his or her students can be captured on a standardized test. Nonetheless, if you follow Brown and Smith for three or four years, their effect on their students test scores starts to become predictable: with enough data, it is possible to identify who the very good teachers are and who the very poor teachers are. What's more β€” and this is the finding that has galvanized the educational world β€” the difference between good teachers and poor teachers turns out to be vast.

This last paragraph kinda sums up what's wrong with what happens in our schools. First, the discrepancy between what we think we're trying to teach, compared to what we are trying to teach, compared to what we should be trying to teach, is large. We think we're teaching our kids to be intelligent, creative thinkers, to be smart, to be independent, but nothing could be further from the truth. We make them wear uniforms, we make them sit in neat rows in the classroom, same as kids have had to do for the past few centuries. We urge them to be keen and enthusiastic and excited about whatever subject they happen to be studying at any given moment, until the bell rings at the top of the hour and they have to immediately drop everything they're doing, rush off to another room and instantly be keen and enthusiastic and excited about the next subject. And God forbid anyone should dare question (a critical component of being intelligent and creative) the status quo or any authority figure, dare do anything outside what has explicitly been deemed permissible by Those In Charge.

Instead, what we're really teaching them is how to pass tests. Tests are the raison d'Γͺtre of our schooling system [2] since they're what we use to decide who gets to go to college, who goes to trade school, who gets what job. And yes, there's not much of real value that can be captured on a standardized test. Trigonometry and conjugating verbs is just memory work, learning how to do processes [3]. Where is the school curriculum that includes the things that are really important (and strategies for determining how well they are being learned): doing your best, generosity, curiosity, courage, determination, passion?

And can it really be true that the educational world has been "galvanized" by the notion that some teachers are significantly better than others? Good grief, isn't that the case of just about any activity you care to name?! The idea of the "rock-star programmer" [4], someone who is 10x as productive as a "normal" programmer (whatever that might mean), has been doing the rounds in the IT world for a few years now, to little controversy since it's generally accepted that such people really do exist. But teachers and schools have always been resistant to the idea of performance testing and rankings, partly, I suspect, due to the ambiguity of what it is they're supposed to be doing, and even if that were completely clear, the difficulty in measuring something as squidgy as education. That, and the Lake Woebegone effect: pretty much every teacher would grade themselves as average or above but by definition, 50% of them are going to be wrong and nobody wants to be told that πŸ™„

Eric Hanushek, an economist at Stanford, estimates that the students of a very bad teacher will learn, on average, half a year's worth of material in one school year. The students in the class of a very good teacher will learn a year and a half's worth of material. That difference amounts to a year's worth of learning in a single year. Teacher effects dwarf school effects: your child is actually better off in a "bad" school with an excellent teacher than in an excellent school with a bad teacher. Teacher effects are also much stronger than class-size effects. You'd have to cut the average class almost in half to get the same boost that youd get if you switched from an average teacher to a teacher in the eighty-fifth percentile. And remember that a good teacher costs as much as an average one, whereas halving class size would require that you build twice as many classrooms and hire twice as many teachers.

No comment on this bit, I just thought it was an intriguing analysis and worth highlighting πŸ™‚

Hanushek recently did a back-of-the-envelope calculation about what even a rudimentary focus on teacher quality could mean for the United States.
After years of worrying about issues like school funding levels, class size, and curriculum design, many reformers have come to the conclusion that nothing matters more than finding people with the potential to be great teachers.

Um, isn't this freaking obvious? Obviously not :wall:

This is the same kind of thinking that leads companies to think that CMM is the be-all and end-all, the road to success. If you're in manufacturing then sure, having a well-defined, repeatable process is a Good Thing but it relies on a key assumption that the people carrying out the work are interchangeable [5]. While that may be the case for a peon on the assembly line, for anything that requires the slightest amount of intelligence or decision-making, there's absolutely no substitute for having people who know what they're doing and are good at it. These kinds of people are most definitely not fungible.

Picture a young preschool teacher, sitting on a classroom floor surrounded by seven children. She is holding an alphabet book, and working through the letters with the children, one by one: "A is for apple. . . . C is for cow."
[W]hat distinguishes her from other teachers is that she flexibly allows the kids to move and point to the book. She's not rigidly forcing the kids to sit back.

[The University of Virginia's Curry School of Education] team has developed a system for evaluating various competencies relating to student-teacher interaction. Among them is "regard for student perspective"; that is, a teacher's knack for allowing students some flexibility in how they become engaged in the classroom.

This touches on one of my (many) pet peeves: adults talking down to children. "Regard for student perspective," "allowing students some flexibility" are all about showing the kids some respect instead of taking a shut-up-and-do-as-you're-told approach. Sure, the teacher is still the boss but in an office, the boss would be expected to treat his staff with some respect and so similarly, a teacher should relate to the students as people, even if they are only little ones πŸ™‚

Pianta pointed out how the teacher managed to personalize the material. "'C' is for cow" turned into a short discussion of which of the kids had ever visited a farm. "Almost every time a child says something, she responds to it, which is what we describe as teacher sensitivity," Hamre said.

Same thing: respect. The teacher is talking with the children, not at them. I'm just gob-smacked that they have a special term for a teacher who responds to a child speaking to them :blink:

I've long held the belief that the language you speak influences the way you think [6]. For example, Asian languages often use different pronouns depending on the relative social status of who you're talking to (or about), so you're constantly thinking about where everyone fits into the social hierarchy so that you can speak with the correct amount of respect. In English, "teach" is often seen as a transitive verb i.e. something you do to something else e.g. I teach them. So we think of teaching as something that people (teachers) do to someone else (students). But really, the important thing is not that teachers are teaching but that students are learning, and "learn" is intransitive, it's something you do by yourself. The teacher is not the star of the show, the student is and a teacher should be there to guide and assist, not "teach".

Educational-reform efforts typically start with a push for higher standards for teachers β€” that is, for the academic and cognitive requirements for entering the profession to be as stiff as possible. But after you've watched Pianta's tapes, and seen how complex the elements of effective teaching are, this emphasis on book smarts suddenly seems peculiar.

This is a tough one. Clearly, simply being good at a subject doesn't necessarily mean that you will be good at teaching it. Lots of students struggle with maths because their teachers don't really understand it well enough themselves (!). Sure, they can do it well enough, but they don't have a deep enough understanding of it to be able to explain how it all works in such a way that the students "get it." And regardless of what you think our schools are trying to achieve, there's no escaping the fact that the people we hire to get our kids through school are the ones who didn't do particularly well at it themselves πŸ™„ The top students certainly don't queue up to become teachers, nor do most of the second-tier students, so the people who become teachers are usually those who were, at best, average students themselves. And we wonder why things aren't going too well in our schools :wall:

But even academic success doesn't help identify potentially good teachers. Quite apart from the importance of non-academic skills (such as empathy and patience), our entire school system is based on the fallacy that if a student does well on a test, it means they understand the material [7]. But this is simply not true. I know from personal experience πŸ˜€ that it is quite possible to do well on tests and not have a clue what's going on. I'm lucky to have a good memory and realized early on that I could get away with just regurgitating what my teachers wanted to hear πŸ™„ Real learning requires a deep understanding and we don't have a good way to test for that.

[A teaching certification or a master's degree] are expensive, time-consuming credentials that almost every district expects teachers to acquire; neither makes a difference in the classroom. Test scores, graduate degrees, and certifications β€” as much as they appear related to teaching prowess β€” turn out to be about as useful in predicting success as having a quarterback throw footballs into a bunch of garbage cans.

I used to work with a guy who swore blind that formal training (i.e. a degree) was an essential requirement to be a good teacher (although IIRC, his wife was a teacher so he may have been a bit biased πŸ™„ ) but funnily enough, he agreed with me that a Computer Science degree has almost no bearing on whether somebody will be a good programmer or not.

For my money, the most important requirement is motivation. You have to care enough to want to do a good job, you have to care enough to want to learn and practice to get better. Pair that with a willingness to work hard, hard enough to pick up the necessary skills, and as long as you're not a drooling moron, that should be enough for you to become at least reasonably good at whatever you might try [8].

Gladwell finishes up with this outrageous proposal:

In teaching, the implications are even more profound. They suggest that we shouldn't be raising standards. We should be lowering them, because there is no point in raising standards if standards don't track with what we care about. Teaching should be open to anyone with a pulse and a college degree β€” and teachers should be judged after they have started their jobs, not before. That means that the profession needs to start the equivalent of Ed Deutschlander's training camp. It needs an apprenticeship system that allows candidates to be rigorously evaluated. Kane and Staiger have calculated that, given the enormous differences between the top and the bottom of the profession, you'd probably have to try out four candidates to find one good teacher. That means tenure can't be routinely awarded, the way it is now. Currently, the salary structure of the teaching profession is highly rigid, and that would also have to change in a world where we want to rate teachers on their actual performance. An apprentice should get apprentice wages. But if we find eighty-fifth-percentile teachers who can teach a year and a half's material in one year, we're going to have to pay them a lot β€” both because we want them to stay and because the only way to get people to try out for what will suddenly be a high-risk profession is to offer those who survive the winnowing a healthy reward.

Food for thought, indeed. But he hits on the crux of the problem, even as it stands now: we don't value what our teachers do. We say we do, but we don't, not really. Jennifer Lopez gets mega-bucks and our love and adulation for some lousy acting (and even worse singing), teachers get stuck with class sizes of 30 or 40 or 50 and having to pay for photocopies out of their own pockets :wall: Parents with money give some of it to private schools who use some of it to attract better teachers but the underlying problem remains; we don't really care about what's happening in our schools, not as much as we do the next J-Lo record 😐

What does it say about a society that it devotes more care and patience to the selection of those who handle its money than of those who handle its children?


To finish on a completely unrelated note, the first thing I thought of when I saw the cartoon at the top of the article was the movie La Cité des Enfants Perdus (The City Of Lost Children) which features a pair of conjoined twins as one (two?) of its characters. I looked it up on IMDB and was surprised to find that the director (Jean-Pierre Jeunet) also did Alien: Resurrection so I dug it out and watched it again. La Cité is visually amazing and you can see a lot of the same style in the Alien movie. I certainly saw it in a different light (instead of in the context of the dreadful Alien 3). Check 'em out, they're great films 😎

[1] The two are often diametrically opposed 😐
[2] And don't get me started on No Child Left Behind, a marvelous example of the incredible distortion an over-emphasis on test results can have on what happens inside the classroom.
[3] For students learning maths, I call it the "sausage machine" approach. When they're looking at how to solve a particular type of problem, they simply have to remember how carry out each step of the process. Then they just feed the numbers in, turn the handle and out pops the answer. Note that they don't necessarily have to understand what they're doing, they just have to be able to turn the handle.
[4] Can't stand the term myself 😐
[5] McDonald's are CMM 5. 'Nuff said 😐
[6] In programming as well as in the real world. A Java programmer tends to approach a problem in a much different way to someone used to thinking in Prolog or assembly language.
[7] Consider the case where if A is true, then B is also true. It does not therefore follow that if B is true, then A must be true i.e. from "If an animal is a dog, then it has four legs", you can't turn it around and say "If an animal has four legs, then it is a dog." But we do this all the time in schools; we take "If a student understands the material, they will do well on the test" and conclude "If they do well on the test, they understand the material."
[8] Gladwell has a new book, "Outliers", out now whose main premise is that success is as much a result of hard work and other factors (say, dumb luck πŸ™„ ) as it is raw talent. "Blink" and "The Tipping Point" were thought-provoking, even if you didn't necessarily agree with his ideas, but with "Outliers", he's definitely preaching to the choir here.

2 Responses to this post

Thanks Taka, nice discussion. For those who have a love of educational reform, the theosophical ideas proposed in the 1880s were radical but also commom sense. They had quite a widespread effect and inspired the classics "What is and What might be" by Edmund Holmes and its sequel "In Defence of What might be".

Enjoy the extract below, yours, Will W

from "The Key to Theosophy" by H.P.Blavatsky (1889)

ENQUIRER. Yes; but I have heard you congratulate yourself at the ever-increasing number of the Agnostics and Atheists in our day, so that it appears that even people trained in the system you abuse so heartily do learn to think and reason for themselves.

THEOSOPHIST. Yes; but it is rather owing to a healthy reaction from that system than due to it. We prefer immeasurably more in our Society Agnostics, and even rank Atheists, to bigots of whatever religion. An Agnostic's mind is ever opened to the truth; whereas the latter blinds the bigot like the sun does an owl. The best -- i. e., the most truth-loving, philanthropic, and honest -- of our Fellows were, and are, Agnostics and Atheists (disbelievers in a personal God). But there are no free-thinking boys and girls, and generally early training will leave its mark behind in the shape of a cramped and distorted mind. A proper and sane system of education should produce the most vigorous and liberal mind, strictly trained in logical and accurate thought, and not in blind faith. How can you ever expect good results, while you pervert the reasoning faculty of your children by bidding them believe in the miracles of the Bible on Sunday, while for the six other days of the week you teach them that such things are scientifically impossible?

ENQUIRER. What would you have, then?

THEOSOPHIST. If we had money, we would found schools which would turn out something else than reading and writing candidates for starvation. Children should above all be taught self-reliance, love for all men, altruism, mutual charity, and more than anything else, to think and reason for themselves. We would reduce the purely mechanical work of the memory to an absolute minimum, and devote the time to the development and training of the inner senses, faculties and latent capacities. We would endeavour to deal with each child as a unit, and to educate it so as to produce the most harmonious and equal unfoldment of its powers, in order that its special aptitudes should find their full natural development. We should aim at creating free men and women, free intellectually, free morally, unprejudiced in all respects, and above all things, unselfish. And we believe that much if not all of this could be obtained by proper and truly theosophical education.

>>> For those who have a love of educational reform, the theosophical ideas proposed in the 1880s were radical but also commom sense.

And over a hundred years later, common-sense still appears to be lacking... 😐

Completely side-stepping the question of the effects of religion on how we think... πŸ™„

>>> Children should above all be taught ... to think and reason for themselves.

I've never really been a big fan of John Taylor Gatto since his basic premise is that the failure of our education system is one big conspiracy but nevertheless, I think that it's not completely off the mark either. We don't *really* want kids who think for themselves and act independently - virtually no teacher wants their students doing anything other than obediently going down the path that has been mapped out in a meticulously prepared lesson plan - and it continues on into adult life. How many bosses or politicians want their people thinking independently, checking on and questioning what they're doing? The failure of our schools to "[teach] self-reliance, love for all men, altruism, mutual charity, and more than anything else, to think and reason for themselves" may not be deliberate but it may be kinda inevitable given that those in power don't have much incentive for an alternative outcome.

Have your say