Making data-driven education reform decisions is great — unless the results come from flawed questions.
The results of an opinion poll will vary, and not by a little, as a function of how the questions are phrased. “Do you favor special preferences for minorities in the form of affirmative action?” will attract many fewer favorable responses than “Do you favor efforts to help minorities get ahead in order to make up for past discrimination?” And then, of course, there are “push polls,” which only pretend to sample people’s views while attempting to influence them: “Would you be more or less likely to vote for Congressman McDoodle if you knew he was a practicing Satanist?”
I find myself thinking about how much more—and less—there is to polling than meets the eye whenever I come across one of those surveys that school administrators like to distribute to parents. I have to assume these are not intended as the equivalent of push polls, that there’s a sincere desire to be responsive to the community and an honest pride in being able to cite “data” to judge the effectiveness, or at least the popularity, of school policies. (Data good.)
Too often, though, survey questions reflect a set of hidden assumptions about what’s desirable, or inevitable. Moreover, they help to cement that view of education into place. The issues about which people are invited to express their opinions are most revealing for what’s not being asked: the underlying ideological commitments that aren’t open to question. The more we’re asked to offer feedback about how well the school is doing x, the less apt we are to ask why x is being done in the first place, and what might be done instead.
Noam Chomsky put it this way: The smart way to keep people passive and obedient is to strictly limit the spectrum of acceptable opinion, but allow very lively debate within that spectrum—even encourage the more critical and dissident views. That gives people the sense that there’s free thinking going on, while all the time the presuppositions of the system are being reinforced by the limits put on the range of the debate.
So how does this play out in a school setting? Well, here’s a question from a parent survey that someone just forwarded to me from an Illinois elementary school: “Overall, the teaching (delivery of instruction) at _________ School is successfully preparing my child for his/her next academic level.” (Strongly agree / agree / unsure / disagree / strongly disagree)
At least two premises inform this question, each of them a subliminal lesson for the parents who read it. The first is that teaching can be defined as the delivery of instruction. Students are the recipients of whatever is given to them, which means they are regarded—and expected to remain—fundamentally passive. Teachers, meanwhile, can be evaluated on the basis of how many facts or skills they have succeeded in delivering. When education is conceived this way, it would follow that traditional methods of conveyance (lectures, worksheets, textbooks, etc.) and evaluation (tests and grades) are entirely appropriate. Indeed, their absence might even seem suspicious. If, instead, the parenthetical phrase in the survey question defined teaching as, say, “facilitation of active discovery of ideas,” then parents might ask why their children were still being made to fill out worksheets or regurgitate answers on quizzes. But it doesn’t. So they don’t.
The second premise of this survey item is that what happens in each of the school’s classrooms should be judged on the basis of how well it prepares kids for whatever’s being done in other classrooms. Whether there’s any merit, any intrinsic value, to those practices is apparently beside the point. Excluded from consideration are various other possible criteria, such as whether students are becoming more excited about learning, whether the curriculum is responsive to their interests and questions, and so on. (In fact, the particular school survey I’ve been reviewing includes a second question that asks whether parents believe the curriculum—as opposed to the teaching—is successfully preparing children for what comes next. Such repetition underscores the message that preparation is the point.)
I’ve written elsewhere about this rather curious rationale for subjecting younger children to dubious educational practices, including standardized testing, competition, grades, and homework. It doesn’t matter that these things actually aren’t beneficial, particularly for kids of this age. It doesn’t even matter if they’re downright harmful. All that counts is that people are going to subject children to these things later, so our obligation is to get them ready by doing these same things to them now. I call this the BGUTI (Better Get Used To It) defense, and the survey in question not only fails to consider its implications; it discourages such reflection by subtly communicating that BGUTI is perfectly legitimate. Indeed, it’s a key criterion by which teachers and schools are supposed to be evaluated.
Once you start noticing the implications of what’s being asked, these parent surveys come to seem not merely less useful but positively insidious. You see a question about whether such-and-such is “promoting student success,” and you wonder why there isn’t a question about how the school is defining student success: Is it anything more meaningful than grades and test scores?
Or you come across a survey item that asks you to agree or disagree with the statement “My child clearly understands school routines and expectations” (another actual question), and you want to replace it with: “Children are invited to participate in creating school routines and expectations rather than just doing what they’re told.”
You begin to suspect that parental feedback about issues like the latter is almost never sought in schools where there’s little doubt about what the answer would be. (Parents might write in an option to the right of “strongly disagree” called “Ha!”) And your stomach sinks as you anticipate the pride with which administrators will soon report that an overwhelming majority of our parents believe we’re doing a damn fine job! Eighty-seven percent agree or strongly agree that we cram the facts into their children that the middle schools expect them to have been taught! Ninety-one percent report that their children can recite the prohibitions and punishments we’ve unilaterally devised and imposed on them! Is this a great school, or what?
One of the topics about which parents seem to be surveyed with particular frequency is homework— and here we find a striking illustration of Chomsky’s point about limiting meaningful discussion by eliciting opinions only within a narrow range. Probably the most common item—usually asked of parents rather than of the students themselves—is what might be called the Goldilocks question: "Do you think your child receives too much / too little / about the right amount of homework? "The underlying assumption, of course, is that it's necessary to assign at least some homework. The question is designed to exclude critical responses to the whole idea of making kids work a second shift of academics after the school day is over.
One might be tempted, then, to right the balance by creating some mischievous alternative survey questions:
1. Given that research fails to find any benefit to homework for students who are younger than about 15, do you think they should be assigned homework anyway? Why?
2. Should children be required to devote their afternoons and evenings to academic tasks—even at the expense of their social, artistic, or physical development—or do you believe six or seven hours a day spent on such tasks is sufficient?
3. In your opinion, who should determine what happens during family time: the families themselves or the schools?
4. How likely do you think it is that homework will lead to optimal learning (and enthusiasm for learning) if all the students in a class—regardless of differences in their backgrounds, interests, and aptitudes—are required to complete the same assignment?
Too controversial? Maybe. But these questions are no more biased than the usual ones. It’s just that the typical bias is harder to detect because it reflects and sustains the status quo. In any case, here are some questions that aren’t loaded one way or the other and really ought to be asked:
* To what extent does your child's homework seem designed to deepen his or her understanding of important ideas? In your opinion, is it actually having that effect?
* Many educators and parents believe that the most important criterion by which school practices should be evaluated is whether they help children to become more excited about a given topic and about learning in general. So is the effect of your child's homework on his or her desire to learn generally positive, neutral, or negative?
* Do you think it makes more sense to assign homework on a regular basis or only when it’s truly needed?
* Would you favor a voluntary system whereby families that want additional academic assignments after school could receive them, while families that would rather allow their children to pursue other activities could opt for no homework?
Have you ever seen comparable questions about traditional education practices on a survey? Neither have I. Yet thinking about such practices and the values lurking behind them is exactly what parents and educators should be doing: not merely assessing how well the school is pursuing the agenda that’s been set for it but reconsidering the agenda itself —whom it benefits and whether it’s consistent with our long-term hopes for our kids.
A slightly abridged version of this article appeared in the Washington Post. Copyright 2011 by Alfie Kohn. Reprinted with the author's permission. For more on this topic, please see www.alfiekohn.org.