Some tiny creature, mad with wrath,

Is coming nearer on the path.

--Edward Gorey

Location: Pittsburgh, Pennsylvania, U.S. Outlying Islands

Writer, lawyer, cyclist, rock climber, wanderer of dark residential streets, friend.

Tuesday, March 08, 2005

Teaching to the Test

Joshua Benton, a friend of a friend (it's a one-directional connection, though, so if he sees this he's going to wonder what in God's name I'm talking about), who blogs here, directs readers of his blog to his Dallas News column (subscription required, but log in as deanna[at], PW: deanna, care of bugmenot) concerning the new writing section of the SAT. Here's the heart of the column:

You'd think writing teachers – used to being shunted aside in recent years while reading and math get all the hype – would be thrilled their subject is getting the extra attention.

But some of them worry that the tests could be encouraging bad writing, not good.

"The risk is that the kind of writing that does well on a test becomes the only kind of writing that gets taught," said Richard Sterling, executive director of the National Writing Project, which trains writing teachers. "It ends up being incredibly reductive, and that does real harm to children."

Here's what I mean: The kind of writing that gets you a high score on most standardized tests isn't necessarily pleasant to read. It's overstructured, stilted and dry.

People have figured this out. For instance, the Princeton Review has analyzed the new SAT writing section to figure out how best to boost the scores of its teenage customers. Its advice: Be boring.

"The grading of these essays is going to be very superficial," said Andy Lutz, Princeton Review's vice president of program development. "It's easy to coach kids how to write in a formulaic way that will score well."

If a kid wants to do well on the test, he says, the key is to write in neat handwriting, have clear structure, and don't take any chances. Attempts to be interesting are at least as likely to be punished as rewarded.

In conclusion, he asks:

"These tests reward you for being straightforward and long-winded," he says. "In the real world, it can be good to take rhetorical chances and write something different. But on these tests, following the formula gets rewarded."

Is that the formula to create good writers?

Elsewhere, I have mounted a spirited attack on the problems inherent in the increasing prominence of standardized tests. Unfortunately, I have done so under my real name, so I can't send you there. It's not terribly important, however, what I said there, because I'm about to contradict myself.

My prior objections have focused on the fact that, with a teacher's tenure and a school's funding tied to students' performance on standardized tests, pedagogical innovation is stifled and education homogenized; time that might be spent tailoring lessons to the particular needs of a given class, according to the inclinations of a professional educator who knows better than anyone the needs of the students before whom he stands each morning, is squandered in ensuring students are prepared to cough up the right answers to multiple choice tests. Weeks, or months, and to a lesser extent entire curricula, are spent teaching to these tests with necessarily unfortunate results. For those of us who have studied for standardized tests -- in my case, most memorably, the LSAT and GRE; I don't recall preparing very intensively for the SAT -- we know that study resources and classes teach one strategies to glean answers one doesn't actually know, to guess intelligently, to massage the epiphenomena of the test into the best possible result, notwithstanding a dearth of actual knowledge or the general irrelevance of the material at hand. Why would it be different when the standardized test in question is administered to younger children, if a teacher's success is evaluated by reference to student test results.

I don't know an educator who doesn't despise this trend, this renaissance in the value placed on standardized tests that recalls the initial enthusiasm for standardized tests that roughly coincided with World War II and the draft. Contrary to what defenders of the testing regime on the right suggest, this antipathy of front-line teachers for the testing regime in my experience has little to do with the fear it engenders in the hearts of the many incompetent teachers the right would have us believe have infected our school systems, but rather arises because teachers are not being allowed to do the things they expected to do as teachers. They are assembly line managers, making sure a given chip is placed in the correct socket. There are other jobs like that; people who study to be teachers didn't want them.

But writing is different than knowledge. Set aside, for a moment, the logistical issues of grading hundreds of thousands of exams each SAT season; we'll assume for purposes of discussion the best case: that each exam is given the tender attention of a trained reader such that creativity might be rewarded. We also can take as granted that what Benton and will probably often agree is good writing also can be effective writing -- like his, for example, and hopefully mine.

My question is, to what extent can we expect schools to take responsibility for turning out good writers, and in what sense? We don't ask elementary schools to turn out good painters or good clarinetists, do we? Writing, as I use the word, principally refers to an artistic endeavor -- even, God willing, in the professional sphere, in law, in journalism, in advertising. Of course, in each of these areas and dozens of others what constitutes good writing varies, but I would submit that a truly talented writer can adapt successfully to virtually any context. Whether I am "truly talented" or not, I have, in my adult life, written everything from lab reports to users manuals to fiction (long and short) to direct mail pieces to corporate annual reports for shareholders to trade magazines and, most recently, law, lots and lots of law. In all of these situations, whatever difficulty I encountered was associated with the non-writing aspects of the position; writing itself is always a pleasure, and in the worst of my professional situations has provided safe harbor in the storm, a few hours out of a miserable day when I can do what I love to do most, negotiating puzzles of expression with nothing in my quiver but words and a vague, often conjectured, sense of audience.

I don't think that can be taught. Not really.

What can be taught, however, is organized thinking and expression. Benton, and others, bemoan the five-paragraph essay most of us remember from school, which is now evidently the format of choice for SAT takers: an Introduction, in which the thesis is set forth and three supporting / subordinate propositions are introduced; three body paragraphs, in which each of these ideas is developed, one to a paragraph, in a way that leads to and supports the thesis, and a conclusion paragraph, which in effect is merely a reconfiguration of the introduction paragraph. There's nothing wrong with this exercise, in my view.

Occasionally, someone accuses me of being a little weird to listen to because I always have an introduction, a middle, and a conclusion when I tell a story or argue a point. Among academics in the humanities, I've found that this morphs into the compliment that I have a good sense of narrative. Such as it is, this is because I often unconsciously adopt something very like the five-paragraph format when I speak.

Sure, it's restricting, but so many other things are in this life as well. A talented writer should have no problem being one of the highest scoring students in a five-paragraph essay-writing endeavor, and an unskilled writer who knows how to follow instructions still, I imagine, will not be scored quite so well. Writers may be made, rather than born, although I'm not convinced -- but in any case, I dare say that writers have been formed long before they sit down to take an SAT, even if they haven't figured it out just yet. The test won't change anything, nor will the teaching to the test that might occupy a significant fraction of the time a junior in high school spends in English class make a good writer a lesser one. The few weeks allocated otherwise, obversely, won't turn a middling writer into a good one.

I think the worst you can say about the putative rigidity of the SAT's new writing section is that there may be a leveling effect, inasmuch as it may be possible for mediocre writers to score highly given adequate preparation. But if they can write an adequate five-paragraph essay, why shouldn't they score well? After all, in real life, talented writers tend not to be valued for their talent, or rather their talent goes largely unexamined. Most positions require writing competency, as do most programs in higher education. If a student can learn how to write a coherent five-paragraph essay, it's safe to send the message to universities that the student is a competent writer, as manifest in a reasonably high essay score on the SAT. And most universities are far more interested in assessing a student's writing competency than anything else, with obvious exceptions for certain departments and programs that center on skillful writing.

I don't retreat from my generalized objection to teaching to tests. It's an insidious consequence of No Child Left Behind, and the general trend toward what is so glibly called "accountability" (would that the government were held accountable for its own poor performance -- say, taking away $100B from the defense budget in years where it is misappropriated to misguided wars or corrupt defense contractors). But I think the issue identified by Benton specifically concerning the new writing requirement of the SAT is a tempest in a teapot.

It's also crucial to remember that the SAT's role is to aid universities in assessing a student's aptitude for advanced study. Most programs require nothing more than the writing competency discussed above. The same people lining up against the writing section now are the ones whom bemoaned the cultural bias and the insufficiency of the purely multiple-choice SAT of old.

Furthermore, nothing I have read makes the current situation sound any different than what I was taught in high school. Sure, I had a creative writing elective along the way, and I was active in my high school's literary magazine (I published more angsty screeds and poetry in my senior year than any other author, and I use the word loosely). I also began to read classics in my AP English course. These were the places I began to learn to write in the artistic sense -- although none of them was as important to my development as a writer as my voracious reading from a young age was. In most of my classes, I learned what Benton derides schools for teaching now -- to communicate the given material concisely and cogently, which is the most important skill for the vast run of people.

It's not that I don't like Benton's column; I do. Furthermore, as a great lover of skillful writing, emotionally I tend to agree with him. Whatever little bit teachers can do to improve students' writing they should do. We're going to be stuck reading reports and memoranda and briefs written by these kids later in our careers; even if they're coherent and grammatical, if they're boring we're going to be pulling our hair out and reminisching about how it was when we were young and on the make, even if our memories are largely confabulated and soft-focus. The reality, however, is that artistic writing, truly creative writing, is and has been and will always be hard to find. And that's got very little to do with the schools, the SAT, or any other easy scapegoat. A lack of literacy that spans the generations, two generations raised on the cathode nipple, these are the culprits. And we can't unring that bell.


Blogger Josh said...

See, I disagree. :)

First off, I don't think it's accurate to say writing is some black art that can't be taught. It's not some ingrained skill you're born with or doomed to live without. Personally, I'm a thousand times better now than I was five years ago -- thanks to a lot of practice (for work and on crabwalk) and good editors. Your line of thinking seems awfully defeatist: If writing can't be taught, by the reasoning, I guess we shouldn't be wasting our time teaching it, right? Think of all the recess time that'd free up!

Are most strong writers going to do fine on the SAT, and did they do fine on the old TAAS test here in Texas? Sure. They'll adapt. I'm not that concerned about them. I'm much more concerned about the impact on the teaching of writing.

As I said in the column, teachers figure out really quickly how to get their kids to score well on a test. If the test is a good one, that might mean that they learn to do a better job teaching the areas their kids are weaker in. That would be a positive outcome.

If the test is a skewed one -- as I argue a lot of these writing tests are -- teachers figure out they can stop teaching some skills that aren't going to be tested.

I can tell you that in the TAAS era, some urban schools stopped all instruction of writing other than the five-paragraph model. They were doing other, more interesting stuff before, but they cut it out as soon as they figured out it wasn't going to help their kids pass the test as much as spending another six weeks on the 5-graf model would.

That's my complaint: That building a test around boring writing leads teachers to teach only boring writing. Teachers see there's no return on their investment in teaching interesting writing, and they adjust their class time accordingly.

What's bothersome about all this is that it's a pretty solvable problem: Don't make the test so predictable. Don't have the writing prompt be in the exact same format every year, so teachers can't strip away their entire curriculum to the one skill they know their kids will be tested on. One year have a persuasive essay, the next year something descriptive, the next year something else entirely. Or just stop asking overtly for crappy writing -- when Texas shifted from the TAAS test to the TAKS, they did this, and the writing test is better as a result.

You're right, of course, that we don't test students to ensure that they're good clarinetists or good painters. But creating good clarinetists isn't one of the main objectives of public schools. Creating good writers is. And I guess my main beef is that some writing tests actively hurt that cause instead of helping it.

11:44 AM  
Blogger Josh said...

And how do we know each other, anyway?

11:46 AM  
Blogger Moon said...

If writing can't be taught, by the reasoning, I guess we shouldn't be wasting our time teaching it, right? Think of all the recess time that'd free up!It's not that I think writing can't be taught. Rather, I suspect that, in the aggregate, the writing that can be taught is the sort of writing that will score well on the SAT. Logical development of ideas, ordering of premises and subordinate conclusions, these are fundamentally mechanical tasks -- not so (or not so much) the vagaries of varied usages and structures, of rhythm and emphasis. This is true, to my thinking, not only because these nuanced matters are better the topic of MFA programs, but because this sort of deeper appreciation of the writerly arts can't be imposed -- like the clarinet, or painting, a student has to really want to learn it, a desire I suspect is only present in a small minority of students, which minority of students will figure it out on its own.

You point out that you are more concerned about the effect the pedagogical imperatives the testing regime creates will have on the teaching of writing. I'll grant you this; the sad fact is that some teachers, like people in most jobs, will tend to choose the path of least resistance, opting to make an effort only when it will clearly advance their professional interests. But is the SAT, in any form, to blame for this? Is the requirement of standardized testing so monolithic that it can transform otherwise virtuous and creative, driven teachers into diffident automata?

My already award-winning younger brother teaches elementary school in our hometown. Although these students don't face the SAT's as yet, they do face a battery of standardized tests each year, by which are assessed the school's and teachers' effectiveness. He faces pressure, explicit or im-, to teach to those tests, and for some portion of the second half of the year that's precisely what he does, to the best of his ability. But having to do so for a spell doesn't change his drive to excel, to innovate and inspire, and instead of cowering in the face of the imposition he does his best to work with it when possible and around when he must.

All of that said, there are caveats:

a) Obviously, you know waaaaay more about this than I, and I trust your research quite a bit more than my conjecture. Taking at face value what you've said about the experience in Texas, I'd be hard pressed to dispute the obvious inferences that warrants.

b) I also think it's a nuance rather than a category difference when I express relative indifference to the administration of a predictable test: if, instead of one all-purpose question format, the SAT were to incorporate an essay test that might come in any of four forms, all that would change is that teachers would prepare their students for each form, teach them how to recognize the cues that signal the correct form and how to vary their writing to satisfy the differing demands. This I know from experience, because it's precisely the approach adopted by Bar exam preparation classes. Bar exam essays come in one of four or five principal forms; we were taught in precisely the way described just above.

Still, I can't argue with you that a test that is less predictable would be more desirable. And it's a critique I imagine the SAT folks will be hearing, and to which they will hopefully respond in the next iteration.

Even so, however, I think the writing that demonstrates aptitude for college (let's not forget what the SAT's are about), and the writing that will demonstrate sufficient competency in the workplace, needs no flourishes or surprises to achieve its purpose: clean, concise prose, that's predictably arranged, that presents cogent arguments in a linear narrative, is adequate to both tasks, perhaps optimal. Long before the five-paragraph essay was being tested in this way, it was being taught to younger students as a paradigm of proper writing. Was it wrong then?

Think of all the recess time that'd free up!I happen to think recess provides a far more fertile ground for learning to write in the artistic sense than any classroom.

Thanks for the thoughtful reply.

2:20 PM  
Blogger Josh said...

I just fundamentally disagree with the idea that the only kind of writing that can be taught is automaton crap. If you have a test-driven curriculum where the only writing being taught is automaton crap, then that's what kids will learn. If you have a curriculum that encourages different (and better) kinds of writing, that's what kids will learn.

One of the people I interviewed in my column said the biggest problem advocates of writing instruction run into is the perception that there are two kinds of writing: automaton crap and, off in some other world, "creative writing," which summons images of unicorns frolicking in a sun-kissed field and lengthy self-absorbed blog posts.

His point -- and it's one I agree with -- is that there isn't a clear line between them. The line is something imposed by things like these tests. And as long as there are tests that in effect say, "don't be creative or try to be interesting, it'll only hurt you" -- I'll think that sucks.

I also think that, while it's clearly not your intention, it's the same sort of argument that supports stereotypes. I mean, if you don't try to teach kids how to write well, it's always going to be the poor kids and the black kids and the Hispanic kids who aren't going to write as well. If writing ability is just something you "have," it's always going to be the privileged who are going to seem to "have" it. I'd rather think that everyone's writing can improve and that the vast majority of kids can get beyond the five-paragraph essay.

It just reminds me too much of the old days, when upper-middle-class white kids were automatically put into a college prep curriculum and poor minority kids were put in shop so they could "learn a skill." A lot of those kids could learn Shakespeare just as well as the white kids, but they were shoved into voc-ed because it was assumed they were somehow beyond the reach of teaching.

Finally, re: the SAT: I don't think the SAT writing sample is as bad as some other tests. (Like the old TAAS.) It's bad, but it's not the worst. Plus, unlike state tests, teachers don't change their entire curriculum to match the SAT. So I doubt the SAT will change instruction nearly as much as a bad state test would.

The other saving grace is that colleges will be able to read the essays when they get students' SAT scores. Since competitive colleges will be looking for good writing, not crap, that will reduce the incentive for kids to produce high-scoring bad writing.

3:53 PM  
Blogger Josh said...

Oh,and: I happen to think recess provides a far more fertile ground for learning to write in the artistic sense than any classroom.I like me some recess too, but I hardly think replacing writing instruction with time on the swings will create a generation of great writers. No one's saying cancel recess to improve writing -- except the teachers who will do anything to get their kids' test scores up. And I can guarantee you those score-hungry ones are drilling and killing the five-paragraph model, not anything good.

And just to be clear: I'm not talking about (and my column wasn't about) writing "in the artistic sense." I'm not talking about developing little Hemingways and Faulkners. I'm just talking about teaching people to write in clear, interesting ways. I mean, my newspaper stories (and aren't by any stretch of the imagination "art." I just hope they're good.

4:01 PM  

Post a Comment

<< Home

eXTReMe Tracker