TheStar Standard tests: More questions than answers

TheStar Standard tests: More questions than answers

This article is part of a series that The Star is running. i am including the entire article below, because i find that it is a great read. You can click the link above to read it in its entirety from the original source.

Cookie-cutter school testing.

By Rick Salutin Columnist

If your kid recently soldiered through Ontario’s province-wide “EQAO” tests for reading, math and science, in Grades 3, 6 or 9, then you’ve been part of a worldwide homogenization of education techniques. This kind of testing is meant less to measure how kids are doing than how their teachers and schools are doing, so they can be held “accountable.” Now there’s nothing wrong with accountability. And testing is a necessary tool. The problem is accountability based on high-stakes, standardized tests.

Standardized means the same for everybody, set by a central authority — a government department or private company. But kids aren’t the same. A test can tell you what a kid scores, not what the score means for the kid. It depends on where s/he started from, what his abilities are and what’s important for her to know. A low mark for one kid might be a better sign than a high mark for another. Teachers know this and can adjust the lesson (and the mark’s meaning) to the learner. But anonymous test scorers can’t. So standardized tests are poor indicators of how kids and teachers are doing.

It gets worse when you tack the accountability piece onto standardized testing, as they’ve done all over the U.S. It may seem plausible and clear-cut. But when test scores become the basis for rewards and punishments like hiring, firing, teacher pay and school funding or closing, the tests grow vulnerable to, and even create an incentive for, cheating or gaming the system.

This is done by obtaining widely distributed test copies in advance and giving them to kids, shifting pass/fail levels on tests at state or local levels to meet criteria for receiving federal funds, faking results or “counselling out” weaker students by pressuring them to leave school on a variety of pretexts so the school or class average rises. All this is documented in U.S. education historian Diane Ravitch’s book The Death and Life of the Great American School System. She gets credibility because she served as an assistant secretary of education under George W. Bush, bringing in and arguing for standardized tests.

She now says, “We were wrong . . . testing actually makes the schools worse.” Any learning gains made are shaky and can “evaporate” quickly once the testing pressure is off. Learning in a school can even decline because of testing. There’s a technical term for it: Campbell’s law, proclaimed by U.S. expert on research methodology Donald Campbell. It says that if you base accountability on measurements, what you’re measuring may get worse instead of better. It sounds weird but it happens, for instance, if hospital ERs are rated by the number of patients treated quickly: they might rush people through rather than provide good care in order to raise their scores.

The list of objections to standardized testing is almost endless. The saddest come from teachers; many are cited by former New York Times education columnist Richard Rothstein in his book on accountability. They’re painful to read, the way TV crime dramas about child abuse can be painful to watch. Teachers say they now lack time to do things they loved, like taking kids on trips, or teaching trigonometry by going outside and measuring shadows in the sun. It all loses out to prepping for tests.

Besides, as philosopher of education John Dewey said, nothing kills the joy of learning like failing to have a real-life reason for the lesson. If you’re told you need it for a test, or as training for something later in life, it drains the life and joy from the present, and kids specialize in the present. If you’re not having fun, it’s really hard to learn, says a kid I know. Without fun it’s probably hard to teach, too.

Are there other ways to assess teachers and schools than numerical test scores? Yes, there have been many; they just aren’t as easy to measure. The U.K. began obsessively quantifying during the Thatcher years, but it still sent teams into schools to evaluate various elements beyond test scores until the 1990s. In its early years, the U.S. National Assessment of Educational Progress used a wide range of methods to judge success. If you take the trouble, you can assess outcomes in areas like personal growth, civics, health or art. You can judge by essays, artwork, physical challenges or public service. An Australian evaluation asked kids: Are there political causes worth fighting for? What a great question to judge whether a school successfully “taught” a sense of civic responsibility. In fact, the current fixation on “measurables” to the exclusion of everything else starts to look like an aberration from the long-term trend.

How did the notion of accountability get so narrow? Partly, it suited the business culture of our era, which prefers anything that can be quantified down to a fine powder. Bill Gates, in his new incarnation as education expert, says any teaching that you can’t measure is useless. (I wonder how you’d measure that claim.) The testing at certain points becomes manic. The U.K. under Thatcher reported on about 1,000 skills per kid, leading eventually to a huge revolt by teachers and headmasters. The U.S. version was George Bush’s No Child Left Behind, followed by Barack Obama’s Race to the Top. (The latter program sent millions in extra funding to Washington, D.C., schools, partly because of their high test scores. Teachers and principals also received individual bonuses. But those scores are now under investigation because of abnormally high rates of “erasures” — wrong answers being rubbed out and replaced with correct answers on computerized tests — as reported recently by USA Today.) In Ontario, it hasn’t been as drastic. But we’re partway down that road.

A small but apparently potent unit called the Literacy and Numeracy Secretariat has been set up in the Education Ministry. Its mandate is to improve teaching generally, but classroom teachers and others in the field who’ve dealt with the secretariat say it’s relentlessly focused on boosting those EQAO scores. In a way, once the tests exist, it’s hard not to fix on them. They’re so concrete; they tend to take over from other forms of assessment.

It hasn’t reached the point, as in the U.S., that they’re the basis for hiring, firing or funding. But they do serve to guide parents in choosing (and rejecting) schools. Ontario has a “School Information Finder” that gives EQAO results and socio-economic data for parents to check. It implies a sort of competition model that pits schools against each other based on their test scores, in a contest to impress parents. Every member at the provincially created Education Partnership Table — parent groups, principals, deans of education, student reps, etc. — objected to it but Premier Dalton McGuinty refused to dismantle it. He has an election coming, against a Mike Harris-type opponent. He wouldn’t want to look soft on “basics” like tough testing, which polling shows support for. So politically, he treads carefully.

There is some recent resistance to the testing-accountability recipe. In the U.K., the “Celtic fringe” — Scotland, Wales and Northern Ireland — is dropping or limiting standardized tests. Even England has canned them for the early grades, after a revolt including strike threats by headmasters in 2009. As for us, B.C. was big on the tests but has recently wobbled about whether they’ll be imposed, in response to objections from parents, teachers and principals.

The most intriguing counter-case is Finland. Since the Organization for Economic Co-operation and Development began ranking countries in 2000 by their scores on Program for International Student Assessment (PISA) tests, it’s been the star performer. It always scores at or near the top in all categories. Yet it does no standardized testing till the end of high school (“the first and last national test,” said a teacher in Helsinki). So the results have shocked everyone, starting with the Finns. Their country shot onto everyone’s list of most successful countries, especially since it also led in prosperity (the British think-tank Legatum, 2009), competitiveness (World Economic Forum, 2003-5) and perceived lack of corruption (Transparency International, 2010). Delegations of educators and journalists began arriving in the tens of thousands. Literally. When I entered school staff rooms there, teachers kind of rolled their eyes, then got helpful. It especially miffed them since they don’t value being near No. 1 on tests and didn’t aim for it. “We hate those results,” says educator Pasi Sahlberg, a prominent Finnish author/educator who has represented his country at the World Bank, the OECD, the EU and elsewhere. “It’s not a competition, it’s about building community,” he says. Finns do enjoy beating out Sweden, which ruled them for centuries. But that’s it.

Their big school reform began 40 years ago and wasn’t about scores; it was about creating a single school model for all students up to high school. Till then, students had been streamed in academic or vocational directions. In other words, the reform was about diminishing school choice for the sake of greater equality and social unity. There was a long, raucous debate. Many people, including teachers, argued against the reform, saying it would lower standards. The 2000 PISA results finally ended that debate. Everyone could see the reforms weren’t only socially just, they were academically brilliant.

I sat with a university researcher in the small city of Kokkola, 200 kilometres south of the Arctic Circle, as she explained her theory as to why Finland’s approach yields such great scores. But all she did was restate the elements of the approach, like teacher autonomy and an emphasis on social equality, without explaining why they produce those results. So here’s my theory: Everyone knows that what tests really test is how good you are at taking tests. The tenser you are, the lower you’re likely to score. Since Finns don’t worry about tests, they’re loose going in, and score well. This jibes with a report that found only 7 per cent of Finnish students feel anxiety doing math homework, versus 52 per cent and 53 per cent for French and Japanese kids.

It’s not that Finns don’t test. They just don’t do standardized, high-stakes tests. There were lots of tests in classrooms I visited, and in the halls, where individual kids or small groups might be taking tests with the aid of a teacher or assistant. But teachers set the tests themselves. They do it, one told her students, “because I want to know how you’ve learned and I want to know how I’ve taught.” Most good teachers I know feel this way: they value tests, as long as they exist to serve the learning process, and not vice versa.

As for accountability, the Finns also reject rewards and punishment based on externally set performance targets. Peter Johnson, director of education for Kokkola (whose job includes museums, libraries, youth sports and filling in for the mayor when he’s away) says with quiet pride, “Finland saw its last inspector in 1985.” We were having dinner in late November. It was dark and cold outside. The sun rose at about 10:30 a.m. and set four hours later. “We believe in self-evaluation,” he went on. He called it inside-out or self-service accountability. Others call it smart accountability. The Finns agree with assessing success based on results. They just don’t think test scores should be the results used. They also think self-assessment is faster and more efficient. If you find a problem, you fix it. You don’t have to wait for inspection reports. Johnson says he once told this to a group of 30 visiting Europeans: “They were so silent.” Then one asked if it was true Finland has no inspectors. It suddenly occurred to Johnson to ask if any of them were inspectors, and “half raised their hands.”

This may sound odd to Canadians. Why should anyone trust teachers with self-accountability? When Ontario teachers criticized the EQAO tests, a Globe and Mail editorial said, “They don’t wish to be held accountable.” That’s the common, U.S.-style response: attack teachers. Finland shows there are other kinds of accountability. Their version relies on the rest of society trusting educators. But how do you get to that level of trust? I’ll return to this question later.

When I mention Finland to educators here or in the U.S., they’re often dismissive. They say it’s not comparable: It’s too small (5.3 million), white, homogeneous and middle-class. But it does far better than comparable Scandinavian nations like Norway and Denmark, so something more is going on than size and uniformity. Its diversity is also increasing due to EU membership; its foreign-born citizens have doubled in the past decade. There are schools with more than 40 per cent immigrant students; in Helsinki (pop. 580,000) schools, almost 10 per cent of the kids are immigrants and more than 40 languages are spoken. Based on PISA data, those kids perform better than immigrant kids in similar places, so again, Finland is doing something right. As for size: education is controlled by Canadian provinces and U.S. states so it’s fair to compare to them rather than the whole country: Finland is smaller than Ontario but bigger than the other provinces, many of which are pretty homogeneous. Finland is also a northern land that was historically dominated by larger powers, like us. Its obsession, like ours, was survival. It makes as much — no, more — sense to compare ourselves to them, rather than to imperial behemoths like the U.S. and the U.K. Besides, to get there you fly for endless hours and when you arrive, it feels like you never left here.

There are differences, of course. Finland has few resources aside from fresh water and trees. From this they have concluded they have to be smart and well-educated to succeed economically, which they’ve done. Even from this difference, I’d say we can learn something: that being resource-rich, like us, can make you stupid and slow off the mark.

Nor are the people I met there complacent. “I fear this will all end badly,” says Pasi Sahlberg, meaning the hoo-ha about Finnish test scores. He thinks the EU will use Finland’s educational success as a reason to reduce some economic benefits it now receives. He also has darker fears: that the passion for justice and independence that motivated the reforms will wane and be replaced merely by some fickle international acclaim and admiring foreign delegations. What will inspire future generations of Finnish educators? he and Peter Johnson both wonder.

Meanwhile Ontario continues along its same course of standardized testing, little concerned, it appears, about the perils and precedents.

The real basics: building character

Testing for reading, writing and math sounds like the three Rs from the good old days of pubic education.

But Egerton Ryerson had a larger role in mind for the system he created in Ontario in the 19th century. He wanted schools to counter the anti-monarchy and republican influences that came with heavy immigration from the U.S. and had led to the rebellion of 1837, and to construct an Anglo balance to the French fact in Quebec. This amounted to a “citizenship” agenda, alongside the training he wanted to provide for every child.

In the U.S., Founding Fathers like Benjamin Franklin and Thomas Jefferson also supported public education for moral and civic ends, not just or even mainly for academic skills. In 1830 a Pennsylvania committee warned that poor and working children should get more than “a simple acquaintance with words and ciphers;” they should also acquire “a just disposition, virtuous habits and a rational, self-governing character” — what we now think of as character education or a citizenship agenda. As late as the 1930s, philosopher John Dewey said it was important not just to know how to read, but how to distinguish between the “demagogue and the statesman.”

In fact, in the old days, there was probably less emphasis on the basics than there is now. Put another way, the basics didn’t used to be so basic.

International testing

The Program for International Student Assessment, or PISA, run by the Organization for Economic Co-operation and Development (OECD), measures educational achievement in the 34 OECD members, though it also surveys other countries and some cities. Every three years it tests about 5,000 15-year-old students per country, in reading, math and science, with special emphasis on one of those for each round. The tests are handwritten, part multiple choice and part short essays. There is also a questionnaire on background, study habits, etc. PISA’s methods are highly regarded. The first three test years (2000, 2003 and 2006) showed Finland at the top in the area emphasized that year. Canada did well, placing second twice and fifth once.

In 2009, Finland scored first or second in all categories among OECD members. Canada was fifth, fifth and third. Some countries have done surprisingly poorly; the U.S. and Germany were generally in the middle or at the back of the pack.