A couple of weeks ago the DfES released its annual school league tables (don’t switch off, American readers, this matters to you). The tables have a new component: a ‘value added’ score, which is supposed to show not how well the children performed but how well the school taught them. The presentation of the league tables in the papers I saw stayed with the ‘raw score’ ranking, but included information about the ‘value-added’: presumably some papers did it the other way round.
The ‘value-added’ score is represented by a single number. This, in itself, makes it difficult to understand. The statisticians have devised a way of weighting vocational and academic achievements against each other, and any such weighting is going to be subject to dispute. The use of a single number also masks within-school inequalities: a school with a lousy Maths department can still get a good score if it has an excellent English department. But for an individual parent choosing a school this difference might matter a great deal: my own daughter’s writing and reading skills will develop fine if she is taught English by gibbons; her math and science skills need teaching by a good teacher.
There are also more technical reasons for being skeptical about the scores. Harvey Goldstein of the Institute of Education, the world’s expert on value-added evaluation of schools, has finally posted his commentary on the new tables, and it is (like all his commentaries) essential reading. The central problem, as I read it, is the issue of pupil mobility — we know that the extent of pupil mobility affects learning, but we don’t know how much, and furthermore we have no way of evaluating the extent to which schools themselves are responsible for pupil mobility. The DfES tables simply ignore the problem.
Why does this matter for Americans?
No Child Left Behind requires participating states to evaluate school performance. Most States have adopted raw-score methods of evaluation, but not all have — Ohio, I know, is pursuing a crude version of value-added. The pressure for value-added is increasing, and my guess is that more and more States will adopt it. On the one hand, with bigger schools and less scope for school choice, the evaluations should be more accurate than for British schools. On the other hand, schools have children for less time (typically they attend 3 stages of school, rather than 2), funding levels are much more diverse, and (I assume that) student mobility is greater (just because Americans move more). The UK experience is valuable for American school officials and researchers — and Goldstein’s commentaries are essential to understanding the UK experience (IMHO).
The article skirts the issue of state vs private schools. For background reading try Tooley. With articles such as “Private education: the poor’s best chance?”, I’m sure the readers of this blog will be enthralled but not necessarily educated.
Darren, I know James’s work well, and have written a thorough response to the ideas in that piece, published in the most recent issue of Journal Of Philosophy of Education. I can, in fact, email you a PDF file if you’re interested, and will summarise the part of it that responds specifically to his comments on India and Geeta Kingdon’s findings in a future post. Thanks for the idea!!
Curiously enough, it is my son’s writing and reading (and science) skills [that] will develop fine if he is taught English by gibbons (in fact, in the sixth grade he was); it is his math skills that need teaching by a good teacher…
All of you in school districts with gibbon-led English departments really need to consider relocating….
I would think that student mobility would actually be helpful in evaluating schools. Say you have a group of students that all attend the same elementary school and then all attend the same middle school. When they get to middle school their test scores begin to drop. That suggests that the middle school isn’t as good as the elementary school, but it doesn’t tell us anything about how those two schools relate to any other schools. You might have a decent middle school and an excellent elementary school or a decent elementary school and a terrible middle school. The measurement procedures might be a little more complicated when students move around a lot, but I think the end result would be more informative.
This might be indicative of a bigger problem. If students in one school on average do no better or worse than in their previous school, does the algorithm take that to mean that the new school is just as good as the old school or does it mean that the new school is of average quality? It should be the former, but from what I’ve read I think it’s the latter.
That would be a big problem. If a district has consistently good schools K-12, it will look like the district has good elementary schools, but only average middle and high schools.
Obviously we need different measures for different purposes.
I do worry that there is potential here to disguise inequality. If schools in poorer areas are adding just as much “value” we shouldn’t kid ourselves that the pupils are getting just as good an education.
Value-added is a terrific way to evaluate schools. I have had educators in the richest schools tell me that they do absolutely nothing to add to kid’s education—that it all happens at home. To which I say, then why are you taking in $10,000 per kid in tax dollars…for babysitting?
In disadvantaged schools, with challenged populations, value-added removes the ugly broad hurdle of AYP as a simple standard, and allows the improvement to be measured against the starting point.
À Gauche
Jeremy Alder
Amaravati
Anggarrgoon
Audhumlan Conspiracy
H.E. Baber
Philip Blosser
Paul Broderick
Matt Brown
Diana Buccafurni
Brandon Butler
Keith Burgess-Jackson
Certain Doubts
David Chalmers
Noam Chomsky
The Conservative Philosopher
Desert Landscapes
Denis Dutton
David Efird
Karl Elliott
David Estlund
Experimental Philosophy
Fake Barn County
Kai von Fintel
Russell Arben Fox
Garden of Forking Paths
Roger Gathman
Michael Green
Scott Hagaman
Helen Habermann
David Hildebrand
John Holbo
Christopher Grau
Jonathan Ichikawa
Tom Irish
Michelle Jenkins
Adam Kotsko
Barry Lam
Language Hat
Language Log
Christian Lee
Brian Leiter
Stephen Lenhart
Clayton Littlejohn
Roderick T. Long
Joshua Macy
Mad Grad
Jonathan Martin
Matthew McGrattan
Marc Moffett
Geoffrey Nunberg
Orange Philosophy
Philosophy Carnival
Philosophy, et cetera
Philosophy of Art
Douglas Portmore
Philosophy from the 617 (moribund)
Jeremy Pierce
Punishment Theory
Geoff Pynn
Timothy Quigley (moribund?)
Conor Roddy
Sappho's Breathing
Anders Schoubye
Wolfgang Schwartz
Scribo
Michael Sevel
Tom Stoneham (moribund)
Adam Swenson
Peter Suber
Eddie Thomas
Joe Ulatowski
Bruce Umbaugh
What is the name ...
Matt Weiner
Will Wilkinson
Jessica Wilson
Young Hegelian
Richard Zach
Psychology
Donyell Coleman
Deborah Frisch
Milt Rosenberg
Tom Stafford
Law
Ann Althouse
Stephen Bainbridge
Jack Balkin
Douglass A. Berman
Francesca Bignami
BlunkettWatch
Jack Bogdanski
Paul L. Caron
Conglomerate
Jeff Cooper
Disability Law
Displacement of Concepts
Wayne Eastman
Eric Fink
Victor Fleischer (on hiatus)
Peter Friedman
Michael Froomkin
Bernard Hibbitts
Walter Hutchens
InstaPundit
Andis Kaulins
Lawmeme
Edward Lee
Karl-Friedrich Lenz
Larry Lessig
Mirror of Justice
Eric Muller
Nathan Oman
Opinio Juris
John Palfrey
Ken Parish
Punishment Theory
Larry Ribstein
The Right Coast
D. Gordon Smith
Lawrence Solum
Peter Tillers
Transatlantic Assembly
Lawrence Velvel
David Wagner
Kim Weatherall
Yale Constitution Society
Tun Yin
History
Blogenspiel
Timothy Burke
Rebunk
Naomi Chana
Chapati Mystery
Cliopatria
Juan Cole
Cranky Professor
Greg Daly
James Davila
Sherman Dorn
Michael Drout
Frog in a Well
Frogs and Ravens
Early Modern Notes
Evan Garcia
George Mason History bloggers
Ghost in the Machine
Rebecca Goetz
Invisible Adjunct (inactive)
Jason Kuznicki
Konrad Mitchell Lawson
Danny Loss
Liberty and Power
Danny Loss
Ether MacAllum Stewart
Pam Mack
Heather Mathews
James Meadway
Medieval Studies
H.D. Miller
Caleb McDaniel
Marc Mulholland
Received Ideas
Renaissance Weblog
Nathaniel Robinson
Jacob Remes (moribund?)
Christopher Sheil
Red Ted
Time Travelling Is Easy
Brian Ulrich
Shana Worthen
Computers/media/communication
Lauren Andreacchi (moribund)
Eric Behrens
Joseph Bosco
Danah Boyd
David Brake
Collin Brooke
Maximilian Dornseif (moribund)
Jeff Erickson
Ed Felten
Lance Fortnow
Louise Ferguson
Anne Galloway
Jason Gallo
Josh Greenberg
Alex Halavais
Sariel Har-Peled
Tracy Kennedy
Tim Lambert
Liz Lawley
Michael O'Foghlu
Jose Luis Orihuela (moribund)
Alex Pang
Sebastian Paquet
Fernando Pereira
Pink Bunny of Battle
Ranting Professors
Jay Rosen
Ken Rufo
Douglas Rushkoff
Vika Safrin
Rob Schaap (Blogorrhoea)
Frank Schaap
Robert A. Stewart
Suresh Venkatasubramanian
Ray Trygstad
Jill Walker
Phil Windley
Siva Vaidahyanathan
Anthropology
Kerim Friedman
Alex Golub
Martijn de Koning
Nicholas Packwood
Geography
Stentor Danielson
Benjamin Heumann
Scott Whitlock
Education
Edward Bilodeau
Jenny D.
Richard Kahn
Progressive Teachers
Kelvin Thompson (defunct?)
Mark Byron
Business administration
Michael Watkins (moribund)
Literature, language, culture
Mike Arnzen
Brandon Barr
Michael Berube
The Blogora
Colin Brayton
John Bruce
Miriam Burstein
Chris Cagle
Jean Chu
Hans Coppens
Tyler Curtain
Cultural Revolution
Terry Dean
Joseph Duemer
Flaschenpost
Kathleen Fitzpatrick
Jonathan Goodwin
Rachael Groner
Alison Hale
Household Opera
Dennis Jerz
Jason Jones
Miriam Jones
Matthew Kirschenbaum
Steven Krause
Lilliputian Lilith
Catherine Liu
John Lovas
Gerald Lucas
Making Contact
Barry Mauer
Erin O'Connor
Print Culture
Clancy Ratcliff
Matthias Rip
A.G. Rud
Amardeep Singh
Steve Shaviro
Thanks ... Zombie
Vera Tobin
Chuck Tryon
University Diaries
Classics
Michael Hendry
David Meadows
Religion
AKM Adam
Ryan Overbey
Telford Work (moribund)
Library Science
Norma Bruce
Music
Kyle Gann
ionarts
Tim Rutherford-Johnson
Greg Sandow
Scott Spiegelberg
Biology/Medicine
Pradeep Atluri
Bloviator
Anthony Cox
Susan Ferrari (moribund)
Amy Greenwood
La Di Da
John M. Lynch
Charles Murtaugh (moribund)
Paul Z. Myers
Respectful of Otters
Josh Rosenau
Universal Acid
Amity Wilczek (moribund)
Theodore Wong (moribund)
Physics/Applied Physics
Trish Amuntrud
Sean Carroll
Jacques Distler
Stephen Hsu
Irascible Professor
Andrew Jaffe
Michael Nielsen
Chad Orzel
String Coffee Table
Math/Statistics
Dead Parrots
Andrew Gelman
Christopher Genovese
Moment, Linger on
Jason Rosenhouse
Vlorbik
Peter Woit
Complex Systems
Petter Holme
Luis Rocha
Cosma Shalizi
Bill Tozier
Chemistry
"Keneth Miles"
Engineering
Zack Amjal
Chris Hall
University Administration
Frank Admissions (moribund?)
Architecture/Urban development
City Comforts (urban planning)
Unfolio
Panchromatica
Earth Sciences
Our Take
Who Knows?
Bitch Ph.D.
Just Tenured
Playing School
Professor Goose
This Academic Life
Other sources of information
Arts and Letters Daily
Boston Review
Imprints
Political Theory Daily Review
Science and Technology Daily Review