Education data sorely need a transformation. As commentators, boosters, and sometimes critics of education reform, we have witnessed policymakers struggling to make decisions in the face of incomplete information; school leaders in search of clearer data about the performance of their teachers and pupils; taxpayers and public officials puzzled by why more resources keep pouring into a system from which little more pours out by way of learning; and fellow analysts frustrated by muddy or outdated statistics. The question now, of course, is how to launch such a transformation.
It is true that the data available today are superior to those available in the past. No Child Left Behind has led to important strides in the availability of student achievement data. Emerging technologies are changing how such information is collected and are easing data entry, analysis, and dissemination. And many groups have been pressing for further improvements, such as the Data Quality Campaign doggedly nudging states toward longitudinal databases; the Schools Interoperability Framework Association, enabling seamless data sharing; Greatschools.net and SchoolMatters.com providing parents and policymakers with school-level data; funders such as Gates, Walton, and Broad supporting these and kindred reforms; and, in the public sphere, the U.S. Department of Education and its National Center for Education Statistics, the Council of Chief State School Officers, and numerous forward-thinking states, districts, and schools, collaborating on data reforms.
Yet we still have incredibly far to go. Today's education data systems are exceedingly slow and frequently non-comparable from place to place or level to level, as pre-K information systems typically don't "speak" to the K-12 systems, which in turn don't "speak" to the higher education systems. Some important information (e.g., the cost of teacher benefits) isn't even systematically gathered. Seemingly obvious questions (e.g., where does the money come from and how is it spent) are all but unanswerable. And because most data systems are institution- rather than student-based, they're ill-equipped to "follow" individuals who move from school to school or "graze" their way through college on multiple campuses, and they're also ill-suited to such innovations as charter schools, "virtual" learning, proprietary colleges and part-time students (or faculty).
With this week's release of A Byte at the Apple: Rethinking Education Data for the Post-NCLB Era, we at the Fordham Institute hope to shine a light on these problems and outline creative solutions and alternatives.
Some of the problems seem so basic yet prove so tough to fix. Why can we not easily track the performance of students over time? Why is it so challenging to understand how well a school is educating its students? Daniele Vidoni and Kornelia Kozovska offer examples of education data systems in three other countries. In the United Kingdom, for example, "league tables" provide useful information on school performance, "allowing comparisons based on exam scores and value added (within a region or city)." This information is published regularly by newspapers, and this year Prime Minister Gordon Brown announced his plan to enhance this transparency by enabling parents to use the internet to track the "attendance, behavior, and performance of their children."
Frederick M. Hess and Jon Fullerton also draw data-driven lessons from elsewhere--in this case from business--that could help reformers and policy makers boost educational performance in U.S. schools and districts. How could they better manage themselves? Much of the answer lies in better use of data to inform decisions. Just as companies use "balanced scorecards" and customer satisfaction surveys to track their organizational performance beyond the bottom line, schools must look far beyond achievement data. This will allow them to evaluate such basic necessities as lighted classrooms, efficient bus routes, safe hallways and classrooms, and teacher hiring practices. In other words, schools need "measurement for performance" as well as "measurement of performance," in order to be well-run enterprises that enable rather than hinder effective teachers and dynamic classrooms.
Technology can enhance school practices while generating valuable data, too. Yahoo! Teachers, for example, meant to be a platform for sharing lesson plans, holds great potential for generating data on what works and what is most popular. Likewise, TeacherTube already allows teachers to share and rate videos for instructional purposes. Much more could be quickly learned about which lesson plans and teaching techniques are effective if more of the millions of daily student-teacher interactions could be captured as data--perhaps by the increased use of computers and PDAs as teaching tools.
If the best thinking from outside education were brought to bear within it, a data revolution might follow. Bryan Hassel explains how firms such as credit card providers have long used data mining techniques to understand the tendencies of their customers and assess the profitability of a prospective customer or new product. As Hassel asks, "what if the data inherent of millions of daily teacher interactions could be harnessed to give teachers real insight into whether method X or Y is better?" And what if education leaders could mine that data to ask, "Which of my teachers are doing what is needed to improve their instruction, or which of my district's schools are most in need of an infusion of new leadership"? Good questions, all.
Hassel also draws education applications from the "wisdom of crowds." Amazon book ratings, for example, go beyond "expert" reviews to assist potential readers to choose their purchases with the advice of hundreds of previous readers. Similar "crowd" based reviews exist for restaurants, vacation resorts, consumer electronics, and more. Amazon goes even farther by also providing recommendations based on the shopping patterns of customers with similar preferences.
Perhaps the most interesting use of crowd wisdom is in "prediction markets," where hundreds of people can speculate on the outcome of elections and other events. Hassel explains how Best Buy asked its employees to predict holiday card sales; their collective wisdom proved more accurate than the experts that the company might have otherwise relied upon. For parents and other observers of schools, prediction markets could be harnessed to foresee which schools are poised for improvement and which are in decline; for principals, they could provide new information on the quality of their teachers and staff.
Radical changes in our handling of student achievement data could also foster overdue improvement. For example, Margaret Raymond envisions a "Data Backpack," by which information about a student accompanies him or her throughout his or her education, controlled and populated in part by schools and in part by parents. Already an improvement on the cumbersome data processes currently used in many districts, its greatest power could be its potential to engage parents in their children's education by providing readily accessible information and interactive features, such as "parent discussion boards, informative videos about parenting or child development, immunization and health record keeping, or planning tools to track their child's progress."
Some of these ideas may not make it into widespread classroom use tomorrow. But we're patient. The day after tomorrow is time enough.