Editor's note: On Tuesday, February 2, Fordham hosted the ESSA Acountability Design Competition, a first-of-its-kind conference to generate ideas for state accountability frameworks under the newly enacted Every Student Succeeds Act (ESSA). Representatives of ten teams, each from a variety of backgrounds, took the stage to present their outlines before a panel of experts and a live audience. We're publishing a blog post for each team, comprising a video of their presentation and the text of the proposal. Below is one of those ten. Click here to see the others.
Toward A Next-Generation School Accountability System
Design Priorities
Next-generation accountability systems must inspire schools and their communities to lift the achievement of all graduates to college- and career-ready levels. School rating systems must pinpoint challenges, spur more innovation, and inspire much broader local support if these ratings are to provoke lasting change.
We’ve learned from the current accountability system that we’ve got a long way to go. Our system is designed to protect the state’s role in monitoring school quality, to foster more educator and community engagement, and to avoid taking a short-sighted view based on today’s backlash-driven malaise. Our design builds upon the belief that the academic achievement and growth of every child is most important—especially so for children of color and those from low-income communities—and that non-test academic indicators should round out the picture of student and school success. We have imagined a system to guide resource allocation and rally school teams, school boards, communities and funders to the cause.
Our two design priorities:
1) We must give all students our attention, but focus on closing achievement gaps.
A high-quality education is the best way to improve the odds for all young people. No gaps are more pernicious or at odds with the promise of America than gaps among whites and people of color and gaps among youth at opposite rungs of the socioeconomic ladder. Given our nation’s shifting demographics—in which people of color are on a trajectory to surpass whites as the majority (some estimate within twenty-five years)—there are few goals that should be more important to our nation than closing these persistent gaps.[1] We place our democracy, economy, and standing in the world at risk if we do not. With more than 51 percent of American students on free and reduced-price lunch, we must also attend to the needs of low-income students.
2) Local communities should have real decision making in accountability system design.
Past accountability systems were the darlings of policy makers, think tanks, foundations, editorial boards, and advocates; they rarely had the support of educators, school communities, and the public writ large. They were too often equated with excessive testing that many parents tell each other “takes time away from learning.” Our design provides school communities the opportunity to select additional indicators and measures in every component through discussion of what matters most to them, to share that publicly, and to commit to work that addresses goals that the community develops. Our design also acknowledges that one size does not fit all. It allows communities to apply for waivers to experiment with new assessments and processes, such as measuring growth more frequently or measuring growth to competency.
Below, we detail how the system works for an elementary school with grades pre-K-5, in a mid-sized exurban school district with real socioeconomic diversity.
System Overview, Including Summative Scoring and School Ratings
Our system relies on state-determined indicators (“base points”) and locally determined indicators (“local points”). Both the state and local indicators are in the four categories required by ESSA: academic achievement, growth, progress toward English language proficiency, and student success/school quality.
Schools acquire up to one hundred “base points” through growth and proficiency on state assessments (eighty points) and reductions in chronic absenteeism and increases in social-emotional learning (SEL) indicators (twenty points). Separately, school districts select the indicators for schools to acquire up to twenty “local points” by performing against goals set for each of the same four components. Districts decide, in collaboration with their school communities, what indicators and targets to set. Districts submit plans to the state for approval.
Acquiring local points is not an opportunity to recover lost base points. Rather, schools are assigned a performance tier (green, yellow, or red) and a level (1-10 stars) using both their base score and their local score. A green, eight-star school can score between ninety and ninety-two base points and at least fifteen local points. Another school might score fifteen on the locally selected measures, but have its rating dragged down by a score of sixty on its base indicators—making it a red, 3-star school. Schools that have any racial/ethnic subgroup of students consistently below the state average for that subgroup, across all indicators with a three-year rolling average, cannot be rated above red/three stars.
We envision a clickable “dashboard” reporting results by each component, with clear delineation between performance on state indicators and local indicators, and enabling users to click through for more detail.
Schools are assigned colors and stars/levels with the table below. Red schools will be subject to interventions determined by the state; yellow schools are the responsibility of local school boards.
TABLE 1. SUMMATIVE SCORING & CATEGORIES[2]
Components and Indicators: Base Points
Component 1: Academic Achievement
This component accounts for thirty of one hundred base points and relies on rigorous state-developed assessments in English language arts and mathematics in grades three, four, and five, and a state-developed assessment in science in grades four or five. There are four sub-indicators: all students, targeted students, gap-closing, and pre-K-2 literacy. In the first three sub-indicators, points are weighted (or “pro-rated”), with more points for students achieving above standard and fewer for below standard. The school receives points for the performance of “all students,” and it receives additional points for targeted subgroups (defaulted as racial subgroups unless the “n” size is too low or the district can justify to the state other subgroups) and for gap closing between historically high-performing and low-performing students. Districts select a locally administered measure for pre-K-2 literacy from a state-approved list.
Component 2: Academic Growth
This component accounts for thirty of one hundred base points. While measuring growth to proficiency might be ideal, our imagined state cannot do this at scale; our default design uses student growth percentiles and awards points equally to mathematics and English language arts for grades three, four, and five, and focuses on subgroups and gap closing. The school receives points for the performance of “all students,” and it receives additional points for the growth of targeted subgroups (defaulted as racial subgroups unless the “n” size is too low or the district can justify to the state other subgroups) and for gap closing between historically high-performing and lower-performing groups. Our state allows a waiver process (discussed below) that allows school districts to experiment with growth to proficiency.
Component 3: Progress toward English Language Proficiency
This component accounts for twenty of one hundred base points for schools with significant English language learner (ELL) populations, defined as having at least 5 percent of all students designated as ELLs. Schools with ELL populations below 5 percent will count this component as ten base points (five for each indicator). Schools with no reportable ELLs will not include this component. For schools below 5 percent or with no ELLs, the points—either ten or twenty—will be distributed equally across components one and two. This component awards schools points for two equally weighted indicators: ELL proficiency rates, as measured by scores on a state-specific or consortium test (e.g., WIDA’s ACCESS), and English learner re-designation rates, measured by the percentage of ELLs who are reclassified as English learners within five years of being designated as ELLs. Like the proficiency component, the ELL proficiency indicator is weighted based on proficiency (i.e., one point for "at proficiency," .06 for "approaching proficiency," etc.).
Component 4: Student Success/School Quality
This component accounts for twenty out of one hundred base points. Students can’t achieve if they aren’t showing up, nor if their teachers are absent,[1] so our system emphasizes reducing teacher absenteeism (five points) and reducing chronic student absenteeism (ten points). The school also earns up to an additional five points by meeting a district-determined SEL goal measured by a district-selected assessment, such as DESSA.
Components and Indicators: Local Points
For each of the four components, school districts select one indicator and one measure. Districts are responsible for ensuring these local measures meet state-determined criteria for quality. An indicator for component one might be fifth-grade performance in fine arts and the measure a performance task scored using a district rubric. In component two, the district might measure growth in science. Component three could include bi-literacy (language proficiency in English and in another language) or reducing ELL designation rates in three years. In component four, school climate could be measured by questions on the TELL Survey and/or district-led surveys administered to families. Or schools might measure additional SEL goals. For each indicator, the district sets a goal, and up to five points are awarded based on performance against the target. Ideally, the district plan lays out school-specific indicators, targets and goals.
Waivers for Innovation
Our waiver process is for districts that have the capacity and commitment to innovate. It invites districts to go beyond the growth and achievement components of the default state plan, while still using some of the state’s year-end assessments as a public check. The waiver process requires districts to:
- Select one state assessment in grades 3-5 for English language arts, one in grades 3-5 for mathematics for component one (plus science) and component two.
- Identify measures in one or more subject areas beyond the state assessments. The district could use portfolios and/or competency-based assessments in pre-K-5. It might experiment with assessments administered more frequently than annually or measure growth against competency, not age.
- Propose new calculations and weights for each measure.
- Submit a sampling of tasks, rubrics, and processes the school/district will use to score measures, such as procedures for quality assurance, multiple scorers, etc.
- Submit its quality control plan for scorers and to assure validity and reliability of the measures.
State responsibilities include:
- Approving the waiver plan and publicly reporting on the district’s tools and processes.
- Inspecting the results to make sure the assessments are being scored accurately.
- Providing technical assistance on assessment development/selection and scoring.
Conclusion
We began by imagining that next-generation accountability systems can be an inspiration to school communities, engaging them in efforts to improve achievement for all while making sure that we are delivering on the promise of America. Here is how we think the system we’ve presented does that:
Gap Closing
- Both achievement and growth components weight gap closing. Nineteen of one hundred base points are earned by gap closing. This means that low-income students and youth of color count more than middle- or upper-class white students. We don’t apologize for that. Lifting people out of poverty and making sure that all students regardless of race achieve are among our greatest priorities.
- The system requires each school to address racial and socioeconomic gaps as subgroups as well, unless there are none or there is too little racial or socioeconomic diversity in the school.
Local Choice and Engagement
- Districts decide the details of its plan and submit the plan to the state.
- Districts and school communities choose a pre-K-2 literacy measure and goal.
- Districts and school communities select additional indicators and goals for academic achievement, growth, English language proficiency, and student success/school quality. Within the district, some schools might use common measures for discipline/suspension rates; other schools might share common measures on working conditions.
- Districts can apply for a waiver to use additional measures for growth and achievement.
[4] Each “all students” and “targeted subgroups” indicator for Component 1 is weighted based on Table 4 below. Schools can earn more than three points per indicator if many students score at the advanced (i.e., highest) achievement level.
[5] The state default system selects race/ethnic subgroups against white students and low-income against all students.
[6] As measured by the change in the difference in percentage of students at or above proficiency for the subgroup with the highest proficiency and the subgroup with the lowest proficiency. This indicator uses a five-year average gap between each subgroup as the base for comparison.
[7] As measured by: The number of all ELLs who re-designate at the school in the current year (no matter how long they’ve been English learners), divided by the number of all ELLs who re-designate at the school in the current year + all five-year-plus non-redesignated ELLs at the school.
[8] Emphasis is reducing chronic absenteeism. Measured by the percentage of student population that has been absent 10 percent or more of the school year (excused and unexcused combined). This indicator will use an index that awards points (up to 10) based on the school’s rate (e.g., 0-1% = 10 points, 1.1-2% - 9 points) and based on state historical data for absence rates. Same calculation for teachers.
To calculate weighted points for each “All students” and “Targeted subgroups” indicator in Component 1:
(total points from Table 4 above) 86
divided by
(total possible points if all students scored at level three) 100
= 0.86
(indicator’s weight) multiplied by
3
= 2.58 points in indicator (out of 3)
[1] In a recent study of the fortylargest metropolitan areas, 16 percent of teachers are chronically absent, missing at least eighteen days. A study by the National Bureau of Economic Research estimates that each ten days of teacher absences reduce students' mathematics achievement. A growing body of evidence shows that student chronic absenteeism reduces literacy.
[1] Brookings reported that, as of the 2010 Census, fourteen states already had majority-minority toddler populations, including Florida, Georgia, Mississippi, New Jersey, New Mexico, and New York. Populations in these states’ schools will be majority-minority quite soon.
[2] The Appendix charts 2-4 detail all points and calculations for base points and local points.
