Contract talks are to resume in Youngstown this afternoon. The administration insists that ADC-related language cannot be struck from the contract by law; the union disputes this. There’s some weird cryptic talk from the supe about the union wanting to “go… back to old antiquated practices” as well, which union reps deny. Given all this, it remains to be seen if anything can really be settled at this point. Remote learning continues, but it does not appear that parents (and, by extension, their children) are fully bought in to Zoom School 2023 at this point. (WFMJ-TV, Youngstown, 8/30/23)
I am going to put these two stories together, and leave them here without much comment. I feel like they are thematically linked, and relevant to one another, despite big differences. See what you think. First: A discussion of stressed out students in one Ohio district, along with discussion of possible solutions. (Newark Advocate, 8/29/23) Second: A discussion of the challenges facing Black students in the Dayton area, along with a discussion of possible solutions. (Dayton Daily News, 8/29/23)
Passions ran high at this week’s meeting of the elected board of Akron City Schools, with a dozen people rising to sing the praises of the I Promise School and the LeBron James Family Foundation. At least one of those said that previous public discussion of objectively very bad academic data from the school amounted to “bullying” and “belittling” of school staff and families. Elected board members pushed back on that characterization. Unfortunately (?) for everyone, a planned report out on additional academic outcome data did not go forward due to illness. (Akron Beacon Journal, 8/29/23)
“Excellence gaps” are the disparities in advanced academic performance that exist between student groups. These gaps have important implications for both academic equity and American economic competitiveness, as the most lucrative jobs often go to those who perform at the highest levels.
Although considerable work has evaluated how and why these excellence gaps occur, what’s not been examined closely is what excellence gaps look like for students of different races/ethnicities within the same socioeconomic group. This new report uses NAEP data on eighth graders over the last two decades to trace the performance of America’s highest-achieving students by both race/ethnicity and socioeconomic status (SES).
Specifically, Fordham’s research associate Meredith Coffey and national research director Adam Tyner examine:
The extent to which racial/ethnic excellence gaps can be explained by differences in SES;
Whether excellence gaps still exist when racial/ethnic groups are compared within the same socioeconomic groups; and
How excellence gaps by race/ethnicity and SES have shifted over the past two decades, including since the COVID-19 pandemic.
This new analysis reveals five key findings and several implications for education leaders and policymakers who want to increase the numbers of students from all backgrounds whose academic performance rises to the top.
Besides suffocating heat, humidity, and wildfires, the summer of 2023 also brought a seismic shift to higher education: the Supreme Court’s striking down of affirmative action in college admissions.
Putting aside the rancorousdebates about the rationale and implications for the decision, at the heart of the Harvard case was the clear evidence that Harvard was discriminating against Asian students.
An incriminating 2022 study about Harvard admissions found a “substantial penalty against Asian American applicants relative to their white counterparts.” Scholars estimated that, given the overall admissions rate for Asian Americans at Harvard was around 5 percent, removing what amounted to a handicap would increase their admissions chances by at least 19 percent.
What’s more, the researchers took on a surprisingly candid tone when noting the differences between the Asian and White applicant pool:
While it is widely understood that Asian American applicants are academically stronger than whites, it is startling just how much stronger they are. During the period we analyze, there were 42 percent more white applicants than Asian American applicants overall. Yet, among those who were in the top 10 percent of applicants based on grades and test scores, Asian American applicants outnumbered white applicants by more than 45 percent.
Startling indeed.
Findings from this new study, Excellence Gaps by Race and Socioeconomic Status, reminded us of this eye-popping imbalance. Authored by Fordham’s Meredith Coffey and Adam Tyner, the report digs into how race and socioeconomic status (SES) interact to shape academic “excellence gaps”—disparities in performance among groups of students achieving at the highest levels.
Their analysis utilizes nearly twenty years of eighth-grade reading and math assessment data (from 2003 to 2022) to document the progress of America’s highest-performing students, meaning those who earned “Advanced” scores on the National Assessment of Educational Progress, a.k.a., “The Nation’s Report Card.” Among other things, it finds that fewer Black and Hispanic students from the highest-SES group (those with college-educated mothers) are achieving at Advanced levels than we would expect given their socioeconomic status. That’s a disparity clearly worth our attention.
But so are their findings on Asian American high achievers—who deserve our attention for a different reason. Two decades ago, Asian American and Pacific Islander students (AAPI) were already crushing it at the Advanced level, and they’ve only made more progress since then (Tables FW-1–2). Part of that progress is due to raising the floor: Coffey and Tyner find that, among students in the lower-SES ranks (those whose mothers have a high school diploma or less), there’s a substantial rise over time in the proportion of AAPI students who are Advanced.
Add it up and we can see that the AAPI advantage has only grown.
Table FW-1. Percentage of students scoring at the Advanced level in math in 2003 and 2022
Source: NAEP Data Explorer
2003
2022
Percent Increase
AAPI
13
25
92%
Black
1
1
0%
Hispanic
1
2
100%
White
7
9
22%
Table FW-2. Percentage of students scoring at the Advanced level in reading in 2003 and 2022
Source: NAEP Data Explorer
2003
2022
Percent Increase
AAPI
5
12
140%
Black
1
1
0%
Hispanic
1
2
100%
White
4
5
25%
Now let’s put these numbers into a context that is familiar to admissions officers at highly-selective colleges. If we consider both the percentage of students in each racial subgroup achieving at the Advanced level and their share of the student population, what does the racial composition of students scoring Advanced look like? Here are the pie charts for our top students in 2003 and again in 2022.
Figure FW-1. Racial composition of eighth grade students at the Advanced level in math in 2003 and in 2022
Source: NAEP Data Explorer
As anyone can plainly see, the proportion of Advanced students who are White dropped significantly, from 82 to 61 percent (Figure FW-1). Yet most of the diversity gains came from Asian students (who went from 10 percent to 22 percent) and, to a lesser degree, Hispanic students (from 3 percent to 8 percent). Unfortunately, the proportion of Advanced students who are Black decreased over that time, from a tragically low 3 to 2 percent.
Now let’s see how it looks for reading in 2003 and then in 2022:
Figure FW-2. Racial composition of eighth grade students at the Advanced level in reading in 2003 and in 2022
Source: NAEP Data Explorer
The pattern in Figure FW-2 is largely the same: big declines in the proportion of White students, with large gains for Asian and Hispanic students—once again, more gains for the former than the latter when it comes to reading. The Black proportion is again down, from 5 to 3 percent.
*******
So what can we take from all of this, particularly when it comes to Asian high achievers?
First, they are making solid gains and their success deserves to be recognized.
Second, although our high-achieving students are a more diverse group than twenty years ago, most of this growing diversity is driven by gains by Asian and Hispanic students. For Hispanic students, that largely tracks the growth of their population as a whole, which has nearly doubled over the past two decades. That’s part of the story for Asian students, too (their numbers are up by a third), but it’s also due to their improved performance. Case in point: Our new report finds that Asian students are so high achieving that even those in the lowest-socioeconomic-status group often equal or outperform higher-SES students of other racial and ethnic groups.
Third, we need to learn from the success of AAPI students and their families—not be threatened by it or seek to depress their chances of gaining admission to prestigious institutions. At the national, state, and local levels, policymakers and educators should ask: Are there observable practices among Asian students that could apply more broadly? For instance, are they more likely to participate in extracurricular activities, sign up for more challenging classes, or take part in academic tutoring, clubs, or competitions? Are these behaviors helping AAPI students to reach the highest level of academic achievement? If so, how could smart policies expand those opportunities to students from other communities?
Education reformers spend an inordinate amount of time, energy, and resources (rightly so) on supporting low-performing students. But high performers are often leftto fend for themselves. Let’s just say this: It’s not right. We can do better. And we should start doing better today.
Introduction
Piling on to months of alarming findings on learning loss, NWEA reported in July 2023 that progress toward academic recovery continues to lag, despite the return to in-person schooling. Last year’s achievement gains were significantly smaller than typical prepandemic gains, leaving students over four months behind in both reading and math.[1] Exacerbating decades-old achievement gaps, the losses have been largest for Black, Hispanic, and low-income students. In other words, despite renewed national attention to issues of race and equity, American schools are still disproportionately failing their most marginalized students. And the stakes are high: kids in school during the pandemic could end up earning nearly 10 percent less than peers just a few years older,[2] with lifetime earnings reduced by 1.4, 2.4, and 2.1 percent for White, Black, and Hispanic students, respectively. And other generations will suffer, too, with a projected annual GDP loss of as much as $188 billion.[3]
What’s more, learning loss has occurred across the achievement spectrum: the academic performance of even the highest-potential students has either plateaued or declined.[4] And historically underserved student groups are no exception.[5] On the 2022 National Assessment of Educational Progress (NAEP), for example, the number of Black and Hispanic eighth graders achieving Advanced math scores each dropped by half compared to 2019, declining to only 1 percent and 2 percent, respectively.[6] Ensuring that all students—including the highest achieving—have the opportunity to fulfill their academic potential is urgent for both fairness and economic reasons: individuals deserve the opportunity to reach their potential, and American economic competitiveness depends on the cultivation of innovations founded on advanced academic achievement.[7]
To this end, a growing body of research has called attention to the urgency of combating what Jonathan Plucker, Nathan Burroughs, and Ruiting Song termed “excellence gaps”: disparities in academic performance among groups of students performing at the highest levels.[8] These gaps are evident in comparisons by race/ethnicity (Figure 1) and by socioeconomic status (Figure 2). Studies of excellence gaps have found disparities in advanced performance among students of different genders,[9] students in urban/rural settings,[10] and students with varying immigration histories.[11]
Figure 1. There are pronounced excellence gaps across racial/ethnic groups.
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. AAPI refers to Asian American and Pacific Islander students.
Figure 2. There are pronounced excellence gaps across socioeconomic groups.
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. FRL is an indicator of lower SES that means the student qualified for free or reduced-price lunch. AAPI refers to Asian American and Pacific Islander students.
To date, much education policy has focused on overall achievement gaps, or average achievement differences between student subgroups. Researchers have increasingly explored not just the impact of individual demographic factors but also how different factors interact with one another to influence student achievement.[12] That is, if White students earn higher average math and reading scores than their Hispanic peers (Figure 3), is that simply because the families of White students are wealthier on average than those of their Hispanic peers? Academic success is, of course, strongly associated with socioeconomic status (SES).[13] Indeed, considerable research suggests that both race and SES—at least as they are typically measured—correlate with achievement gaps. Studies show, for instance, that Black students, on average, underperform their White peers at each socioeconomic level.[14]
Figure 3. Widely documented gaps appear in educational proficiency by race/ethnicity.
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. AAPI refers to Asian American and Pacific Islander students.
What education research has not yet examined closely, however, is what excellence gaps look like for students of different races/ethnicities within the same socioeconomic group, particularly in the wake of the COVID-19 pandemic.[15] If differences in SES can explain racial disparities at the top end of academic achievement, then the critical action steps are to address economic inequalities and improve schools serving the lowest-income students. But if SES doesn’t fully explain the excellence gap—as it doesn’t for the broader racial achievement gap—then we must investigate why our schools are failing to serve other underrepresented students (e.g., Black and Hispanic students) with the greatest potential.[16] Solving any problem first requires understanding its cause.
This study is the first to focus on the question of how race and SES interact to shape excellence gaps. Using NAEP data on eighth graders over the last two decades, we trace the performance of America’s highest-achieving students by both race/ethnicity and SES.
To illuminate the interaction of race and SES in predicting excellence gaps, this analysis addresses the following questions:
To what extent can racial/ethnic excellence gaps be explained by differences in SES?
Are excellence gaps still apparent when racial/ethnic groups are compared within the same socioeconomic groups?
How have excellence gaps by race/ethnicity and SES shifted over the past two decades, including since the COVID-19 pandemic?
Data and methods
We acquired data on high achievers using the NAEP Data Explorer (NDE). The NAEP is administered by the National Center of Education Statistics (NCES), which is housed within the U.S. Department of Education’s Institute of Education Sciences. Reading and math assessments have typically been administered every two years for eighth-grade students. Other subjects are assessed less frequently.[17] The next most frequently tested subject is science, in which eighth graders are assessed about every four years.[18]
We use publicly available, national NAEP reading and math assessment data from 2003 to 2022 (Table 1), as well as science data from 2009 to 2019 (the latter is in the Appendix). With tests and student surveys typically covering more than 100,000 eighth-grade students during each administration year,[19] the NAEP provides a large-scale, representative picture of how American students are performing across time and in various subjects.[20] The high quality of the NAEP reading, math, and science assessments are part of what make the NAEP the “Nation’s Report Card.”
Table 1. Math and reading are assessed regularly by the NAEP (eighth grade).
Note: ✓ = Included in this report’s analysis of time trends. ✓✓ = Included in this report’s static analysis of student performance and analysis of time trends.
Year
Math
Reading
Science
2003
✓
✓
2004
2005
✓
✓
2006
2007
✓
✓
2008
2009
✓
✓
✓
2010
2011
✓
✓
✓
2012
2013
✓
✓
2014
2015
✓✓
✓✓
✓✓
2016
2017
✓✓
✓✓
2018
2019
✓✓
✓✓
✓✓
2020
2021
2022
✓✓
✓✓
Our analysis examines eighth-grade results, which provide the most useful data for our purposes. Fourth graders also take the NAEP at the same intervals, but their surveys provide less socioeconomic data because NCES has found their responses on questions like mother’s education to be unreliable.[21] Twelfth graders, the other age group that participates in the NAEP, take the assessments less frequently and may supply less indicative results due to potential lack of academic engagement at the end of high school.[22]
Identifying race and socioeconomic status
Our analysis focuses on excellence gaps among the four largest racial/ethnic groups: Asian American and Pacific Islander (AAPI) students, Black students, Hispanic students, and White students.[23] We focus on mother’s education as a proxy for SES,[24] although some results by self-reported number of books in the home[25] and whether the students qualify for free or reduced-price lunch (FRL)[26] are also included for context in the Appendix (the results are similar regardless of the SES variable used).
Mother’s education is measured by a question on the NAEP questionnaire in which test-takers select from the following options: “She did not finish high school,” “She graduated from high school,” “She had some education after high school,” “She graduated from college,” or “I don’t know”[27] (the final group is excluded from the analyses).
Note that the available proxies for SES are not comprehensive and, in NDE, cannot be used in conjunction with each other (while including race/ethnicity as well). Not only do both variables rely on student knowledge or estimation, but what the term “socioeconomic status” encompasses is contested and may include family financial resources, family structure, and other factors for which data are unavailable. Moreover, there is race- and ethnicity-specific variability in the SES distributions even within SES categories. For example, among households where the mother has completed college (our highest-SES group), other socioeconomic differences, such as differences in income or family structure, likely remain.[28] Although this is the first study to focus on the intersection of race/ethnicity and SES as those factors correlate with excellence gaps, if more nuanced data were available, our results could be somewhat different. For example, it is possible that a more comprehensive picture of SES would have more predictive power and diminish the separate predictive power of race/ethnicity, as has been shown in some other studies.[29]
Identifying advanced achievement
To identify the highest-performing students, we evaluate the proportion of students who earned “Advanced” scores on the NAEP. This is the highest achievement level and indicates that a student has demonstrated full competency in the subject via “superior performance.” For example, for an eighth grader to earn the Advanced designation in math, she must demonstrate high-level skills, such as analyzing whether a certain data visualization will be helpful for solving a given problem.[30] In reading, Advanced skills include, for example, the ability to synthesize textual evidence from multiple sources in order to support a claim.[31]
Using data from NDE covering all eighth-grade students for each subject and year, we created crosstabs with our race variable and each of our SES variables, including data on the percentage of students in each racial/socioeconomic group who earned Advanced scores (for example, the percentage of Advanced-scoring White students whose mothers did not complete high school).[32] It should be noted that NDE cannot conduct statistical significance tests for the pooled samples and trends over time used in this report. Fortunately, the large size of the NAEP samples makes identifying statistical significance trivial in most cases; if there is a substantive difference between groups, that difference is very likely to be statistically significant.
Results I: The intersection of race and SES in excellence gaps
In this section, we examine the most recent NAEP data collections in order to evaluate how the intersection of socioeconomic and racial/ethnic categories relates to excellence gaps.
Finding 1: Even within SES groups, racial/ethnic excellence gaps persist.
As shown in Figure 4, differences in rates of Advanced achievement remain when students are disaggregated into SES groups. Specifically, AAPI students achieve Advanced at the highest rate, followed by White students, then Hispanic students, and then Black students, who are sometimes tied with Hispanic students.
Figure 4. Excellence gaps by race/ethnicity persist within student socioeconomic groups.
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. The measure of SES is the mother’s highest level of education. AAPI refers to Asian American and Pacific Islander students.
At the same time, within each racial/ethnic group, students of the highest SES achieve Advanced at the highest rate, with a consistent decrease in the share of students achieving at the Advanced level as SES decreases.
Yet there is also some overlap among the racial/ethnic groups of higher and lower SES. For example, in math, the highest-SES Black students are more likely to achieve at the Advanced level than one of the groups of White students and two of the groups of Hispanic students. Specifically, 3.0 percent of the highest-SES Black students achieve at the Advanced level, outperforming White students whose mothers did not finish high school (2.5 percent), Hispanic students whose mothers finished high school (2.0 percent), and Hispanic students whose mothers did not finish high school (1.5 percent).
Similarly, in reading, although White students are more likely to achieve at the Advanced level than their Hispanic counterparts within each SES category, the highest-SES Hispanic students are more likely to achieve at the Advanced level (3.0 percent) than the two lower-SES groups of White students, whose Advanced achievement rates were 2.0 percent and 1.3 percent.
AAPI students, however, are so high achieving that even those in the lowest SES group often equal or outperform higher-SES students of other racial/ethnic groups. In math, 13.0 percent of the lowest-SES AAPI students reach the Advanced level, a figure surpassed only by the highest-SES White students (and other groups of AAPI students). The same pattern holds in reading.
We present these same data in descending order by Advanced rates in table form below.
Table 2. Table presentation of Figure 4.
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. The measure of SES is the mother’s highest level of education. AAPI refers to Asian American and Pacific Islander students.
Student Groups
% Advanced Rate (Math)
Student Groups
% Advanced Rate (Reading)
College Graduate
AAPI
38.3
College Graduate
AAPI
15.5
Some College
AAPI
20.5
Some College
AAPI
8.8
College Graduate
White
16.3
College Graduate
White
7.3
High School Diploma
AAPI
15.0
High School Diploma
AAPI
5.8
No High School Diploma
AAPI
13.0
No High School Diploma
AAPI
4.3
Some College
White
8.5
Some College
White
4.0
College Graduate
Hispanic
6.8
College Graduate
Hispanic
3.0
Some College
Hispanic
4.5
High School Diploma
White
2.0
High School Diploma
White
4.0
Some College
Hispanic
2.0
College Graduate
Black
3.0
College Graduate
Black
1.3
No High School Diploma
White
3.0
No High School Diploma
White
1.0
High School Diploma
Hispanic
2.5
High School Diploma
Hispanic
1.0
No High School Diploma
Hispanic
2.0
Some College
Black
1.0
Some College
Black
1.5
No High School Diploma
Hispanic
1.0
High School Diploma
Black
0.5
High School Diploma
Black
0.0
No High School Diploma
Black
0.5
No High School Diploma
Black
0.0
As indicated, we use mother’s education as our primary measure of SES. But the SES and racial/ethnic patterns of Advanced achievement on eighth-grade math and reading NAEP assessments are similar when either FRL (Appendix, Figure A.1) or self-reported number of books in the home is used instead (Appendix, Figure A.2). Science has much lower levels of Advanced achievement, meaning that some racial/ethnic groups do not reach 1 percent Advanced achievement, regardless of SES (Appendix, Figure A.3).
Finding 2: Racial/ethnic excellence gaps are larger among higher-SES students than lower-SES students.
Another way of examining the intersection of race and SES in producing excellence gaps is to see how racial/ethnic excellence gaps vary by SES. Consider Figure 5, which shows the reading excellence gaps in percentage-point differences between groups overall (“all”) and within SES groups. If racial/ethnic gaps were driven by SES, we would expect the gaps to be much smaller—or even nonexistent—within SES groups. But in fact, excellence gaps are not necessarily smaller within SES groups. Indeed, within the highest-SES group (college-educated mothers), excellence gaps are always larger than the overall gap between racial/ethnic groups. On the other hand, for the two lowest SES groups, the White-Black and White-Hispanic excellence gaps are small, ranging from 0.3 to 2.0 percentage points.
For example, Figure 5 shows that the overall AAPI-Black excellence gap in reading is 10.7 percentage points. Yet the gap between AAPI and Black students whose mothers graduated from college is even larger (14.5 percentage points). Likewise for the White-Hispanic excellence gap: the gap for students in the highest SES group (4.3 percentage points) is larger than the overall 3.7 percentage point gap. In contrast, for students whose mothers have a high school diploma or less, the White-Hispanic excellence gap is at most 1.0 percentage point. The generally low rates of Advanced achievement for lower-SES students helps explain why excellence gaps are relatively small for lower-SES students and much larger for high-SES students.
Figure 5. Reading excellence gaps are largest for students in the highest socioeconomic group.
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. The measure of SES is the mother’s highest level of education. AAPI refers to Asian American and Pacific Islander students.
In math (Figure 6), we see a very similar pattern: racial/ethnic excellence gaps are largest within the highest-SES group and are larger than the overall racial/ethnic gaps in Advanced achievement.[33] As with reading, racial/ethnic excellence gaps diminish within SES groups, and for the lowest-SES group (those students whose mothers do not have high school diplomas), the White-Hispanic and White-Black excellence gaps are minimal—1.0 to 2.0 percentage points, respectively (all of these patterns are similar when using self-reported number of books in the home to operationalize SES; see Figure A.4 and A.5 in the Appendix).
Figure 6. Math excellence gaps are largest for the highest-SES students.
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. The measure of SES is the mother’s highest level of education. AAPI refers to Asian American and Pacific Islander students.
Finding 3: SES has a similar effect on all racial/ethnic groups.
When evaluating the intersection between racial/ethnic identification and SES, it is helpful to consider the extent to which SES moderates racial/ethnic excellence gaps. Figure 7 uses the highest-SES group (students whose mothers are college graduates) as the reference group and shows how the bottom three SES groups compare to the top SES group in terms of Advanced achievement rates. Calculated separately for each racial/ethnic group, each bar shows the percentage of students in that SES group divided by the percentage in the high-SES group, such that if the rates of Advanced achievement were equal, the value would be one.
Figure 7 shows that the effect of SES on Advanced achievement is similar across groups. For math, the values for students with mothers who attained “some college” are all between 0.5 and 0.6, meaning that a little more than half of those students achieved at the Advanced level compared to students in the highest-SES group. For the lower-SES groups, SES has a similarly consistent effect, with the exception of lower-SES AAPI students—who, as indicated earlier, outperform other racial/ethnic groups. Although the effect of SES on Advanced reading achievement appears to be erratic for Black students, that is likely a rounding issue based on the fact that just 1 percent of the highest-SES Black students achieved at that level in all four years of NAEP data analyzed (these patterns are similar when using self-reported number of books in the home to operationalize SES; see Figure A.6 in the Appendix).
Figure 7. The effect of SES on Advanced achievement is generally consistent across racial/ethnic groups.
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. The broken line represents 1.0, meaning that a value is equal to the value of the highest-SES group for that racial/ethnic category. The measure of SES is the mother’s highest level of education. AAPI refers to Asian American and Pacific Islander students.
Results II: Excellence gaps by race and SES over time
The patterns in excellence gaps described above appear in most of the recent years of NAEP data (2015 to 2022). In this section, we examine trends in these data over the past two decades, once again using the highest level of education attained by the student’s mother as the SES variable (additional results may be found in the Appendix).
Finding 4: The last two decades have generally seen rising Advanced achievement for all racial/ethnic and SES groups but particularly for AAPI students.
For all but the lowest-SES Hispanic and Black students, Advanced achievement is appreciably higher in the most recent data collections than in the early 2000s, the earliest years of this study. This pattern holds for both reading (Figure 8) and math (Figure 9).
Figure 8. Over the past two decades, reading excellence gaps by race/ethnicity within socioeconomic groups have endured.
Note: These are the authors’ calculations from data representing eighth-grade achievement on the reading portion of the NAEP. The measure of SES is the mother’s highest level of education. AAPI refers to Asian American and Pacific Islander students.
Among the lower-SES ranks, a particularly positive development is the substantial rise in the proportion of AAPI students who are Advanced achievers. In the early 2000s, these students typically outperformed other low-SES groups by only a percentage point or two, and no racial/ethnic group’s lowest-SES students had more than 3 percent achieving Advanced in math or 1 percent in reading. But lower-SES AAPI students broke away from the pack beginning in the early 2010s and strongly outperformed their counterparts in all other racial/ethnic groups in recent years. By 2019, 11 percent of the lowest-SES group of AAPI students reached Advanced in math, and 2 percent notched Advanced in reading. Their improvement is notable and encouraging, despite the fact that it does equate to widening excellence gaps during the study period, as AAPI students were slightly outperforming other groups even in the earlier years of the data.
Some of these patterns of Advanced achievement over time appear for science, but the lower overall rate of high achievement means that the gaps are mostly apparent for higher-SES groups. For more on this, see Figure A.7 in the Appendix.
Finding 5: The pandemic era saw declining Advanced achievement for all racial/ethnic groups.
Another—more troubling—trend is the reversal of earlier progress in Advanced achievement between the 2019 and 2022 NAEP data collections, particularly in math (Figure 9). Especially among higher-SES students, Advanced achievement rates generally rose from 2003 to 2019 for all racial/ethnic groups. Between 2019 and 2022, however, among the students whose mothers attended college (whether or not they graduated), Advanced math achievement dropped for all racial/ethnic groups. These declines in Advanced performance are, of course, likely attributable to the challenges of the COVID-19 pandemic, which other studies have shown took a toll on the performance of lower achievers even more than that of high achievers.[34] These patterns are similar when using self-reported number of books in the home as the SES variable (see Figure A.8 and Figure A.9).
Figure 9. Over the past two decades, math excellence gaps by race/ethnicity within socioeconomic groups have not closed.
Note: These are the authors’ calculations from data representing eighth-grade achievement on the math portion of the NAEP. The measure of SES is the mother’s highest level of education. AAPI refers to Asian American and Pacific Islander students. The year 2022 represents the only year of NAEP score data affected by the COVID-19 pandemic.
Summing up
We show that excellence gaps are predicted independently by student race/ethnicity and SES. Controlling for SES with the data available in NDE mitigates—but does not eliminate—excellence gaps by student race/ethnicity. That means that even when various racial/ethnic groups are compared within same-SES categories, excellence gaps largely remain. Still, race is not the entire story: higher-SES Black and Hispanic students sometimes outperform lower-SES White students. We also find that because Advanced achievement rates are generally low for lower-SES students, racial/ethnic excellence gaps are even larger within the highest-SES group than they are for students overall.
In addition, we see that all racial/ethnic groups had higher shares of students achieving at the Advanced level in the 2010s than the 2000s, although the lowest-SES Hispanic and Black students saw little growth in Advanced achievement. On the other hand, AAPI students of all SES categories have experienced soaring rates of Advanced achievement, and more research is needed to understand what factors are behind this success story.
Lastly, we find that excellence gaps have grown during the period of this study (2003 to 2022), particularly in math. The period coinciding with the COVID-19 pandemic saw some declines in Advanced achievement, although these are larger in math than in reading.
Policy implications
So what might policymakers and practitioners do to boost Advanced achievement for all student groups while working to narrow excellence gaps?
Happily, a recent report from the National Working Group on Advanced Education, on which one of us served, offers thirty-six concrete recommendations for policymakers and practitioners attempting to build a wider, more diverse pipeline of advanced learners (see Summary of recommendations of the National Working Group on Advanced Education).
The Working Group did not attempt to differentiate between recommendations focused on economically disadvantaged students versus those targeting underrepresented racial/ethnic groups. The present study indicates that, on the whole, that strategy was wise. Given the large and persistent impact that SES plays across all racial groups, actions that support lower-SES groups will support most underrepresented racial/ethnic groups, too. For example, making sure that every elementary school in the country, including high-poverty Title I schools, has a robust gifted program in place should dramatically increase the number of Black, Hispanic, and lower-SES students entering the pipeline of advanced learning opportunities.
However, this study also indicates that focusing on lower-SES groups alone won’t be enough to narrow racial/ethnic excellence gaps, because those gaps are particularly large among the highest-SES group. Put another way, policymakers and practitioners need to wrestle with the fact that fewer high-SES Black and Hispanic students, in particular, are achieving at Advanced levels than we would expect given their SES.
Several of the Working Group’s recommendations should be helpful to high-SES Black and Hispanic students, such as its calls for culturally relevant programs and curricula, “universal screening” as a check on racial bias in assigning students to gifted programs and to advanced courses, and stronger data-reporting requirements.
But even more aggressive actions might be necessary. Staffing policies provide a useful lever. Studies have shown that same-race teachers improve academic achievement for Black students.[35] A smaller body of research suggests that same-race teachers may also have a positive, if more modest, effect on Hispanic students’ achievement.[36] Having teachers of color for gifted and advanced programming has been shown to increase the enrollment of students of color in advanced courses.[37] To this end, state- and district-level policymakers can expand and invest in teacher career pathways that educators of color are most likely to pursue, including Grow Your Own teacher-prep programs and alternative teacher career pathways. Leaders will need to take care, however, not to run afoul of constitutional limitations on explicitly race-based policies, especially in light of the Supreme Court’s recent affirmative action decision.
Finally, leaders should invest in better understanding the remarkable success of even the lowest-SES AAPI students in reaching the highest levels of achievement. For example, are high-achieving AAPI students more likely to participate in after-school tutoring and academic enrichment programs? If so, how could policy expand those opportunities to students from other communities?
Shrinking excellence gaps, which are continuing to expand, is a tall order. The good news is that even seemingly small improvements in percentages will mean that thousands more children have gained the opportunity to excel.
Summary of recommendations of the National Working Group on Advanced Education
(Excerpted from the 2023 report “Building a Wider, More Diverse Pipeline of Advanced Learners: Final Report of the National Working Group on Advanced Education”)[38]
We believe that all children deserve an education that facilitates their growth toward their full potential and that fostering learning of students at advanced levels of achievement benefits both individual students and our society at large. We acknowledge that students’ likelihood of excelling academically is a function not only of their promise and ability but also of their background, past experiences, and opportunities and that, as a result, today we observe many excellence gaps and disparities in the proportions of students from various racial, ethnic, and socioeconomic backgrounds who participate and excel in advanced levels of academic learning.
Our schools have both an opportunity and a responsibility to create learning experiences that meet the needs of all students, part of which is creating and supporting advanced learning. Equity, done right, means opening up advanced education to all students who could benefit from it. And excellence, done right, means doing the hard work to help all students achieve at high levels—not just the students who come to school with great advantages.
To achieve these goals, we recommend that all school districts and charter networks build a continuum of advanced learning opportunities, customized to individual students’ needs and abilities, that spans the K–12 spectrum.
Recommended district and charter network policies and practices
Accessible front-loading (grades pre-K–5)
Provide accessible front-loading programming—either to all students or to those from historically underrepresented groups—starting in pre-K or Kindergarten
Adopt universal screening to identify students with potential for high achievement
Use data from universally available assessments
Use assessment data to identify additional students for advanced education services in every grade
Use local (i.e., school-based) norms
Acceleration (grades K–12)
Allow early entry to Kindergarten
Allow children who are ready for advanced material in all subjects to skip entire grade levels
Allow children to skip grade levels in particular subjects
Offer “grade-compressed” pathways for students
Embrace “concurrent enrollment”
Offer advanced courses in as many subjects as possible in grades 6–12
Automatically enroll students participating in elementary school advanced education programs in subsequent advanced learning opportunities in middle and/or high school
Intentionally recruit underrepresented and underserved students for external advanced learning opportunities, such as college for dual enrollment and/or online courses
Equitable achievement grouping (grades K–5)
Frequently and equitably evaluate all students
Ensure that teachers alter the complexity and pace of the curriculum
Err on the side of inclusion
Selective enrollment schools (grades 6–12)
Expand the supply of seats in such schools
Base admission on multiple indicators, including but not limited to exam score
Marry culturally responsive advanced instruction and prior preparation of students who are admitted
Social and emotional learning (SEL) and mental health supports for advanced learners (grades K–12)
Make programs culturally relevant to all students
Foster a positive school culture
Implement a plan for supporting advanced students’ mental health comprising triage, trauma informed practices, targeted intervention, and faculty support
Well-prepared educators
Provide high-quality professional-learning opportunities about evidence-based advanced-learning strategies
Inform feeder teacher-preparation programs
Empower teachers with actionable data on students’ abilities
High-quality instructional materials
Supplement high-quality instructional materials
Recommended state policies and practices
In school and district accountability systems, place significant weight on student-level progress over time
Eliminate any policies that bar early entrance to Kindergarten, middle school, or high school
Mandate the identification of students with advanced learning needs, providing services for said students, and the use of local, school-based norms for identifying students for advanced programs, particularly at the elementary level
Implement specific requirements about the services provided to advanced learners
Mandate that districts and charter networks allow for acceleration (including grade skipping) for students who could benefit from it
Publicly report on the students participating in advanced education, including their achievement and growth over time, as well as their demographic characteristics
Ensure that preparation and in-service professional-development programs offer evidence-based instruction in advanced education, both for district-level coordinators and for teachers
Enforce the federal requirement that states explain how teacher-preparation programs are addressing education of special populations, including advanced learners
Expand funding and other incentives to encourage schools to frequently and equitably evaluate all students and provide a continuum of services to every student who could benefit
Appendix
In this Appendix, we provide some additional results, including presentation of excellence gaps using alternative SES measures and excellence gaps in science.
Figure A.1. Math and reading excellence gaps by race/ethnicity persist within socioeconomic groups (using FRL).
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2019. FRL is an indicator of lower SES that means the student qualified for free or reduced-price lunch. AAPI refers to Asian American and Pacific Islander students.
Figure A.2. Math and reading excellence gaps by race/ethnicity persist within socioeconomic groups (using number of books reported in the home).
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. AAPI refers to Asian American and Pacific Islander students.
Figure A.3. Science excellence gaps by race/ethnicity persist within socioeconomic groups (using mother’s education).
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2019. AAPI refers to Asian American and Pacific Islander students.
Figure A.4. Math excellence gaps are largest for the highest-SES students (using number of books reported in the home).
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. AAPI refers to Asian American and Pacific Islander students.
Figure A.5. Reading excellence gaps are largest for the highest-SES students (using number of books reported in the home).
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. AAPI refers to Asian American and Pacific Islander students.
Figure A.6. The effect of SES on Advanced achievement is generally consistent across racial/ethnic groups (using number of books reported in the home).
Note: These are the authors’ calculations from data representing eighth-grade achievement on the NAEP, averaged from 2015 to 2022. The broken line represents 1.0, meaning that a value is equal to the value of the highest-SES group for that racial/ethnic category. AAPI refers to Asian American and Pacific Islander students.
Figure A.7. Over the past two decades, science excellence gaps by race/ethnicity have persisted within socioeconomic groups (using mother’s education).
Note: These are the authors’ calculations from data representing eighth-grade achievement on the reading portion of the NAEP. AAPI refers to Asian American and Pacific Islander students.
Figure A.8. Over the past two decades, math excellence gaps by race/ethnicity have persisted within socioeconomic groups (using books in the home).
Note: These are the authors’ calculations from data representing eighth-grade achievement on the reading portion of the NAEP. AAPI refers to Asian American and Pacific Islander students.
Figure A.9. Over the past two decades, reading excellence gaps by race/ethnicity persist within socioeconomic groups (using books in the home).
Note: These are the authors’ calculations from data representing eighth-grade achievement on the reading portion of the NAEP. AAPI refers to Asian American and Pacific Islander students.
[4] Scott J. Peters, Meredith Langi, Megan Kuhfeld, Karyn Lewis, “Unequal Learning Loss: How the COVID-19 Pandemic Influenced the Academic Growth of Learners at the Tails of the Achievement Distribution,” EdWorkingPaper: 23-787, Annenberg Institute at Brown University, June 2023, https://www.edworkingpapers.com/index.php/ai23-787.
[5] The one exception occurred in Black students’ performance in civics. Advanced performance in civics, however, remains dismal. The percentage of Black students earning Advanced civics scores previously rounded to zero; the 2022 percentage was only 1 percent. See Brandon Wright, “High-achieving middle schoolers fare poorly in U.S. history and civics, finds NAEP, but that’s nothing new,” ADVANCE, Thomas B. Fordham Institute, May 5, 2023, https://edadvance.substack.com/p/high-achieving-middle-schoolers-fare.
[7] Tyler Clark and Julia Roberts, “What are excellence gaps and how can we close them? An interview with Jonathan Plucker and Scott Peters,” Gifted and Talented International 33, no. 1–2 (2018): 64–70, https://doi.org/10.1080/15332276.2019.1656565.
[8] Jonathan A. Plucker, Nathan Burroughs, and Ruiting Song, Mind the (Other) Gap! The Growing Excellence Gap in K–12 Education (Bloomington, IN: Center for Evaluation and Education Policy, February 4, 2010), https://files.eric.ed.gov/fulltext/ED531840.pdf.
[9] David Rutkowski, Leslie Rutkowski, and Jonathan A. Plucker, “Trends in education excellence gaps: a 12-year international perspective via the multilevel model for change,” High Ability Studies 23, no. 2 (2012): 143–66, https://www.tandfonline.com/doi/full/10.1080/13598139.2012.735414.
[11] Carla B. Brigandi, “First-generation students in rural communities: A study of effective programming components that support closing the excellence gap,” Gifted and Talented International 35, no. 1 (2020): 3–15, https://www.tandfonline.com/doi/full/10.1080/15332276.2020.1774944; David Rutkowski, Leslie Rutkowski, and Jonathan A. Plucker, “Trends in education excellence gaps: a 12-year international perspective via the multilevel model for change,” High Ability Studies 23, no. 2 (2012): 143–166, https://www.tandfonline.com/doi/full/10.1080/13598139.2012.735414.
[12] For work on the interaction of race, socioeconomic status, and gender, see Rebecca McGraw and Sarah Thuele Lubi, “A Closer Look at Gender in NAEP Mathematics Achievement and Affect Data: Intersections with Achievement, Race/Ethnicity, and Socioeconomic Status,” Journal for Research in Mathematics Education 37, no. 2 (2006): 129–50, https://pubs.nctm.org/view/journals/jrme/37/2/article-p129.xml, and Laia Bécares and Naomi Priest, “Understanding the Influence of Race/Ethnicity, Gender, and Class on Inequalities in Academic and Non-Academic Outcomes among Eighth-Grade Students: Findings from an Intersectionality Approach,” PLOS One 10, no. 10 (2015), https://doi.org/10.1371/journal.pone.0141363. For work on the interaction of race, socioeconomic status, and sexuality, see Sung Tae Jang, “The schooling experiences and aspirations of students belonging to intersecting marginalisations based on race or ethnicity, sexuality, and socioeconomic status,” Race Ethnicity and Education (2020), https://www.tandfonline.com/doi/abs/10.1080/13613324.2020.1842350. For work on the intersection of race and community poverty rates, see Mellissa S. Gordon and Ming Cui, “The Intersection of Race and Community Poverty and Its Effects on Adolescents’ Academic Achievement,” Youth & Society 50, no. 7 (2018), https://doi.org/10.1177/0044118X16646590, and R. Aaron Wisman, “Operationalizing the Intersection of Racial and Socioeconomic Diversity in Predicting School-Level Academic Achievement,” Education and Urban Society 52, no. 6 (2020), https://journals.sagepub.com/doi/10.1177/0013124519894989.
[13] Jonathan A. Plucker, and Scott J. Peters, Excellence gaps in education: Expanding opportunities for talented students (Cambridge, MA: Harvard Education Press, 2016).
[14] Sarah Theule Lubienski, “A Closer Look at Black-White Mathematics Gaps: Intersections of Race and SES in NAEP Achievement and Instructional Practices Data,” The Journal of Negro Education 71, no. 4 (2002): 269–87, https://www.jstor.org/stable/3211180; Katherine W. Pascall, Elizabeth T. Gershoff, and Megan Kuhfeld, “A Two Decade Examination of Historical Race/Ethnicity Disparities in Academic Achievement by Poverty Status,” Journal of Youth and Adolescence 47, no. 6 (2018): 1164–77, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8209685; Daphne A. Henry, Laura Betancur Cortés, and Elizabeth Votruba-Drzal, “Black–White achievement gaps differ by family socioeconomic status from early childhood through early adolescence,” Journal of Educational Psychology 112, no. 8, 1471–89, https://doi.org/10.1037/edu0000439.
[15] Jonathan A. Plucker and Scott J. Peters briefly touch on the intersection of race and socioeconomic status in their seminal 2016 book Excellence gaps in education: “Even when controlling for socioeconomic status, excellence gaps by race/ethnicity may still exist. . . . Any interventions to address excellence gaps that focus on poverty reduction may not fully address persistent excellence gaps.” See Plucker and Peters, Excellence gaps in education.
[17] At the most recent NAEP science assessment in 2019, 31,400 eighth graders participated. See U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, “Science,” December 12, 2022, https://nces.ed.gov/nationsreportcard/science.
[18] In 2009, the science framework changed such that the NCES has found that results prior to 2009 are not comparable. The current framework assesses students’ knowledge of physical, life, and earth and space sciences, as well as their abilities to identify and use science principles, apply scientific inquiry, and implement technological design. See U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, “What Does the NAEP Science Assessment Measure?”, May 11, 2021, https://nces.ed.gov/nationsreportcard/science/whatmeasure.aspx.
[20] Since 2003, Title I funding requirements have required the biannual participation in eighth-grade reading and math testing in all fifty states, as well as jurisdictions including Washington, D.C., the Department of Defense, and the Bureau of Indian Education. Students are randomly chosen for participation, which remains voluntary at the individual level. U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, “National Assessment of Educational Progress (NAEP),” NCES Handbook of Survey Methods, n.d., https://nces.ed.gov/statprog/handbook/naep.asp; National Center for Education Statistics, “NAEP Assessment Sample Design,” NAEP Technical Documentation, December 6, 2022, https://nces.ed.gov/nationsreportcard/tdw/sample_design.
[23] Hispanic students may be of any race, and the other groups are of non-Hispanic students. For our race/ethnicity variable, we used “race/ethnicity used to report trends, school-reported,” since it is the only race variable that has been collected consistently throughout the span of our study. Schools report student race/ethnicity, although this information is sometimes supplemented by student self-reported data. See The Nation’s Report Card, “Variable,” NAEP Data Explorer, n.d., https://www.nationsreportcard.gov/ndecore/xplore/NDE. In 2011, the category “Asian/Pacific Islander” was replaced by two categories: “Asian” and “Native Hawaiian or other Pacific Islander.” The latter remains too small for statistical analysis, and the NAEP Data Explorer treats “Asian/Pacific Islander” and “Asian” as equivalent categories across time, so we do as well. Similarly, also in 2011, the race category “Other or unclassified” was replaced by “Two or More Races.” Again, because the NAEP Data Explorer treats these two categories as equivalent across time, our analysis does, too. See U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, “Student Groups,” October 14, 2022, https://nces.ed.gov/nationsreportcard/guides/groups.aspx.
[24] Parental education is often used as an indicator of socioeconomic status, and limited research suggests that maternal education may be more strongly associated with academic performance than paternal education. See Gary Neil Marks, “Are Father’s or Mother’s Socioeconomic Characteristics More Important Influences on Student Performance? Recent International Evidence,” Social Indicators Research 85, no. 2 (2008): 293–309, https://www.jstor.org/stable/27734583; Elena Arias Ortiz, “What are the Factors of Success at University? A Case Study in Belgium,” CESifo Economic Studies 54, no. 2 (2008): 121–48, https://www.researchgate.net/publication/46513074_What_are_the_Factors_of_Success_at_University_A_Case_Study_in_Belgium.
[25] The results by the number of books reported in the student’s home are similar to those reported by the mother’s education and are presented in the Appendix. This variable has four values for which NAEP takers can respond: “Few (0–10),” “Enough to fill one shelf (11–25),” “Enough to fill one bookcase (26–100),” or “Enough to fill several bookcases (more than 100).” For ease of interpretation, we refer to the values ordinally as “Very Few Books,” “Few Books,” “Some Books,” and “Many Books.” Some studies note that students of higher SES may be better at estimating the number of books in their home. For example, see Kimmo Eriksson, Jannika Lindvall, Ola Helenius, and Andreas Ryve, “Higher-achieving children are better at estimating the number of books at home: Evidence and implications,” Frontiers in Psychology 13 (2022), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9623014.
[26] Although FRL is the most common indicator of socioeconomic status used in K–12 education research, it is a binary and therefore cruder measure than the other measures we employ.
[29] Paul L. Morgan, George Farkas, Marianne M. Hillemeier, Yangyang Wang, Zoe Mandel, Christopher DeJarnett, and Steve Maczuga, “Are students with disabilities suspended more frequently than otherwise similar students without disabilities?” Journal of School Psychology 72 (2019): 1–13, https://www.sciencedirect.com/science/article/pii/S0022440518301171.
[32] The NDE automatically rounds percentages to the nearest whole number. As a result, it’s possible that 1.6 percent of a certain student group earned Advanced scores and 1.4 percent of another group did so—a small difference of 0.2 percent—but the results suggest that twice as high a percentage of the first group earned Advanced compared to the second. Given that the percentages scoring Advanced are so low in many cases, the effect of automatic rounding could be more than marginal, but our approach, which averages rates for the static analysis, mitigates this concern.
[33] As discussed in the methodology section, there is variability in the SES distributions for different racial/ethnic groups.
[34] Peters, Langi, Kuhfeld, and Lewis, “Unequal Learning Loss.”
[35] See Seth Gershenson, Michael Hansen, and Constance A. Lindsay, Teacher Diversity and Student Success: Why Racial Representation Matters in the Classroom (Cambridge, MA: Harvard Education Press, 2021).
[37] Jason A.Grissom, Luis A. Rodriguez, and Emily C. Kern, “Teacher and principal diversity and the representation of students of color in gifted programs: Evidence from national data,” The Elementary School Journal 117, no. 3 (2017): 396–422, doi:10.1086/690274.
This report was made possible through the generous support of our sister organization, the Thomas B. Fordham Foundation. In addition, we are grateful to external advisers Scott Peters, senior research scientist at NWEA, and Jonathan Plucker, the Julian C. Stanley Endowed Professor of Talent Development at Johns Hopkins University, for their multiple rounds of insightful feedback; to Pamela Tatz for her thorough copyedit; and to Dave Williams for designing the figures and tables.
At Fordham, we thank Meredith Coffey and Adam Tyner for authoring this analysis, Chester E. Finn, Jr. for reviewing drafts, Jeanette Luna for handling funder communications, and Victoria McDougald for her role in dissemination.
Back from a little vacation. Sorry (for you) that’s it’s over!
The biggest thing we missed talking about while I was gone: The teachers strike in Youngstown. After last minute negotiations couldn’t break whatever impasse persisted to that point, the strike began on Wednesday, August 23, wiping out days one and two of the new school year. (Ideastream Public Media, 8/23/23) School year started in earnest in the district on Friday, August 25, fully remotely. School buildings are to remain closed to students and community members for the duration of the strike, although meal pickups are happening outside the buildings every day (parents being required to cross picket lines to do so, I would imagine). We learn here that full-time substitute instructors will use prepared curriculum, and district administrators and non-striking teachers will also be helping with online instruction. (WKBN-TV, Youngstown, 8/24/23) I was hoping to have an update for you on how the first remote day went, but nothing has been published in the media as yet. How odd. Instead, we have this weekend update from the teachers union talking about what they are hoping for when/if negotiations resume. (And it’s a twist which will likely make my long-time Gadfly Bites subscribers—Happy Monday to all eleven of you!—say “of course”.) While we had some contradictory information last week about what the sticking point might be—district officials said money, union officials denied it—but the answer is, as it always seems to be up here, HB 70. To wit: Union officials say that they want language removed from the contract that allows the superintendent to keep what is termed “extraordinary power”, similar to what the CEO had when the district was run by an Academic Distress Commission. District leaders say they cannot legally do so since, in reality, that governance is still in place for at least two more years, despite what day-to-day operations may have looked like since the ADC offramp legislation went into effect in 2021. (WFMJ-TV, Youngstown, 8/27/23)
An unmistakable fault line is emerging between much of public education and many of those it serves, particularly parents, on transgender issues. Put bluntly, a strong majority of Americans—57 percent in a recent poll conducted by the Washington Post and Kaiser Family Foundation—simply don’t buy the idea that a person can be a gender other than the one “assigned at birth.” The fissures grow even wider on related issues that schools must contend with. Sixty-nine percent oppose transgender females competing against biological females in youth sports. And overwhelming majorities, more than three-fourths of those polled, believe it’s “inappropriate for teachers to discuss trans identity in elementary schools.” Other polls have found strong majorities of parents think transgender students should use bathrooms and locker rooms to match their sex rather than their chosen gender identity.
Big swaths of public education are moving aggressively against these sentiments. The most egregious examples are policies that say school district personnel can or should keep a student’s transgender status hidden from parents. The advocacy group Parents Defending Education counts over 1,000 school districts, which collectively educate more than 10 million children, that encourage teachers and staff to allow students to “socially transition”—change their name, pronouns, or gender expression—without parental consent or notification.
It is not possible to overstate the level of distrust, even contempt, reflected in the practice of excluding parents from discussions about their child’s gender preference, even deceiving them if students claim their parents are unsupportive. Moreover, such policies, typically justified as protecting students from harm, are wholly unnecessary. Teachers are “mandated reporters” in every U.S. state. If they have reason to believe children are unsafe at home, they are required by law to alert local child protective services. In no other conceivable instances is there any justification for excluding parents from profoundly life-altering decisions about their own children. If schools insist on inserting themselves between parents and their children, they do so not because they are compelled to, but because they choose to. The Supreme Court has recognized for over a century, going back to Pierce v. Society of Sisters, that “the child is not the mere creature of the state.” Keeping a child’s gender switch a secret from parents as a “best practice” is the state saying, in effect, “Oh, yes he (or she) is.”
It is not news that Americans’ trust in major institutions has been in freefall for some time. In the case of public education, it’s a two-way street. A December 2021 poll conducted for EdChoice showed that only about a third of teachers (36 percent) say they trust parents, a level below their confidence in their principals, union leaders, and the U.S. Department of Education. This level of institutional disregard creates ripe conditions for public school personnel to assume that their judgment is superior to a child’s parents. And that’s exactly what appears to be happening. Coming at a time when Americans have more opportunities to educate their children outside of traditional public schools and a growing inclination to do so, the consequences of aggressive anti-parent policies could easily become public schools’ undoing. The increased tempo of debates over masking during the pandemic and current issues with ideologically-tinged curricula and activism have already alienated a significant number of parents. Enshrining into policy and practice ideological notions about student gender to which a majority of Americans do not subscribe could easily be fatal to support for public education at large.
It will be argued that these policies are rarely enacted, but that’s beside the point. A government-run institution granting itself permission to withhold life-changing information from parents about their own children is both profoundly alarming and a massive overreach. These policies effectively demolish parental authority and allow the state to assume a role for which it has no rightful or reasonable claim. There is simply no credible evidence to support the belief that parents do not have in mind the best interest of their transgender children. Moreover, keeping such information from parents almost certainly violates the Family Educational Rights and Privacy Act (FERPA), which gives parents virtually unlimited rights to review their child’s school records. By circumventing this law in spirit, if not in practice, schools not only undermine parental rights, but also risk legal consequences. Today there are some twenty active court challenges to these policies, and as Luke Berg, a lawyer at the Wisconsin Institute for Law & Liberty, who is litigating three of these cases, describes the situation, “It’s hard to imagine a faster way to destroy the public education system than to tell parents that one of the conditions of sending their children to public school is ceding their parental role to teachers.”
Indeed, there are no words adequate to capture this level of institutional hubris. If teachers in our nation’s public schools wish to restore and maintain Americans’ trust in education, they must be willing to acknowledge a simple fact about their profession: They’re not free agents, not activists or ideologues, but government employees with no reason or right to usurp parental authority.
A public education system that ignores or overrules the fundamental role of parents, placing the state in a position of superior authority over children will not be accepted for long. If this is the hill public education chooses to die on, don’t be surprised if it gets its wish.
For at least a decade, schools have been using online credit-recovery (OCR) courses to award bogus credits that satisfy graduation requirements, and thus inflating graduation rates. In spite of well documented abusesand comprehensive research studies showing negative outcomes associated with OCR use, schools persist in using them to push students—often the most disadvantaged—across the finish line.
The pandemic made this worse by ushering in a practice that is ripe for cheating: schools allowing students to take OCR courses entirely at home. Worse, some school systems have continued the practice post-pandemic, including three of the largest school systems in my home state of Georgia: Atlanta, Fulton, and Gwinnett.
I recently undertook substantial efforts to confront the problem in my own district (documented in this video), the Paulding County School District, which revealed a massive failure of accountability that may have implications outside of Georgia.
My experience began when I was unexpectedly assigned mid-year to oversee online credit recovery courses. I soon discovered that a few seniors assigned to me never showed up for class and were taking their course at home so they could graduate on time. I was expected to give them access to their assessments by unlocking them on Edmentum’s (the course provider) teacher platform. Knowing that the answers to the test bank were posted on third-party websites in a massive crowd-sourced cheating effort (see Figure 1), I refused to supervise these students and commenced my mission to rid the district of this ill-advised and harmful practice.
Figure 1. An example of how easy it is to cheat on Edmentum’s online credit-recovery courses
I first appealed to my school principal, urging that OCR exams be supervised on campus. But he rejected my proposal on grounds that the practice was a district norm. My first dead end.
Next I contacted my district administrators, but they sanctioned the practice and would not require exams to be proctored. A second dead end.
Realizing that this wasn’t going to be fixed internally, I made a video documenting the cheating and followed my own ethical obligations to report it to outside authorities.
I began by calling the U.S. Department of Education’s fraud, waste, and abuse hotline. For the agency to have jurisdiction, the program must be federally funded. I had reason to believe that Covid relief funds may have been used, but could not verify this. The U.S. DoE declined to get involved, but referred me to Georgia’s state education agency. A third dead end.
I contacted the state DoE’s Accountability Division, whose mission is “to provide all stakeholders with important information on the performance and progress of Georgia schools,” which it does mainly in the form of a school rating system in which graduation rates comprise 10 percent. Since the division’s mandate is limited to measurable outcomes, and not the means by which a school attains them, staff demurred and referred me to the agency’s External Affairs division. There I got some traction, bringing the matter to the attention of officials at the highest levels, but no action was taken. This may have been because, under Georgia’s “Strategic Waiver Initiative,” the DoE has permitted local school boards to waive all sorts of state regulations, provided that they attain certain annual increases in their district’s ratings. Either way, it was my fourth dead end.
I then turned to the Governor’s Office of Student Achievement, which exists to support “accountability and transparency through strategic data use to advance student success.” Since it has a legal mandate to “inspect academic records” to ensure that schools are “faithful to performance accountability requirements,” I requested an academic audit of the Paulding County School District’s credit recovery program. But their mandate extends only to standardized assessments required by the state. Lacking the authority to investigate local assessments, they also declined to get involved. A fifth dead end.
My next step was Cognia, the non-governmental accreditors that are supposed to ensure that accredited online courses are effective and credible, which would include verifying that the assessments are valid, i.e., not gameable. Cognia’s current president distinguishes the purpose of accreditation from that of government accountability: “unlike compliance measures, accreditation identifies what graduation rates, test scores, and other indicators cannot tell on their own—what takes place in the school that leads to its results.” In other words, accreditation focuses on the means by which an increased metric is attained. Since enabling online cheating betrays public trust and obliterates the credibility of academic credits, I assumed Cognia would get involved, so I filed a complaint supported by extensive documentation.
But Cognia sent an email to the Paulding County School District about my complaint, explaining that the accreditor was declining to investigate due to “insufficient evidence.” This was surprising, considering that I was a first-hand source who gave them email evidence of the district’s approval of unsupervised testing, as well as multiple assessment-provider records showing how students aced tests in impossibly low amounts of time.
Cognia, it seems, has been enabling schools to engage in such practices for over a decade by giving OCR courses its stamp of approval, in spite of the various ways they fall short of basic accreditation standards. Accrediting ineffective and easily gameable courses should damage Cognia’s credibility. But that doesn’t seem to have happened. A sixth dead end.
The last institution I could think of that may have the power and will to bring some accountability was Georgia’s Professional Standards Commission (PSC). The PSC oversees teacher certification, and is charged with enforcing the state’s ethics code.
Of the ten standards that comprise the code, two seemed germane to this scheme: honesty and professional conduct. Honesty is violated when schools misrepresent student achievement. Professional conduct is a catch-all category for any number of behaviors that are detrimental to education and to the “welfare or morals of students.” (There is a third standard on test integrity, but this is only limited to state-mandated assessments. When it comes to the integrity of local assessments in Georgia, anything goes.) Enabling students to earn graduation credits fraudulently certainly misrepresents student achievement and harms the profession while damaging students’ consciences.
Alas, the PSC complaint form is focused on discrete violations by specific individuals and is not set up for reporting systemic cheating where administrators are complicit for creating a permission structure, and guidance counselors and teachers are complicit for carrying out the scam by unlocking tests knowing that students had access to the answers. My seventh and final dead end.
—
One of the largest cheating scandals in U.S. history unfolded from 2009–2011 in Atlanta Public Schools, when over 100 educators were implicated in changing students’ wrong answers on state-mandated assessments A few were sentenced to prison. (The Fulton County District Attorney who indicted Trump last week was the lead prosecutor!) There were severe consequences because there are definitive regulations for state-mandated assessments and clear sanctions for cheating. Educators thus take great pains to administer these tests with integrity.
Yet when it comes to increasing graduation rates, rightly called the most “fungible statistic in education,” educators can practice—or allow their students to practice—all sorts of gaming behaviors with impunity because they are held accountable by the state only for the outcome. Online-credit recovery is a prime example of this.
To deter this behavior, we need a better system that also holds everyone—education departments, accreditors, administrators, principals, and educators—accountable for the means, as well. Yes, it’s vital that more high school students earn diplomas. But those diplomas will become worthless unless they mean that graduates are prepared for college and career.
Some of Georgia’s largest school systems fall short of this standard—and that’s very likely true of many more across the country. The biggest victims are the students themselves, failed by their schools and thrust into adulthood with false promises and a lack of skills. It’s high time for our leaders to put these young people first.
Editor’s note: Jeremy Noonan resigned his teaching position in Paulding County after the district refused to address the problem. In response to an Atlanta Journal-Constitution article and video based on his experience, he was put on paid administrative leave in order for the district to conduct their own ethics investigation into his use of student data.
Ohio recently passed a historic state budget that includes, among other components, ambitious literacy reforms that require schools to follow the science of reading—an instructional approach that emphasizes phonics for building foundational literacy skills and knowledge-rich curricula to support vocabulary and comprehension. These are much-needed changes, and similar movements are also spreading across the country. But effective implementation of legislative reforms will be a crucial next step. If Ohio leaders—and officials in other locales working to execute science of reading laws—successfully follow through on implementation, these promising literacy reforms will have a much better chance of boosting reading proficiency.
To recap, the legislation requires Ohio districts and charter schools to adopt curricula that aligns to the science of reading starting in fall 2024. Backed by a large body of research, this approach emphasizes explicit and systematic phonics instruction, as well as knowledge-rich curricula that builds vocabulary and comprehension. To support the transition, lawmakers set aside some $170 million to replace outdated curricula and provide professional development for teachers. The overarching goal is to ensure that all children are taught to read via proven methods by well-trained teachers, ultimately leading to stronger reading proficiency statewide.
Under the new policy, schools that have previously embraced popular but debunked approaches such as “three-cueing” or “balanced literacy” will need to change course. Doing so is crucial, but it could also invite pushback from those wedded to the status quo. Some schools may openly defy state requirements. More likely, however, resisters will seek to undermine state policy in subtler ways, such as claiming to follow scientifically based instruction but continuing to use disproven methods behind closed doors.
If state leaders aren’t attentive and hard-nosed about implementation, Ohio’s promising literacy efforts could turn into mush. How can they ensure rigorous implementation? Let’s take a look at seven ways.
1. Ensure a complete and thorough system-wide survey of curricula materials. While weak curricula are assuredly in use across Ohio, there is no systemic data on how many schools use them. This leaves us uncertain about the heft of the implementation. It’s going to be much heavier lift if three-quarters of Ohio schools are using disproven methods than if only half of schools are doing so. Fortunately, the budget bill requires the Ohio Department of Education (ODE) to field a reading curricula survey, with districts and charters required to respond. Recognizing the urgency of collecting these data, ODE, to its credit, sent out the survey this week. The agency should now make certain that schools respond promptly and completely. On the latter count, ODE should ensure that school-level information is obtained, as curricula may differ across schools within a larger district. And, as recommended in this piece, it should also insist on specificity, making sure to collect not only the title of the material, but also publisher and year, as older editions may be of differing quality.
2. Keep the bad stuff off the high-quality instructional materials lists. The budget bill tasks ODE with creating two lists of high-quality instructional materials: one for core curricula and the other for intervention programs. All districts and charter schools must select curricula and programs from these state-approved lists (with one exception, discussed in #5 below). Curating carefully vetted lists of materials is a crucial implementation step, as the whole effort could be undermined if state officials—perhaps under lobbying pressure from publishers—include low-quality programs such as Fountas & Pinnell’s Classroom, Lucy Calkins’ Units of Study, or Reading Recovery. Timeliness is also key, as schools need to know this year which materials have the green light. The good news is that EdReports, a well-regarded national organization, has conducted detailed evaluations of reading curricula that Ohio policymakers can rely on. States such as Colorado, Louisiana, and Massachusetts have also developed solid lists of high-quality materials.
3. Smartly allocate instructional materials funding. Lawmakers set aside $64 million to subsidize schools’ purchase of high-quality materials. These funds are critical, but the bill doesn’t provide any direction to ODE about how to allot them. Moreover, while the overall set-aside is significant, it may not cover curricula upgrades in all Ohio schools. What this means is that ODE will likely need to develop an allocation method that prioritizes funds. At the front of the line should be the districts and charters that absolutely need to change curricula because their current ones do not make the state-approved lists. If there is a sufficiently large number of schools that must change curricula, ODE may need to further prioritize dollars, perhaps by providing subsidies to higher-poverty schools first. (In this event, it should also request additional funding from the legislature.)[1] One final issue that ODE may need to iron out is whether to provide a per-pupil subsidy up front or reimburse districts after purchase. A per-pupil amount might be more sensible, as reimbursement could end up incentivizing unduly expensive purchases.
4. Bolster teacher professional development (PD) requirements. Retraining teachers who are accustomed to using debunked teaching methods is essential to the science of reading effort, as they’ll be the ones shifting to a different approach and using the new materials. To build the knowledge and skills of Ohio’s teaching force, the budget bill sets aside $86 million over the biennium to pay teacher stipends for completing PD.[2] Implementation details are left to ODE, so the agency will need to sort out several issues:
Who must participate: The bill requires all administrators and teachers, regardless of grade or subject, to complete a whole “course” in the science of reading. But it also provides an exemption to those who have “previously completed similar training, as determined by the Department.” Thus, ODE will need to set criteria for the exemption, and it should set a high bar to qualify—perhaps exempting only those who have taken one of the state-approved PD courses before. One aim of the initiative is to get Ohio educators on the same page around effective methods, and a more inclusive statewide PD initiative would better achieve that goal.
Who may provide PD: The bill requires ODE to identify vendors that provide PD to educators. Much like the vetting of instructional materials, ODE should carefully screen prospective vendors. It might be wise to approve just a handful of vendors—Colorado approves a half-dozen, for example—rather than a long list of variable quality.
What counts as “course completion”: Teacher PD is notorious for being more of a box-checking activity than rigorous, skills-building work. ODE should put some meat on the “completion” bones and require PD vendors—as a condition of approval—to include an end-of-course assessment that educators must pass to complete the course and receive their stipend. If an educator falls short, he or she should have an opportunity for a retake; failing that, they should have to redo the course.
5. Scrutinize waiver requests. While the legislation requires schools to use state approved curricula and includes an explicit proscription on “three-cueing,” it also includes a loophole that could allow a school to use three-cueing in two circumstances: (a) if it receives a waiver from ODE to use it for a particular student[3] or (b) if a student’s IEP calls for the use of this method. To guard against abuse, ODE should carefully review waiver requests and likely reject most, if not all, of them. If it doesn’t, it runs the risk of becoming a rubber stamp that allows schools to circumvent the state’s science of reading requirements. ODE should also publicly report the number of waiver requests from each district and school, as well as how many were approved. The sunlight will also provide another safeguard against abuse.
6.Publicly report schools’ reading curricula on an ongoing basis. Beyond the survey mentioned above, Ohio’s new literacy laws require districts and charter schools to report core reading curricula and intervention programs on an ongoing, annual basis to ODE. While the legislation doesn’t explicitly require public reporting after data are sent to ODE, the agency can and should make them public à la Colorado’s curriculum transparency dashboard. This tool would provide communities a check on whether their local schools are following state law, and it would clearly flag any obvious cases of non-compliance. It may also allow communities and parents to advocate for changes if their schools are using programs that, while state-approved, are not up to their exacting standards. Lastly, public reporting could allow for analyses that link schools’ reading performance to their curricula selections, potentially shedding light on which are associated with the strongest learning gains.
7. Strictly enforce state literacy requirements. State officials shouldn’t turn a blind eye if schools are ignoring state law. As agencies in other states have done, ODE should step in and take corrective action if a district or school is using disapproved programs. The agency may also need to periodically conduct curricula reviews of schools (perhaps randomly selecting a small percentage each year) to verify implementation of high-quality curricula. All students deserve to learn via effective reading methods, and ODE should honor the intent of the literacy law through strong enforcement.
* * *
Led by Governor DeWine, the state budget bill greatly improves Ohio’s early literacy laws by requiring schools to follow the science of reading. For the effort to succeed and benefit Ohio students, implementation will be key. As this piece indicates, there’s a lot on the plate of state officials. But if Ohio can get the details right and rigorously implement its new policies, the payoff will be great: a better educated, more literate next generation of Ohioans.
[1] It’s also possible that fewer schools than expected will need to change curricula, allowing the $64 million set-aside to go further. If that’s the case, the state might consider creating another tier of instructional materials—“exceptionally high-quality”—and subsidize schools that seek to upgrade from a state-approved curricula to a top-tier curricula.
[2] All teachers in grades K–5; English teachers in grades 6–12; and intervention specialists, English learner teachers, reading specialists, and instructional coaches in grades pre-K–12 are eligible for $1,200 stipends upon course completion. Grades 6–12 teachers in other subjects (e.g., math or science) are eligible for $400 stipends. Though required to take PD, administrators are not eligible for a stipend.
[3] Provided he or she is not on a state-required reading improvement and monitoring plan.
Future Forward began in Milwaukee in 2005 as SPARK—a small-scale, local effort to combine family engagement with intensive tutoring to help low-income elementary-age students improve their literacy skills. It has since expanded significantly, rebranded, and moved under the aegis of national nonprofit Education Analytics, Inc. The SPARK pilot has been studied extensively, and a new report, from researchers at the University of Wisconsin-Milwaukee, gives us the most comprehensive analysis yet.
This analysis extends previously-published research, following 576 students who were entering kindergarten, first grade, and second grade in seven Milwaukee Public Schools in fall 2013. The children, nearly all of them Black or Latino, were randomly assigned to receive either two years of Future Forward literacy intervention along with traditional business-as-usual reading instruction, or simply business-as-usual instruction. Students with IEPs were excluded from the analysis (except for one child with a speech-language plan), as were English learner students. Those in the treatment group were offered thirty minutes of phonics-focused, one-on-one tutoring from a paraprofessional or trained volunteer three times each week via pullouts from noncore classes. Informal learning opportunities were also included in afterschool programming provided by the Boys and Girls Clubs of Greater Milwaukee. But the program’s most innovative aspect was family engagement, which included regular communication from an engagement coordinator (typically another school parent) about the child’s progress, home visits where parents were provided with “development opportunities” that could help their child’s literacy growth during the school year and in the summer, plus monthly community events that included learning opportunities, as well as fun activities to attract families.
The prior research study found that, after two years of active participation—comprising an average of thirty hours of in-school tutoring actually attended per student per year, sustained communication to parents, dozens of community events, hundreds of home visits, and two years of afterschool opportunities—treatment group students saw positive impacts on foundational literacy, reading achievement,, and school attendance. The impacts were greatest for students starting at the lowest levels of reading skills. So far so good.
The new analysis follows the original participants for five years beyond the end of the active treatment, through winter 2020 when children were in middle school. Attrition occurred in both treatment and control groups due to students moving out of state or switching to private schools, but the researchers note that the percentages are within the norms set by the What Works Clearinghouse to avoid attrition bias. Data are provided using the same sources as previously, although the district did change its reading assessment vendor beginning in 2015–16. The nationally-normed scores were converted to grade level equivalents to allow direct comparison to previous scores.
Treatment students continued to notch higher test scores than their control group peers through middle school. However, positive impacts were now concentrated around students who had started SPARK at the highest levels of reading skills, reversing the trends observed during active treatment and providing solid evidence of a fading out of effects, at least for certain students. Furthermore, one year after active treatment, Future Forward students were almost exactly at grade level in reading (0.03 years below), while control group students were 0.29 years below. However, five years after active treatment, the typical Future Forward student was reading 2.03 years below grade level, and even those who had started at a higher skill level were reading 1.17 years below. The situation was even more dire for control group students, with the worst performers reading more than three years below grade level. Neither is desirable, but at least participants did not fare as poorly as their non-participant peers.
Another tiny bright spot: The average Fast Forward student whose attendance data could be reliably tracked across all years under study recorded 5.9 fewer absences than the average control student. However, the researchers suggest that the incomplete data for a large number of students could be skewing those results. These positive effects were also concentrated among students who started at the higher end of the literacy skills distribution. No discussion is provided about possible mechanisms at work here.
These findings accord with similar analyses over the years of HeadStart, Success for All, and kindred interventions for disadvantaged young children: early positive impacts that begin fading out in one or two years. And if the educational systems in which students spend the rest of their primary and secondary education are underperforming, no amount of early boost will get them—and keep them—on par with their more-advantaged peers. The authors conclude here that it is unreasonable “to expect time- and resource-limited programs to fundamentally transform the educational terrain,” and that their findings “highlight the need for systemic change.” Programs like Future Forward are part of the solution, but are not sufficient on their own, even with the addition of home- and community-based efforts.
“Teens need to start school later. No more excuses, experts say.” —Education Week
In recent years, sixteen states have passed laws that will hold back third-grade students who cannot read. —Wall Street Journal
Where other states have seen declines in Catholic school enrollment, Florida’s school choice scholarships have allowed enrollment to grow. —Wall Street Journal
While many science of reading laws overlook the importance of background knowledge, more policy-makers are starting to recognize its importance. —Matt Barnum (Chalkbeat)
“Chinese parents around the country are launching programs and schools to defend traditional values like hard work and meritocracy.” —City Journal
Jeers
The poorest schools in New York City already have smaller-than-average class sizes, so affluent schools will likely benefit most from a new class-size law. —Chalkbeat
Los Angeles and New York City are limiting co-location, the practice of housing traditional public and charter schools in the same building. —The 74
School choice legislation has allowed more businesses to offer and more families to use hybrid school options, mixing home- and micro-school models. —Washington Post
Many schools are finding success with various initiatives such as tutoring, extended school years, expanded mental health services, and better data tracking. —McKinsey
The Pacific Legal Foundation asked the Supreme Court to weigh in on a prestigious high school’s revamped admissions policies that purportedly discriminate against Asian-American students. —National Review
Despite the rising skepticism of college credentialing, degrees maintain a positive return on investment. —Ben Wildavsky
When classes moved abruptly online at Iowa State University in March 2020 as part of Covid-mitigation actions statewide, psychology professor Jason Chan expected big changes in student behavior. Specifically, he worried about his students being easily able to cheat on unproctored online exams. But he saw little evidence of that, with his students producing a fairly typical distribution of scores, comparable to the proctored, in-person exams he gave earlier the same semester. Intrigued, he and associate Dahwi Ahna undertook a deeper analysis of university-wide test scores to see whether his experience was typical.
Chan and Ahna obtained data from eighteen courses offered during the spring 2020 semester—everything from large introductory lecture classes to smaller, specialized major courses for upperclassmen. They calculated an average score for the in-person exams (i.e., first half of the semester) and an average score for the online exams (i.e., second half of the semester) for more than 2,000 students. They then computed the correlation between the two halves using a meta-analytic approach which treated each course as an individual study.
Across the board, scores from the unproctored online exams closely correlated with those from the traditional in-person exams. A positive correlation was observed for every course, and correlations did not vary significantly by types of questions asked on the exams, field of study, course level, exam duration, or enrollment. They ran a standardized effect size analysis to confirmed that score inflation via relaxed grading for Covid-impacted online exams was not driving the correlation.
The researchers also ran the same first-half/second-half comparison for a set of courses that were all in-person in the spring semesters of 2018, 2019, and 2021. Overall, these courses showed stronger correlation between first half and second half exam scores than the split in-person/online courses of 2020. When the comparison was restricted to courses taught by the same instructors in both sets of semesters, the difference became smaller and was no longer statistically significant. However, the sample size was just nine courses, which could have impacted the outcome of the analysis.
The data indicate that cheating was likely uncommon when students had to pivot and take exams online in that first Covid-disrupted semester, though it’s possible that cheating was actually widespread but simply not effective at boosting student outcomes beyond typical norms. The researchers equate the situation to the difference between a closed-book versus open-book proctored test: Students who have missed lectures, paid little attention in class, or have low understanding of the material despite attending will likely not to do well on a typical exam even if they have a textbook, notes, or even the entire internet at their fingertips. The corollary is typically true as well: Students with good attendance and a decent understanding of the material will generally fare well on an exam, with or without notes and other materials before them.
This feels like good news as far as it goes. Whatever effort these college students were putting forth prior to pandemic disruption appeared to continue immediately after their entire educational experience turned upside down. However, this does not mean that extended use of online exams—and indeed virtual teaching and learning writ large—will result in the same behavior when it is the everyday norm. Authors Chan and Ahna conclude from their research that online exams can provide a valid and reliable assessment of learning just the same as in-person exams, but that seems like too large a leap given the unique circumstances involved in spring 2020. Professors and institutions sticking to online exams in the absence of force majeure need to leverage the technology to permanently hinder cheating. After all, exams have traditionally been proctored for a reason.
