Ruth Wattenberg is a former member and president of the State Board of Education.

Before the pandemic shut down D.C. schools, each public school, like each student, got a report card. Every fall, the school report card included a STAR rating, from one through five. 

The rating was based on a formula designed and used by the Office of the State Superintendent of Education, D.C.’s education agency. Federal law requires OSSE to identify the “bottom 5 percent” of District schools, so that they can receive additional funding. 

In effect, OSSE’s STAR Framework ratings used a measurement of need to indicate a measurement of quality.

And as a measurement of quality, the formula failed.

The report card is now due for revision. OSSE is scheduled to bring its new proposal to the State Board of Education for a vote on Feb. 21. Many in the community had high hopes for much-needed changes.

When I served on the SBOE, we invited experts to analyze the formula and the data it used. One expert showed us how high-poverty schools disproportionately got low ratings, even when test scores reported that their students had learned more than average. Education researcher and D.C. public school parent Betsy Wolf concluded that “our accountability system measures family income more than it measures school quality.” Based on these findings, the SBOE resolved in 2022 that the rating system was “fundamentally flawed” and recommended eliminating it.

In D.C., where families can choose to send their kids to any public school in the district, this flawed rating system is especially consequential. “Many kids have left their neighborhood schools” because of the ratings, says Sheila Carr, a Ward 7 resident, grandparent of current D.C. students, and co-founder of Decoding Dyslexia – DC.

A small exodus can trigger budget, staffing, and program cuts that have the potential to drive more families away from a particular school, triggering yet more cuts. A decade ago, Carr remembers, this meant multiple school closings. Although DCPS has avoided more closures recently, enrollment at some schools are way down. Anacostia High School enrolls just 287 students. 

At the SBOE’s early January meeting, some parents’ hopes of pushing to revamp the report cards faded. OSSE surfaced its new report card, and, instead of labeling schools with stars, the new proposal assigns each school a number, one to 100, called an “accountability score.” 

The number will still be highlighted on each school’s online profile and on the central School Report Card, where it will be among the first and primary impressions of a school that parents will see. The formula that produces the new accountability score, while slightly revised and less toxic, is still biased against low-income schools. It is still the same formula OSSE uses to identify the neediest schools for the U.S. Department of Education.

Rebranding efforts aside, the scores, and the report cards where they’re embedded, lack much of the information that families want. 

***

A significant chunk of data that underlie the new accountability score’s formula comes from students’ scores on a single test, attendance rates at a moment in time, and graduation rates. 

Students in grades three through 10 take end-of-year tests, while high schoolers also take other standardized tests such as the SAT, and Advanced Placement and International Baccalaureate exams.

These one-time results are important data to know and report, but they don’t tell us about school quality. An unfortunate fact of American life and education is that students from low-income families typically enter school multiple years behind their middle- and upper-class classmates. If these students attend schools of “average” quality and make “average” progress every year, their individual test scores, and the aggregate scores of high poverty schools, will be much below those of equally “average” schools attended by more affluent students. And tests aren’t the only problem. For many reasons, children from lower-income families tend to have higher absence rates. Even if higher poverty schools can improve their attendance rates, and any school scores based on them, their scores will still be lower on average than at more affluent schools. 

A fairer way to grade schools is to measure students’ test score growth or attendance improvement.

Education and poverty expert Sean Reardon says that average test scores “are the results of all the opportunities kids have had to learn their whole lives, at home, in the neighborhood, in preschool and in the school year.” 

“So it’s misleading to attribute average test scores solely to the school where they take the test,” he adds. “If you want to know how good the schools are, a better, but not perfect, measure would be the learning rates because those are measuring how fast are kids learning while they’re in school, regardless of where they started.”

Less biased data on school quality measures educational practices and conditions known to promote student learning, such as teacher retention and the extent to which a school offers instruction on a variety of subjects, including social studies, science, and the arts, rather than an overly narrow focus on math and reading (which is what end-of-year tests focus on). Survey data showing student perceptions, such as the extent to which students feel academically challenged and supported is also an effective metric.

The formula that OSSE uses to produce each school’s score is complicated, but it is anchored by a set of indicators (such as test scores and attendance rates) that are each weighted differently. In 2019, OSSE researched the relationship between a school’s level of student poverty and the school’s score on every indicator in the formula. 

End-of-year test scores, chronic absenteeism rates, and high school graduation rates are heavily related to poverty, according to OSSE’s 2019 survey. For example, at the high school level, end-of-year test scores correlate as high as .794 and .818 on a scale of zero to one. 

On the other hand, test growth and attendance growth scores generally have low correlations to poverty. At the elementary level, attendance growth correlates with poverty at a near-zero .006.

According to Wolf, in 2019 the correlation between a school’s overall score and its poverty level was .72. 

But what about the new formula? In her new Substack, Wolf calculates that under the new formula (used this past year to identify schools for the Department of Education), the overall correlation to poverty as .48—an improvement over the previous iteration of the rating system but not good enough. The correlation at high schools under the new formula is much worse: .763. 

OSSE defends its new proposal based on feedback from parents who say they want a single number that’s highly visible on a school’s online profile page.

I attended some of those sessions and over the years have spoken to many, many parents about this. In my experience, it’s true that many parents are looking for a single number, but they’re looking for a single number that reflects school quality. It’s unlikely that any would say that they wanted a number that was clearly biased against schools with higher concentrations of poverty.

If OSSE wants a single number, it should construct a formula and produce a score that reflects quality, not poverty (though all measurements will have some error).

***

There is a better way. Surveys from the State Board of Education and the advocacy group EmpowerEd suggest that parents and educators want to see data that reflects school culture and climate. That includes the satisfaction levels of students, teachers, and parents, as well as student safety and teacher retention, test score growth—all of which are also strong indicators of school quality—and in-school and extracurricular programs. It is also important for the report card to include a measure of how much time, especially in elementary schools, is spent on subjects other than reading and math, which would encourage schools to pay attention to history and social studies, science, and the arts.

All of the information currently contained in the accountability number should still be reported. Whether they reflect school quality or not, one-time school test scores, graduation rates, and all of the data points that contribute to the current ratings are important to know.

The SBOE proposed a color-coded Report Card in 2022 with no ratings or scores but that makes it easy for families to discern how well a school is doing based on key indicators. It enables families and parents to see a big picture and then dig deeper according to their interests. EmpowerEd has also recommended a revised report card. Both samples provide a more accurate and less biased picture of our city’s schools than the revisions OSSE is proposing. 

OSSE will bring its penultimate proposal to the SBOE meeting on Wednesday, Feb. 7. Board members can respond with comments and recommendations. The board will vote on the final proposal on Feb. 21. Under D.C.’s governance rules, the SBOE cannot amend the proposal, and members can only vote to approve or disapprove the new report card. OSSE says if the report card is disapproved, the old report card will remain in place this fall.

Ward 1 SBOE Rep. Ben Williams says he has serious concerns about OSSE’s current proposal. 

“It’s incomplete, misleading, and confusing,” he says, adding that he would prefer the old report card stay in place rather than approve inadequate revisions. “The majority of the public wants useful information to make school decisions, and the score misleads them around assessing school quality. Let’s get it right,” he says.

Carr, the grandparent and co-founder of Decoding Dyslexia, says parents “want to know whether schools are good, not whether they serve low-income students. Just give us that.”