Much-anticipated national test scores released last week seem to show a country stagnating across the board in reading and math, with scores of almost every demographic group and region slipping since 2013.
Most experts agree that the new NAEP scores, derived from testing done last spring in 2015, present some difficulty for educational reformers who had become accustomed to touting progress in NAEP scores as evidence that new curriculum and accountability systems were paying dividends.
"We've gotten so used to NAEP scores going steadily up that these results are striking, and cause for concern," said Rick Hess, a senior fellow at the American Enterprise Institute.
"Combined with last week's testing reversal by the White House, this could lend momentum for those pushing back against Obama-era centerpieces like test-based teacher evaluation," Hess said.
But hasty most experts also warn against conclusions based on a single test year. "We're not sure whether it is a blip or a trend, or what to do about it," said Michael Petrilli, president of the Washington D.C.-based Fordham Institute.
Petrilli notes that, though it is very tempting, it is wrong to draw too much out of a single testing year like this. "We'll have to wait to 2017 to see what direction this is heading," he said.
Across the board
Math scores on the National Assessment of Educational Progress came down in all regions, in most states, and, in math, for both 4th and 8th grades. On reading, 8th grade scores fell, while 4th grade scores held steady.
Drops were also evident across most states. Only three states improved their 4th grade math scores since 2013, for example, with 16 states declining.
Still, some experts warn against sweeping conclusions.
"There is no one solution for a given community," said Chris Minnich, executive director of the Chief State School Officers, speaking at the Tuesday press conference announcing the test results.
Minnich argues against the temptation to seek nationwide solutions to what appears to be nationwide performance slip. "The District of Columbia needs a different conversation than does rural Montana. Both have similar gap problems, but the solutions will be very different," Minnich said.
Achievement gaps along socioeconomic and ethnic lines remain a vexing problem in the new scores. In 8th grade math, for example, Asian students continued to excel at 61 percent proficient. White students struggled to keep up at 43 percent, with Hispanics, at 19 percent, and African Americans at 13 lagging far behind.
Data abuse
NAEP scores reflect careful statistical samples from across the country, taken not every year but at set intervals, which vary depending on the subject and the age group. This 4th and 8th grade reading and math sequence, one of the most closely watched, is done every two years.
Unlike state-level scores tied to federal No Child Left Behind, NAEP samples do not judge the performance of any one school let alone any single teacher. And because no individual careers hinge on the outcomes, there is less incentive to devote class time to test-aligned test preparation.
But while the high quality of the test is widely acknowledged, interpreting the data is always subject to dispute. The misuse and abuse of NAEP data has become so common that stakeholders and experts in the field now even have a name for it. They call it "misNAEPery."
One of the strongest critics of data abuse is Matt Chingos at the Urban Institute, who points out in a new report that raw NAEP scores are highly deceptive when used to compare state performance. Those raw scores, he notes, do not account for demographic imbalances in the number of at-risk, poor and minority students a state teaches.
Test results in Texas and Florida, for example, appear to be mediocre at best when looked at with raw data. But when adjusted for demographics in Chingos' data, both states climb near the top. Meanwhile, states like Utah, Iowa and Connecticut, which appear to be well above average in raw data, fall to middling or worse when adjusted.
All of that said, no one, including Chingos, is is suggesting that the drop in performance from 2013 to 2015 can be explained by shifting demographics alone.
Uncertainties
Experts did offer some possible explanations as food for thought, however.
One would be that there is a mismatch between the new Common Core curriculum and the NAEP scores. "There are some areas where the assessment and the framework are not exactly aligned," acknowledged Peggy Carr in the Tuesday press conference.
Misalignment would occur when a particular skills is tested and one age, based on earlier curriculum, but is now taught at a later age, so the students have not been taught the material yet. Another misalignment could happen if the material is tested using very different keywords and concepts from those used in teaching it.
Petrilli also points to Common Core misalignment, noting that if this is the case, it would not be an indictment of either NAPE or the curriculum. It may be simply that things are now being taught in a different sequence than they are being tested.
And Carr also noted that in some content areas and age groups, there is already 100 percent alignment between NAEP and Common Core, "and we still saw declines in those areas."
And Minnich noted that even in states not using common core standards saw some slippage. "We need to look at the nation as a whole and make sure that what is being taught across the country is close to what NAEP measures," Minnich suggested.
Another possibility Petrilli raises is that lingering economic uncertainty presents a "stiff headwind," with many communities and vulnerable groups still struggling to find or hold good jobs, with repercussions moving from the home to the school.
Most experts agree that the new NAEP scores, derived from testing done last spring in 2015, present some difficulty for educational reformers who had become accustomed to touting progress in NAEP scores as evidence that new curriculum and accountability systems were paying dividends.
"We've gotten so used to NAEP scores going steadily up that these results are striking, and cause for concern," said Rick Hess, a senior fellow at the American Enterprise Institute.
"Combined with last week's testing reversal by the White House, this could lend momentum for those pushing back against Obama-era centerpieces like test-based teacher evaluation," Hess said.
But hasty most experts also warn against conclusions based on a single test year. "We're not sure whether it is a blip or a trend, or what to do about it," said Michael Petrilli, president of the Washington D.C.-based Fordham Institute.
Petrilli notes that, though it is very tempting, it is wrong to draw too much out of a single testing year like this. "We'll have to wait to 2017 to see what direction this is heading," he said.
Across the board
Math scores on the National Assessment of Educational Progress came down in all regions, in most states, and, in math, for both 4th and 8th grades. On reading, 8th grade scores fell, while 4th grade scores held steady.
Drops were also evident across most states. Only three states improved their 4th grade math scores since 2013, for example, with 16 states declining.
Still, some experts warn against sweeping conclusions.
"There is no one solution for a given community," said Chris Minnich, executive director of the Chief State School Officers, speaking at the Tuesday press conference announcing the test results.
Minnich argues against the temptation to seek nationwide solutions to what appears to be nationwide performance slip. "The District of Columbia needs a different conversation than does rural Montana. Both have similar gap problems, but the solutions will be very different," Minnich said.
Achievement gaps along socioeconomic and ethnic lines remain a vexing problem in the new scores. In 8th grade math, for example, Asian students continued to excel at 61 percent proficient. White students struggled to keep up at 43 percent, with Hispanics, at 19 percent, and African Americans at 13 lagging far behind.
Data abuse
NAEP scores reflect careful statistical samples from across the country, taken not every year but at set intervals, which vary depending on the subject and the age group. This 4th and 8th grade reading and math sequence, one of the most closely watched, is done every two years.
Unlike state-level scores tied to federal No Child Left Behind, NAEP samples do not judge the performance of any one school let alone any single teacher. And because no individual careers hinge on the outcomes, there is less incentive to devote class time to test-aligned test preparation.
But while the high quality of the test is widely acknowledged, interpreting the data is always subject to dispute. The misuse and abuse of NAEP data has become so common that stakeholders and experts in the field now even have a name for it. They call it "misNAEPery."
One of the strongest critics of data abuse is Matt Chingos at the Urban Institute, who points out in a new report that raw NAEP scores are highly deceptive when used to compare state performance. Those raw scores, he notes, do not account for demographic imbalances in the number of at-risk, poor and minority students a state teaches.
Test results in Texas and Florida, for example, appear to be mediocre at best when looked at with raw data. But when adjusted for demographics in Chingos' data, both states climb near the top. Meanwhile, states like Utah, Iowa and Connecticut, which appear to be well above average in raw data, fall to middling or worse when adjusted.
All of that said, no one, including Chingos, is is suggesting that the drop in performance from 2013 to 2015 can be explained by shifting demographics alone.
Uncertainties
Experts did offer some possible explanations as food for thought, however.
One would be that there is a mismatch between the new Common Core curriculum and the NAEP scores. "There are some areas where the assessment and the framework are not exactly aligned," acknowledged Peggy Carr in the Tuesday press conference.
Misalignment would occur when a particular skills is tested and one age, based on earlier curriculum, but is now taught at a later age, so the students have not been taught the material yet. Another misalignment could happen if the material is tested using very different keywords and concepts from those used in teaching it.
Petrilli also points to Common Core misalignment, noting that if this is the case, it would not be an indictment of either NAPE or the curriculum. It may be simply that things are now being taught in a different sequence than they are being tested.
And Carr also noted that in some content areas and age groups, there is already 100 percent alignment between NAEP and Common Core, "and we still saw declines in those areas."
And Minnich noted that even in states not using common core standards saw some slippage. "We need to look at the nation as a whole and make sure that what is being taught across the country is close to what NAEP measures," Minnich suggested.
Another possibility Petrilli raises is that lingering economic uncertainty presents a "stiff headwind," with many communities and vulnerable groups still struggling to find or hold good jobs, with repercussions moving from the home to the school.
