When kids struggle with poverty and neglect at home, can they get a boost at a boarding school? According to a new study of a prominent effort in the nation's capital, the answer is maybe not.
The SEED school of Washington, D.C., is a public charter with a twist. Beginning in 1998, the charter school has been teaching, housing and feeding middle schoolers and high schoolers five days a week, sending them home on weekends. Because it is a public school, it operates tuition free, funded by the District of Columbia and foundation support.
The theory of the boarding model is that intensive mentoring and a secure environment during the week would break cycles of poverty and unhealthy family and neighborhood pressures at home.
But a new study by MRDC, one of America's most prestigious education research institutions, casts doubt on whether SEED works. The study suggests SEED may not raise test scores, increase high school graduation rates or enhance other life outcomes, such as a reduction in risky behavior. This disappointing finding comes after a 2014 study found significant positive test results, particularly in reading.
"If I were the SEED people, I'd be disappointed in the result," said Martin West at the Harvard Graduate School of Education. "But I wouldn't conclude that we had failed and go home and drink whisky."
This is American education reform in a nutshell: innovate, research, explain, dispute results and repeat. A quarter century into America's education re-invention adventure, how much do we really know about how to close the achievement gap?
The boarding model
There are now three SEED schools: one in Washington, with 300 students, another in Baltimore, which started in 2008 and now has 400 students, and third in Miami started in 2014, which enrolled 60 students the first year and is adding 60 more each year on its way to 400.
SEED is nearly the ultimate in "wrap around services," a term for the current trend in education circles to provide more services traditionally provided at home for children from underprivileged backgrounds. These include after school programs and breakfast, in addition to lunches. In some cases, schools literally merge with community centers, becoming, by design, the focus and locus of students' entire lives.
Because SEED is a charter school, students pay nothing. But it is a boarding school, and it costs roughly twice as much as traditional schools, or nearly $40,000 per pupil per year. The school's founders had to lobby for special rules from both Congress and the District of Columbia to get the public funding.
Almost all SEED students are both low-income and racial minorities, but the rules in Washington, D.C., require that the school open its lottery to all students, according to Christina Brown, director of communications for the SEED Foundation. In Maryland and Florida, however, the schools have been allowed to prioritize more vulnerable students based on family income status and other risk factors.
Achievement gaps
The SEED model is designed to address the "achievement gap," or the persistent phenomenon that low-income students, English language learners and black and Hispanic students consistently trail behind white and Asian students in standardized tests, high school graduation and college attendance.
In a 2014 study of the SEED model, Vilsa Curto, a Stanford Ph.D. student, and Roland Fryer, a Harvard economics professor, note that the persistence of the "racial achievement gap" has raised questions about whether schools can overcome the overwhelming effects of students "being reared in dysfunctional families and failing communities."
Curto and Fryer point to one of the iconic documents in education reform, the 1966 Coleman report, a study commissioned by Congress as part of the 1964 Civil Rights Act, and one of the first serious studies of the "achievement gap."
The Coleman Report diagnosed the problem but offered few solutions, concluding that schools cannot overcome a childs background and general social context and that inequalities imposed on children by their home, neighborhood and peer environment are carried along to become the inequalities with which they confront adult life."
Even if boarding schools were effective, they're very expensive. At $40,000 per pupil, cost becomes a major factor in replicating the program in other cities, something MRDC noted. Justifying that high cost, MRDC argued, would require fairly strong measurable results.
"Although SEED DC showed a couple of positive behavioral effects," the report concluded, "it did not show an impact on the key non-academic outcomes, such as teen pregnancy or interaction with the criminal justice system, that could justify its higher cost."
MRDC reports that students were slightly more likely to have engaged in "risky behavior" such as "skipping school, arguing with parents or hitting someone" in the past three months, when compared with non-SEED group students.
Why the spike in reported behavior problems? Rebecca Unterman suggests there may be what researchers call a "reference bias" problem. Simply put, that means that students at SEED may hold themselves to a higher standard, reporting more severely on their own behavior than their peers in public schools.
Jay Greene, a professor of education reform at the University of Arkansas, though, says he found the higher rates of risky behaviors reported in the study troubling. The program is heavily structured, he notes, which may have unintended side effects. "They may have figured out how to move scores in the near term, but heavily controlling students' lives may infantalize them," Greene said. "That's my fear."
Boarding schools, Curto and Fryer note, have potential to break a cycle of chaos and deprivation at home or in the neighborhood. But, they also note, the model has hidden risks. Students might have a hard time adjusting to being away from parents or siblings, and stress from isolation may actually cancel out some of the benefits of the 24/7 environment.
An ephemeral boost
One of the bright spots in the SEED study was a short-term boost in test scores.
Students who began in seventh grade were tested two years later. Students were found to have gained 33 percent of a standard deviation in reading, and 23 percent in math.
Those results are "pretty big in the education world," Greene said. By one popular measure, Greene said, a .25 percent standard deviation translates into 180 extra days of class time. "That," he said, "is what a student would learn in an entire school year."
"If we judged the quality of schools entirely based on short-term changes in test scores, as many reformers would like to do, wed say this school was doing a great job," Greene concludes.
In fact, this is precisely what Curto and Fryer found in their own study of SEED, which also looked at the 2007-08 results and stopped after two years. In their paper, they report impressive findings in both reading and math scores. But because their study did not continue for the whole six years, they did not pick what came next.
But once the students got to 10th grade, the reading score boost disappeared, and the effect in math fell below statistical significance. In other words, the gains were short-lived.
The lesson Jay Greene draws from this, though, is not that SEED is failing. Greene argues that test scores may not be a proper measure of a school's effects, and this would hold true whether the scores looked good or looked bad. Either way, he sees it as shamanism, a kind of hocus pocus.
"We have built an entire system around a model that has not been validated," he says. "Instead, we should trust parents to make choices. They have better instincts than all these instruments for bureaucratic control."
Worth pursuing
There is a clear tone of reluctance in the MRDC about drawing strong conclusions on its SEED research. No one wants to torpedo a bold and promising program that has a plan to tackle the achievement gap.
SEED's Christina Brown seizes on Jay Greene's skepticism about short-term outcomes. "We actually don't think that this was a long-term study. The standard we hold ourselves to is college graduation," Brown said.
She points out that 94 percent of their graduates enroll in college within 18 months of leaving high school, and that three times as many SEED graduates complete college as their peers.
Brown and Unterman also noted multiple areas where SEED improved during the MRDC study, including home visits to help secure more parental engagement.
SEED has also begun an intensive reading and writing program. In the first year of the program in the Miami SEED school, students jumped an average of two grade levels in reading, Brown said.
For his part, West thinks the SEED experiment is worth continuing, emphasizing that we don't really know how to look for some of the key hard-to-test factors.
"I would be reluctant to consider SEED a failure," said Harvard's Martin West. "They may not be helping more students graduate, but it may be that those who do graduate are better prepared. When a school dramatically changes the high school experience, just looking at test scores or graduation rates may not tell you everything that is happening."
The SEED school of Washington, D.C., is a public charter with a twist. Beginning in 1998, the charter school has been teaching, housing and feeding middle schoolers and high schoolers five days a week, sending them home on weekends. Because it is a public school, it operates tuition free, funded by the District of Columbia and foundation support.
The theory of the boarding model is that intensive mentoring and a secure environment during the week would break cycles of poverty and unhealthy family and neighborhood pressures at home.
But a new study by MRDC, one of America's most prestigious education research institutions, casts doubt on whether SEED works. The study suggests SEED may not raise test scores, increase high school graduation rates or enhance other life outcomes, such as a reduction in risky behavior. This disappointing finding comes after a 2014 study found significant positive test results, particularly in reading.
"If I were the SEED people, I'd be disappointed in the result," said Martin West at the Harvard Graduate School of Education. "But I wouldn't conclude that we had failed and go home and drink whisky."
This is American education reform in a nutshell: innovate, research, explain, dispute results and repeat. A quarter century into America's education re-invention adventure, how much do we really know about how to close the achievement gap?
The boarding model
There are now three SEED schools: one in Washington, with 300 students, another in Baltimore, which started in 2008 and now has 400 students, and third in Miami started in 2014, which enrolled 60 students the first year and is adding 60 more each year on its way to 400.
SEED is nearly the ultimate in "wrap around services," a term for the current trend in education circles to provide more services traditionally provided at home for children from underprivileged backgrounds. These include after school programs and breakfast, in addition to lunches. In some cases, schools literally merge with community centers, becoming, by design, the focus and locus of students' entire lives.
Because SEED is a charter school, students pay nothing. But it is a boarding school, and it costs roughly twice as much as traditional schools, or nearly $40,000 per pupil per year. The school's founders had to lobby for special rules from both Congress and the District of Columbia to get the public funding.
Almost all SEED students are both low-income and racial minorities, but the rules in Washington, D.C., require that the school open its lottery to all students, according to Christina Brown, director of communications for the SEED Foundation. In Maryland and Florida, however, the schools have been allowed to prioritize more vulnerable students based on family income status and other risk factors.
Achievement gaps
The SEED model is designed to address the "achievement gap," or the persistent phenomenon that low-income students, English language learners and black and Hispanic students consistently trail behind white and Asian students in standardized tests, high school graduation and college attendance.
In a 2014 study of the SEED model, Vilsa Curto, a Stanford Ph.D. student, and Roland Fryer, a Harvard economics professor, note that the persistence of the "racial achievement gap" has raised questions about whether schools can overcome the overwhelming effects of students "being reared in dysfunctional families and failing communities."
Curto and Fryer point to one of the iconic documents in education reform, the 1966 Coleman report, a study commissioned by Congress as part of the 1964 Civil Rights Act, and one of the first serious studies of the "achievement gap."
The Coleman Report diagnosed the problem but offered few solutions, concluding that schools cannot overcome a childs background and general social context and that inequalities imposed on children by their home, neighborhood and peer environment are carried along to become the inequalities with which they confront adult life."
Even if boarding schools were effective, they're very expensive. At $40,000 per pupil, cost becomes a major factor in replicating the program in other cities, something MRDC noted. Justifying that high cost, MRDC argued, would require fairly strong measurable results.
"Although SEED DC showed a couple of positive behavioral effects," the report concluded, "it did not show an impact on the key non-academic outcomes, such as teen pregnancy or interaction with the criminal justice system, that could justify its higher cost."
MRDC reports that students were slightly more likely to have engaged in "risky behavior" such as "skipping school, arguing with parents or hitting someone" in the past three months, when compared with non-SEED group students.
Why the spike in reported behavior problems? Rebecca Unterman suggests there may be what researchers call a "reference bias" problem. Simply put, that means that students at SEED may hold themselves to a higher standard, reporting more severely on their own behavior than their peers in public schools.
Jay Greene, a professor of education reform at the University of Arkansas, though, says he found the higher rates of risky behaviors reported in the study troubling. The program is heavily structured, he notes, which may have unintended side effects. "They may have figured out how to move scores in the near term, but heavily controlling students' lives may infantalize them," Greene said. "That's my fear."
Boarding schools, Curto and Fryer note, have potential to break a cycle of chaos and deprivation at home or in the neighborhood. But, they also note, the model has hidden risks. Students might have a hard time adjusting to being away from parents or siblings, and stress from isolation may actually cancel out some of the benefits of the 24/7 environment.
An ephemeral boost
One of the bright spots in the SEED study was a short-term boost in test scores.
Students who began in seventh grade were tested two years later. Students were found to have gained 33 percent of a standard deviation in reading, and 23 percent in math.
Those results are "pretty big in the education world," Greene said. By one popular measure, Greene said, a .25 percent standard deviation translates into 180 extra days of class time. "That," he said, "is what a student would learn in an entire school year."
"If we judged the quality of schools entirely based on short-term changes in test scores, as many reformers would like to do, wed say this school was doing a great job," Greene concludes.
In fact, this is precisely what Curto and Fryer found in their own study of SEED, which also looked at the 2007-08 results and stopped after two years. In their paper, they report impressive findings in both reading and math scores. But because their study did not continue for the whole six years, they did not pick what came next.
But once the students got to 10th grade, the reading score boost disappeared, and the effect in math fell below statistical significance. In other words, the gains were short-lived.
The lesson Jay Greene draws from this, though, is not that SEED is failing. Greene argues that test scores may not be a proper measure of a school's effects, and this would hold true whether the scores looked good or looked bad. Either way, he sees it as shamanism, a kind of hocus pocus.
"We have built an entire system around a model that has not been validated," he says. "Instead, we should trust parents to make choices. They have better instincts than all these instruments for bureaucratic control."
Worth pursuing
There is a clear tone of reluctance in the MRDC about drawing strong conclusions on its SEED research. No one wants to torpedo a bold and promising program that has a plan to tackle the achievement gap.
SEED's Christina Brown seizes on Jay Greene's skepticism about short-term outcomes. "We actually don't think that this was a long-term study. The standard we hold ourselves to is college graduation," Brown said.
She points out that 94 percent of their graduates enroll in college within 18 months of leaving high school, and that three times as many SEED graduates complete college as their peers.
Brown and Unterman also noted multiple areas where SEED improved during the MRDC study, including home visits to help secure more parental engagement.
SEED has also begun an intensive reading and writing program. In the first year of the program in the Miami SEED school, students jumped an average of two grade levels in reading, Brown said.
For his part, West thinks the SEED experiment is worth continuing, emphasizing that we don't really know how to look for some of the key hard-to-test factors.
"I would be reluctant to consider SEED a failure," said Harvard's Martin West. "They may not be helping more students graduate, but it may be that those who do graduate are better prepared. When a school dramatically changes the high school experience, just looking at test scores or graduation rates may not tell you everything that is happening."
