Wednesday, June 3, 2009
MCAS Scaled Scores
Over at Gotham Schools, Skoolboy, the resident educational researcher and statistician, has been talking a lot about how the most statistically valid way of comparing test data is to look at average scaled scores, instead of proficiency rates. He argues that proficiency rates can be misleading because they don't tell you if kids are near the top or bottom of each category range. In New York, they publish both proficiency rates and average scaled scores for each test in each grade.
In Massachusetts, the DOE publishes the percentage and numbers of students at each score range, which they also compile into a number called the CPI. I've been looking around on the DOE website to see if I can find scaled score data with no luck. If someone knows how to find this data, it would be great if you could post it below.
Update: From Bob Lee, who is MCAS Chief Analyst at the MAESE, responded on the comments page:
"We don't have an interval scale in Massachusetts. In other words the difference between a 216 and a 218 is larger than the difference between a 222 and a 224 so our Technical Advisory Committee (and I agree) doesn't want us to compute average scaled scores. Use the CPI, it has 5 intervals that are equal. Otherwise you have to use medians or the scaled score equivalent of the average scaled score."
Update II: My understanding of CPI is that is separates kids into 5 groups (Advanced and Proficient, High Needs Improvement, Low Needs Improvement, High Warning and Warning). The problem that I see is that it does a good job of accounting of sorting the kids and schools on the low end of the spectrum but does not allow for differentiating schools on the top end. Here's an example: School A has 80% of kids scoring proficient and 20% advanced on the 8th grade math test. School B has 20% of kids scoring proficient and 80% advanced on the same test. Which school and kids are performing better at math? CPI says they are equal; I say they're not. CPI sets an important, but low bar. As a teacher, I want all my kids to be proficient or advanced, but I want the advanced section as high as possible, too.
In Massachusetts, the DOE publishes the percentage and numbers of students at each score range, which they also compile into a number called the CPI. I've been looking around on the DOE website to see if I can find scaled score data with no luck. If someone knows how to find this data, it would be great if you could post it below.
Update: From Bob Lee, who is MCAS Chief Analyst at the MAESE, responded on the comments page:
"We don't have an interval scale in Massachusetts. In other words the difference between a 216 and a 218 is larger than the difference between a 222 and a 224 so our Technical Advisory Committee (and I agree) doesn't want us to compute average scaled scores. Use the CPI, it has 5 intervals that are equal. Otherwise you have to use medians or the scaled score equivalent of the average scaled score."
Update II: My understanding of CPI is that is separates kids into 5 groups (Advanced and Proficient, High Needs Improvement, Low Needs Improvement, High Warning and Warning). The problem that I see is that it does a good job of accounting of sorting the kids and schools on the low end of the spectrum but does not allow for differentiating schools on the top end. Here's an example: School A has 80% of kids scoring proficient and 20% advanced on the 8th grade math test. School B has 20% of kids scoring proficient and 80% advanced on the same test. Which school and kids are performing better at math? CPI says they are equal; I say they're not. CPI sets an important, but low bar. As a teacher, I want all my kids to be proficient or advanced, but I want the advanced section as high as possible, too.
To Lift the Cap or Not?
Last week, MassINC published a report on the impact of the last 15 years of education reform in Massachusetts. You can find the full report here. While the primary focus was not charter schools, the report pushes for lifting the cap for a number of reasons. The full comment is below:
"Closer to home, there has also been a recent and influential study that compares the performance of students at Boston’s charter, pilot, and traditional schools.25 This research finds consistently positive effects for the city’s charter schools on student achievement in all MCAS subjects at both the middle and high school levels. Currently, the state places several limits on the number and location of charter schools. Nationally, the Secretary of Education is targeting federal stimulus money toward states that embrace education reform, including removing the cap on charter schools. If Massachusetts raises or eliminates the cap on charter schools, this change could lead to new charter schools in a dozen communities, such as Boston, Holyoke, and Fall River, that are currently at the limit. In addition, the state should consider allowing effective charter schools serving high-poverty students to operate additional schools and thus allow for greater efficiencies and economies of scale. At the same time, the state should also be more aggressive about closing charter schools that are not working."
A couple of days later, Scott Lehigh followed up with praise in his Globe column.
I also commented at Gotham Schools supporting lifting the cap. My comment is copied below:
"Lift the cap with the following caveats:
1) Charter schools that are underperforming in relationship to their home district should be closed at the end of their charter. Not enough weak charter schools are closed down. Weak charter schools are a financial drain on the system and they hurt kids.
2) Charter schools that are significantly outperforming their home districts should be allowed to replicate. If your school has 70% low income kids and is outperforming the wealthiest districts in the state, you’re doing something right and should give more kids the opportunity.
3) Charter schools with new ideas or models should be allowed; new schools that mooch off older models (especially unsuccessful ones) should not be authorized. This promotes innovation.
4) Districts that don’t want more charter schools can minimize expansion of new charters (though not the replicas), by adopting the practices of charter schools that work. That forces failing schools to try to learn from their neighborhood charter schools that are currently kicking their rears. I still can’t figure out why district schools aren’t visiting my school to learn from what we do here…and why they won’t let us visit their schools that are successful.
5) The cap should be lifted with priority in districts with the greatest need.
Just some random thoughts…"
#4 is my personal addition to the current dialogue that's going on in the charter school community about how to get the cap lifted.
"Closer to home, there has also been a recent and influential study that compares the performance of students at Boston’s charter, pilot, and traditional schools.25 This research finds consistently positive effects for the city’s charter schools on student achievement in all MCAS subjects at both the middle and high school levels. Currently, the state places several limits on the number and location of charter schools. Nationally, the Secretary of Education is targeting federal stimulus money toward states that embrace education reform, including removing the cap on charter schools. If Massachusetts raises or eliminates the cap on charter schools, this change could lead to new charter schools in a dozen communities, such as Boston, Holyoke, and Fall River, that are currently at the limit. In addition, the state should consider allowing effective charter schools serving high-poverty students to operate additional schools and thus allow for greater efficiencies and economies of scale. At the same time, the state should also be more aggressive about closing charter schools that are not working."
A couple of days later, Scott Lehigh followed up with praise in his Globe column.
I also commented at Gotham Schools supporting lifting the cap. My comment is copied below:
"Lift the cap with the following caveats:
1) Charter schools that are underperforming in relationship to their home district should be closed at the end of their charter. Not enough weak charter schools are closed down. Weak charter schools are a financial drain on the system and they hurt kids.
2) Charter schools that are significantly outperforming their home districts should be allowed to replicate. If your school has 70% low income kids and is outperforming the wealthiest districts in the state, you’re doing something right and should give more kids the opportunity.
3) Charter schools with new ideas or models should be allowed; new schools that mooch off older models (especially unsuccessful ones) should not be authorized. This promotes innovation.
4) Districts that don’t want more charter schools can minimize expansion of new charters (though not the replicas), by adopting the practices of charter schools that work. That forces failing schools to try to learn from their neighborhood charter schools that are currently kicking their rears. I still can’t figure out why district schools aren’t visiting my school to learn from what we do here…and why they won’t let us visit their schools that are successful.
5) The cap should be lifted with priority in districts with the greatest need.
Just some random thoughts…"
#4 is my personal addition to the current dialogue that's going on in the charter school community about how to get the cap lifted.
Wednesday, May 27, 2009
To Bus or Not?
Boston Public Schools wants to save money by reducing the amount of busing that happens. To do this, they plan to have more, smaller school zones and remove the city-wide status of two of their most popular schools (Hernandez K-8 and Timility Middle). As part of this plan, they are proposing to stop busing charter school students outside of their BPS zones. This would effectively limit the reach of Boston charter elementary and middle schools by taking away transportation to those students that live outside the zone in which the school exists. There's a piece in today's Globe that discusses the pros and cons of this proposal.
Monday, May 25, 2009
Menino Grands at Boston Collegiate?
This website purports to have information that Mayor Menino's grandchildren received a lottery spots for Boston Collegiate. I have no idea how reliable this information is, especially given the anti-Menino slant of the site, but it's interesting that they chose a non-BPS option for their kids after having them in BPS for elementary school. It'll be interesting to see if they use BCCS as a bridge to the exam schools.
Saturday, May 16, 2009
Codman College Speed Dating
Almost 2 months since Codman Academy's last flashy piece in the Globe, they're back. This time the article is about an event in which colleges and Codman juniors try to learn more about one another using the principles of speed dating. It's an interesting concept.
This is Codman's fourth Globe article of the year. You can check out their PR record here.
This is Codman's fourth Globe article of the year. You can check out their PR record here.
Saturday, May 9, 2009
KIPP Wants In
In an op-ed in Friday's Globe, Scott Lehigh reported on an event at the Boston Foundation at which KIPP CEO Richard Barth was the keynote speaker. Barth discussed KIPP's interest in expanding to Boston and its inability to do so because of the 9% charter cap. Lehigh posited that the potential of Boston KIPP schools should motivate local politicos to lift the cap. I doubt this is how our local pols think.
Lehigh's overall view is that the cap should be lifted and KIPP should be welcome here in Boston. I wholeheartedly agree. But he sets KIPP up as saviors for both charter school expansion and for the kids of Boston. It's a shame that he doesn't include in his argument the successful charter schools here in Boston that are itching to serve more kids and can't because of the cap. You would think that Bostonian politicians would be more inspired by local kids' exceptional success at schools like Excel or Roxbury Prep than the results of kids in other states.
Michael Goldstein "Goes Wild" of MATCH says that the risk of losing federal "Rae to the Top" money is a much better motivator and that changing the cap will allow local success stories to expand. He's probably right. Money trumps all.
Subscribe to:
Posts (Atom)
