Education Week, which calls itself American education’s newspaper of record, publishes an annual report called “Quality Counts” that ranks the states according to education policies and outcomes. This year, Maryland came out on top overall and the self-congratulation has been in full bloom, some of which is well-deserved. But a closer look at the data shows student achievement doesn’t match the hoopla.
To measure student achievement, the researchers at Education Week used the percent of public school students who scored “proficient” on the latest National Assessment of Educational Progress (NAEP) in 4th grade reading and 8th grade mathematics.
Ranking the states by Ed Week’s own measure of student achievement, Maryland was far from number one. Maryland didn’t even make it into the top 10. In fact, Maryland ranked 17th in reading and 14th in math. To be fair, some of the underlying NAEP scores are so close as to make the differences between some states negligible, but the best that can be said is that Maryland’s public school students scored higher, to statistically significant degree, than their counterparts in only about half of the 50 states.
So, how can Maryland be first in education, when student performance is just above average? The folks at Education Week say student achievement is among 35 factors that go into their overall rankings.
It’s helpful to look at those 35 factors by separating them into two broad categories: inputs and outcomes. The inputs are the variables that may affect outcomes. Outcomes are results, such as measurements of student achievement.
Aside from the results in math and reading on the NAEP, the only other factor used for the Education Week rankings that could be considered an outcome of the public school system is the percent of public high school students who graduate with a diploma. Maryland tied for 23rd place.
All of the other factors measure inputs, and that’s how Maryland made the grade. Some factors, like family income, parent education and parent employment are not under the direct control of states or school systems. Other factors measure state financing or how well school systems smooth the transitions from pre-k to college – important factors which policy makers do control. Still others grade the states on indicators of debatable merit, such as whether college prep is a prerequisite for a high school diploma (should it be??) and whether the state has written a “formal definition for school readiness” in addition to having established standards.
If the factors Ed Week used to rank the states create better outcomes for students and Maryland is the model, why aren’t the outcomes better in Maryland?
Maybe it’s because Ed Week didn’t even consider some pretty meaningful inputs like the quality of the curriculum, where Maryland earned a grade of C+ from the most recent Fordham Institute’s State of the Standards Report.
The disparity between Maryland’s headline-grabbing rank and actual student achievement may have a variety of causes. But if I lived in Massachusetts, which had the top scores in reading and math by a good margin, I’d demand a recount.
It’s fine for Education Week put out a report like “Quality Counts”, but student achievement should carry a lot more weight in the rankings or else be ranked in a separate report. That way, policy makers, parents and other taxpayers can distinguish between what goes into the nation’s public schools and what comes out. And we can be sure that the investments made in education create outcomes worth crowing about.
Great catch. Maryland’s public school system is still a tough case. And certainly one that doesn’t need false aggrandizations. These students teachers and parents need to know exactly where they stand.
Maryland’s # 1 ranking is relative to where the US school system ranks with other school systems in the world?
If the US schools are ranked # 1in the world, and Maryland schools are ranked the best of the best then we have something to celebrate. But if the US schools are ranked anywhere other than # 1 in the world, then we really don’t have much to celebrate. Unless you consider that we could be ranked lower than # 1 in the US.
If the above is true, and Harford County Schools are ranked in the top tier in a school system that is ranked the best in the US, whose worldwide ranking is below # 1, are we providing our children with a “World-Class” education?
My concern quite frankly is where doe the schools that serve the Route 40 communities rank in all of this?
Jansen, the U.S. is not #1 in international rankings. http://www.sciencedaily.com/releases/2008/12/081210171906.htm
even when measuring our top students against students from around the world.
At the state level, standards have changed over time which makes long term analysis impossible. But currently, MSA and HSA scores are the internal standard by which MD schools are assessed. Comparisons and rankings of schools (for example, by the percent of students who passed these tests) would have to be calculated by hand. Adding to the complexity is the fact that administrators at each school can elect not to test certain students right away if they are unlikely to be successful and students who have failed a test once can re-take it up to several times per year. All this data is mixed together, making year to year comparisons less meaningful. Here’s a link to the datatbase http://mdreportcard.org/
I can say that the performance in the schools you mention are below the average for HCPS, and the lack of adequate yearly progress has prompted oversight by the Maryland State Department of Education in some cases. Also, in many of the Rt. 40 high schools, over half of the students got an E in at least one course, which makes them ineligible for extra-curricular activities after school.
As for external measures of achievement, I have tried to get SAT scores broken out by high school, but I have been told to file a Freedom of Information Act request to get this data. The info used to be on the schools’ web sites, it is available in other counties, but no longer in Harford.
The questions you ask are good ones and the answers should be readily available to parents and other taxpayers. But as you can see, the standards keep changing, data is not always available in useful formats and comparisons are difficult because uniform measurements are often non-existent.
I was really surprized by the #1 ranking of Maryland. But many of the things Ed. Week looks at has nothing to do with how much our kids are learning. Cindy, you’re right, the state makes it very difficult to compare MSA scores year by year. Last year they deleted the nationally normed part of the test, which makes it impossible to compare how we’re doing with the rest of the country. They also shortened the test. No surprize, everyone’s scores went up.
In the latest TIMMS test, an internation test of science and math, since 2004 the US made improvements in math, but science stayed the same from 1995. Minnesota and Massachusetts made really big improvements in Math. I’d love to know what math curriculum they’re using! Massachusetts science scores were #3 behind Singapore and Hong Kong. Boy they really should be #1.
I find articles such as this with links to the cold hard data invaluable. I hate to sound cynical but when I first heard this announcement from Nancy Grasmick I couldn’t help but question its source and the criteria used.
As a parent of three children who have attended or are still attending HCPS I have many positives to share about their education-particularly teachers who have influenced my kids. Without these talented professionals our school system would be a bust. But then I wonder why these same professionals are not listened to when their insight is offered ( the CSSRP for one example).
I know there are deficiencies in their education opportunities and development. Could they have been worse?? Unquestionably. But is there room for improvement? Yes, absolutely. So when results like this are shared and left for interpretation –as to how we, the State of MD, is faring overall when compared to other states’s educational measurements, then I have to be cynical. All most people heard was that MD was ranked #1 in education. They didn’t get the specifics.
Measuring student achievement is critical to evaluation of our students’ abilities and preparation. It is telling that as taxpayers we can’t even get our HCPS high schools’ SAT report.
Thanks for this perceptive and constructive article. This should be in the Aegis!
Cindy,
Some (not all) of the High Schools post thier SAT scores under the “school profile.” I did notice that the info on those profiles is much slimmer than when we moved here 5 years ago. They used to list what AP classes they offered and the % pass rate on AP tests. Trying to compare schools (like someone moving here from out of state would do) takes a lot of work, boring down through about 6 pages on the HCPS website. These profiles seemed to have lots of useless data, like attendance, and with the real data (like SAT’s) pushed to the bottom. How about PSAT scores? AP test pass rates. Much better indicators of performance than GPA.
Statistically, high family income correlates closely with high student test scores. So a school filled with — on average — students from rich families will have high test scores almost regardless of how good (or poor) the teaching, management and support staff are at that school.
So the honest way to evaluate teaching effectiveness is to find out how well students are doing compared to other schools with students of similar affluence. We may be pleasantly astonished to learn, say, that Edgewood High actually has better, more effective teachers than Fallston High.
Without considering family situations it’s impossible to either credit or blame teachers and principals for students’ test results. At present, we’re not measuring the school. All we’re probably measuring is family income.
Margaret, thanks for the heads up. The school profiles on hcps.org have been updated. As of January 2009, the profiles include 2008 SAT scores.
Jansen asked about performance along the Rt. 40 corridor. From the link below, click on a high school and the profile will pop up. SAT scores are on the chart at the bottom. Data for 2008 and 2007 are included and you can compare each school’s results to the national, state and Harford County averages.
http://hcps.org/Schools/SchoolProfiles.aspx
On the AP test issue. Some schools (private) require that a student who takes an AP course take the test (they also have to be recommended to get into the class and don’t think that doesn’t increase the pass rate). Not being required to take the test does several things: students who know a teacher gives an easy A can boost his/her grade point; an A on the transcript looks good without the corresponding AP test grade, which could be a 2 or 3 (I know this happens at BAHS); students who should perhaps not be in the AP level classes take them and may dilute the quality of the class–yeah, yeah, elitest, I know. The test verifies the quality of the instruction to a certain extent.
Any AP figures should include how many students take the classes, how many take the tests, and what percentage achieve a 3 or better on the tests. These figures are included in a school profile that private schools send with the transcripts when students apply to colleges. When I inquired about a school profile to accompany a Bel Air High School transcript, I was told that that information was located on the Web site. I asked the person I was talking to whether she really thought that an admissions counselor would go look at the BAHS Web site and she said that if the college was interested in a student it would. Gee, I am certain that the Princeton rep, sitting among her 17,000 applications will take the time to find a Web site. And people wonder why kids get rejected. There are very simple services HCPS kids are not receiving.
And Maryland tests have always been a big gripe of mine. I don’t care how my kids do compared to Maryland kids. I want to know how they rank nationwide–on a nationally normed test. As far as I am concerned a Maryland test, given in Maryland only to Marylanders is a waste of money.
The good news that Maryland ranks about #3 in the nation (behind Mass. and NJ, I think) for National Merit Scholar scores (Maryland kids have to score higher than other states to qualify). The bad news is that out of the approximately 300 students who will be National Merit Scholars from Maryland this year, only about six–maybe fewer–are in HCPS with none at Bel Air High School. If you look at top public schools in the country (Thomas Jefferson in Fairfax, VA, for example), they have a handful of NMS each year.
And Jansen is right, money is the magic ingredient. Higher incomes, higher scores. That is why PSAT scores have to be higher; more kids have the opportunity and cash to take prep classes and buy study guides.
Ooops, sorry that was Mort with the money talk.
Jansen:
Some disturbing statistics for the Edgewood and Joppatowne schools on the school profiles out today. Not only are the SAT scores down, but it looks like there are many students not passing portions of the HSA. It appears as though English 11 and Biology are the two most problematic at all of the schools. The other disturbing number was the GPA. Overall averages are really low and frankly quite sad.
Aberdeen seemed to fair a bit better but part of that may be the Science and Math Academy boosting scores a bit.
You are correct in your assertion that what really matters is how your school and your child(ren) are performing. Even if your child is in a school that has good test scores that doesn’t mean they are doing well. What is being done differently to help the students in these schools? My fear is that they are in high school and have trouble reading which effects every subject and there is a high level of frustration that manifests itself in undesirable ways and kids give up.
There should definitely be more opportunities for vocational training and maybe that would help some students have a purpose and feel capable. We are not serving many of the kids who really aren’t college bound and we are not serving our county by not doing more to help prepare students for other vocations. This was something that should have been addressed more comprehensively when HCPS undertook CSSRP but instead they made them do “career clusters.” Not the same thing at all. Now we have built Patterson Mill, rebuilt Bel Air, renovated North Harford and Aberdeen, adding capacity to Edgewood and not one vocational program only “Magnet Programs” at schools like Bel Air that don’t need the extra students.
Whatever happened to the Aberdeen North Building for a vocational school? How about working with the manufacturing companies throughout Belcamp like Sephora, TIC Gums, Mercedes, Rite-Aid, etc. Has the school system talked about that lately? I share your frustration …
so let me get this straight, school in rich districts do better, so they get more funding? shouldn’t the schools who are failing because of a lack of funds be the ones who get the help?
As with mst statistics, they are open to interpretation and manipulation. Although there may be a correlation between school scores and income levels of the community, is this because money gets higher scores or is it because the same things that bring the higher income levels also bring higher school scores. Not being elitist but being pragmatic, wouldn’t you think that the parents of kids in higher income areas would have factors related to intelligence and ambition that they would pass to ther kids? Wouldn’t you think that the home environment of the higher income areas would be more conducive to better educated kids? These would only be trends for the larger group to which there would be many exceptions: poor kid who is a genius, rich kid that is a slacker, etc..
Add to this the scholastic atmosphere that this breeds. Why do you think housing prices jump or drop as you cross school boundary lines? Is it because the house one block down is that much better or is it because parents feel that the school environment and atmosphere will get their kid a better educational opportunity?
On top of all of this, how much of a student’s education is done in the home either directly or by example? Do they study at home? Are the parents involved with their schooling? Is education seen as the most important thing they do?
Kids who see their parents reading are more likely to read themselves. Kids that were read to regularly at an early age are more likely to be better readers at an earlier age. And the ability to read is the biggest contributor to educational success (not to be confused with general intelligence which we will hold for another discussion). After watching a child with dyslexia struggle through the first few grades of school until he got the extra help to overcome the problem. And seeing the extreme success of that child in high school and knowing friends that have gone on to master’s degrees after overcoming the same problem in middle school, the importance of reading ability is paramount.
Don’t let superficial statistical analysis blind you into throwing money at a problem. Keep asking why is there a relationship.
You are all raising so many important points!
I totally agree that we should support a true vocational school. We don’t have one now – Harford Technical HS changed its main purpose from vocational school to college prep. It’s extremely competitive to get into that school. This doesn’t help the hands-on, non-college-bound students who are important too. If you agree, it’s time to start telling this to the School Board. The Board will need repeated reminders as they seem to have a hard time hearing public voices.
Regarding figuring out what the problem is before trying to fix it, that’s always a good approach. “jj” points out many reasons WHY rich kids usually do better than poor kids, and we as a society need to figure out how to address the resulting problems faced by the poor students. Are they malnourished and need an expanded food program? Are parents working multiple jobs that stop them from helping with homework and good habits, so kids need after-school tutoring? Do they need lessons in respect and teamwork, which is also something that can be taught in school and in after-school programs?
Pulling yourself up by your bootstraps only works if you HAVE bootstraps. Kids don’t, and they deserve help to become productive citizens. If parents can’t or won’t do it, then what do WE do? Throw their future away, along with our own (think about the cost of crime, unemployment, jails, etc)? Or are we smart enough to help the kids so they become taxpayers instead of tax users? Helping those who need help not only saves our own future, it’s also the morally correct thing to do.
This story of Cindy’s, “Quality Counts? It Depends On Who’s Counting,” was picked up and ran in yesterday’s edition of the Baltimore Examiner.
http://www.baltimoreexaminer.com/opinion/0126emsopedmumby.html