Executive Director of Secondary Education David A. Volrath gave an update on high school reform at a recent Board of Education meeting and it was a whopper in more ways than one. The sixty-page, chart-laden report was handed to the Board just prior to the meeting rather than the Friday before as is standard practice. Unusual, especially given the size, the import and the level of public interest in the reforms also known as CSSRP.
Hopefully the Board will put this report back on a future agenda, because it was hard to dissect all the data on the fly and a closer look has revealed some misleading first impressions. Kind of like watching a magic show; you want to believe the illusion, so it’s a let down when you notice the trick.
One example is the analysis of Advanced Placement (AP) Exams. High school students who choose AP classes can also take an end-of-course AP Exam, a national test administered by the College Board. Looking at the report’s soaring bar charts, you might conclude AP results in Harford County are improving rather dramatically. It’s quite a show, but there’s a sleight of hand.
Look closely and you’ll see that all of the charts tell you the same thing. That is, more AP exams are being administered. That’s especially good if it means more students are participating in AP work. But high-performing students often take more than one AP class, so the increase may simply be double- and triple- counting a small group who are beefing up their resumes in light of stiffer competition for college slots. More than likely the increase is due to some combination of the two, but it’s a distinction that should have been made.
Even more than the increase in the number of exams, what matters is student performance. A score of 3 out of 5 on the AP Exam is the usual cutoff to earn college credit. But the only charts devoted to test scores measure the number of “exams at 3 or higher” within a rising number of exams given, giving the distinct impression of improving achievement.
But the measurement is misdirection. To demonstrate student performance, you’d need to show how many exams were scored 3 or better as a percentage of the total number of exams given. With all the bar charts flying around, you’d think at least one of them could have been devoted to that display.
The information is certainly chart-worthy. Students who take the AP Exam plunk down $86 for the privilege and hope to get college credit or at least impress college admissions officers. A poor AP score has the opposite effect, so students who expect a score below 3 tend to skip the exam and save the cash.
So, is performance on AP exams rising or falling in Harford County Public Schools? The chart below would have answered that question if anyone had bothered to put it together: The blue bars are the years prior to CSSRP; yellow is after:
Turns out the percent of exams with a score of 3 or better is basically flat. It hovered between 63% and 65% for the past 5 years and it has gone down slightly in the two years since CSSRP was implemented.
How could the Board know any of that from the presentation they were given? They’d have better luck getting Houdini to share his secrets.
The full CSSRP Update:
http://hcps.org/boe/docs/hottopics/cssrp/CSSRP_Update_Sept08.pdf
This is so typical of what is going on and it worries me as a taxpayer. You put so must faith and trust into the school system as they have a huge infuence on our kids. Why can’t we get the honest report of what is going on? So the AP scorres stayed about the same. Is that the end of the world?
Must they manipulate everything to show that the CSRRP is having a positive effect on test scores.
Why doesn’t the Board of Ed ask the tough questions or figure out that many of these reports with stats are manipulated?
The eligibility numbers are definitely off, but if you report on something different to the board, the numbers look good and the CSRRP looks great (AKA the School System).
What happens now with initiatives being stopped due to budget shortfalls??? Does that mean we stop looking criitically at the Block Schedule or LICW or School Uniforms, etc. If so, the school system and Board somewhat won because they are able to keep the CSSRP initiatives going without changes, which is what it seems they want.
On the flip side, I do wonder what will get cut? Any ideas or thoughts??
The Board is overwhelmed with “Baffle them with BS” presentations. I went through that 60ish page presentation and many stats seem “inconclusive” at best. On these not so favorable charts it is mentioned that they just need to be studied and tweaked to get improvements on the charts where improvement is shown CSSRP lovers smile and take credit for that. Quite frankly, HCPS is a master of manipulating numbers in their favor. As I look at agenda and exhibits presented at each board meeting, I wonder how the board can give prioritized effort to any of the so called “new initiatives” much less the “BAD” new initiatives”
Carl:
Watch the Board meeting from 10/6 and you will see how Mark Wolkow always gives the Board so much credit for looking at new initiatives. He claims that is what makes the Harford County BOE so much better than other Boards in other counties. I think it’s fine to look at new initiatives but if you implement bad policies than what?? By the time the current seniors graduate, there will be no high school students that have had anything else but the block schedule and the school system is holding out for that. There is more than an adequate amount of data to support the fact that students needs math, science and foreign language EVERY DAY to be the most successful but we continue to ignore it.
Most of the arguments for CSSRP can be challenged. Schools on same schedule – no. Kids needs to be prepared for college schedule – no. The flagship campus of the state of Maryland DOES NOT operate on 85-90 minute classes nor do many colleges in Maryland or elsewhere. Career clusters?? We need LICW to make kids decide on a career and maybe if they put on a uniform that will help them decide what they want to do too… The only way students can learn what they may want to do is by taking classes and seeing what they are good at and what they like, and be educated about what requirements there are and what type of opportunities are available. Not nearly enough career or vocational counseling in any of the schools. That is one of the main issues parents brought up at the last town hall meeting that school system had. Save the money from LICW and have the kids meet periodically with a counselor to discuss goals. My son never saw his guidance counselor even though I asked him to. Pretty sad…
Anyone who dares question the school board is acccused of being self serving and a rabble rouser. Somewhere along the line, they decided to tune out any constructive criticism and embark on new “initiatives” to help improve student achievement and promote social equality. If they would listen more to the staff, parents, and students they would learn a lot about what is going on in the schools. Interestingly, the survey on CSSRP showed huge discrepancies in perception from the administrators and the rest of the school community. Wonder why?? Why is Dave Volrath so afraid of showing the real numbers here. Is is because it may hurt is chances of going to another school system as a “consultant”? The Board should demand he come back and give another presentation because that first one was insulting.
So instead of standardized tests you have a choice of wether or not to take the test depending on how you feel you will do….
Waaa ?????????????????????
Go Dagger !
Yes, Kate, I don’t have a life. I happened to be channel surfing last night, left the board meeting on and heard Mark Wolcott’s comments. It appears the uniform issue has them riled up as well. the new gentleman on the board’s name escapes me as I write this, but it looks like he may ruffle some feathers on that issue. He said ” we already have uniforms and a uniform policy” it just needs to be enforced. At least, it gave me a laugh, whether you are in favor or against the uniform issue, it was nice to see a little challange thrown out. Although, it appeared that Hess and Rich did not like it. And by the way, I’m with you on the CSSRP–MS RABBLE ROUSER!! The Nerve of you—questioning the Board!!
Well said Ren about the AP test scores. Why go to such lenghts to obsure the facts? I’m a parent in this school system and I want the results to be good, but I have to wonder when so much energy goes into dumping data on the Board without giving them useful information.
In the past, the Board let it happen at the same time insisting they are already accountable and shouldn’t have to face the voters. We have some new leadership and some new members now, maybe they will dust off that CSSRP report and insist on follow-through.
About the budget issues, I am hearing that the enrollment numbers in HCPS are way down, making increased funding requests very difficult to justify. That might explain some of the panic. Middle School Reform and Magnet Schools are two big initiatives that will cost money and could be put on hold.
Extremely insightful article. The CSSRP review looks like it is good bedtime reading. Ask the teachers and the guidance counselors..they will readily admit that Career Clusters along with the Block schedule are not moving our kids forward. The AP test results reinforce this-an obvious concern is as HCPS continues to mandate reform like the ones mentioned, and our students remain stagnate…how are they going to measure up against the rest of their peers from other counties, States etc., who by the way are moving ahead.
What is even more disconcerting is why slanted reports such as this are prepared to begin with. It is an insult to the BOE and the public–as taxpayers we deserve better and so do HCPS students.
Fortunately we have committed teachers who care about their students and work hard to effectively teach, granted in the High School this is now in a shorter amount of time than they had 3 years ago, and with less frequency as well. But they get to teach more- 6 periods out of 8 vs 5 out of 7 previously. Hmm I can’t help but wonder if this isn’t why the block schedule was pushed so hard.
I also wonder what is going to come of future initiatives. First the State has to make up its mind about the HSA’s for the Class of 2009!
Thank you for this post.
Even a 3 on the exam is not a good indicator since many colleges will not accept a 3 as acceptable towards credit of the first level course in a subject. This is especially true in the areas of science and math where a 4 is more the norm for credit and occasionally a 5 is required.
Also keep in mind that the most popular of AP classes, calculus, has multiple levels for whihc credit can be recieved – AB and BC. There are some schools that will not accept AP Calculus AB towards Calc I in an engineering or science major.
The proliferation of AP courses has diluted there importance and is starting to show in the college credit approvals given. Can you believe there are AP courses for things like “Art History”?
jj – Please do not knock Art History. It is a very tough, detailed history class involving politics, governance, war culture and all other facts commonly found in history classes. All of this history is related to the paintings, their details, creators and how they were influenced by the events around them. Some college art history courses are known as “freshman flunk out courses” because students sign up for them to fill a requirement and find that they are overwhelmed with the immensity of the course content and class requirements.
As for HCPS, Mr. Volrath has a national reputation to uphold. Remember his presentation to a national educational conference where he shopped himself around as the person who shoved school reforms onto an unwilling school population and got away with it? Does anyone know – do HCPS Board members ever take home the reports they receive and do their own verification analyses?
They should because this Central Office has a proven track record of fudging numbers. Active and questioning School Board oversight is the only way to keep accountability. Please School Board members (I am sure you read this site), please start reading info given to you with questioning eyes and always be wary when you hear the phrase spoken, “I have a power point presentation.”
I would like to pursue the comment you made “A poor AP score has the opposite effect, so students who expect a score below 3 tend to skip the exam and save the cash.”
So my questions would be:
How many children take an AP class now, as compared to 2 years ago? (my guess is that the number has gone up because students can move faster with the block scheduling, especially in math)
Out of that number, how many are actually signing up to take the exam? The statistics used below do not show how many are “opting out” . My daughter graduated last year. That was the first year of really having a lot of juniors in the AP Calc class, as they had been able to move quickly in the math track with the block scheduling. Many struggled and many did not take the test. The system is not adequately preparing the students to take the test. While you will always have a certain percentage who do not take the test, my guess is that that percentage has increased. What is the point of taking the class if you are not taking the exam? I think you could say that those that are opting out of taking the exam are doing so because they don’t think they will do well on the exam. How would that change the numbers?
Some AP courses are taught for 5 days in the fall semester. The test is administered in the spring. Does that affect performance? The AP scores seemd to mirror the SAT scores, flat for the last 5 years.
I’ve talked with several high school students who told me they felt “totally unprepared” for the AP test they took last spring. It sounds to me like HCPS is teaching courses they call AP, but aren’t committed to preparing the students to pass the test.
For some comparison, my daughter is taking an AP course in private school. Her teacher annouced he had a 97% pass rate on the AP test.
Some schools will not accept the AP tests for all subjects. All three of my children graduated from the same college where incoming freshmen are required to submit a portfolio of five writing samples. When they arrived at freshman orientation they were told which English class to include in their freshman course selection. One of my children opted out of the two beginning classes and was given 6 credits for doing so. All of my kids took AP English but taking the AP test would have been a waste of time and money for them.
Good questions DeLane. The closest the CSSRP report comes to answering you is this chart: http://hcps.org/boe/docs/hottopics/cssrp/CSSRP_Update_Sept08.pdf
which shows an increase in test takers as a percent of “participants”.
On the surface, it seems that a greater percentage of students are opting to take the AP test, but if one student is taking 3 AP classes and takes all 3 exams, does that mean he or she was counted 3 times? I think it would be helpful to separate that out. It’s great to have kids choose challenging work, but what is still unclear is how much of the increase is due to some students loading up on AP courses/exams.
Also, the CSSRP report has a series of charts showing the number of students taking each AP test offered, but the information is not very useful because it doesn’t relate that as a percentage of those taking the course. The answer to your question about Calculus could be found if that had been done. The enrollment numbers were not provided, so we can’t do the calculation ourselves.
I know many teachers go out of their way to compensate for the lost instruction time in the block schedule by tutoring before class and after school. Sometimes teachers are the ones who compensate for bad policy.
I agree that students who don’t think they’ll do well will likely opt out, but I know of at least one school where administrators purchased additional AP exams, contacted students and even called parents at home encouraging them to have their students take AP tests last year. The parents I talked to said their students were not prepared and felt pressured to take the test anyway, but I don’t know how wide-spread that was.
In reference to my above comment. “opted out” was the wrong term. My daughter earned 6 credits because of the quality of the portfolio writings. I feel that this approach is a better way to assess student performance than a one day test.
Margaret-
You are speaking a select few high school students. I’ve been teaching an AP course for the last 7 years. I started with 18 students and peaked to over 150 last year alone. An overwhelming majority get 5’s not 3 but 5 on their AP exam and I have a 97% pass rate with all but approximately 3 kids opting out. I encourage ALL kids to take the test. I’d ask your daughter’s teacher how many of the kids in the course actually took the test. The AP teachers and AP programs are audited by the college board. A course will not be granted AP status if the course and teacher to not meet rigorous standards.
Great article to open insightful discussion and hopefully a better approach on reporting on and reviewing the AP test scores. It says a lot about the current HCPS situation showing that they are apparently incomplete and inadvertantly misleading. Hopefully they were not intending to be devious on the face of it, but a track record is growing in the wrong direction.
Giving the Board the large report late is poor management or a strategic way to avoid important dialogue. The questions you ask in your article should have been anticipated and answered by the staff with a chart as you say or the obvious compare and contrast questions should have been asked by the Board if not part of the presentation. It’s an appropriate question. I have to believe that sometime since the second year after AP tests were ever conducted someone had to be interested in the answer of “how are we doing (in various data comparisons and contrasts) compared to past/last years?” So to not anticipate it and then present it is to close one’s eyes to the obvious need to know information for informed decision making.
On the current scores situation – it is what it is. We should not expect it to be going up leaps and bounds as that would call into question the validity of the test and the efficacy of the AP program (it’s too easy, not all the beautiful children in Harford County are certified geniuses). And your article did not do that – you aren’t down on the performance of the students or the program. It should be a hard test or the colleges would look at HCPS with jaundice. 60 some percent is probably pretty good in the state (if you are not cheating and teaching to the test).
The central office curriculum people’s job is to report the results fully for consideration. If we do or don’t like the results that is the next step in the discussion. They should not be afraid to report the results – – unless someone thinks that their current or future job or promotion is attached to these outcomes and wants to obfiscate/deny the outcome. The central office staff don’t teach it – they structure and promote AP courses to a certain group that takes the material. AP is not for all students. They (HCPS) have to get over this idea that if a student doesn’t immediately graduate and go on to college or pick a career path by the end of their freshman year in high schol that somehow the school or the system is failing. Not true and no one expects or blames them.
It again points out that the gears at HCPS grind rather than mesh smoothly. Thanks for your analytic efforts. All the PTA’s in the county should be sent a copy of your article as an FYI about late thick reports and lack of discussion/presentation of analysis (and cc Volrath and Haas).
It is good people are paying attention to these gentle central office folks at HCPS, then they know parents care and are involved. The web site at HCPS says it is important to have parent involvement – that has in recent years come to mean similing and nodding not asking questions. They need to know the rabble are aroused. If they would more openly and expediciously try to prevent, mitigate and fix problems in a non-defensive approach “we could all get along”. But when questions are asked about incomplete work then the walls go up from HCPS – it doesn’t need to happen, but their spirit or leadership needs to change.
I spent some time today looking at the report. Here’s some quick number crunching. If you look at the percentage of students that made a 3 or above on AP tests, HCPS’ best year was 2005-2006 when 65% of students who took the test passed it. This is a pretty good number considering nationally only about 50% pass. For 06-07 it was 64% and 07-08 it was 63.8%. Is this good or bad, who knows, but the trend isn’t good.
There are many charts in the presentation that I would characterize as “misleading.” Especially when you look and the red and green boxes at the end. Mr. Volrath says that GPA was the highest in 07-08, but in fact it’s the same as it was in 05-06. He just didn’t include the data in the summary charts in the back. The summary charts are especially revealing, the lowest data points are mostly before CSSRP, the highest were after CSSRP. But my quick look at the data doesn’t support that conclusion.
I hope the Board will take a closer look at the report.
Thanks for taking a closer look Margaret. According to Education Week, 57% of AP exams in 2007 scored a 3 or better nationwide. So our rate is still higher than the national average, which is good.
However, it appears that less than 60% of our students in AP courses opted to take the exam last year compared to 74% of AP students nationwide, according to the College Board.
Is that because more of our students are not prepared? Is cost a factor in Harford County vs. the rest of the nation? These are the kinds of questions the Board would be able to explore if meaningful data had been included in this report. Like you, I hope the Board will insist upon it.
I find it very disheartening that Mr. Volrath would try to slip this in so the Board didn’t have a chance to evaluate it ahead of time. It seems like he knew there was not substantial data to back his statement that test scores are up, and so by giving the Board this at the last minute, he expected them to think things were going better. Thanks for deciphering it for us and taking the time to see that the real data we need is not even given. I sure hope the Board takes the time to evaluate this report and see it for what it really us – misleading. It once again shows you can make data mean whatever you want if you present it your way.
The slipping in of data at the last minute is not new and is very much a pattern, particularly if you are trying to find info on the HCPS website. The board and hcps staff have often scolded us for coming in at the last minute. The problem is that we get much —to all of the info to decipher at the last minute. Not to bring up a sore subject, but during the redistricting, the figures, many which were innacurate and were brought to the staffs attention were still not corrected and were left on the website as if they were gospel. When I mentioned a gross innacuracy concerning one community to one of the staff at a public meeting, the response was, oh I guess we must have put in the wrong query. Guess what though, it never was changed. Although this is not an elected school board or staff, I feel like we are many times dealing with politicians who can speak out of both sides of their mouths in order to make their point. Really, have you ever seen either a politician or hcps staff or board member who have ever said that they may have made a mistake? How many times have you seen the board members take the staff or particularly Dr. Haas to task–ever?