Sunday, November 3, 2013

Where to start?

The battle over the A to F grades for schools has been going on since the law was first passed a couple of years ago.  Politicians think it is a great thing and an improvement over the former system, which graded schools on a scale between 0 and 1,500.  Educators know that these systems use exactly the same data, just manipulated in different ways - neither of which gives a clear idea about the worth of a school. 

The big difference is that the API score system didn't pretend to be anything other than what it was.  It took some data from the schools (mostly test scores), put them into an incomprehensible formula and came out with a number.  High number = good, low number = bad.  Pretty clear.  However, politicians were not happy with that because - well, who knows why.  They felt that parents could not possibly understand the "high number = good" concept I suppose.  Maybe they felt that the "incomprehensible formula" part was a problem.  Although, if that is the case, they made a drastic error in their choice of solution.  At any rate, they came up with a supposedly simpler formula that "everyone could understand" and announced that we could now know the true worth of a school.

There are only a few minor problems.  Let's just look at reliability.  Not once, but twice now, scientific research and evaluation has proven that not only are these calculations misleading at best, but that they in no way reflect the value of a school.  See the latest report here.  I can't find a link to the first one right now, but I'm sure someone who reads this will have it and can post it in the comments.  However, here is one of the many important parts to their study of the system:


 So, all of the parts to the grading system have flaws, and when you combine them they are even more useless.

I could go into detail on a variety of issues with the actual formulas, starting with the fact that only a very small portion of our students are actually counted in any way for the calculations.  Or that our lowest students count 2 or 3 times while our highest students count once.  Or that the "growth" is not comparing one group of kids from one year to the next, but different groups of kids who happened to take the same test.***  However, that's a post for another time.  Suffice it to say, the formula itself is very, very limited.

***Edited:  This was my first understanding of the system.  That they looked at the bottom 25% of the tests taken this year and matched them to last year.  However, after looking at some additional information, I think they look at the bottom 25% from the previous year and try to match tests from this year.  So, if they have moved to a different school, they are dropped.  If they take a different test, they are dropped.  For example, if they took the OMAAP last year and the OCCT this year, they won't count.  So instead of counting a different group of kids, it seems they are just not counting the entire bottom 25%.  I think.  For such a simple formula, it sure seems to leave a lot of questions about who fits where.  I'll be interested to see what happens this year, now that there is no OMAAP.  All of them will be taking the OCCT.

I could also make a very long list of things that affect the success of a student outside of a classroom, but again, that's a post for another time.

I could even point out (again), that the tests themselves are not reliable for much, let alone determining the worth of a school.  However, I would be repeating what a lot of people have done much better than I.  There are entire books on the subject.

What I really want to talk about it this:  Governor Fallin Counters Critics of A - F Grading System for Schools.  I admit.  I voted for her.  Throughout all of the hoopla of the last couple of years, I had planned on voting for her again.  I know....many of my friends would think I'm crazy, but I do believe in some of the things she does.  I am a Republican after all.  I don't know who will run against her, but I have to say that it would take some serious flaws for me not to vote for him or her now.  I don't believe that our leaders should be bullies.  Threatening not to fund schools because teachers and administrators exercise their right to speak freely?  No.  That I can not accept.

Let's look at this article a bit closer, shall we?


I suppose it's not helpful to their cause, but it is certainly helpful to ours and to the kids'.  What a silly thing to say.  As for undermining the credibility of the system, we don't actually have to do that.  It does that all on its own.  We just speak out about events as they happen and publish widely exactly what is used to calculate these grades.  The SDE has done more to undermine its credibility through its administration than any teacher or administrator ever could.  Nine changes to the grades in a week?  Really?  Throw in the e-mail one administrator received telling him his bottom 25% kids were randomly selected.  Then add in the necessity of having to re-score the writing tests because they were incorrectly graded the first time.  Ooops...and don't forget the complete disaster of the actual testing sessions due to wrong tests being sent to schools, computer systems (at the testing manufacturer, not the schools) being unable to handle the actual testing sessions, etc.  I guess Governor Fallin thinks these things are credible and it is only our telling the public about them that mars that credibility.

The last little bit about the law being the law is not only ridiculous to spout, but hypocritical coming from our Governor's office.  She has already determined that at least two laws are not appropriate for Oklahoma to follow - Obamacare and the payment of benefits to gay National Guard soldiers.  Whether you agree with her stance or not is irrelevant.  The fact is, she cannot say we must follow the law because it is the law and she does not have to.  In addition, there is not a single district (that I know of) that is not following the law.  In fact, we are bending over backwards to do it.  So much so that teachers are leaving the profession rather than put up with all of the problems and stress.  They are tired of working so hard to implement something detrimental to the students.  We are only being vocal about how wrong the law is.  We have that right.  It's in the Constitution.


Am I supposed to feel bad that she is "dismayed?"  I am "dismayed" by what they are doing to our students.  My "dismay" wins.  I have to wonder exactly what they want this report used for.  My guess is that they would like to see it thrown in the trash.  You know what really stands out to me about this report?  They've had over a year since the first one to come up with their own analysis that contradicts it.  They haven't, and I don't believe for a minute that it is because they haven't tried.




Deliberately misunderstanding what educators are saying seems to be a habit of this administration.  Schools can (and do) have tremendous effect on student performance.  It also is not the authors' belief we are responsible for 20 to 30% of student achievement - it has been researched and there is ample evidence to back them up.  What it means is not that schools are worthless.  It means that grading schools solely by looking at test scores is worthless.  They need to look at the entire effect a school has on a child, not just academic.  For some of these kids, school is the only place where they get fed.  For some, it is the only place they are spoken to.  For some, it is the only place they are safe.  We address all of the issues that come with children, not just their academics.  Politicians need to recognize the value of that, and if they are going to create a grading system of any sort, those things also need to be included.

Therein lies the problem.  She really believes this.  Despite all of the research and evidence to the contrary, she really does.  It is this that makes me truly understand that she does not belong in her position.  If this is how she looks at an issue - completely disregarding the facts and multiple sources of evidence to back it up - then I want her out of there.  It doesn't matter what the issue is.  Her ability (or willingness) to think and to analyze an issue is not sufficient to allow her to do the job.

If what she had said was "I realize it is not the best way, but for now it is the best we can do," I would disagree.  However, I would at least know she was somewhat grounded in reality.  To say she really believes it is adequate?  She's lost my confidence that she can do her job.


I've tried to keep most of my natural snarkiness out of this post.  However, what else can you say to this but "well, duh!"?  Increased resources, properly administered can improve just about anything.  The fact is, I don't believe she has any intention of actually increasing funds to education.  If she had any such intention, she would have done it.  Instead, she cut taxes.  Schools lost millions of dollars.  We will never have extra funds under her leadership, so it is an empty promise.  It's a weak one at that.  She'll try.  Yoda said it best.  "Do, or do not.  There is no try."  She has the power to do so, she has no intention of actually doing it.


These quotes are quite telling.  A) If the formula as it stands is not complex, then 9 different grade releases should not have been necessary to get it "right."  So, either the SDE is incompetent or the formula is more complicated than they are letting on.  Either way, it doesn't work.  B)  The report card has 3 parts, not 2.  Plus a bonus section.

Now, I can understand why they want you to think there are only two, equally weighted, portions.  It makes it seem much more reasonable.  But the reality is, the bottom 25% of kids are used three different times.  Once in overall performance.  Once in overall growth.  Once in growth of the bottom quartile.  That's three.  Not two. 

Since parents are becoming quite vocal about the ridiculousness of this system (look at the Tulsa Area PLAC), I'd say they understand this system quite well.

I'd like to see his research showing they are successful.  Everything I've read says they are not.  Here is an NPR article on some of the problems with Florida's system (which we copied).  This one is an opinion piece from Utah, so take it for what you think it is worth.  I think it's worth pointing out that the system doesn't seem to be overly successful there either, and it backs up the opinion that Florida's system is a mess.  Indiana also seems to be struggling.  All of the states considering it or implementing it (including ours) seem to do so based on Florida's flawed system.  Even their leaders admit that it is invalid.  I did come across one article (I believe from Michigan) that puts forth an almost compelling reason to change to an A to F scale.  Their current system uses rainbow colors.  I'd be hard pressed to tell you the difference between green and lime, let alone what that meant for my school.

In the end, I suppose it depends on your version of successful.  If you believe that getting rid of veteran teachers and replacing them with TFA recruits, closing schools in the poorest neighborhoods, opening up unprecedented amounts of online and charter schools (who operate under different rules and are no more successful than public schools), and driving teachers to leave the profession in droves is successful, then I suppose he is right.


I don't know if anyone could make the Governor sound more ignorant about education if he tried than this guy has in this interview.  It would take some work.  First of all, "we're concerned about the outcomes, not the inputs."  I spend my days teaching my students that you get out what you put in.  Not all people will be successful at all things.  I get that.  However, most of the time, effort brings success.  Lack of effort generally brings lack of success.  Garbage in, garbage out. 

I'm trying to make sense of his argument here.  First, he admits that class size and teacher salaries are important.  Then he says those things affect student performance.  Then he says that really smart kids could succeed in large classes.  He follows this with the fact that they only care about the results, not about what goes in to getting the results.  Okay.   This says to me that the Governor's office admits that educators and administrators are correct in saying other things influence student performance, but they really don't care because the only thing they want to see are good test results. 

On second thought, given the actions of our "leaders," I guess he is stating their position quite clearly.


1 comment:

  1. That's right. They are concerned about the outcomes, not the inputs. The inputs are children - not numbers on a piece of paper. It's blatantly obvious that the focus is in the wrong direction.

    ReplyDelete

These comments are moderated and will not show up until they are approved.