Advanced Teaching Metrics

I have been teaching for 5 years and every year I feel like my principles and district people get more and more into data. We all give the same assessments and EOCAs so they can look at the data and make decision.  The worst part is most of these people have taken the single statistic class that is required to get their masters and don’t understand much about data.  (funny side note: At a meeting one time, our principle was showing predictive data.  The data said our students were predicted to do 4% better on our high-stakes test this year.  My principal asked the following question to the staff, “Do you guys know why we want to get this percent higher?  Because the margin of error is ALWAYS +/- 5% and we want to be out of the margin of error.”)  Regardless, because our bosses value the data from these assessments, these affect the way teacher teach.  This is no new secret, bad high-stakes tests and bad summative assessments can lead to bad teaching.  What is interesting, is that this is not limited to teaching.  The NBA for years used statistics that did a bad job at assessing the effectiveness of a player.  Michael Lewis wrote an article in 2009 about the NBA player Shane Battier titled, The No-Stat All-Star.  The gist of the article is that Battier doesn’t accumulate a lot of the standard statistics (points, rebounds, assists, ect.), but because of his unselfish play, he is as effective as some of the more well known NBA all-stars such as Carmelo Anthony or Vince Carter (This is 2009 Vince Carter).  Here is a quote from the article, where Daryl Morey (The GM of the Houston Rockets, the team Battier played for in 2009) discusses how they do a better job at measuring the effectiveness of players:

“…the big challenge on any basketball court is to measure the right things. The five players on any basketball team are far more than the sum of their parts; the Rockets devote a lot of energy to untangling subtle interactions among the team’s elements. To get at this they need something that basketball hasn’t historically supplied: meaningful statistics. For most of its history basketball has measured not so much what is important as what is easy to measure — points, rebounds, assists, steals, blocked shots — and these measurements have warped perceptions of the game. (“Someone created the box score,” Morey says, “and he should be shot.”) How many points a player scores, for example, is no true indication of how much he has helped his team. Another example: if you want to know a player’s value as a ­rebounder, you need to know not whether he got a rebound but the likelihood of the team getting the rebound when a missed shot enters that player’s zone.”

Morey goes on to show other instances of how standard statistics create selfish players.

“Taking a bad shot when you don’t need to is only the most obvious example. A point guard might selfishly give up an open shot for an assist. You can see it happen every night, when he’s racing down court for an open layup, and instead of taking it, he passes it back to a trailing teammate. The teammate usually finishes with some sensational dunk, but the likelihood of scoring nevertheless declined. “The marginal assist is worth more money to the point guard than the marginal point,” Morey says. Blocked shots — they look great, but unless you secure the ball afterward, you haven’t helped your team all that much. Players love the spectacle of a ball being swatted into the fifth row, and it becomes a matter of personal indifference that the other team still gets the ball back.”

I feel this same problem occurs in education.  When we value simple statistics, such as test scores, we create selfish teachers.  We say things such as, “Don’t worry about that, its not on the test”, or “The best way to get the answer is to just plug in the 4 multiple choice answers.”  Maybe the more detrimental part of selfish teaching is it limits the types of tasks we do in our class.  For example, a teacher might say,  “I am not going to do that Mathalicious lesson because it doesn’t exactly go over the problems that are on our district created tests.”  or, “I am not going to do Estimation 180 because my students don’t need to estimate on the AIMS test. I am going to work on adding/subtractice integers instead.”   When we value simple statistics like test scores, we reward teachers for selfish teaching.  The NBA has become better at finding effective, less-selfish players by developing advanced metrics to measure the effectiveness of players like Shane Battier.   Here are some examples of new NBA advanced statistics:

Opponent Field Goal Percentage at the Rim (Opp FGP at Rim): This stat measures an opponent field goal percentage at the rim when you are defending them.  If there is a low percentage, that means you defend well at the rim.  If there is a high percentage, it means you do not defend well at the rim.   The lowest percentages for Opp FGP at Rim is Bismack Biyombo at 38%.  Biyombo rates 21st in blocks which is usually the statistic we use to measure great rim protectors.  On the other hand, DeAndre Jordan who is second in blocks, rates 36th in Opp FGP at Rim.  To the casual fan, we think of Jordan as a better rim protector because of his high number of blocks, but really, it is  Biyomo.

Another method NBA teams have taken to better evaluate players is by installing SportVu cameras in all of the NBA arenas.  These cameras track all of the player’s movements.  Teams can use this for multiple purposes such as seeing how teams defend pick and rolls or how much distance a player travels in a game.  SportVu cameras also allow teams to see how fast a player is moving at all times.  This information can tell them how often a player is sprinting, jogging, walking, ect. which can factor into a player’s effort.
So, let’s say we were able to record every single math class and chart the data, what are the advanced metrics that we would want to collect?  Here are some of my ideas and I would love to hear some of yours.
1. TQ (Teacher Questions): Number of questions a teacher asks per 60 minute class period.
2. SQ (Student Questions): Number of questions students ask per 60 minute class period.
3. CQPTNQ (Challegeing Questions per Total Number of Questions) Number of challenging questions a teacher asks in comparison to the total number of questions they ask.
4. Helping: Number of times a teacher gives students the answer per 60 minute class period.
5. Estimations: Percentage of times a teacher requires a student to make an estimation prior to starting a problem.
6. Less Helpfuls: Percentage of times a teacher answers a students question with another questions.
7. LP (Lecturing Percentage): Percentage of time a teacher is talking.
8. SSDP (Student to Student Discussion Percentage): Percentage of time on topic student to student discussion occurs.
9.  TeacherVu: Track the movement of a teacher.  For example, how much time is a teacher spending with struggling students. Is a teacher standing in front of the room or at their desk.
10. DOK average: Percentage of problems used in class that are DOK levels 1, 2, or 3.
11. Leinwands:   Number of times per 60 minute class period a teacher asks questions of the type, “Why?”, “Can you explain?”, “How do you know?”
12. Feedback Length: Length of feedback on a assessments (By letter count).
We are probably never going to get to the point of collecting this type of data, but if we did, would it change change anything?  Does an NBA player knowing he is being measured by Opp FGP at Rim instead of blocks change the way he plays defense?  Maybe. Shane Battier was making all of the same unselfish plays before he got to the Rockets.  Just like Fawn Nguyen is doing all of the right things in her classroom without advanced metrics guiding her instruction.  But the truth is, not all NBA players or Teachers, are playing unselfish.  The advanced metrics have had an affect on the play of other NBA players.  For example, the Houston Rockets believe the best shots to take on a NBA court are 3 points and layups.  These are the shots they tell their players that they value.  They believe that long 2-point jump shots are not valuable.   This year the Rockets took the fewest shot from 15-19 ft.
This is important because it shows when you let your employees know that you take certain statistics seriously, you can change their behavior.  If a school decided to make other statistics more important than test scores, teachers would change their behaviors and try to meet the new statistics.
The counter argument to advanced metrics in the NBA is that it doesn’t give you a complete picture of a player’s value.  You cannot measure the competitive spirit that Jokim Noah plays with.  You cannot measure the mentoring a veteran player does with a young player.   With teaching, there are so many things that matter that can’t be measured by statistics.  For instance, the personal connections that teachers make with students. Or, the guidance a veteran teacher provides to a new teacher.   While there are definitely limits to statistics, advanced metrics paint a better picture of what is going on in a teachers classroom.   And with the right advanced metrics, you can positively change what happens in a teachers classroom.
I feel like for the most part math teachers are happier with the Math CCSS than with our previous state created standards.   However, we are all waiting to see the PARCC and Smarter Balance assessments to see if they will actually assess the CCSS.  If they do, the belief is that teachers will have to change the way we teach.  But if this were true, then NBA teams like the Rockets, could have just used wins and loses to motivate their players to play in an unselfish way.  Some teachers need good advanced metrics, that measure specific things, in order to change the teaching that occurs in their classroom. If we don’t have these advanced metrics in teaching, then I believe no statistics are better than simple ones.