I recently read "The Prize: Who's in Charge of America's Schools?" by Dale Russakoff. A book about Mark Zuckerberg's $100 million philanthropic gift to Newark Public Schools and it's impact on district's school reform initiative. It's a rich story with many characters including Cory Booker and Chris Christie. If you're interested in the complexity and nuance of school reform, it's a book I strongly recommend. Howard Fuller, the former Milwaukee superintendent had a quote that succinctly sums up my primary takeaway from the book:
“I think a lot of us education reformers—and I include myself—have been too arrogant,” he said. “It’s not even what you do sometimes, it’s the way you treat people in the process of doing it. If your approach is to get a lot of smart people in the room and figure out what ‘these people’ need and then we implement it, the first issue is who decided that you were smart? And why do you think you can just get in a room and make decisions for a community of people? You don’t think they’ll respond the way they responded? I’m not saying you can ever create this level of change without resistance, but I don’t see how this is politically sustainable over time.”
A persistent topic in school reform is data. How can teachers and administrators be better equipped with data to drive learning outcomes in the classroom? How can data be used to assess which teachers are successful versus those that are not?
Dr. Damian Betebenner, a statistical analyst mentioned in the "The Prize", developed the system "Student Growth Percentile". It is implemented in New Jersey schools. It was designed to measure student gains or losses. Administrators would assume that it was solely the responsibility of the teacher to increase the Student Growth Percentile. And if this data point fell, it must have been because of a bad teacher. Dr. Betebenner commented on this:
“Simply focusing on teachers as being the only potential cause of growth of students is pretty obviously myopic,” said Betebenner. The data would be more useful, he said, as a starting point for discussions with teachers on the reasons individual students are improving or losing ground, which could include many factors in and out of school.
“A lot of high-stakes accountability has become self-defeating—focusing solely on the identification of bad schools, the bad teachers, as opposed to creating a signal and involving teachers in processes that lead to investigations and changes,” he said.
As we develop data points to track, we should consider why those data points were created and what can directly and indirectly influence them. Was the Student Growth Percentile created as a primary signal of teacher performance? It was not, yet it was being used as such. Instead of the metric being used as one type of signal to student performance, it was being used in it's absolute value form to measure a teachers effectiveness.
Sometimes a data point must be absolute (e.g. a boxer's weight before a fight). At other times it's just a signal. Is the signal trending up? Down? What influenced the data point to trend that way? What other questions should we ask? As Dr. Betebenner stated: analyze the signal, establish a process for investigations and changes.
In the tech world, companies are notorious for setting percentage increase goals for certain data points. For example we want to increase revenue by 50% by the end of the year, or increase the number of visitors to our website 4x. In certain instances it can be helpful to set absolute goals (e.g. I will spend no more than $2000 this month on ads), but in other instances, such as the goals I just stated, it could be pointless. Say you increase revenue by 42% by the end of the year, did you fail? The signal is positive (42% growth!) even though the absolute number came short.
So as industries (like public schools) become more data-driven, step back and consider the data points we collect and how we use them to assess people. Consider treating data points not as absolute numbers, but as signals. Explore those signals, understand what's driving the trends, and determine how you can influence them.