Yesterday, I put out a tweet as part of a bigger project I’m working on.
What would be your answer: I wish I could measure a hockey players ability to ___________
— Alison (@AlisonL) December 15, 2020
I got very good (and mostly serious) answers that are going to be a big help to me, but I also got plenty of cheeky responses (y’all think you’re funny) saying things like “grit” or “want it more.” “Be good in the room” was in there, too.
You know these phrases. They are supposed hockey cliches. “Intangibles.” Things we “can’t measure” that have been bandied about so much we often mock them, a behavior the analytics community is often assumed to perform the loudest.
But, as the answers kept pouring in, that trend of “intangibles” being mentioned increased in more and more thoughtful ways.
Take lessons learned from one on-ice context to another.
— Murat Ates (@WPGMurat) December 15, 2020
Essentially: their ability to solve new problems (and their ability to get better at solving new problems.) https://t.co/d0E67YOo4S
Lead. Specifically, to improve the quality of play of the players around them just by being present. I think if this was quantified it could allow us to see players' soft skills as a part of what makes them a positive or negative contributing player.
— ryan (@RyanDHobart) December 16, 2020
Now, regardless of whether you agree that these concepts (and others like them) are valuable to evaluating hockey players, what most people might all agree on is “sure, but you can’t measure that!”
This has been a crux of the argument critiquing analytics for years, not just in hockey but in many sports: “you can’t measure heart, but I know it matters.” How do you quantify whatever it was that propelled the Blue Jackets to come back from a 3-0 deficit in Game One of the Stanley Cup 2019 playoffs against Tampa Bay and not just win the game but sweep the series? That wasn’t what the data laid out – there had to be something more.
This is why “watch the game” became a common chirp directed at us hockey nerds – it was presumed that if you didn’t watch players play you couldn’t infer that often things happen that don’t make sense or don’t show up on a spreadsheet. You couldn’t enjoy the magic of sport that goes beyond completing a pass, making a save, or scoring a goal.
But here’s the dirty secret. Or at least, here’s my dirty secret.
EVERYTHING can matter. And just because you can’t measure it now, doesn’t mean it doesn’t matter.
Let's rebrand "intangibles"
— Carolyn Wilke (@Classlicity) July 29, 2015
"unmeasured characteristics" is really what they are. pic.twitter.com/3eL2aNou6e
A little bit about me. I graduated with a degree in Leadership Studies from the University of Richmond. And it’s still something I’m very proud of. We learned about the science (emphasis mine) of leading, following, team building, communications, and more. There are methods to the madness. There are ways of studying behavior and quantifying them.
As I cited above, back in 2015, the brilliant Carolyn Wilke was challenging us to change what we called “intangibles.” There’s literally a note on my iPhone from a conversation with her from October of that year (do I take notes when I have conversations with my friends? Yes. I have very smart friends. They teach me things.)
And going one step further than understanding that we can quantify things like leadership, there have been important voices that have challenged us with the how.
Someone who will always hold a place as a thought leader in my book is Stefan Wolejszo. In the midst of the 2014-2015 data revolution in hockey, Wolejszo was writing on his now shuttered blog about bringing in quantifiable ways to measure leadership, to measure motivation, to measure grit.
He spoke on the subject at hockey conferences, including a talk “Rank Ordering the ‘Intangibles,’” (Slides HERE) that outlined how data on things like resilience and grit could be collected and evaluated.
Wolejszo also talked about the importance of folding in “intangibles” to player evaluation and integrating it as part of a true analytical approach, something he outlined at the Rochester Hockey Analytics Conference (Slides HERE).
Slide by Stefan Wolejszo, 2015
So, if we follow this line of thinking, intangibles don’t live in isolation, nor do “analytics,” they meld together for comprehensive evaluation.
The other side of the coin is that some intangibles can tie to measurable data depending on how an organization so chooses to do so. At last year’s Blue Jackets Hockey Analytics conference, Colorado’s Arik Parnass made this point during panel discussions.
Parnass: "Heart and compete is foundational to our game..." but "the players who work harder... are going to spend that time to become better players." so it shows up in the numbers. #CBJHAC
— Hockey Graphs (@HockeyGraphs) February 8, 2020
So popular to what some might think “analytics” and “intangible” evangelists believe, both analytics and intangibles deserve to have a seat at the table when it comes to evaluation; perhaps the challenge with the latter is that we may not always have a quantifiable way to measure them (yet) – and if we don’t, that is when the path ahead is fraught with peril.
And maybe that’s the problem we’ve been facing. Instead of debating if a big hit or a massive intermission speech matters, we should start to ask not “what matters?” but “how can we define how it matters?” There are tools out there…and it may not be easy going, but it’s possible.
Do we run the risk of taking the magic, or even the joy, out of the game if everything is quantified? I say no. I believe the true caution underlying concerns over unmeasured characteristics isn’t that they may not matter, but rather, that absent true quantifiability, we may over-value things. Is the ability to “lead” worth having a not so good player on the ice? To answer that question, we need scales of measurement because sometimes the value of leadership does compensate “enough” for the “not so good.” But sometimes it does not. And it’s those decisions that we still can’t make with authority.
Maybe in 10 years – perhaps sooner! – dismissing certain behaviors as “intangibles” will be a moot point. But until then, we owe it to ourselves to investigate things that we think matter, recognize that all things can be measured, and continue to evolve the way we evaluate the game.
