Why Goaltending is Basically Random and Will Always Make You Look Stupid (In 5 Graphs)

Expecting consistency from year to year will probably leave you disappointed.

There might be nothing more satisfying on social media than catching somebody in a take that really didn’t age well and ridiculing them for their stupidity and lack of foresight. I gave Jackets fans the opportunity to do this a week ago based on a tweet I sent on August 2nd, the morning of game 1 of the qualification round.

Korpisalo, of course, would go on to register a 0.939 save percentage and stop 5.3 goals above expected in nine games, eliminating the Maple Leafs and breaking the NHL record for most saves in a single game with 85 in the process. Oops.

Here’s the thing though: it was a fair take at the time. Since 2017 Korpisalo had been very poor in exactly those stats that he would go on to excel in. While I had jokingly allowed for the unpredictability of goaltending in a subsequent tweet (in which I said that he could win the Conn Smythe), I was operating under the assumption that it’s a lot easier to be consistently bad than consistently good. In other words, I tried to think logically about goaltending. That was my mistake.

That’s because any way you look at it, goaltending is not predictable from season to season. Whether you use basic save percentage-based stats like goals saved above average or sharper metrics that factor in shot quality, it’s hard to conclude that goaltending isn’t basically random.

Take this graph for example. If you’re not super stat-inclined your eyes might be glazing over, but bear with me. This shows the relationship between a starting goalie’s save percentage above expected (based on quality of shots faced) in one season and the next season. It asks whether you can rely on a goalie’s performance to repeat the next season. The answer is basically no; the correlation (r) between the two is very low at 0.12. And this model (built by Cole Anderson) has the highest repeatability of any publicly available above-expected model.

If you’re clever, you might be saying “Well that’s just one guy’s model. Goaltending isn’t just random because a certain model isn’t predictive.” Which is fair. (If you’re wondering if the fancy proprietary above-expectations models might be more predictive, they probably are, but based on what people like Garret Hohl have said about them it is probably not by that much.) So what if we cut out the expected goal stuff and just stick with regular old save percentage (expressed here as goals saved above average per 60 minutes, which as a rate will function as Sv% above average)? Let’s do the same test, but with that stat:

Well, crap. Even save percentage, which should theoretically be much more repeatable because of the team defence factor, isn’t reliable at all either. Coincidentally, the correlation here is also 0.12.

So maybe predicting next year’s performance is an impossible task. But at least within a season things will be reliable, right? I mean, Korpisalo might have been pretty nuts, but overall a goalie will perform similarly in the playoffs compared to how they did that season, right?!

Should’ve known better. Correlation is a measly 0.15. Barely any better than year over year. This is the universe in which Joonas Korpisalo can look like Dominik Hasek, Jordan Binnington can look like Jim Carey, and Connor Hellebuyck follows up an MVP season by looking mortal.

Okay, that’s enough scatter plots. I promise. Let’s look at some actual goalies:

These are the eighteen goalies who have been starters in all of the past three seasons. Two things should stick out:

1. Goaltenders are insanely inconsistent from year to year. At most, four goalies have been anything approaching “reliable” in that time period, and they’ve been reliably above-average - not elite. The rest of the list is a mess. I mean, if Matt Murray is considered to be “above average” in terms of year-to-year consistency, that’s a pretty good sign that goaltender performance is a crapshoot.

2. The best goalies of three seasons ago are, to say the least, not looking good. Keep this firmly in mind when talking about goalies as “elite” and assuming that they’ll stay that way. Among the goalies on this list who ranked in the top ten three seasons ago, only Semyon Varlamov and Connor Hellebuyck were even even above average this season (and keep in mind that both of them struggled in 2018-19.)

So what have we learned from this? Expecting your team’s starting goalie to have anything resembling consistency is an absolute fools errand. Want to know how Braden Holtby will play next year? An assessment based on hours spent combing through stats and game tape is about 12% more likely to be reliable than just guessing. This is why paying big money (especially over the long term) for a goalie is a fool’s errand. As the Panthers learned this season, it is risky to the point of insanity to bet that a starter will be able to maintain even an above-average level of play throughout a contract, let alone a high level.

What if your favourite starter was the best in the league this year? Obviously that makes him an Elite Goalie™. Except that’s not really how it’s worked in the past.

We’re talking the cream of the crop here, and it’s still basically random. Two goalies remained “elite,” five out of nine finished below-average, and one was even bottom-three. The Jets should probably get that defence sorted out this summer or they might be in for some trouble next season.

Conclusion

It’s tempting to treat goalies like skaters, ranking them and debating who qualifies as “elite.” But unlike top skaters, who can generally be reliably expected to perform and produce at a certain level year over year, goalies almost never maintain a high calibre of play for even two or three years at a time, let alone a decade. Imagine if in the next five seasons, Connor McDavid went from scoring 110 points to 30, and then to 65, and then to 120, and then to 10. No wonder takes like these age so badly:

This doesn’t mean analyzing goalies is a waste of time, or that some goalies aren’t more technically talented than others. It certainly doesn’t mean that teams should fire their goalie coaches and just play a procession of ECHLers until one of them goes on a hot streak. Sharpening advanced goalie stats like save percentage above expected allows us to better understand how these players have performed and apply some method to the madness. Analysts like Cole Anderson (@ice_cole_data on Twitter) and Micah McCurdy have made incredible progress in furthering our understanding of how goalie results come to be. For example, check out Cole’s analysis of Martin Jones and Micah’s amazing new shot type/location visualizations.

Goaltending is probably the most consequential position in the sport, and it’s a wild card that can doom otherwise beautifully built teams. Jeff Marek says if you have the goalie it’s 70 percent of your team, if you don’t it’s 100 percent. The problem is that if you think you have the goalie, there’s a very good chance that you actually don’t.