In November, I introduced a preliminary version of my work on “Zone Transition Times” (ZTTs) at the Pittsburgh Hockey Analytics Workshop.  The slides for and video of my presentation can be found here.

In December, I submitted a paper on ZTTs to the Sloan Sports Analytics Conference; the paper was not selected as a finalist in the research paper competition.  A slightly modified version of this paper can be found here.  The results and text are unchanged from the December submission, except for minor typos.

Since then, I’ve identified some flaws with this work that I didn’t (have time to) explore in November/December:

• While there are modest in-season correlations between different team ZTTs and future positive outcomes (e.g. Corsi%, Goals-For%, GoalsFor/60), these correlations are lower than those of more established metrics like Corsi, Fenwick, and scoring chances.
• Team ZTTs are only moderately repeatable across seasons.  The correlations between previous-season-ZTTs and next-season-ZTTs are positive but usually less than 0.5.  This varies substantially depending on which ZTTs are being examined.  For example, fast transitions out of the team’s defensive zone (“good puck moving”) are typically more repeatable than are slow transitions out of the offensive zone (“good forecheck”).  The repeatability of ZTTs is lower than that of more established metrics for team evaluation.
• While we can calculate ZTTs for players, there just isn’t enough data to come to any reasonable conclusions about which players have better/worse ZTTs.  That is, the standard errors on player ZTTs are very high, so that differences in players’ ZTTs are almost never statistically significant within a given season.

Thanks to everyone who gave feedback on earlier versions of this work.  Please feel free to share your own feedback with me on Twitter.  I hope to revisit this in the summer, or when player tracking data allows us greater precision to evaluate players and teams with ZTTs.

Finally, I’d like to conclude this post with a comment on null results in quantitative research.  For those unfamiliar with scientific jargon, the term “null result” is typically associated with completed scientific studies that are “unsuccessful” in proving a hypothesized claim (statistically, studies where there is not enough evidence to reject the null hypothesis).

I’m not sure I would call my findings with ZTT a “null result,” but the results I did find were not as grandiose or game-changing as I had originally hoped they would be; they could best be described as “weak.”  I’m sure that others have experienced similar results when trying to further research into hockey analytics and other fields.  For those people, I encourage you to publish your null/weak results!  They are interesting on their own.

For example, if I found that a team’s ability to quickly transition the puck out of their defensive zone had absolutely no effect on that team’s ability to suppress goals or shots in the future, would you think that was interesting?  If I found that a team’s ability to keep the puck in the offensive zone for longer periods of time had no effect on a team’s ability to score in the future, would you think that was interesting?  I would.  And if I was someone interested in exploring this topic in the future, I’d want to know what previous researchers have found.

Publish all of your results, regardless of how “strong” or “weak” they are.  It can only serve to benefit the research community by putting this information out there.