December 27th, 2015, Jets vs. Patriots at MetLife stadium. It’s one of the most important regular season games in the last decade that no one ever talks about. If someone brings up to you, it’s likely to notate the last time the New York Jets defeated the New England Patriots in a game of professional football. It literally cost the Patriots home field advantage in the 2015-16 NFL season, which more likely than not cost them yet another Super Bowl trip (they would go on to lose the AFC Championship Game in a nail biter at Mile High Stadium, one of the toughest fields in the league to play at due to its namesake altitude, against Peyton Manning’s corpse), which then, if you want to start getting really crazy and applying transitive properties to the whole situation, means a single overtime defeat to the soon to be hapless Jets prevented their dynasty from winning an unfathomable 4 Super Bowls in 5 years.
And the game was lost in overtime on the back of one of the most unusual and universally reviled late game coaching decisions in recent memory made by the man who most regularly appears in the google search results for “GOAT NFL head coach.” Bill Belichick chose to kick the ball off in the NFL’s semi-sudden death Overtime, a decision that has occurred in approximately 1 percent of all NFL overtimes.
Here’s the thing. Analytics weren’t particularly perturbed about Belichick’s decision. They weren’t in favor or it to be clear, but depending on which model you were following, it only “cost” the patriots somewhere between 4 and 8 percentage points of win probability. This is not the Steelers punting against the Browns on 4th and one down two possessions in the fourth, of the bevy of other horrifically indefensible conservative coaching decisions made in this year’s playoffs alone.
And this is where context is vitally important. The goal of analytics in football, or at least what the goal should be, is to support a philosophy of decision making that is more evidence driven and less driven by fear and emotion. It is meant to, at its best, provide indisputable evidence that certain decision making philosophies that were considered canon in the NFL coaching community should be abandoned. It is also often devoid of certain contexts, the x’s and o’s, the flow of a given game, the weather (stick a pin in this one), and player matchups and offensive philosophies that may yield higher or lower successes in 4th downs and short yardage situations than the usually generic* league-wide based win probability models do.
*(Aside: The reason most NFL win probability models tend to use generic 4th down probabilties rather than team specific ones is because of sample size. You narrow the scope far too much when you’re cherry picking an individual offense or individual defense’s effectiveness in 4th down or short yardage situations. The sample size becomes too small to be useful. For example, if next year the Josh Allen Bills are 3 and 9 in 4th and short in the regular season, that doesn’t mean the Bills shouldn’t be aggressive in those situations, because they have a quarterback who is elite in power running situations, which is a skeleton key of efficiency for this sort of thing)
And it is that sort of thing that I think made the Belichick overtime kickoff decision much more than just simply a suboptimal decision. Losing a small percentage of overall win expectancy isn’t in and of itself a defense for analytically unsound decision making. For example, the decision to kick the extra point instead of going for 2 after scoring a touchdown down 15 points in the 4th quarter yields a minimal loss of win probability. The reason for that is because being down 15 means your odds of winning are already incredibly low, so there’s only so much lower your odds of winning can go comparatively. The reason that common decision is indefeasible is because it is inherently a scenario which is completely context neutral. There is no schematic, weather, personnel or rule difference between going for 2 with 8 minutes in the game versus going for 2 with no time left. Therefor, there is no reason not to make the analytically optimal decision outside of psychological cowardice.
Belichick’s mathematically sub optimal decision is the opposite of this. He had two very specific reasons for reasonably being able to justify it. The first is that there were winds up to 18 miles per hour during the game, meaning that the usually irrelevant advantage of picking which end zone you defend by kicking off was significantly amplified. The second is that the 2015-16 Jets were a much better defensive team than offensive team. They still had most of the infrastructure and great defensive players from the Rex Ryan era that preceded the last good Jets team. Their offense was spearheaded by NFL career journeyman Ryan Fitzpatrick.
To really understand why even analytics guys were so deeply opposed, we have to go back to ground zero of the analytics conversation in mainstream media discussions, which is yet again, a late game Bill Belichick coaching decision.
Ground Zero
November 15th, 2009, Colts Patriots at Lucas Oil Stadium. NFL Football is about to change forever. Everyone who was around watching remembers this one. There’s 2:08 left on the clock on 4th and 2. The Patriots have the ball and they’re up six. They’re on their own 28 yard line. Converting this makes victory statistically a near certainty. Failing gives Peyton Manning one of the easiest 2 minute drives he’ll ever attempt to complete his epic comeback.
That’s exactly what happens. The Colts win 35-34 after stopping Kevin Faulk by mere inches.
It wasn’t quite the same level of universal hate lobbed at his punting decision against the Jets, nor did he receive the staggering about of vitriol Barry Switzer did in 1995 when he made a similar decision. Still, while over a decade of evolution in how we think about 4th down decisions has made Belichick’s decision seem obviously correct in hindsight, it was certainly unpopular at the time, to say the least. Rodney Harrison and Tedy Bruschi, his own players so recently removed from their original dynasty run with him, ripped him on national television.
But there were defenders. New York Times writer Brian Burke wrote a four paragraph defense of the move for his football analytics website he ran on the side. His argument was brutally simple. Essentially, the average team had about a 52% chance of scoring a touchdown in a 2 minute drill from the field position that would be yielded on a failed 4th down conversion, versus a 30% chance to score a touchdown with an average punt. The Patriots’ chances of converting a 4th and 2 are much higher than 22%, the difference between the two field position scenarios yielded, and therefor the decision to go for it was not only correct, but obviously so. Tony Dungy’s declaration on national television that “the stats clearly backed up punting there,” is frankly, more and more hysterical with each passing day.
But the true beauty of Burke’s piece is the last paragraph, which includes the sentence:
“You’d have to expect the Colts had a better than a 30% chance of scoring from their 34, and an accordingly higher chance to score from the Pats’ 28. But any adjustment in their likelihood of scoring from either field position increases the advantage of going for it.“
He manages not just to explain why it’s statistically a better option with approachable, middle school math that anyone should be able to understand, but to also fit all that context I mentioned within the confines of his four paragraphs. Not only is it clearly the statistically superior choice, the context of having to punt it to Peyton Manning and his elite Colts offense that was already surging a startling comeback makes it the contextually obvious correct choice as well. The simplicity and efficiently of Burke’s piece really is, no exaggeration, one of the best pieces of writing I’ve ever read.
Context Matters
The reason for all of this hoopla about the history of the media’s analytics conversation is so you understand the context that caused the analytics-leaning football analysts of today to loathe Lafleur’s decision so much. The goal of analytics is to strip bias and emotion out of football decision making, but becoming someone who believes in analytics involves tying themselves to the idea that, in the most broad application, means one should take a purely offense oriented approach to winning NFL football games.
Whether it is Belichick’s famous 4th and 2 decision, or the new crusade of stat heads that teams need to pass the ball on first down, the history of football analytics writers is one rooted in fighting the coaching convention of playing for things like field position, time of possession, and really, anything that isn’t oriented directly with amplifying offensive efficiency.
And this is not me making some broader point about how analytics has gone too far or that we need to abandon it. In fact, when I was watching the game, I literally turned to my father “I can’t believe someone is actually dumb enough not to go for it here.” Most analytics guys will tell you that the goal of what they do is to remove the obvious crimes against middle school math that NFL coaches regularly make even today. Most analytics guys won’t tell you that coaching decisions should be made strictly on a chart without even consideration for context.
This is not so much a critique of people who are better at math than me, but a critique and reexamination of my own biases and emotional reactions. It would be really cool if somehow this piece reached the people who are better at math than me and encouraged them to examine their own biases in a similar manner, but well, this is me dusting off my little old blog after nearly a year’s hiatus. Realistic expectations and all that.
The Decision
First, the math of it. The packers in their 4th and goal go-for it scenario had a win expectancy of about 10.8 percent. This went down about 3 percentage points by kicking the field goal. I have seen it analytically argued, including in the linked piece, that similar to my scoring a touchdown down 15 scenario all the way back at the top of this post, that we should consider it a much more analytically egregious decision because the odds of winning are already so low. This piece in particular argues because approximately 28 percent of the “available” winning odds were lost by this decision.
This is a ridiculous argument driven by the same logic of the Biden administration’s argument that 1,400 dollars equals 2,000 dollars because we already gave you 600 dollars argument. Winning or losing a football game is an absolute, binary state. Three percentage points of win probability mean the same thing regardless of where it falls on the spectrum of overall likelihood to win. It’s using galaxy brain logic so they can print a bigger number and point at it.
So given the analytical effect is minimal, let’s examine the context.
With how little time is left in this particular game, we can actually examine every single variable with both decisions and then come to the conclusion after describing victory and defeat with each scenario. In this thought exercise, door 1 is the field goal, the decision Lafleur made in real life, and door 2 is going for it, the scenario that dumb football guy and really smart math guy both wanted Lafleur to make. Consider this a “choose your own adventure” novel where you have to pay me zero dollars.
Door 1:
Here are all the possibilities.
- Mason Crosby misses the field goal. You have now essentially entered door 2, but with slightly better field goal position and minus any of the possibilities where you actually score the touchdown on 4th down. Possibility of occuring: negligible
All other scenarios from this point forward assume Crosby makes the field goal considering how close it was.
- Bucs get the ball back and get two first downs. Just like real life, the time is run out and the Bucs are going to the super bowl. The Packers’ biggest weakness all year was their run defense was their biggest weakness. Therefore , I’ll rank the possibly of this scenario occurring: decent
- Bucs get the ball back and get one first down. Packers get the ball back with 40-50 seconds left and one or no timeouts somewhere between their own 15 and their own 35. One of two things occurs.
- Rodgers scores the touchdown and wins. Granted, he’s actually done this sort of miracle before in the playoffs, but the possibility of accomplishing this is negligible.
- Their 50 second miracle drive fails. The possibility of this occurring is decent
- Bucs get the ball back and go 3 and out. Packers get the ball back with some version of between 1 to 2 minutes and either 1 or 0 timeouts depending on how many pass attempts are involved and how they choose to manage the clock. The field position is probably somewhere between their own 20 and 40. Once gain, one of two things occurs.
- Rodgers scores the touchdown and wins
- He scores the touchdown and loses.
- Stats would suggest he has about a 1/3 chance of scoring a touchdown, but out of respect for Rodgers, I will give him a 50/50 shot. Let’s call both scenarios having a roughly equal possibility of fairly low
But the second door, the one where Lafleur goes for it, that’s where things get pretty wild. Let’s explore that possibility.
Door 2:
It’s actually really hard to find a chance of success calculation for this scenario. The analytics article I criticized earlier set it at about a 1 in 3 chance. That seems quite high to me considering the advantages of knowing they have to pass combined with the advantage of having to defend a very small area, but once again, out of respect for Rodgers I will allow this.
- Rodgers makes it, now Green Bay has to attempt the two point conversion (since they didn’t’ get the field goal, obviously).
- The Packers convert the two point conversion. Now they have to stop Tampa Bay with 2 minutes and 3 timeouts from scoring a field goal. Unfortunately, there are not a lot of great measurements of the likelihood of success here. Burke (in a separate piece) estimates a 2 minute drill’s chance of success or failure at about 40% from touchback position, but this estimate doesn’t filter for available timeouts. Thus, I’ll have to simply extrapolate what Tom Brady, the greatest quarterback of all time, can do with the easiest possible 2 minute drill. I’ll say, based on minimal statistical evidence, he as something in the neighborhood of a 2/3 chance to get 3 points after accounting for the possibility of the kicker gacking a clutch kick. Therefor I would overall rank the odds as follows
- Successful 2pt conversion+Bucs field goal: fairly likely for 2PC success scenarios
- Bucs turnover leads to a Packers score to end the game in regulation: negligible
- Game goes to overtime: somewhat unlikely.
- Once you get to overtime, it’s basically a coin flip. Therefor I would say the packers likely hood of converting this scenario is slightly above negligible
- Failed two point conversions break off into scenarios very similar to the door 1 scenarios above, with the advantage of only having to kick the field goal.
- Bucs get two first downs and end the game: decent for failed 2pc scenarios
- Bucs get one first down and give the ball back to rodgers, but they only need a field goal this time. Incredibly tough, but Rodgers’ skillset is uniquely suited to deal with the challenge of getting in field goal range with 40 seconds and no timeouts, and this one is astronomically easier, so i’ll say slightly above negligible
- Finally, you get the three and out scenario with the failed 2PC. This gives the packers a really good chance to win the game with the various 2 minute drills to score merely a field goal. Not quite as good as the TD+2PC scenario is for brady, but I would rank this one overall as decent for failed 2pc scenarios
That’s what makes this particular decision really fun to analyze and discuss. Fun enough, I guess, to awake my blog from it’s nearly year long slumber. It’s basically like the Dragon Age: Origins second act Landsmeet*. One seemingly simple decision and binary result has all these different little branches involved. It’s also why the immediately, emotional reaction is “why not trust your hall of fame quarterback,” is the easiest response. You really have to break it down like this to come to this conclusion.
*The Venn diagram of people reading a nearly 4,000 word piece on a single coaching decision and the people who get my reference to a 15 year old RPG video game is quite small I’m sure. Sometimes I just can’t help myself.
Why Kicking the Field Goal was Right
Kicking the field goal, as outlined above, generates the best possible win scenario, as scoring a touchdown on a 2 minute drill with no timeouts, or a 1 minute drill with 1 timeout, or some variation of, and then winning the game outright (I softly guestimated his odds of pulling this off as slightly worse than a coin flip). By contrast, really, the only realistic go for it scenario where the packers win requires the touchdown (about the same odds as the prior 2 minute drill scenario just on its own), then convert the two point conversion (basically a coin flip), and then win the overtime (another coin flip) OR complete the first task but easier on top of completing the 4th down.
Why did I not mention anything about having to trust the defense. Because you have to trust the defense no matter what. If Tom Brady gets the ball back with 2 minutes and 3 timeouts, stopping him from scoring a measly 3 points is really, really difficult. In fact, kicking the field goal actually makes Green Bay’s defensive task easier. Let me explain.
Bruce Arians, in spite of his reputation with his air raid inspired offense, is not a very aggressive coach when it comes to decision making. He ranks slightly below average as far as 4th down aggressiveness. He runs the ball a lot on first down, an analytics faux pas.
Why I bring that up is, when your back is against the wall, really, a lot of it comes down to hoping your opponent messes up in some way. For example, to cite yet another New England Patriots moment, the famous 28-3 comeback against the Falcons required two pretty spectacular mistkes in spite of New England’s superlative play. 1) They needed the Hightower sack fumble deep in Atlanta territory, and 2) They needed the sack+the holding call after they got into field goal range before the final touchdown drive of regulation.
Kicking the field goal is, in part, hoping that Bruce Arians will default to his natural tendencies when his season is on the line and run it up the middle three times. To his credit, the very first play the Buccaneers called was a pass.
One last thing to consider. Matt Lafleur cited the two minute warning as another reason he chose to kick instead of go for it on 4th down. There was 2:09 left on the clock for the actual 4th down play. The field goal took 4 seconds. The 4th down play more likely than not would have taken more than 4 seconds, but there wouldn’t be a kickoff if he fails. Given Aaron Rodgers’ propensity to scramble and extend plays, I don’t think it’s unreasonable to assume going for 4th down also increased the possibility, if not necessarily by a lot, on losing the 2 minute warning if they gave the ball back. It’s not enough of a likelihood that I drew out non-2 minute warning scenarios, but it’s just one more small thing in favor of kicking the field goal.
I think when you really dig into the context of the decision, the narrow moat of Packers victory scenarios that truly exist, and the fact that the Packers defense must get a stop in any scenario, I actually think it becomes quite clear that kicking the field goal was the correct decision. Analytics is currently limited, and probably always will to some degree. That’s part of what makes football a beautiful sport, whereas the fun in baseball is being rapidly sucked out by the conversion to the three true outcomes (strkeout, home run, walk) because it is a much easier sport to quantify numerically.
So cheers to you Matt Lafleur, and know that you have at least one offensive minded thinker on your side!
Mason's thoughts 