Rational decision-making process and Astonishing Sports
I recently read Misbehaving, a book on behavorial economics written by Richard Thaler, one of the most important contributors of the field. One thing in particular baffled me. An academic paper written by David Romer had shown that NFL teams punt too often on a 4th down. From the result was even created a bot, using a model developed by Brian Burke, that determines which decision a coach should take on a 4th down.
At the time, it seems it wasn’t really well received by the NFL (it’s apparently evolving, thanks analytics). This is not necessarily surprising. Despite decades of sabermetrics studies, fans, journalists and even players sometimes are often ditching what has been found (if not proven).
It’s also very common in the Astonishing Sports community. Players take irrational decisions for all kind of reasons. Reaching a personal objective, an overweighted belief in things like defense or home runs, or just because they like a player’s nickname.
I’m not judging of course, but one would assume that if your ultimate goal is to win, you should optimize your chance to win. And if numbers say you should do something, it’s probably the right path to follow (except if you have rational arguments to counter of course. But it generally ends in guts feeling reasoning).
A Baseball Experience
That gave me an idea. How the Astonishing Sports community players would react if they were given a tool a bit similar to the 4th down bot. Would they use it? How often? Would they still rely on what they think is better?
With these questions, I created a form where a hypothetical mobile app can predict whether you should give the green light to steal a base or not, in the most reliable way possible, and without any concern about it being cheating. It asks the respondant if she would use such an app, and then how often, and after how many failures would she ditch it? The form also features some space for extra explanations.
A second part asks basically the same questions, but with an actual human being instead of an app. The hypothetical coach would basically have the same knowledge/success rate than the app.
The rational behavior is to use the app, all the time, and never ditch it. And do the same with the human version. If you simply know that the analytics is right, then there’s no reason to not trust it. It doesn’t mean you’ll have a 100% steal percentage, but your actual success rate will be optimal given the team you have. In a world of econs (Term coined by Thaler to describe people who always behave rationally), the results would support this rational responses.
Of course, in practice, it’s not the case.
Disclaimer: only a few people (15) participated in this study. Since I don’t expect to get any scientific value out of this data, I guess it’s okay to just extrapolate a bit for fun from it.
The App Results
Unsurprisingly, most people would use such an app. Only one person said she wouldn’t use it, because it wasn’t playing the game “right”. Fair enough.
It becomes more interesting when looking at the frequency of usage.
Only one person would use it all the time, while most would just use it from time to time. If you know it’s reliable, why wouldn’t you use it as much as possible. At best, you can be at much reliable as it is, but not more. In game theory, using the app would be the dominant strategy without a doubt.
Justifications on this was particularly interesting. Most people explained that they would use it as a second opinion, or when they’re not sure. Which is… Probably better than not using it, but still an unoptimal strategy. One respondant said that it was not gonna be accurate anyway, and another declared she was a master of making good decisions and so the app wouldn’t be very useful. This person is probably a Major League coach now.
When asked after how many failures the respondant would ditch the app, the results are very variable. One person said only one failure would be a death sentence for the tool. Most other people would give just a few tries to it. Given how risky base-stealing is, you would probably need a bigger sample size to judge it objectively. Three people would keep it anyway, which is probably the most rational thing to do (except if you suspect a fault in the app).
When asked about an explanation, most people were okay with giving the app a few tries before taking a final decision. One person said she would try it for at least half a season, which seems reasonable enough to me.
The Amazing assistant
The second part of the questionnary asks basically the same questions as the first one, but with a human to replace the app. It states that this coach assistant is as efficient as the hypothetical app.
Apparently, respondants are more understanding about the failure of the human than of the machine. And some would even listen to him even after many failures. Respondants mostly declared that they would trust a human more than an app. Here are some interesting extracts:
- “Humans deserve more chances than technology”
- “I trust a person to be more successful than a non reliable app”
- “(They are) human and are never expected to be perfect”
It seems humans get the benefit of the doubt because of their nature. Which is probably a good thing, but also a non-rational behavior if you’re in a highly competitive environment.
This study was only there for fun, but results were as I expected. Now let’s develop this app and see how it works in real life!
I should probably find out who this Master of good decisions was to ask for help!