Tags
Aggregation, Bayesian, easy, Forecasting, hard, marketing, Questions, Research, Uncategorized
So far, this blog has discussed a fair number of political applications and even some sports-betting, but today I wanted to briefly bring a business-world application of Bayesian math to the attention of our readers. This short presentation from Math Marketer illustrates a fundamental use for Bayesian logic in identifying which customers are likely to upgrade from a free service as opposed to deadweight ‘freeloading’ customers. The simple power of a few questions…
The example used in the presentation is easy to understand, so I’ll repeat it here. In it, the author likens identifying that ideal market segment to guessing which value of card someone drew from a deck.
In the example, your ‘opponent’ starts the game by drawing a jack from a deck of cards, without showing you the card. You start with a base chance of 4/52, or 1/13, to guess right. Now, suppose you can ask a question or two before trying to guess what card it is.
Some questions are useless for furthering understanding of a certain topic:
“Is it a red card?”
Yes: Still 2/26 = 1/13 odds of guessing the right card
No: Still 2/26 = 1/13 odds of guessing the right
Some questions, however, can drastically shift the odds in your favor:
“Is it a face card?”
Yes: 1/3 odds of being right now, armed with this new information
No: 1/10 odds of being right now, but maybe your next question will narrow it down for you. Perhaps you’ll ask if it’s even or odd, cutting the range in half.
It’s a rough correlation, but the author argues you can use Bayesian modeling/regression to identify those ‘dud’ customers versus the ones who want to throw their money at you for your fine product. Asking the right question lets you focus future efforts more effectively, getting more for your money for less total effort.
This raises an interesting point, then – how do you know if you’re asking the right question? What makes a question “the right question”? It’s often very easy to know after the fact, but knowing ahead of time is much more valuable in most settings – on prediction markets, in the business world, etc. Few settings are as predictable, and few sets are as predetermined as a deck of cards. When you don’t know all the rules of the set, things get a lot murkier, quickly.
Commenters, what are your thoughts – what indicators do you look for in a real-world question (such as the ones many of you are estimating about on our prediction market) to immediately identify the value or usefulness of that question? By the same token, what makes a question a “bad” question in your eyes?
“By the same token, what makes a question a “bad” question in your eyes?”
The worst questions are those with ambiguity. If the answerer doesn’t know if it was a yes/no question or a multiple choice question, for example, they’re likely to get lost in figuring out how to answer rather than considering the actual question at hand. For example, some questions posted on the exchange were of the form: “Will X country receive a score of ___, ___, or ___ on the ….?” When presented with questions like this, I ended up spending more of my time trying to figure out how the question wanted me to answer. Was I supposed to increase the % if I thought it would be one of the two positive responses? Or if I thought it would be one of the three choices. Sometimes a question like this could be even more complicated.
The best questions in a more general setting seem to require some experimentation. By asking some small number of people questions to evaluate which give the best responses, a general idea of those question that are most effective can be found. It’s too difficult to just blindly find a “best” question without some investigation.
Yes, we got surprised by some multiple choice questions before we had finished the web interface, and hacked them (badly) so we could return some answer to the sponsor. We should be good now.
But if you see an unanswerable question, please flag it here or on the HelpDesk.
A good question for me is one that concerns an event that will have a significant material impact, of which knowing the likelihood will allow a person/business/government to prepare a contingency plan.