At UAI last Friday, Robin Hanson gave a walkthrough of the new Markov Engine. Here is a subset of the screenshots he used:
The new site still lets you make regular edits
Just ignore the “Current Assumptions Box (CAB)” in the top left, select a question in the lower left, and edit it in the right-hand pane, as usual. Most of your edits should probably still be regular, unconditional edits like this.
You might notice that the preview widget now gives both the relative gain (+/-) and the expected scores. Also, the graph widget now has labeled axes. J
Related questions, conditional forecasts
Consider the following two questions on the market:
- Will any country officially announce its intention to withdraw from the Eurozone before 1 April 2013?
- By December 31, 2012, will the UK officially announce its intention to withdraw from the EU?
These questions are related. The UK is in the EU but not the Eurozone. However, situations which lead the UK to withdraw from the EU probably increase the chance of another country withdrawing from the Eurozone. I may not know how likely either question is to resolve, but I think that they’re more likely to happen together. In the new combinatorial market, I can make that forecast using Assumptions.
Make an Assumption
On the left-hand side, hovering over the question-number lets me click to add this question as an assumption. Anything you can edit, you can also assume. (But it may not always make sense – see later.)
I can assume either one. Let’s pick the more global one here. We’re going to assuming that someone is going to withdraw from the EU.
Here is the “Assume” in Progress
The question is in the “CAB” on the top left. But no assumption has yet been made.
Notice I have the UK question (Q.131) active, and the Eurozone question (Q.215) in the CAB. With no assumptions made, the UK estimate stands at 54%. Now assume Q.215 is True: some country will announce its intent to withdraw.
Assume Finished
Notice the flag saying my assumptions are currently estimated to have a 19% probability. The lower that number gets, the less difference my edit makes in the larger world. That is, my edits only matter in this assumed world, and if that world is really unlikely, my expected gain back in the actual world is small. The more things you assume at once, the “smaller” your possible world. That’s why the CAB small — we want to discourage assuming too many things at once.
Notice that the probability on the UK question has gone up. Someone has already expressed a correlation between these two. But I don’t think it’s strong enough.
Make a conditional edit
So I’m going to increase the chance the UK withdraws, conditional on Q.215 being True.
The conditional edit succeeds. Notice I have 732 points left in “Balance”. (We’re going to rename that “Available” to be less confusing.)
Make Alternate Assumption
I also think that if one question is False, the other is likely to be False. So assume that Q.215 is False:
This is the Status Quo, and appropriately has a high probability (81%) to begin with.
Notice that in this possible world, the probability of the UK withdrawing is (still) 51%. I’m the first one to edit – excellent!
Also notice in this possible world, my Balance (Available) is higher! I get to reuse the points I spent assuming Q.215 was True – because they can’t both happen! Keeping track of assets across possible worlds is a big part of the combinatorial logic.
Make Alternate Edit
I hadn’t noticed that one question was about the EU and the other about the Eurozone. So I thought the UK could withdraw (by December 31) only if some other country withdraws by April 1. Here’s my overconfident edit:
After the Edit
OK, now we’ll clear the assumptions (by clicking the (X) on the top right of the CAB).
Back in the actual world
The combined effect of my edits was to reduce the overall chance of the UK withdrawing to 15%. But each of my edits is conditional, and only gets paid off if that condition arises.
Flip the Conditional: Bayesian Inference!
Just for fun, now assume UK announces an EU exit, and select the Eurozone question for editing. Notice it’s now 95%!
Nice to know the numbers work. J
Would you like to play a game?
We’re running a restricted Beta test soon, and looking for about 10 volunteers. Leave comments if interested.
I am in for the test!
I like it! This will definitely help express some of the correlations I already use to make my estimates. I’m definitely interested in the beta.
I wrote, “But it may not always make sense – see later.” Then I forgot to add the “later” so here it is. While all questions could be interdependent, any given question will only have a few “neighbors” that are directly connected. The first version of the Markov Engine can only handle conditional edits among neighbors. Calculation time is exponential in the size of the largest neighborhood, and we will start small.
When you make an assumption, the questions outside its neighborhood will be grayed out. If you assume a grayed-out question, the system wlll note that you intended a conditional edit, but then ignore the condition and make a regular edit. We will track these for suggestions about links to add. (Think of it as crowdsourcing related variables or key factors.) Comments would be helpful too.
Let’s see an example. Suppose we had 5 questions arranged like this:
A---B---C---D---EWhen editing A, you could assume B. When editing B, you could assume A or C, or both. And so on. Other assumptions would be ignored.
This is a computational restriction to deal with combinatorial explosions. Later in Year 2, we plan to add approximations and projections to find a “best fit” to out-of-neighborhood edits. But one step at a time.
@scott, @DAGGRE_BG: Noted. We’ll keep you posted.
This sounds great, I’m excited.
And yeah, I’d like to test it out for sure.
Yes! I would like to be part of this beta testing about assuming if one thing will happen then how likely is it for something else to happen. I can see the logic behind these kind of questions. But if your assumption is wrong then the related question estimates would be probably be wrong also. So how would score that?
The combo system will refund you for edits based on conditions that don’t happen. For example, if you raise the chance that Iran blockades the Strait of Hormuz conditional on an outside force attacking Iran, then you only gain or lose if an outside force attacks Iran.
Until the outcome is known, the points you invested in that edit are unavailable. But then one of three things happens:
* Attack & Blockade: you get your points back plus extra
* Attack & No Blockade: you get nothing back
* No Attack: you get your points back
I’ve emailed the three of you about beta testing, using the email associated with your comments.
Pingback: Forecasting the US Elections « Future Perfect
Pingback: Conditional Reasoning « Future Perfect
Pingback: Software Update This Week: Combo! « Future Perfect
Pingback: Combo Site Live! « Future Perfect
Pingback: Leaderboard, Graph, and Ripple Effects « Future Perfect