Tags
Brier score, DAGGRE, Forecasting, IFPs, Middle East, Politics, Questions, resolutions
Israel elections have been set for the date of January 22, 2013: Reuters. The question we had on the market with respect to the elections in Israel described the dissolution of the Parliament as a potential event that would trigger early elections. Yet our market did not foresee this dissolution to happen before the deadline of November.
Analysis:
We closed question #266 with a Brier Score of 0.63:
The Brier Score (Brier 1950) is a measurement of the accuracy of probabilistic predictions. As a distance metric, lower is the better. The Brier Score ranges from 0..2, and is the sum of the squared differences between the forecast and the outcome averaged over the number of forecasts. On a binary (Yes/No) question, simply guessing 50% all the time yields a “no courage” score of 0.5. We want to be firmly below 0.1.
In this case, the market did not perform very well. From the comments of our participants, we think that the reason for this high Brier Score stems from the instability in the area, the international political issues with Iran and from internal budget disputes.
Participant Comments:
Here are the evolution of the market and some comments from the participants:
Netanyahu has announced early elections – http://www.haaretz.com/news/national/netanyahu-announces-early-elections-1.468944 – So we now have “announced intentions by government figures without dissolution of parliament and the legal establishment of an election date”, but we’ll have to wait for the official dissolution of parliament when it reconvenes on October 15 to officially resolve the question.
As of today http://www.huffingtonpost.com/huff-wires/20121009/ml-israel-politics/. Some sort of announcement is planned later today.
“Prime Minister Binyamin Netanyahu has already made a final decision to seek a February 12 election rather than try to pass the 2013 state budget, politicians who spoke to Netanyahu said on Thursday.” – http://www.jpost.com/DiplomacyAndPolitics/Article.aspx?id=286702 and http://www.upi.com/Top_News/World-News/2012/10/05/Report-Netanyahu-to-seek-early-election/UPI-75321349436302/
“Coalition chairman Ze’ev Elkin (Likud) said he considered early elections inevitable, because none of the Likud’s coalition partners appeared ready to compromise.” – http://www.jpost.com/NationalNews/Article.aspx?id=287048
“Netanyahu is having trouble getting the 2013 budget approved, which could serve as a trigger for early elections.” – http://www.nation.com.pk/pakistan-news-newspaper-daily-english-online/international/08-Oct-2012/hyperinflation-stalks-iran-israel-wavers
“Prime Minister’s Office says: Netanyahu is bent on passing the national budget for 2013 and holding elections on their scheduled date • No official agreement yet on budget by coalition partners.” – http://www.israelhayom.com/site/newsletter_article.php?id=5866
Netanyahu admits possibility of early elections – http://www.jpost.com/Headlines/Article.aspx?id=284836
Just can;t see an election date established, the unrest in this area is constantly growing.
‘Asked if elections will be called early, Yuval Steinitz replied: “I don’t know. 50/50 percent.”‘ – http://www.swissinfo.ch/eng/news/international/Israels_budget_a_political_hot_potato.html?cid=33489362
I don’t believe this will happen before November, as Iran is on the radar and a strike may be imminent before the US presidential election, according to David Rubin an Israeli terrorism expert.
It seems like the momentum is going this way….
We are looking forward for your comments on why the early elections in Israel took us by surprise. We also have more questions on Israel on the market, such as #232 or on Iran issues, such as #236 and #276.
Does IARPA share how the different teams compared on questions like this? Can we see when we did poorly in relation to the other teams, vs. when the question was just legitimately hard to predict?
I agree with Scott that it would be interesting to see how this group does vs. other teams. It would also be interesting to know if speed of response to new information is a factor that IARPA looks at.
Are all the questions set by IARPA? I’m sure I’m not the first to think that having more questions (or at least one new one a week) would encourage more regular participation. Questions that fit within the rules but do not originate from IARPA could be suggested by those in the “all time top 500″. IARPA could ignore data from these but this would be an optional reward/privilege for doing well, it may help to generate additional traffic by increasing content for the site and, the exercise may suggest interesting avenues of inquiry.
Also is there a good reason the length of comments is so limited on the main site, brevity is great but do we really want to restrict discourse to the level of twitter?
@Scott, @HalfPast: We can certainly post how we did with respect to the baseline (an unweighted opinion pool), and should do a summary post soon. It may not be possible to include scores from other teams. That’s up to IARPA.
No, not all the questions are from IARPA. For example, we added the baseball ones for fun (and to test an autotrader). We’re happy for suggestions — yes we should include a “suggest” button on the site, but in the meantime use comments, email, or the blog.
We’d like to release some every week, but also not end up with so many that the official IARPA questions get lost.
Comments need to have some limit to avoid insanity, but there’s nothing special about the limit we have. We can try doubling it.