Background of the Forecast
On May 7, 2025, I released a data-driven forecast titled “What do prediction markets say about the next Pope?” The analysis blended odds from three prediction markets – Paddy Power, Ladbrokes, and Polymarket – to estimate each papabile cardinal’s chance of being elected. It also used an AI-powered deep search (GPT-4o) to categorize all 133 cardinal electors by ecclesial faction (Progressive, Moderate, Conservative, Traditionalist) and Traditional Latin Mass (TLM) stance (Supportive, Indifferent, Opposed). The forecast’s aim, written from a Traditionalist Catholic perspective concerned with liturgical policy, was not to pinpoint the exact papal successor (acknowledging that is nearly impossible given the secrecy and 133 candidates) but to illuminate “broader patterns” – specifically, “what faction is most likely to produce the next pope” and “will he be supportive, indifferent, or opposed to the Traditional Latin Mass”.
The conclave indeed began on May 7, and the very next day the College of Cardinals elected Robert Cardinal Prevost of Chicago as Pope Leo XIV, the first American pope. Below we evaluate the forecast’s accuracy against this outcome, assess its methodology, and discuss the reliability of its sources and framing.
 |
| Pope Leo XIV. |
1. Accuracy of the Forecast vs. Outcome
The actual outcome – Cardinal Robert F. Prevost becoming Pope Leo XIV on May 8, 2025 – was only partially anticipated by the prediction-market model. In terms of specific identity, the model did not flag Prevost as a top-tier candidate. Prevost was a relative dark horse: having been a cardinal for just over two years, he was not prominently featured in pre-conclave shortlists or betting favorites. In fact, betting markets heavily favored well-known figures like Italian Cardinal Pietro Parolin (Vatican Secretary of State) and Filipino Cardinal Luis Tagle, each commanding ~20–30% implied chances, whereas Prevost hardly registered in odds discussions. This underscores the model’s own caveat that prediction markets “cannot forecast the exact individual” pope in such a secretive, one-off election. Indeed, the short conclave’s surprise choice of Prevost – “a man who had been a cardinal for only a little more than two years” – defied conventional expectations.
**However, the forecast proved accurate in predicting the new pope’s broad ideological profile and liturgical stance. Siegmund’s model concluded “The new Pope will be moderate and indifferent to the Latin Mass” – a prediction borne out by Pope Leo XIV’s profile. Prevost is widely described as a moderate consensus figure within the Church. For example, The Guardian’s profile of Leo XIV explicitly labeled him “the moderate, good-humoured first US pope,” noting he is a calm bridge-builder between factions. His election was “welcomed by progressive factions” in the Church and not the result hoped for by arch-conservatives, confirming he is neither a Traditionalist nor a hardline conservative.
Crucially for the Traditional Latin Mass question, the forecast’s expectation that the next pope would be “indifferent” (i.e. not an active champion of the old rite) was correct. As a close ally of Pope Francis, Prevost is seen as a continuity candidate on liturgical policy. In its analysis of the new pope, The Washington Post flatly stated that “no one should expect a return of the Latin Mass” under Leo XIV. In other words, Prevost is unlikely to reverse Francis’s 2021 Traditionis Custodes restrictions on the pre-Vatican II liturgy. This aligns with the “indifferent to the Latin Mass” label – he has not been a vocal proponent of expanding the Tridentine Mass, though nor is he known as a vehement opponent beyond enforcing Francis’s line. (Notably, Cardinal Prevost even reportedly celebrated the old Latin Mass privately in his chapel at times, reflecting a personal openness but a public stance of maintaining the status quo.) In summary, while the markets and model did not foresee Prevost by name, they correctly anticipated that the 2025 conclave would produce a centrist-moderate pope with no strong inclination to undo Latin Mass restrictions. This outcome validates the forecast’s value in capturing “broader patterns” if not precise details.
2. Evaluation of Methodology and Statistical Soundness
The forecasting methodology combined quantitative market data with qualitative AI analysis in an innovative way. From a statistical standpoint, the approach was sensible but not without limitations:
-
Blending Multiple Prediction Markets: The use of three independent markets (two bookmakers and a crypto prediction exchange) was intended to dilute the bias or blind spots of any single source. By converting betting odds into implied probabilities for each cardinal and then averaging those probabilities, the forecaster obtained a composite likelihood for each candidate. This blended probability approach is straightforward and ensures that if a candidate was highly rated in at least one market, they would appear in the combined ranking. The subsequent renormalization to make the probabilities sum to 100% is a standard step for consistency. In principle, this aggregation is sound; it mirrors techniques used in meta-predictions (akin to averaging polls in political races). However, we should note that papal betting markets are relatively thin and speculative. Unlike election polls, these odds are driven by gambler sentiment and limited expert knowledge (since no insider information from the conclave is available). Thus, the combined probabilities still reflect the conventional wisdom and biases of bettors. In this case, most bettors gravitated toward famous papabili (Parolin, Tagle, Zuppi, etc.), so the blended odds largely mirrored that consensus, which, as discussed, missed the long-shot Prevost. Statistically, the averaging did little harm, but any errors or blind spots (e.g. undervaluing dark horses) present in all markets would persist in the result. The forecast appropriately acknowledged this uncertainty, cautioning that the information asymmetry in a conclave is huge. Overall, using prediction markets as inputs was justified – such markets can sometimes outperform pure guesswork and force consideration of diverse candidates. Yet, given the small sample of markets and the event’s rarity, we should not overstate the confidence in those probabilities. The method is data-driven, but the data itself were noisy and incomplete (with some candidates not even listed in certain books).
-
AI-Driven Categorization (GPT4o “Deep Search”): A novel aspect of the methodology was deploying an AI (GPT-4o’s Deep Search feature) to classify each cardinal along the two dimensions of interest: faction and TLM stance. Essentially, the forecaster leveraged NLP to scan “official sources and blogs” about each cardinal and assign them labels like Moderate/Progressive etc., and Supportive/Indifferent/Opposed to the old Mass. This approach adds qualitative depth to the raw odds. Methodologically, if done carefully, it’s a clever way to incorporate expert knowledge without manually researching 133 individuals. The soundness of this approach depends on the AI’s accuracy in summarizing each cardinal’s reputation. In broad strokes, these categories are well-defined: for example, only ~24 of the 133 electors had expressed clear support for the Tridentine Mass, while the majority (~92) had not spoken on it (effectively indifferent). Similarly, a handful of cardinals (like Burke or Sarah) are well-known Traditionalists, a group of ~10–15 are noted progressives, and many others fall in between. It’s plausible that GPT4o, parsing news articles, Vatican statements, and Catholic commentary, correctly identified the obvious cases (e.g. Cardinal Burke as Traditionalist & TLM-supportive, Cardinal Cupich as Progressive & TLM-opposed, etc.) and labeled the rest as moderate or no known stance by default. This yields a rough ideological mapping of the conclave electorate.
There are, however, questions of reliability in using GPT for this task. AI text models might misinterpret tone or take biased blog opinions at face value. The source selection is critical: reputable sources (Vatican News, NCR, Crux, etc.) would provide accurate info, whereas partisan blogs might mischaracterize a cardinal’s views. Without knowing the curation method for “Deep Search,” there’s a risk of classification error or bias. For example, one cardinal’s nuanced view on liturgy might be oversimplified as “opposed” if an article criticized him from a Traditionalist viewpoint. The model’s three-tier schema (Progressive/Moderate/Conservative/Traditionalist) also forces simplification of a spectrum of beliefs. That said, given the clear trend that emerged – most high-probability papabili (Parolin, Tagle, Zuppi, etc.) fell into the Moderate or center-left faction, and very few strong Traditionalists had any odds – the categorization likely captured reality well enough. Summing the probabilities by category to find, for instance, the total probability of a “Moderate” pope, is a valid calculation under the assumption that one and only one category will apply to the eventual winner. This is statistically sound (treating the categories as mutually exclusive outcomes and the cardinals as partitioned into those bins). Indeed, the forecast determined the highest cumulative probability lay with a Moderate candidate who is Indifferent toward the Latin Mass, which was its final prediction.
In summary, the methodology was innovative and mostly sound. The integration of multiple market signals provided a quasi-“wisdom of crowds” input, and the AI-driven categorization injected substantive domain knowledge (church politics and liturgical leanings) into the forecast. The main limitations come from the inputs themselves – market odds on a secret conclave are volatile estimates, and AI interpretations of ideology may err on subtleties. Despite these, the method succeeded in producing a coherent forecast (correct on key attributes), illustrating the value of combining quantitative and qualitative techniques in forecasting unique political events.
3. Source Reliability and Interpretive Framing
Reliability of Prediction Markets: The forecast relied on odds from Paddy Power and Ladbrokes (traditional bookmakers) and Polymarket (a crypto prediction market). These sources are credible in the sense that they represent real-time aggregated expectations, but we must recognize their inherent limitations for papal elections. Unlike public political contests, a conclave has no polls or campaigning, and cardinal electors are sworn to secrecy. Thus, betting odds are driven by media speculation, expert opinion, and gambler hunches, rather than hard data. In the lead-up to this conclave, the markets clearly coalesced around a few high-profile candidates (with Parolin given roughly a 1 in 3 chance, etc.), reflecting conventional wisdom. This turned out to be a poor guide to the actual vote — a reminder that even widely followed markets can miss outcomes when information is scarce or insider-only. In fact, some “outsider” candidates saw their odds surge only late (e.g. the oddschecker report noted long-shots like Cardinal López Romero and Cardinal Tolentino gaining betting interest during the conclave), showing how reactive and speculative these markets were. That said, the aggregate signals did correctly point to a moderate outcome, since all the top-ranked papabili were of the moderate or centrist ilk. The markets’ credibility is bolstered by the historical observation that they sometimes predict papal outcomes (they correctly identified Cardinal Ratzinger as a favorite in 2005 and had Cardinal Bergoglio as a 50/1 outsider who gained traction in 2013). The forecaster’s use of three different markets added reliability by cross-verifying odds; and renormalizing ensured no market’s omissions skewed the total. In conclusion, while prediction markets are not foolproof (especially for an event so opaque), using them was a reasonable and data-backed choice. They provided a baseline probability distribution that, when interpreted cautiously, gave a better foundation than pure guesswork or partisan wishful thinking.
AI Classification and Ideological Framing: The decision to categorize cardinals by faction and liturgical stance reflects a Traditionalist Catholic framing of the papal question. Dr. Siegmund, writing as a self-identified Traditionalist, focused on whether the next pope would undo or uphold Pope Francis’s restrictions on the old Mass. This focus is understandable – for Traditionalist observers, the TLM issue is a key fault line – but it is a particular interpretive lens. By using GPT-driven research on each cardinal, the forecast attempted to ground this ideological categorization in factual evidence (statements, writings, reputation of each cardinal). The reliability of this approach depends on the quality of sources. Official sources (e.g., Vatican press, biographies) would reliably tell a cardinal’s positions only in broad terms. Many cardinals do not publicly discuss the TLM, and as one analysis noted, roughly 92 electors had not spoken either way on it. In such cases, the model effectively labels them “Indifferent,” which is fair (they haven’t taken action to either encourage or suppress the old liturgy). A Traditionalist viewpoint might interpret “indifference” as negative (i.e., not an ally for expanding the Latin Mass), whereas a progressive might simply call it being “mainstream” or focused on other issues. The GPT classification likely mirrors the Traditionalist author’s interest by explicitly flagging who is friendly or hostile to the old rite. This doesn’t invalidate the approach, but it’s worth noting that the framing emphasizes a specific policy stance as a measure of the pope’s ideology. Other analysts might have categorized cardinals by, say, social theology (progressive vs conservative on social issues) or governance style (curial vs pastoral). Here the lens was faction (a general left-right within Church) and liturgical traditionalism, reflecting concerns of Traditionalist Catholics.
The credibility of using AI “Deep Search” for this is cautiously positive. GPT-4 can synthesize large amounts of text, meaning it could read biographies, interviews, and commentary far faster than a human, potentially surfacing each cardinal’s record on, for example, implementing Traditionis Custodes or supporting traditional orders. If the AI’s outputs were reviewed for obvious errors, this could be a very effective way to classify 133 individuals. However, one must be cautious: AI might inadvertently incorporate bias from sources (for instance, a liberal Catholic blog might brand a moderate cardinal as “conservative” which the AI could take at face value). Ideally, the forecaster would cross-check contentious classifications. For a specialist audience, the takeaway is that AI can be a force-multiplier in aggregating qualitative data, but its answers are only as good as the data fed in. In this forecast, the categories chosen and the conclusions drawn (moderate, Latin-Mass-indifferent pope likely) were in line with independent assessments from seasoned Vatican watchers. That suggests the AI categorization, coupled with the probability weighting, succeeded in capturing the reality.
In terms of interpretive framing, the forecast clearly speaks to concerns of Traditionalist Catholics: it asks “Do Traditionalists need to prepare for further restrictions?”. The analysis concluded yes – the next pope would probably be indifferent (at best) toward the TLM, implying no relief for Traditionalist liturgical aspirations. This framing is valid for that constituency and was borne out (Prevost’s election did signal continuity with Francis, which means continued strict oversight of the Latin Mass). For a broader forecasting audience, the framing does not skew the data so much as it selects which data to highlight. The forecast did not, for example, dwell on whether the next pope would be “committed to social justice” or other dimensions – it focused on faction and liturgy. This is a feature, not a bug, given the author’s intent. It demonstrates how forecasters can tailor their analysis to the questions their audience cares about, provided they remain honest about the data. In this case, the Traditionalist framing honed in on a dimension (traditional vs moderate vs progressive) that proved very relevant to the conclave outcome.
Conclusion
In conclusion, Robert F. Siegmund’s prediction-market-based forecast was insightful and largely vindicated. While it did not single out Robert Prevost as the next pope (an unsurprising miss, since neither did most experts or bettors), it accurately foresaw the type of pope the 2025 conclave produced: a centrist moderate who will likely maintain Pope Francis’s approach to the Traditional Latin Mass (neither actively promoting it nor being a staunch traditionalist opponent). The methodology – blending odds and leveraging GPT-driven research – was a sound experimental template for forecasting low-information events. It combined the quantitative rigor of markets (ensuring every candidate’s chance was considered proportionately) with qualitative insight into Church politics. This hybrid approach allowed the forecaster to extract meaningful signals (like the high probability of a moderate, status-quo pope) from noisy data.
The sources and framing, oriented to a Traditionalist viewpoint, did not detract from the analysis’ credibility; rather, they clarified the implications of the forecast for a specific stakeholder group (i.e. those worried about the Latin Mass). In fact, by classifying cardinals’ ideological and liturgical leanings, the forecast translated the opaque conclave probabilities into a narrative about the likely direction of the Church. This proved prescient – Pope Leo XIV’s profile confirms that prediction markets, cautiously interpreted, can shed light on outcomes even when they cannot name the victor. For specialists in political and religious forecasting, this case exemplifies the value of combining data from diverse sources and the importance of framing results in context. The forecast of May 7, 2025 will be remembered as a successful call on the big picture, if not the specific name, reinforcing that in Vatican politics patterns often prevail over individual prognostication.
Sources:
-
Siegmund, R.F. What do prediction markets say about the next Pope? (Forecast presentation, 7 May 2025).
-
The Guardian – Profile of Pope Leo XIV (Robert Prevost).
-
The Washington Post – Analysis of Pope Leo XIV’s election and stance.
-
National Catholic Reporter – Data on cardinals’ views of the Latin Mass.
-
Oddschecker/Forbes – Pre-conclave betting odds for Next Pope (Parolin as favorite, etc.).
