By Megan Czasonis, Mark Kritzman, and David Turkington
Statistics and intuition converge with the concept of “relevance”.
We introduce a prediction system based on assessing the relevance of prior outcomes for future predictions, and describe the advantage it brings to both simple and complex quant models.
It is hard to make good predictions about the future, which is why it pays to use every tool we have. One way to predict is with data, building “quant” models and scrutinizing the statistics of many variables. Alternatively, we can rely on qualitative judgement and human experience. Ideally, we can combine the benefits of both styles, but doing so requires that we reorient our approach to data. Rather than focusing on variables, we propose focusing on experiences and carefully assessing their relevance in order to extrapolate the future from the past. Relevance allows us to see exactly how much each prior outcome contributes to today’s prediction, and how that contribution changes over time. And it allows us to evaluate the confidence we should have in each specific prediction, which also changes. Classical statistics and machine learning do not offer these views. The future of prediction, we argue, must be ever more transparent and flexible in its use of data.
