A group of researchers backtested a range of AI trading strategies over 20 years and more than 100 stocks, and found (in a paper that is not yet peer-reviewed) that most of them still couldn’t beat plain buy-and-hold investing.
The researchers are from the University of Edinburgh, the University of California, Los Angeles (UCLA) and South Korea’s Sungkyunkwan University, and they plan to present it at the ACM SIGKDD Conference on Knowledge Discovery and Data Mining in August.
They claim from their findings that the advantage AI tools seemed to have in earlier research was mostly an illusion, and a result of how it had been tested.
Why a six-month test made the bots look brilliant
Most AI trading studies have been run on a small group of winning stocks over a short stretch (sometimes three months on as few as three stocks). If you test Tesla over six of its blockbuster months, almost any strategy can start to look brilliant.
That creates three common traps:
- survivorship bias, where the analysis only counts stocks that survived and won, and drops the ones that failed out of the data.
- look-ahead bias, where the model uses information it wouldn’t have had yet
- data-snooping, where enough tests on the same data can make a trading strategy look brilliant by luck
Earlier studies often relied on a few stocks over short windows, Mihai Cucuringu, a mathematics professor jointly affiliated with UCLA and the University of Oxford, and one of the paper’s authors, told the Wall Street Journal. His team widened the test to cover the 2008 financial crisis, the Covid-19 crash and the bull markets between them, and also added delisted stocks back in so the failures still counted. Cucuringu says the apparent advantage “largely disappears” once the test of AI tools runs longer and wider.
What 20 years and 100-plus stocks showed
The study found that plain buy-and-hold (buy, then wait) was one of the best investing strategies. In one setup that screened for steadier, low-volatility stocks, buy-and-hold posted the best risk-adjusted result in the field and an annual return of about 7.9%, beating every AI agent the researchers tested.
To be fair, the bots weren’t hopeless. In one setup, an AI agent posted the highest raw annual return in the group, at nearly 14%, but it also had weak risk-adjusted numbers and steep drops along the way, which is exactly the kind of risk a headline return can hide.
A 55-year-old statistical model called ARIMA also beat the AI tools once the study accounted for risk. The two AI trading agents at the center of the study, both built on commercially available large-language models, showed no reliable edge over the market once the test was run fairly. There was no clear sign they added real skill beyond what the market already gave them.
Bigger models didn’t solve the problem either. The team tested models of different sizes and found the larger ones didn’t reliably beat the smaller ones.
