Advances in Financial Machine Learning

The opening claim is that machine learning on market data is its own field, not standard machine learning pointed at prices. López de Prado says investment books tend to fail one of two ways — elegant maths about a world that does not exist, or practice with no theory, overfit and failing in use — and that he means to sit between them.

His diagnosis of why quant projects fail is organisational. Discretionary managers work alone because they cannot work as a team; firms copy that model for quants, hire dozens of researchers and ask each for a strategy within months. He calls this the Sisyphus paradigm and says it ends in one of two places: a false positive from an overfit backtest, or a crowded factor strategy with a low Sharpe ratio.

His alternative is a production line, where each researcher masters one station — data, features, backtests, execution — and discoveries come from the team. He states it strongly: “Every successful quantitative firm I am aware of applies the meta-strategy paradigm”. The book is written as a manual for such teams.

The warning that runs through the rest is short: “When misused, ML algorithms will confuse statistical flukes with patterns.” Add the low signal-to-noise ratio of markets, he argues, and careless users will produce false discoveries faster than ever. The chapter includes a table pairing each common pitfall with the chapter that treats it.

López de Prado argues against sampling prices at fixed clock intervals, the habit of most practitioners and academics. Markets do not trade at a steady pace — the hour after the open is far busier than midday — so time bars over-sample quiet periods and under-sample busy ones. He adds that series sampled by the clock show serial correlation, uneven variance and non-normal returns.

He prefers to sample by activity: a new bar after a set number of trades, shares, or dollars changes hands. Dollar bars get his case. A stock that doubles needs half as many shares for the same dollar trade, and share counts shift with issuance and buybacks, so counting dollars gives a steadier clock. He shows it on E-mini S&P 500 futures, where bars per day vary far less by dollars than by trades or volume.

He then builds bars that fire on imbalance — when signed trades run further to one side than expected. These form more often when informed traders are active, so each bar carries roughly the same amount of new information.

The common academic label — up or down beyond a fixed threshold after a fixed time — gets two objections. A fixed threshold on clock bars ignores changing volatility. More basic, it ignores the path: every real strategy has a stop, whether set by the manager, the risk desk or a margin call, and a label that counts a trade the market would have stopped out is not a realistic one.

His replacement is the triple-barrier method, which he says he has not found elsewhere in the literature. Set a profit-taking barrier above and a stop-loss barrier below, both scaled to estimated volatility, and a time limit on the right; the label is whichever barrier the price touches first.

Meta-labeling splits the bet in two. A primary model — a machine, an equation, a fundamental view or a human manager — picks the side. A second model learns only whether to act on that call and how much to stake, including nothing. He gives four reasons it helps: it can sit on top of a model that is understood, it limits overfitting because the machine never picks the side, it allows separate models for longs and shorts, and it puts effort on sizing, where accuracy on small bets cannot make up for errors on large ones.

He extends the idea to discretionary funds. A meta-model trained on a manager’s past calls could learn when those calls tend to be good — near a regime change, say — or poor, as under stress. He predicts that discretionary funds will become “quantamental” firms and that this tool will be their route.

Standard machine learning assumes each observation is independent. In finance, labels overlap in time: if one label spans days one to five and another days three to eight, both depend on the same returns. López de Prado treats this overlap as a central defect of financial data sets.

He measures each label’s “uniqueness” — at each moment of its life, one over the number of labels alive at once — and averages it over the label’s span. Labels that share their window with many others count for less in training and in sampling.

His sequential bootstrap draws observations one at a time and lowers the chance of drawing one that overlaps those already drawn. The aim is samples closer to independent, which in turn makes an ensemble’s members less alike and its average more useful.

The chapter starts from a dilemma. Price levels carry memory but are not stationary, which most models require; returns are stationary but have discarded almost all memory. López de Prado’s way out is to difference by a fraction rather than a whole step, removing only as much memory as stationarity requires.

His E-mini example makes the point with numbers. The standard unit-root test passes at about d = 0.35, and at that setting the transformed series still correlates 0.995 with the original price. Plain returns (d = 1) correlate 0.03. Across 87 liquid futures, he reports that every one reaches stationarity with d below 0.6, and some need no differencing at all.

The conclusion he draws is that almost every finance paper that works in returns has over-differenced its data — thrown away more memory, and so more predictive information, than the statistics demanded.

On the choice between bagging and boosting, López de Prado sides with bagging for finance. Boosting cuts both bias and variance, but correcting bias raises the risk of overfitting, and with markets this noisy, overfitting is the greater danger. Bagging also runs in parallel; boosting runs in sequence.

He ties this back to Chapter 4. Overlapping observations make the resampled training sets nearly identical, so averaging the models no longer cuts variance, and they inflate the out-of-bag score, because the held-out examples closely resemble the training ones.

Ordinary k-fold cross-validation leaks in finance. Because labels overlap, a training example may share information with a test example next to it in time, and the test score looks better than it should.

The cure has two parts, both named by López de Prado. “Purging” removes from training any example whose label overlaps in time with a label in the test set. An “embargo” also drops training examples that come just after the test set, since many features carry serial correlation.

He gives a test for leakage: if scores keep improving as the number of folds rises towards the number of bars, the model is likely profiting from leaks. And he says purging and embargo belong in every split — tuning, backtesting and performance evaluation alike.

The chapter opens on the error López de Prado calls among the most pervasive in financial research: run data through a model, backtest, adjust, and repeat until the backtest looks good. He notes that statisticians treat this as misconduct, and puts a number on it: “It typically takes about 20 such iterations to discover a (false) investment strategy subject to the standard significance level (false positive rate) of 5%.”

His alternative is to ask which inputs drive a model before asking whether it makes money. He sets out three measures. Mean decrease impurity is quick and built into tree models. Mean decrease accuracy shuffles one input at a time and measures the loss of out-of-sample skill. Single feature importance tests each input alone.

The first two suffer from “substitution effects”: when two inputs carry the same information, each can look less important than it is. The third avoids that but misses inputs that matter only in combination. He recommends using them together, since each fails in a different way.

When tuning settings for an investment model, López de Prado prefers log loss to accuracy as the score. Accuracy counts a confident wrong call the same as a hesitant one, but a strategy sizes positions by confidence, so a confident miss costs far more. Log loss charges for that.

He opens with poker and a warning: “Your ML algorithm can achieve high accuracy, but if you do not size your bets properly, your investment strategy will inevitably lose money.”

A worked example shows why. Two strategies make the same correct call on a price that halves and then ends 25 percent above where it started. One starts at half size and adds on the dip; the other starts at full size and is forced to cut. The first makes money and the second loses it. Sizing should keep cash back for a signal that strengthens.

He then sets out ways to turn a model’s predicted probability into a position between full short and full long, and to average the signals that are live at once so the position does not swing on every update.

López de Prado insists that a backtest is not an experiment. It is one historical simulation; it proves nothing and promises nothing. Its use is as a sanity check on sizing, turnover, cost and behaviour in a scenario. He cites the Deutsche Bank team led by Yin Luo for seven common sins: survivorship bias, look-ahead bias, storytelling after the fact, data snooping, transaction costs, outliers and the cost of shorting.

His sharper point is in the title of section 11.3: even a flawless backtest is probably wrong. Only an expert produces a flawless backtest, and an expert has run tens of thousands, so the one on the table may be the fluke that multiple testing guarantees. The better a researcher gets, the more false discoveries appear.

Hence his rule, which he calls the Second Law of Backtesting: “Backtesting while researching is like drinking and driving. Do not research under the influence of a backtest.” And: “The purpose of a backtest is to discard bad models, not to improve them.”

His practical advice: build models for whole asset classes, since a pattern found in one security alone is likely false; use bagging, and distrust a strategy that bagging degrades; do not backtest until the research is done; record every backtest run on a data set so the final Sharpe ratio can be deflated by the number of trials; test across simulated scenarios rather than the one history that happened; and if a backtest fails, start again rather than reuse it.

The standard walk-forward backtest tests one path through history, and López de Prado argues one path is easy to overfit — a researcher can rerun it until a false positive appears. He adds that a few observations carry heavy weight in a walk-forward result, so its variance is high.

His answer is combinatorial purged cross-validation: split the data into groups, form every combination of training and test groups, purge leaks, and stitch the test pieces into many distinct backtest paths. The output is a distribution of Sharpe ratios rather than one number.

He grounds the need in a known result: if many strategies with a true Sharpe ratio of zero are backtested, the best of them will still show a positive Sharpe ratio, and the more trials and the more variance, the higher it goes.

The chapter catalogues what a backtest report should contain, from holding period and turnover to drawdown and “time under water” — how long the strategy takes to regain a previous high.

The two measures it is known for adjust the Sharpe ratio. The probabilistic Sharpe ratio estimates the chance that the true ratio beats a chosen benchmark, given the track record’s length and the skew and fat tails of its returns; it rises with longer records and positive skew and falls with fatter tails. The deflated Sharpe ratio raises the bar further to account for how many trials preceded the one reported.

López de Prado observes that every strategy exits through a profit target or a stop — and that even one with no declared stop has an implicit one, where the investor can no longer fund the position or bear the loss. That lets him model outcomes as a run of two-way bets.

From that he derives the probability that a strategy misses a target Sharpe ratio, as a function of its hit rate, betting frequency and the sizes of its wins and losses. In his example, a fall in hit rate from 70 to 67 percent wipes out the profits. His lesson, in italics in the original: “Strategy risk should not be confused with portfolio risk.” The risk officer watches the holdings; the question that matters to the chief investment officer is whether the strategy will fail.

The chapter introduces Hierarchical Risk Parity, aimed at three faults López de Prado finds in Markowitz-style optimisers: instability, concentration and underperformance. The root cause, he argues, is that those methods invert the covariance matrix, and the inversion magnifies small estimation errors.

He names the bind: “Markowitz’s curse is that the more correlated investments are, the greater is the need for a diversified portfolio, and yet the greater are that portfolio’s estimation errors.”

His method clusters assets into a tree by correlation, reorders the covariance matrix to match, and splits weight down the tree by inverse variance — with no inversion at any step, so it works even on a singular matrix. He reports Monte Carlo tests in which it delivers lower out-of-sample variance than the minimum-variance optimiser, and suggests using it to allocate across a set of machine-learning strategies.

López de Prado argues that the shift from one regime to another — mean reversion giving way to momentum, say — offers some of the best risk and reward, because most traders are caught out and react badly: holding losers, even adding to them, until forced out.

He divides the tests into two families: CUSUM tests, which flag drift from a baseline, and explosiveness tests, which look for prices rising or falling exponentially — bubbles, including sell-offs. The supremum augmented Dickey-Fuller test runs a unit-root test over every start date for each end date, so it needs no assumption about how many breaks there are. The point of the chapter is to turn these tests into model inputs.

The last feature chapter sorts market-microstructure models into three generations: those using prices alone (the tick rule, Roll’s spread model), those adding volume (Kyle, Amihud), and those built on the sequence of trades, starting with the probability-of-informed-trading theory of Easley, O’Hara and others in 1996.

That theory recast the bid-ask spread as the premium market makers charge for the risk of trading against better-informed traders. VPIN, a high-frequency estimate of it computed on volume bars from the imbalance of buys and sells, is one of the features the chapter offers. López de Prado notes the mixed evidence: one study found it a poor predictor of volatility, while several others found predictive power.

This site reads coverage, filings and prices, not order books, and builds no trading models. The book is here for Chapters 8, 11 and 12, which are the strongest argument we know for distrusting any number produced by trying many things and reporting the best. That applies to a fund’s backtest, a broker’s factor screen, and our own forecasting experiments alike: the honest figure is the one deflated by the number of trials behind it.

Two other ideas carry over to reading the record. Meta-labeling — a second model judging when a first one’s calls are worth acting on — is a fair description of what a reader does with an analyst’s or a fund’s public views. And strategy risk versus portfolio risk is the distinction to hold when a fund’s holdings look safe but its approach depends on a hit rate with little room to fall. The simpler, practitioner’s version of the backtesting chapters is in Quantitative Trading; the view from inside the large firms is in Inside the Black Box.