Inside the Black Box

Narang’s case for the book is that quant trading is widely misunderstood, and that people fear what they do not understand. He blames the quants too: in his view they are often needlessly secretive even about the broad outline of what they do, which breeds the distrust they complain of.

His working definition puts people at the centre. Quant trading, as he defines it, is the systematic carrying-out of strategies that humans devise through research. The machine repeats; the people decide what it should repeat.

The chapter’s main lesson for any investor is about discipline. A well-run quant shop tests a strategy that makes economic sense and then lets it run without arbitrary meddling. Narang argues that people are good at planning and bad at carrying out a plan under strain.

He uses the old trading rule “Cut losers and ride winners” to show the gap. Discretionary investors find losses hard to take and gains easy to take — the disposition effect. A systematic trader writes the rule into the program, so the choice is made, in Narang’s words, “at a time when there is no pressure”. The point is not that quants are better people, but that they decide before the moment that tests them.

He also sizes the field. Citing a combined Newedge and Barclay Hedge database, he reports that almost 90% of the assets run by commodity trading advisers sat with systematic firms as of August 2012, and puts quant futures money at $282.3 billion.

Narang draws the live system as five boxes. An alpha model, a risk model and a transaction cost model feed a portfolio construction model, which passes its target to an execution model. Data feeds the whole thing, and research is how it gets built.

The division of labour is the useful part. The alpha model forecasts; the risk model limits exposure to factors not expected to pay; the cost model says what trading will cost. Portfolio construction weighs the three and sets a target portfolio, and the gap between that target and current holdings is the list of trades.

He insists humans stay in charge. They pick the ideas, the instruments and the speed, and they usually hold what he calls a “panic button” to cut risk when markets move outside what the models can handle.

Narang’s starting premise is that no instrument is good or bad in itself, so none is worth owning or shorting forever. Alpha is skill at timing: knowing when to be long, short or out. Even a value investor is timing — buying cheap and selling once the price is fair. A stock that is always cheap, he notes, is almost certainly not worth owning.

He sorts price-based ideas into two opposite bets. Trend following expects a move to continue; mean reversion expects it to reverse. He grounds trends in the slow building of agreement among investors as evidence piles up. He adds a third, rarer kind he calls technical sentiment.

Fundamental strategies fall under three headings in his scheme: yield (or value), growth and quality. Together with the price-based ones, they make up his map of theory-driven alpha.

On combining signals he describes linear and nonlinear blends and machine learning. His judgement on the last is specific: machine learning works better and is used more for weighting existing forecasts than for finding new signals, though only a small share of quants use it to blend.

The chapter’s example is a value strategy left to its own devices. Stocks in one sector tend to cheapen together, so a hunt for cheap stocks can end up heavily long one industry. Narang points out there is no evidence of a lasting reward for overweighting a sector, so that exposure is risk, not alpha, and the risk model’s job is to find and limit it.

He describes two kinds of risk model. Theory-driven models use named factors that have an economic argument behind them, such as the market and sectors. Empirical models find the factors in historical data with statistical tools such as principal component analysis; in equities these usually rediscover the market first and the sector second.

He adds a caution for anyone assessing a fund: risk control does not always live in the risk model. An alpha model can be built to forecast only the exposures it expects to pay and to avoid the rest, so part of a strategy’s risk management may sit inside its forecasts.

Costs come in three parts: commissions and fees, slippage, and market impact. Fees are small and fixed and set a floor. Slippage — the price drifting between the decision and the trade — and impact — the price moving because of your own order — vary and are much harder to measure.

Narang notes a twist in slippage: the better the forecast, the more likely the price is already moving away from you before the order reaches the market.

He names four shapes of cost model: flat, linear, piecewise-linear and quadratic. Each is a different guess at how cost grows with order size, and the simpler ones are right only near the trade size they were fitted to.

Narang treats portfolio construction as a critical step that gets far less attention than finding new alpha. A trader who gives the most money to the worst ideas will do badly however good the ideas are on average.

He lists four common rule-based methods: equal position weighting, equal risk weighting, alpha-driven weighting and decision trees. Equal weighting has a defence he spells out. Mean-reversion signals are often largest for the names that have moved most, yet big moves often carry real news; capping each position limits the damage — what traders call “picking up nickels in front of a steamroller”.

The alternative is an optimiser, which seeks the most return for each unit of risk subject to limits. Narang calls optimisers simple in concept and harder to understand in detail.

The first choice an execution algorithm makes is how aggressive to be. Aggressive orders, usually market orders, fill at whatever price is there. Passive orders, a kind of limit order, wait in the book at a price the trader chooses.

Narang warns that passive orders carry adverse selection. A trader willing to cross the spread may know something, so a passive order often fills just before the price moves against it. Some exchanges pay for passive orders and charge for aggressive ones, which complicates the choice further.

The chapter opens with an epigraph from an anonymous futures trader: “I’d sell you my kids before I’d sell you my data, and I’m not selling you my kids.” Narang’s own version is “garbage in, garbage out”. He cites the Mars Climate Orbiter, a $200 million craft lost in 1999 when two teams worked in different units.

Cleaning data means deciding what is an error. Narang’s example is the 10-year German bund future, which fell about 1.4% in seconds on 28 March 2008 and recovered at once. A filter for spikes might have thrown that print out, but it really happened, so some quants have a person review flagged prices instead.

He defines look-ahead bias as assuming you could know something before it was knowable — “getting yesterday’s news the day before yesterday”. Quarterly reports are the standard trap: they describe a quarter that ended weeks before they were released, and a test that dates the figures to the quarter’s end trades on news nobody had yet.

Narang frames research as the scientific method: an idea, then a test of it on data held apart from the data used to fit it. In-sample testing gives the model a cheat sheet; out-of-sample testing takes it away. One check he describes compares how well the model explains the new data with how well it explained the old.

On overfitting he gives a clean example. A rule that buys the S&P 500 after a fall of at least 40% and sells at the next high backtests beautifully, but in his words it “would see exactly three trading signals in the past 40 years”. A handful of trades, however profitable, proves little.

He also tests how much results change with small changes in the settings. A strategy that works with one threshold and fails with its neighbours is fragile, and the fragility says more about its quality than the best result does.

Narang calls model risk the most basic risk a quant system brings — the chance that the model does not match the thing it tries to exploit. It may not show at once: small errors can build slowly and then break on a busy day. He adds exogenous shocks, such as government action or rumour, which carry information no model can measure.

The chapter’s centrepiece is August 2007. He stresses that there was no general panic: U.S. stocks were roughly flat over the first ten days of the month. Instead market-neutral funds sold hundreds of billions of dollars of positions, and the losses came from the cost of selling rather than from being wrong. Index levels barely moved. His conclusion: “For the first time, crowding became a risk of quant trading strategies.”

He traces the causes to how big funds ran money. Multi-strategy firms and bank trading desks chased recent results, pledged their strategies as security for one another, and set leverage by a risk target. When volatility and correlation rose together, the risk measures jumped and forced selling across the board. On the worst day, he reports, covering one short cost as much as 15% in market impact.

His story of one fund shows what holding on meant. Its investment committee could not meet until 13 August because members were on holiday, so the fund stayed fully invested; prices returned to their earlier trends once the selling stopped, and by the time the committee met the fund had recovered much of its loss.

The irony he points to fell on the small firms. Boutique quants running unusual strategies, with little overlap with the large ones, began losing only late in the second week. They then had to sell at the tail of an already huge unwind and paid dearly for it.

Narang takes the charges against quants one at a time: that trading is an art, that quants cause volatility by underestimating risk, that they cannot cope with sudden change, and that only a few large firms can thrive. His general reply is that most of these apply just as well to discretionary traders.

On sudden change he uses the spread between value and growth stocks. A trend favouring growth began in May 2007, and it reversed violently twice, once in July 2008 — among the sharpest reversals in the spread’s history. Most quants suffer in such turns because the pattern their forecasts rest on flips; a few short-term strategies aim to profit from them.

He rejects the idea that size always wins. Large funds pay heavily to adjust big books in a crisis, as 2007 and 2008 showed, and whole classes of strategy — stat arb in smaller markets, high-frequency trading — are too small to be worth their while.

His summary is balanced: some quants do bad science, overfit or crowd into the same trades. But “Done well, quant trading can produce superior risk-adjusted returns and substantial diversification benefits.”

For investors judging a quant, Narang says the edge must sit in one or more of six parts of the investment process, and the investor must find which. Saying “stock picking” names the activity, not an edge. For quants, he finds the edge most often lies in research skill or in getting and cleaning data.

He spends as much time on how a quant reacts to losses as on the strategy. A knee-jerk cut can throw away whatever edge a strategy had; at the other extreme, a quant may miss that a sound strategy is heading for ruin. He reads LTCM’s error as forgetting Keynes’s warning that markets can stay irrational longer than an investor can stay solvent. Good monitoring, he argues, will rarely be the edge, but poor monitoring can lose one.

He is blunt about credentials: “the fact that a quant has a PhD in physics (or anything else) is no indication of quality or skill.” Some of the best quants have no advanced degree, he adds, and some of the worst failures had Nobel Prizes.

Narang separates two things the press runs together. High-speed trading is the infrastructure. High-frequency trading is a set of strategies that need that infrastructure, hold positions for less than a day and usually end the day flat. Execution algorithms use the same fast plumbing without being high-frequency strategies.

Narang explains why speed matters by order type: placing a passive order, placing an aggressive one, and cancelling a passive one. In each case a fraction of a second decides the outcome.

His example for aggressive orders has two traders sending the same limit order to buy at the best offer. The first to arrive takes the shares. The second, a moment later, finds the offer gone and becomes the new best bid instead — the same order, a different result, decided by arrival time.

He sorts high-frequency strategies into four groups: contractual market making, non-contractual market making, arbitrage and fast alpha. Fast alpha uses the same ideas as Chapter 3 — momentum, mean reversion, technical sentiment — on intraday price data.

The chapter asks four questions: whether high-frequency trading is unfair competition, whether it leads to front-running or manipulation, whether it adds volatility or instability, and whether it has social value. On the 2010 Flash Crash his verdict is that high-frequency traders were not to blame, though he grants that software faults at such firms can and do cause trouble.

Narang closes with the view that so-called black boxes are systematic versions of what traders have always done, and that the backlash against them is largely a matter of a generation adjusting.

He points to three directions: hybrids in which analysts feed views into a system that builds the portfolio, sentiment strategies built on internet data such as Twitter and blogs, and “mid-frequency” strategies that hold for an hour or more but need high-frequency infrastructure.

The August 2007 chapter is the reason this book sits here. Narang’s point is that a position can be right and still lose money when many holders own the same names and one is forced to sell. That is the question this site asks when it reads who holds a stock: not only what a fund owns, but who else owns it with them.

The parts list — alpha, risk, cost, construction, execution, data, research — is also a checklist for any claim of edge. When a firm says it has one, ask which box it lives in. The Physics of Wall Street tells the history behind these tools, and When Genius Failed is what crowding and leverage did to one fund in 1998.