The Physics of Wall Street

Weatherall opens with Jim Simons, a geometer whose fund, Medallion, he presents as proof that a scientist can beat the professionals. The numbers he gives: a 2,478.6% return over the fund’s first decade, against 1,710.1% for George Soros’s Quantum fund over the same span. Renaissance, Simons’s firm, staffs itself with about two hundred people, a third of them holding PhDs in physics, mathematics or statistics rather than finance.

The foil is August 2007, when quant funds lost heavily in a single week and Goldman Sachs put $3 billion into its own funds to steady them. The press called it the quant crisis. Weatherall says he began the book in the autumn of 2008, struck by how often the coverage blamed physicists and their models for the wider collapse.

His thesis answers that charge. He states it plainly: “physicists and their models are not to blame for our current economic ills”. But he does not let the industry off. His complaint is that better tools existed before the crisis and that few banks, funds or regulators used them — that even sophisticated funds ran older methods when newer ones were on the shelf.

The chapter starts in Paul Samuelson’s office at MIT around 1955, where he finds a dissertation from 1900 by Louis Bachelier, a Frenchman he had never heard of. Bachelier had worked out much of the mathematics of random price movement half a century before economists adopted it.

Weatherall treats the thesis as a lesson in how ideas get lost. Poincaré admired it, but the committee would not give its top grade to mathematics on a subject outside mathematics. Bachelier got “honorable”, not “très honorable”, and with it lost his prospects as a professional mathematician. His work came back into circulation only when Paul Cootner put an English translation at the front of a 1964 collection on the randomness of stock prices.

The point Weatherall most wants a reader to keep is that randomness was not Bachelier’s end goal. He wanted a price for options, and built it on the idea of a fair bet — a price at which neither buyer nor seller expects to gain on average. Assuming prices move at random was what made the formula work, not an admission that nothing could be known. Weatherall adds the catch: Bachelier gave no method for turning the formula into a trading strategy, and options were too exotic for decades for anyone to care.

M. F. M. Osborne, a physicist at the Naval Research Laboratory, published “Brownian Motion in the Stock Market” in 1959, when Bachelier’s work was still almost unknown outside Samuelson’s circle. Weatherall credits him with two advances: he was the first to show that the log-normal distribution matters for markets, and the first to state clearly how a random model yields probabilities for future prices and returns.

His fix to Bachelier was small and practical. If prices themselves wander at random they can fall below zero, which real share prices cannot. Osborne instead let rates of return wander, and found the data fitted better.

What Weatherall admires most is Osborne’s habit of revising his own model. When volume turned out to vary, and when Osborne and a colleague found in the mid-1960s that a small rise in price was more likely to be followed by a fall than another rise, he changed the assumptions rather than defend them. Weatherall calls this a hallmark of the work and the reason Osborne matters to the story.

The chapter’s sharpest line is about what happens when traders skip that step: “If you continue to trade based on a model whose assumptions have ceased to be met by the market, and you lose money, it is hardly a failure of the model. It’s like attaching a car engine to a plane and being disappointed when it doesn’t fly.”

Benoit Mandelbrot enters through Hendrik Houthakker’s cotton data, which did not fit a normal distribution. Mandelbrot, a student of Paul Lévy’s later work, recognised the pattern as one of Lévy’s stable distributions, with fatter tails than the normal curve. His reading of the result, in Weatherall’s words: “cotton prices were random, all right — far more wildly random than Bachelier or Osborne could have imagined.”

Weatherall explains the scale. Lévy-stable distributions carry a number, alpha, that measures how wild they are: 2 is the normal curve, 1 the far wilder Cauchy curve. Cotton came out at 1.7. Mandelbrot went on to find alphas below 2 in other commodities, stocks and bonds.

The verdict on Mandelbrot is split. Later researchers confirmed his main claim, that normal and log-normal curves miss how often large moves happen: returns have fat tails. His specific claim, that markets follow Lévy-stable distributions with infinite volatility, did not hold up as well. Weatherall also defends the field for carrying on with simpler models in the meantime — start with the simplest theory that works, push it, then ask where it fails.

Edward Thorp worked out how to count cards at blackjack. In Las Vegas, with two backers he later called Mr. X and Mr. Y, he turned $10,000 into $21,000 in a little over thirty man-hours of play, and published the system as Beat the Dealer.

Counting told Thorp when the deck favoured him. Claude Shannon showed him John Kelly’s paper on how much to stake once you know that. The Kelly rule sizes each bet by the player’s advantage relative to the payout, and says not to bet at all when there is no edge. Weatherall sums up Kelly’s result in three words: “Information is money.” Thorp and Shannon also built a concealed device to beat roulette and took it to Las Vegas in 1961, with modest results.

Thorp then turned to warrants. He saw that working out a warrant’s “true” price was like working out the true odds on a horse race: a gap between that price and the market price was an edge, and the Kelly rule told him how to use it. Weatherall states that the method Thorp worked out in 1965 was equivalent to the Black-Scholes formula, though Thorp computed prices with a program rather than writing the closed-form equation, and reasoned from Bachelier’s fair bet rather than from a hedge.

Fischer Black, Myron Scholes and Robert Merton reached the options formula from the hedge: hold the right mix of stock and option and the risk cancels. Weatherall stresses the timing. The SEC approved the Chicago Board Options Exchange on 14 October 1971, and the exchange opened about a month before the Black-Scholes paper appeared. On its first day nine hundred options changed hands on sixteen stocks; well over a million traded in 1973.

The 1987 crash shows what happens when a model’s assumption gives way. Portfolio insurance sold stock automatically as prices fell, which works only if someone is buying. When everyone with the insurance tried to sell at once, there were no buyers, and the computers sold far below the prices their designers had planned on. Weatherall notes that many people think the automatic selling made the fall worse.

His counter-example is O’Connor and Associates. Michael Greenbaum and a teenage hire, Clay Struve, built a version of Black-Scholes in 1977 and 1978 that allowed for sudden jumps in price — fat tails. According to Struve, the firm saw the “volatility smile” in prices before the crash. O’Connor survived 1987. Weatherall’s moral is that the crash was not unforeseeable: a firm that thought about where the simple model breaks could prepare for it.

He does not hide the failures. Long-Term Capital Management, with Scholes among its partners, lost $4.6 billion in under four months in 1998. And he states the general rule in his own words: “Models fail. Sometimes we can anticipate when they will fail, as Greenbaum and Struve did; in other cases, we figure out what went wrong only as we are trying to put the pieces back together.”

Doyne Farmer and Norman Packard began as graduate students building hidden computers to predict roulette, under the name Eudaemonic Enterprises. By the summer of 1978 the devices were working well enough to take to Las Vegas and win. The roulette work drew them into chaos theory, which became their academic careers.

They left academic posts to start the Prediction Company with one goal: to predict financial markets. Their field was nonlinear forecasting — finding short windows when a chaotic process has enough structure to say where it goes next.

Weatherall describes the method as modest. The firm scanned market data for patterns with some short-lived power to predict, traded them while they lasted, and built tools to tell when a pattern had faded. The firm never published results. Weatherall reports what he could gather: it remained a subsidiary of UBS more than a decade on, and one source told him its risk-adjusted return over its first fifteen years was almost a hundred times the S&P 500’s. Farmer left in 1999 and Packard, as chief executive, in 2003.

Didier Sornette came to markets from the study of how materials break. Working for Aérospatiale on pressure tanks, he found in 1991 that a rupture is preceded by a pattern of vibrations whose peaks bunch closer together on a log scale — “log-periodic” — and that the pattern lets you estimate when the break will come. Aérospatiale patented the method.

He then applied the same idea to crashes. Weatherall summarises Sornette’s point that standard economics expects a crash to need some large piece of news, yet for many crashes no such news can be found. That suggests the cause lies inside the market, in a build-up of herding that makes a break likely.

The test case Weatherall gives is the Japanese market. On 25 January 1999, with the Nikkei near a fourteen-year low, Sornette posted a paper saying it was in an “anti-bubble” and would rise 50% by the end of the year. Weatherall reports that it did, by exactly that amount.

Sornette’s term for the extreme events he thinks can be seen in advance is “dragon kings”, set against Nassim Taleb’s black swans. Weatherall puts the difference in one line: “They are tyrannical when they appear. But unlike black swans, you can hear them coming.” Sornette does not claim every crash is predictable — only that many events that look like black swans give warning first.

The last chapter changes subject from prediction to measurement. Eric Weinstein and Pia Malaney took on the index number problem: how to reduce the cost and quality of many goods to one number, such as the consumer price index, that can be compared across time and place.

Weinstein had once doubted that mathematics had much to offer economics. Arrow’s impossibility theorem changed his mind — a proof that no voting system can turn every individual’s ranked preferences into a fair ranking for the community. A field that could produce that result, he decided, could use more mathematics.

Their tool was gauge theory, the branch of physics that compares quantities that seem incomparable — in its first form, the lengths of rulers in different places. Weatherall notes that Jim Simons did much of the early mathematical work on modern gauge theory. Malaney took the idea to her adviser, the Harvard economist Eric Maskin, and made it her dissertation. The question it poses is how to compare what money is worth to a lumberjack in 1950 and a programmer in 1995.

The epilogue restates the method the book has followed: make simplifying assumptions so a problem can be solved, then go back and ask what happens when the assumptions change. Sometimes the first answer turns out useless, sometimes it needs a small fix, and sometimes it is good only within limits that must be learned.

Weatherall rejects the idea that this method marches toward a final theory of markets. The goal, as he puts it, is more modest: “You’re trying to find some equations that give you the right answer some of the time, and to understand when they can be relied on.”

He closes the argument with two firms. O’Connor survived 1987 by using its models with more care than its rivals, and Renaissance returned 80% in 2008. In his telling, both did it the same way — by questioning their own assumptions and hunting for the weak points in their models.

Each chapter is the same story told about a different model: an assumption that held for a while, a moment when it stopped holding, and a question of who noticed in time. Portfolio insurance in 1987 and the funds of August 2007 failed because a model met conditions it was never built for. O’Connor’s jump model is the counter-case. That is the habit this site asks of a reader of any valuation or coverage chart: name the assumption, then watch for the day it stops holding.

The book pairs with two others on the shelf. When Genius Failed is the long version of the LTCM loss Weatherall covers in a paragraph, and Inside the Black Box describes from the inside how quant funds are built.