Requiem for efficient

For the past 30 years, the efficient market hypothesis has been one of the most cherished tenets in those campus buildings where economists live. There are a few tiny discrepancies between theory and reality. Exactly how rational were those people who were eagerly bidding up shares of Amazon.com when it was already selling at a total market cap roughly the size of the GDP of Iceland? When people crammed their portfolios with Dell Computer at a multiple which could only be justified if they expected Dell to grow to become the world’s only computer supplier eight years down the road, were they, perhaps, paying too much attention to recent performance? You might also wonder why hordes of rational investors walk into casinos, play the lottery or seek investment advice from somebody who is not held to fiduciary standards – but despite all this skepticism, the foundations of most of today’s portfolio management activity still rest on the efficient market hypothesis.

In the next few weeks, the publishing industry will have given us two books that, paired together, blow the doors off of conventional market theory. They do it in very different ways, but both acknowledge that if you want to study and understand the markets, instead of examining the movements of stock prices, you have to shift your attention to the people who are actually making the trades.

Money magazine columnist Jason Zweig’s upcoming "Your Money and Your Brain" (You can preorder it from Amazon.com) provides the best and most detailed summary of what the behavioural finance researchers have uncovered about our dysfunctional investment behaviour. But then he goes a LOT further, and searches for the origins of these behaviours in the actual structure and workings of the human mind, interviewing neuroscientists around the world, and also participating himself where possible.

Meanwhile, one of the most important contributors to the efficient market hypothesis, Stanford University professor and Nobel Laureate Bill Sharpe, has created an entirely new model of investor behaviour in a book, "Investors and Markets" that he clearly hopes will become an advanced-level economic textbook. It’s one of those books you have to be familiar with before you can hope to follow the next generation of economic theory and the academic spill over into our profession.

Remarkably, although they take very different paths to reach it, both books lead you toward the same conclusion. Your investment experience is likely to be highly personal, and your chances of outwitting the market are not high – not because the markets are efficient, but because you are not and never will be.

Let's begin with Sharpe. The Nobel-prize winning professor says, in the very first pages of his discussion, that his goal is to "explore the relationships between investors' characteristics and investment opportunities." A few pages later, he breaks from the way traditional investment research has been conducted, telling us that: "Prices are not determined by random number machines. They come from recent or prospective trades by real people. Investors who fail to fully take this into account do so at their peril."

To understand the behaviour of the people at either end of stock trades, Sharpe identifies a set of characteristics that each of us possess in our investment lives.

The first is an individual's "position" – more broadly, the circumstances a person is in, including age, geographic location, financial situation and so forth. Two people in different circumstances might choose very different portfolios, even if they agree on what underlying prices should be. Since supply and demand are different for the two of them, the asset price will most likely settle somewhere in the middle.

The second is the individual's "preferences" – akin to risk tolerance. You may prefer less volatility in your portfolio than I do, and we may have different experiences with investments that lead us to avoid or prefer this or that asset class.

Finally, one investor's "predictions" will differ from another’s. Later, Sharpe introduces the concept of state theory. A state can be thought of as the atomic structure of the expectations of a given investment. The stock might rise from $1 today to $3 next year and continue this rise more or less forever along the same trajectory. (This is apparently what people were expecting when they bought Dell shares during the tech boom.) Along the way, it may pay certain dividends. Or (different state), it might go into bankruptcy and default. Or it may languish at $1 a share for the next 15 years and pay no dividends.

Sharpe defines the expected utility of owning this stock as a composite of each of the possible future states, where you multiply the future economic value of each state by the chances of it happening. Most investors, he says in passing, don’t have a complete inventory of all the possible states in their heads as they buy and sell – in fact, they may be fixated on only one or two, or a narrow range. In that case, they will buy at (or sell for) a price that is different from the stock's expected utility. From an economist’s viewpoint, they have made an irrational decision. And, since all the possible states cannot possibly be known, and all the probabilities are just guesses anyway, there is a wide latitude in the plausible prices an investment can trade for. Today, I buy tech stocks at a premium and the Nasdaq index is booming. A few weeks later, it’s trading at half its former value, and, having adjusted my expectations, I am dumping my shares. All of this is plausible, but it’s not very efficient in the classical sense.

People with different positions, preferences and predictions all have a goal of sharing in future rewards – of, in other words, participating in the beneficial things that happen in the market.

As consumption grows, as you have more of something, you will tend to have less desire for it. Sharpe draws some graphs of curved lines which assume that, for one investor, the desire for something (the marginal utility, more precisely the need for something) drops roughly 1.5% for every 1% increase in consumption (the amount he or she already has). Another person has a marginal utility that decreases by approximately 2.5% for every 1% increase. In both cases, you can define their growing or declining interest in stocks, fish or anything else by an equation which can be modified in a variety of interesting ways.

For example, a wealthy person and a much less wealthy person enter the stock and bond markets. Because she is way out on the marginal utility curve, the wealthy person will have a diminished appetite for stocks, and will tend to choose a less volatile portfolio than the less wealthy person person who aspires to become wealthy and wants the growth stocks offer. A few equations later, both settle on portfolios with minimum variance per unit of expected return (Sharpe calls them efficient portfolios), and Sharpe makes the case that if you can graph the utility function and the efficiency of different portfolio mixes, and assume similar preferences and expectations, you can predict what each investor will ultimately hold.

The book then moves to modeling expectations by, first, making the simplifying assumption that future returns for each state and the probabilities of each, are known (and agreed upon) by all. The underlying security is thus defined by its price per chance (PPC). An insurance agent proposes to sell you a policy that will pay you $1,000 if your computer is stolen this year, and the coverage costs $60 – or six cents per dollar of coverage. If you think there is an 8% chance your computer will be stolen, the PPC is .06/.08, or .75. You are paying 75 cents for each expected dollar of return. If you think there is a 4% chance of your computer being stolen, the price per chance is .06/.04 or $1.50 for every dollar you expect to get back.

Securities markets are not quite that simple, of course. Sharpe talks about market states, each of which may or may not occur, and breaks out the different returns you might expect in four of them – recession, normality, prosperity and boom. In his simplified example, you could buy a combination of securities that would give you a weighted average expected return of x%, and won’t generate a return below y% under any of these future market scenarios. Then, Sharpe wades into territory whose existence the efficient market theorists deny, showing that it is possible to take on additional market risk and come away with a higher expected return.

But what if different investors have different expectations about the future states – either the probabilities of them, or the returns that will be generated in any or all of them? Sharpe introduces this new variable into his calculations, and makes the most controversial assumptions in the book. He cites some anecdotal studies (one in the early 1900s where people at a fair paid a sixpence to judge the weight of a cow) to suggest that although virtually all investors will be wrong in their future assumptions, their aggregate assumption will be approximately correct, and the more participants there are, the more correct they will tend to be.

Modelling his assumptions, Sharpe looks at situations where some of his traders are slightly better than the average at predicting which of the future states will occur, and some, of course, are not as good. They all have different risk preferences, so their returns, created by a Monte Carlo simulator, plot above or below the average line along the risk/return spectrum. Those who are able to make superior predictions sometimes end up above the line, sometimes below, as do those who aren’t quite so prescient. The point? Unless you have an infinite timeline to evaluate, skill is often obscured by luck, and there are times when even investors who know more than the market don’t enjoy the benefits of their superior talent. Sharpe's conclusion, and the best line of the book, is that "it is difficult to be a superior active manager in a world in which inferior active managers are plentiful, error prone, but unbiased."

Sharpe is willing to entertain situations that are heresy in an efficient market universe. He explores the possibility of earning outsized returns if you can somehow identify biases in most investors (when, for example, they are fixated on recent past returns or recent news), and offers a generally skeptical view of downside-protected investments. ("The bad news is that taking all possible markets into account, protected investment strategies may offer lower expected returns than available from equal-beta linear market-based strategies.")

At the end, there is a chapter on how to give investment advice by trying to find the most efficient portfolio given a person’s preferences and position. The most interesting part of this last chapter is a caution about choosing portfolios based on historical returns. Taking 25 years of market data, and creating a thousand possible 25-year histories, Sharpe shows that optimal asset mixes based on historical estimates will turn out to have been significantly suboptimal in future markets more than half the time. He consistently suggests that index funds will provide superior returns, on average, to actively managed portfolios, but he doesn’t recommend a passive allocation strategy. In fact, he says that this is actually a riskier “bet” (his word) than making macroeconomic forecasts and changing the portfolio mix according to your ever-evolving beliefs that future states will occur.

So Bill Sharpe is advocating, in dense academic language, that investment advisers and financial planners become active managers of their client portfolios. As he puts it: "Those who wish to avoid betting should periodically use current market values to form new macroconsistent forecasts, then determine an up-to-date set of asset holdings."

How realistic will this be? We can say, as kindly as possible, that even these advances on the one-dimensional efficient market traders are crude approximations – but we don't have to, since the author makes this point himself, lamenting that "investors have different horizons, owing to differences in age, health, the ages of their children and so on. To adequately take such diversity into account presents a huge challenge to anyone trying to build an analytic model or computer simulation of the equilibrium process." You think?

When Nobel Laureate Bill Sharpe comes out in favour of active decision-making in investment portfolios, and shows the world that his Capital Asset Pricing Model is a simplified (read "incomplete") version of a broader mathematical description of the investment markets, you know that the conventional wisdom is going through a period of radical adjustment.

For academics (and people who want to invest like them), the new game will be to gain a clearer understanding of the probability of future states or future returns associated with each state.

For financial planners, an evaluation not only of preferences, but also position, is already part of the process. We can expect a lot of new attention from academics on how these are defined in the real world. And, perhaps, advisers who are moving away from planning in favour of investment management may rediscover that planning was always a necessary part of the investment process.

My takeaway from a slog through the mathematics is that we are moving into a world where some estimates will be better-informed than others, but we’d almost certainly have better luck finding those superior thinkers by sitting down with them and evaluating their intelligence, rather than trying to parse out luck from skill in the past return data. You’re going to hear a lot of debate among economists around the edges of Sharpe’s model, and this will undoubtedly lead to more sophisticated ways of looking at investment decisions. Sharpe has replaced the paper doll investor model with something very like a three-dimensional robot, with mathematically definable positions, preferences and positions.

Part two of this review, focusing on Zweig’s book, is available to Bob DownUnder subscribers. Bob is one of the most respected and influential financial services industry commentators in the US. In NZ, financialalert has exclusive access to and distribution rights for Bob’s Inside Information e-newsletter service reviewing the most important articles published in US financial planning trade and business magazines, and summarising the best thinking from 10 US industry conferences a year.

Shared membership details

Read the paper