How to forecast markets?

The theory and empirical literature of Finance are the best starting point as they deal directly with asset prices. Next are macroeconomics and statistics. Markets are not Math or Engineering, but a forever learning and adapting system with all of us observing and participating from the inside. Quantitative techniques are indispensable, though, to deal with the complexity of financial instruments and the overload of information we face. Empirical evidence counts for more than theory, but you need theory to constrain empirical searchers and avoid spurious correlations.
The starting point of Finance is the Theorem of Market Efficiency which posits that under ideal conditions what we all know should be in the price. Only new information moves the price. Hence, it is changes in expectations about the future that drive asset prices, not the level of anything.

How to forecast markets?

The theory and empirical literature of Finance are the best starting point as they deal directly with asset prices. Next are macroeconomics and statistics. Markets are not Math or Engineering, but a forever learning and adapting system with all of us observing and participating from the inside. Quantitative techniques are indispensable, though, to deal with the complexity of financial instruments and the overload of information we face. Empirical evidence counts for more than theory, but you need theory to constrain empirical searchers and avoid spurious correlations.

The starting point of Finance is the Theorem of Market Efficiency which posits that under ideal conditions what we all know should be in the price. Only new information moves the price. Hence, it is changes in expectations about the future that drive asset prices, not the level of anything.

How to forecasts view changes? The good news is that changes in opinions about fundamentals such as growth and inflation tend to repeat. This is one driver of momentum in asset prices, and is likely driven by the positive feedback between risk markets and the economy that forecasters naturally find very difficult getting ahead of.

I live by Occam’s Razor: If you can explain the world with one variable, don’t use two. This keep-it-simple rule does not deny that reality is complex, nor does it say anything about simple minds. It forces one to focus on the most important fundamental drivers of markets and to cut out the clutter. It reduces the risk of becoming a two-handed strategist.

The mode and the mean. There is a fundamental difference between an asset price and a forecast. A forecast is a single outcome that you consider the most likely, among many. In statistics, we call this the mode. An asset price, in contrast, is closer to the probability-weighted mean of the different scenarios you consider possible in the future. When our own probability distribution for these different outcomes is not evenly balanced but instead skewed to, say, the upside, the market price will be above our modal view. Asset prices can thus move without a change in modal views if the market perceives a change in the risk distribution. An investor should thus monitor changing risk perceptions as much as changing modal views.

Do markets get ahead of reality? They do, yes, exactly because asset prices are probability-weighted means and the reality we perceive is coded as a modal view. Information arrives constantly and almost always only gently moves the risk distribution around a given modal view. Before we change our modal view of reality, the market will have seen the change in risk distribution and will have started moving already.

Are some markets faster than others? I hear frequently in one market, say equities, that they are monitoring other markets, such as credit or bonds, for early signs on what stocks will do. But I hear the reverse frequently in the bond world. I do not like either view and just assume that all markets react at the same speed as they see all information at the same time.

Levels or direction? In our business, we are asked to forecast asset prices and returns. I have found this very hard but fortunately have had the luxury to be able to stick to forecasting market direction rather than outright asset price levels. In markets that are close to efficiently priced, what we know is already in the price and we cannot really use that same information to make a coherent case for an asset price level that much different from today. All I have been able to do is to make a case that there are mild-to-decent odds in favor of the market going in one direction rather than the other. We have been much more successful in forecasting direction than actual asset price levels, and it is the direction that is more important for strategy.

Top down or bottom up? In assessing the outlook for a market or an economy, should you start judging individual countries, sectors, and companies and then add them up to the overall market, or should you start from the top down? As a macro strategist, I naturally think top down, arguing I sit on top of a tall building, seeing where all the traffic and capital is going. But I know that from that high up, I do not see any potholes. For that, I have been relying on my local analysts to tell what conditions prevail on their street. And they in turn ask me what I can see from high up. I have found that it is the dialogue between bottom-up and top-down thinking that is most fruitful. Our economists do this quite well: they start the global forecast from the country level up, but then look at a host of global signals to put pressure on the bottom-up forecasts.

The US as the indispensable market. Applying this top-down thinking, should we therefore start strategy at the global level and then drill down to regions and sectors, or should we follow the more common approach of starting with the USD market and economy, and then analyze the rest of the world as a spread market? I have done the latter. This is not only because we have the longest return series in the US and the US market and economy have been more stationary than others, but also because dollar assets are half of the investable world as many non-US entities both fund and invest in dollars.

Rules versus discretion? You need both. I have tried to have logical arguments to buy or sell certain assets, based on Finance. And I have tried to corral evidence that the signals I use have in the past had the assumed impact on asset prices. Each of these then became a rule, of the form: If X>0, buy A, and vice versa. As we collected these rules, and published them in our Investment Strategies series, the question came up naturally whether we should not simply make our investment process driven by a number of empirically proven rules, and to banish any discretion (emotion?) from the process. Over time, we converged on a mixture of the two as pure rules ran into the problem that the world is forever changing, partly as everyone else figures out the same rule and then arbitrages away the profit, and partly as economic structures and regimes similarly change over time in a way that we cannot capture with simple rules.

Much as I have been talking a lot about cycles, I do not think of the world as a stationary system described by a set of parameters that we steadily get to know more about. Instead, as economists we think of people constantly optimizing their objectives, under the constraints they face. Aside from truly exogenous shocks to the system, the main difference between today and yesterday is that today, we know what happened yesterday and that information allows us to constantly fine tune and thus change our behavior. That is, we constantly learn from the past, much to try to avoid making the same mistakes. At the macro level, this means that the system is constantly evolving. As Mark Twain said, “History doesn’t repeat itself, but it often rhymes”. As investors, we should look at the market as billons of people all learning and adapting. The best investors are those who get ahead of this by learning faster and understanding better how others are learning.

Expectations are adaptive. Markets should be purely forward looking into the future and treat the past as just that, the past. The problem we have is that the only information we receive is from the past. Ages ago, a debate raged in economics on whether expectations for say inflation are rational, or adaptive. The term rational was meant to denote that investors plug in all the info they have into their model of what will drive the future and derive from that the most efficient forecast. That is, investors do not slavishly extrapolate the past. True in principle. But we also find that as new information rational priors as new data steadily challenge them. In effect, then, market expectations for future fundamentals on earnings, inflation, defaults and such come close to adaptive, moving averages of past performance.

Risk premia are about risk and uncertainty. This sounds obvious, but is frequently overlooked. It means that even when nothing surprising is happening, that by itself is surprising against markets that are priced for a certain volume of surprises. When nothing happens and data come out as expected, the market updates in an adaptive sense its uncertainty, and risk premia comedown.

Flows, positions, and supply and demand.

Economics teaches us that supply and demand determines price. That is true also for asset prices, and explains the high interest in information on flows. Applying this dictum is not easy, though, as we cannot measure future intended supply and demand, aside from governments’ budget plans. All we measure ex post is transactions at a price that then equated supply with demand. For every seller in the past, there was a buyer, with the price moving to create this equilibrium. Only the movement in prices can tell us whether intended demand exceeded or fell short of supply. Given that we know how prices changed, flow data do not tell us much more.

I have a different gripe about position surveys. If you tell me that you are long or OW asset class X, then I must conclude investors are long and advise you to sell. You know that, and thus should not tell me that you are long. I thus do not “trust” survey data.

This is not to say that flow and position data are useless. We instead find that more detailed understanding of how different types of investors, each with their own restrictions and objectives, interact with the plumbing of the system, has allowed us to make better investment decisions.

Central banks and QE do not “cause” asset price inflation. It is often argued, and our own language has come dangerously close to it, that easy money by central banks has massively and artificially inflated asset prices and that a QE unwind will thus deflate them. I do not like to think in those terms. Easy money may be the proximate cause of high asset prices, but is not the ultimate one. All central bankers try to do is to search for the non-inflationary equilibrium level of rates driven by the supply and demand for capital as well as inflation expectations. In this cycle, higher global savings from EM and corporates, depressed capital spending, consumer delevering and public sector austerity have created a surplus of savings over investment that is the real cause of low interest rates and high asset prices. If central money was too easy, we would have also seen much faster growth and higher inflation, which we did not get.

Market volatility is not a mystery but should be thought of as fundamental volatility, of growth, earnings, inflation, plus technical forces which are largely due to leverage, positions, market plumbing and such. Another way of looking at vol is as a function of the number of shocks and surprises hitting the system, the propagation and contagion forces around them (mostly leverage) and the shock absorbers that counteract them (largely central banks).

Where is alpha?

The Theorem of Market efficiency, which implies investors can’t beat the market, implies that asset prices will follow random walks, with drift and that asset price changes will be white noise, with no serial correlation. There are thus only two possible inefficiencies to be exploited: positive serial correlation, which we call Momentum, or negative serial correlation, which we call mean reversion, or Value (to become valuable, asset prices need first to go down, or fundamentals need to improve faster than the price). It is an empirical question which dominates where. At the asset class and sector level, we have found that Momentum dominates, while within the fixed income world, Value is more important.

The Theorem of Market efficiency assumes frictionless markets. Hence, cross-sectionally, we need to focus on areas where there are frictions due to different regulations, business practices, or investment objectives. Most profitable for me have been differences between currencies and industry segmentation between HG and HY, EM and DM, and bonds and equities.

Across time, market momentum at the macro level has been the best way to earn excess returns. I discussed above how some of this is due to the momentum in view changes. More fundamentally, in open markets, we frequently face a Fallacy of Composition according to which rational and equilibrating behavior at the micro level becomes destabilizing at the macro level. The free market is very good at motivating entrepreneurship and rational behavior at the micro level, but is subject to constant booms and busts at the macro level. Central banks try to control this instability through countercyclical policies but can’t undo it all.

Trade the risk bias. Even when markets price in exactly our modal views, I find it useful to consider how prices will move on new information and then try to position on any skew in the outlook. If I find that a particular price or spread will move a lot more on bullish than on bearish news, then I will position bullishly. This works at the portfolio level if I can combine different unrelated risk biases.

Is there now so much information that everyone sees at the same time that alpha is dead? To some extent, yes, as reflected by the inability of the hedge fund world to offer better returns than a simple bond and equity portfolio with the same volatility over the past 10 years. Still, while alpha is weaker, I don’t think it is truly dead, as allocation across asset classes is still working well, even as it seems harder to earn alpha within asset classes.

Is passive investing destroying alpha? No. it should actually make it easier if a lot more investors choose to allocate passively and therefore leave opportunities to the reduced number of active managers. I do feel the move to passive is largely within asset classes (i.e., stock picking) and that the arrival of liquidity passive products (ETFs) has made active asset allocation a lot easier. I think many managers have moved from active stock picking to active asset allocation.

How to analyze risk? Risk is not the same as past vol, but the surprise that will hurt your portfolio. I have never found it useful to make long list of all the things that can go wrong over the next year. Instead, I start from the premise that the big risks that will have an impact at the macro level almost always start as small ones. I have called these local brush fires, of which there are always a bunch and of which I need to decide which will become a wildfire. This does not solve the problem fully but at least reduces the number of risks to monitor.


I have generally ignored these risks, primarily as I do not have a model to understand or project them. When they do become market relevant, they typically hit us so fast that is too late to do much about them.

How to put it together?

I like a Lego approach to TAA of one trade at the time. In theory, an active money manager should translate their ideas into expected returns and risks and then use portfolio optimization to calculate an efficient frontier of the highest return portfolios by levels of risk.

I started that way decades ago as a young strategist and ran into numerous problems of how to assess all the necessary return, volatility and correlation parameters over multiple horizons. I found that the more assumptions you have to make, the greater the probability of putting in numbers for which you have no idea. The well-known Black-Litterman approach tries to deal with this from a Bayesian point of view, starting with the parameters implied by market outstandings, but I had problems with why these parameters would make sense, and how to dynamically change portfolios on constantly incoming new information and ideas.

I then moved to greatly simplify my process of converting views into portfolios in two ways. First was to postulate that any active portfolio is a passive benchmark portfolio plus a number of zero sum deviations of under- and overweights against that benchmark that I think about as indifferent to what benchmark is used. That allowed me to separate the active overlay portfolio from the underlying benchmark and give each global investor the same OW/UW advice, irrespective of their benchmark.

The second simplification was to think of each single active view as a single trade that needs to stand on its own, with its own drivers and logic. If the latter turn, I exit the trade, without changing the other trades.

Does that mean I ignore correlations? Yes and no. When building a portfolio of active trades, I start with a target overall active risk (e.g., 1% VaR). The lower the correlations between my different trades, the higher the VaR I can allocate to each individual trade. But as we actively turn off trades and add new ones, I will not constantly move the whole portfolio around.  

This is partly as I find correlations unstable and hard to forecast. The past correlation between two assets or positions depends on what was driving them. Bonds rallying because of monetary easing will be bullish for stocks and the equity bonds correlation will be positive. Bonds gaining because of low inflation on weak growth will correlate negative with equities. I am very wary of extrapolating past correlations and will generally not base recommendations on them.

Sizing risk by track record and hot hands. It is not only important to have the right trade on but also to make sure to have the right amount of risk allocated to each. I start with a target amount of tactical risk which I think about in Value of risk, in dollar terms or percent of AUM. I then decide whether today is a good time to take a lot of risk, or a bad time. If we are been on a roll making money, then we probably have a better sense of the direction of markets and I then take more than average risk.

Next comes deciding where to take this risk. I look here at track records, both long term and more recent. I have found over the past 30 years that certain areas are “easier” to make money than others. They are broad asset allocation (risk on, risk off), cross country in bonds and FX, and credit spreads. The harder ones are bond duration, and country and sector selection in equities. I aim to make sure I generally take more risk in the easier areas.

Finally, I check where we have been doing better more recently. At times, we have a cold hand in certain areas, and I then reduce their risk budget until performance picks up, and vice versa. In effect, I assume momentum in success.

The conflict between consistency and diversification.

Given how efficient markets generally are and that I do not really have superior information, I try not to get too cocky about my ability to beat the market. I assume my success rate for any individual position will be only just over 50/50. How then to get a portfolio with a success rate that is well above 50/50? The trick is to choose positions and OWs/UWs that are not correlated to each other. That is easier said than done because our mind naturally veers to creating consistency

I have found only one way to create diversification in trades, which is to make them go through different brains and ways of thinking. As a research strategy CIO, I had to make sure I do not dictate all trades, as they would otherwise become highly correlated. Instead, it is important to allocate trading decisions (on paper in our case) to different individuals and ways of thinking.

How long to hold on? I find it nearly impossible to hit the exact top to take profit on a winning trade and thus had to make a choice between exiting while still going up, or only after it is already going down. Most of the time I find myself selling on the way down, and have rationalized this by the observation that we are generally underestimate how far a market can go when we have the direction right.

What is the right investment horizon for active positions? It is almost a truism that successful trades end up becoming longer lived than expected, while bad ones becomes shorter-lived. Beyond that, I find that asset classes with positive feedback with fundamentals, like equities and credit, have much higher longevity (quarter to years) than markets with negative feedback, such as bonds and currencies (weeks, maybe months). This is why our bond floor always feels so short-termist versus our equity floor. It took me a long time to recognize that this makes sense.

How frequently to adjust your portfolio? In theory, every time new information arrives or asset prices move. This is not practical. I have been doing it monthly, but the beauty of our Lego approach and the usage of different brains in our paper portfolio with each managing a different trade is that we are effectively changing small parts here and there of the portfolio virtually on a weekly, if not daily basis.

Final thoughts

Cherish your errors. I have learned ten times more from being wrong than being right. Once you make a mistake, go public with it, analyze it in detail, and learn from it. Be your own devil’s advocate, and spend most time with people who do not agree with you, or who have a different way of looking at things. Not always easy as being with like-minded people is more comforting.