September 1, 2009

SSE Composite Index Bubble II

Posted by John Baez

You’ll recall from my earlier blog entry that on July 10th, a team led by the geophysicist Didier Sornette predicted a crash in the Shanghai Stock Exchange Composite Index. They said it would happen between July 17th and 27th, with 60% probability.

Now some say they were right.

According to the New Scientist:

Physicists successfully predict stock exchange plunge

28 August 2009

With 20/20 hindsight, financial crashes seem inevitable, yet we never see them coming. Now a team of physicists and financiers have bucked the trend by successfully predicting a steep fall in the Shanghai Stock Exchange.

Their model, which employs concepts from the physics of complex atomic systems, was developed by Didier Sornette of the Financial Crisis Observatory in Zurich, Switzerland, and Wei-Xing Zhou of the East China University of Science and Technology in Shanghai. The idea is that if a plot of the logarithm of the market’s value over time deviates upwards from a straight line, it’s a clear warning that people are investing simply because the market is rising rather than paying heed to the intrinsic worth of companies. By projecting the trend, the team can predict when growth will become unsustainable and the market will crash.

Sornette, Zhou and colleagues applied their model to the Shanghai Composite Index, which tracks the combined worth of all companies listed on the Shanghai Stock Exchange, the world’s second largest. Early this year, the index gained 50 per cent in just four months. In July, the team predicted that the index would start to fall sharply by 10 August (arXiv:0907.1827). The index duly began to slide on 4 August, falling almost 20 per cent in the subsequent two weeks.

Anyone hoping to exploit the model for profit should think twice. “If enough investors take action based on our predictions, the evolution of prices will probably be affected,” says Zhou.

I’m not sure how the earlier claim that the plunge would occur “between July 17th and 27th, with 60% probability” squares with the above remark that Sornette’s team predicted it would happen “by 10 August”. But I’m also not sure that matters much.

Presumably there will be better places to find a detailed post-mortem than the frequently inaccurate New Scientist. Does anyone know where?

Thanks go to Robert Schlesinger for pointing out this item.

Posted at September 1, 2009 6:30 AM UTC

TrackBack URL for this Entry:   https://golem.ph.utexas.edu/cgi-bin/MT-3.0/dxy-tb.fcgi/2046

Re: SSE Composite Index Bubble II

I know this is sounding like a stuck MP3 track…:

Regarding the comment “I’m not sure it matters much.” Here the difference with physics asserts itself: there’s only one (up to isomorphisms and modulo unobservable entities) fundamental physical explanation for a given event. In contrast this kind of model building and analysis is deliberately not trying to build a model of the “fundamental explanation” for the phenomena but trying to select just the tiny subset of the influencing “variables” that happen to be highly predictive. In which case you can’t really choose between models on the basis of which is “most true model”, you can only do it based on some statistical measure. The machine learning community has come up with myriad of ways of computing statitics for selecting models (which models often implicitly include training and evaluation algorithms) given multiple competitors. These measures may even vary depending on your purpose: if you’re the Shanghai government trying to predict the taxes you’ll get from trading over the next year then being off by just over a week isn’t significant; if you’re an individual investor then being off by a week may dramatically reduce your trading account balance.

In case this sounds like I’m down on Sornette’s team, I’m not: it sounds like on an intuitive evaluation they’ve done quite well, this just highlights the difference in viewpoint (particuarly with respect to evaluation) that one needs to make when working in these kind of areas.

Posted by: bane on September 1, 2009 4:37 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

bane wrote: “Regarding the comment “I’m not sure it matters much.”” [SH: which I also wondered about]

I think my comment below agrees with your (bane) assessment if it is dummied down a layer. Zhou approximately said, ‘If investors were to read this paper, it would interact with the investment pattern and change its prediction.’ A very similar idea appeared in a stock market book I read. Suppose there is a sharp home investor of stocks who is choosing to invest over 10 candidate stocks. His chances of making a successful pick are a little better than random.

Now suppose that there are 10 investors who subscribe to a stock tip newsletter and follow the recommendation. The analyst who produces the recommendation is also sharp and also selects from the same 10 candidate stocks as the home investor has chosen from.

The home investor has only about a 10% chance of picking the same stock as the analyst and benefit from the buying impact of the 10 investors. The rest of the time, the 10 investors benefit from their mutual buying power. So even though the home investor might pick a slightly superior stock in terms of going up in price (pure value), the chance of his success is usually (9 out of 10 cases) not as great as one of the 10 investors who just follow a shared newsletter tip. I think the impact of the newsletter is quite analogous to the influence of something that works like the arxiv paper, a self-referential factor.

Posted by: Stephen Harris on September 1, 2009 9:37 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

Bane: I completely agree with your remarks, both this time and before.

Just to be clear: when I said “I’m not sure it matters much”, I was not trying to say “I don’t think it matters much”. I would never be so sloppy as that.

I was saying that I don’t really know what we should think when someone predicts a somewhat exciting event will occur with 60% probability between July 17th and July 27th, and then it happens on August 4th. On the one hand it suggests Sornette’s team might be on to something, but on the other hand we can’t get too serious about claiming that some theory has been validated until we makes precise what the theory is supposed to predict, and do a lot more trials (in secret so the theory doesn’t influence the data).

Posted by: John Baez on September 1, 2009 11:38 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

This paper by Sornette was written about 2 weeks after the Shanghai market paper. I especially like the term he coined for the quasi-random outliers, Dragon-Kings.

http://arxiv.org/abs/0907.4290
Dragon-Kings, Black Swans and the Prediction of Crises
Authors: Didier Sornette(Submitted on 24 Jul 2009)

“The following sub-sections present empirical evidence of the presence
and importance of dragon-kings in six different systems. A very
important message is that there is no unique methodology to diagnose
dragon-kings. One needs a battery of tools. Dragon-kings can be
observed sometimes directly, in the form of obvious breaks or bumps
in the tail of size distributions as in the example of sections
3.1 and 3.2. Or they need the construction of novel observables,
which are more relevant to the dynamics of the system, as in the
example of section 3.3. Or it is the comparison of distributions
obtained at different resolution scales that allows one to diagnose
the existence of a population of dragon-kings, as shown in the example of section 3.4.” …
—————————————

As bane has pointed out, this has a lot to do with Machine Learning. John Colt is an emeritus (machine learning)professor. He gives an example of a pattern which has a rule which is difficult for a human to figure out, but not so hard for a computer program. But, he concludes by saying
“Interestingly, it is mathematically proven that there can be no computer program which can eventually find (synonym: learn) these (algorithmic) rules for *all* sequences which have such rules!”
————————————

SH: I think the “mathematical proof” would have to involve recognition that the inability to find “algorithmic rules” means the same thing as Sornette’s “there is no unique methodology to diagnose dragon-kings”. There is a movie, I Robot, starring Will Smith. In a freak accident, his car and another car carrying a 12 year old girl are swept below the surface of a river and drowning is imminent. A passing FAI computes the chances of the girl surviving at 11% and Smith’s chances at 47%, so the robot decides to rescue Smith. After that, Smith hates robots because he says it should have rescued the girl, “any human would have known that”. This situation is an outlier and challenges the average knowledge of a robot which can pass for imitating human behavior. There are unpredictable situations which a program cannot anticipate, especially if the motive is found in outliers of discoverable human knowledge: unconscious instincts which are not listable/tractable, because they evolved over a period of a billion years. Anyway, it’s a good movie even if you don’t like my Turing-Test-like analysis.

Posted by: Stephen Harris on September 2, 2009 2:00 AM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

I’m too busy at the moment to add anything new, but just wanted to reiterate what I said in the last post:

Anyway, reading this paper on an SSE bubble is low on my priority list, but I give it very little credence, i.e. my “BS” detector is going off. I’m guessing they built a model that is telling them,

We are 60% confident that the SSE Composite will lose more than 20% of its value over the next 10 days.

The very same model would probably also say

We are 10% confident that the SSE Composite will gain more than 20% over the next 10 days.

Basically, they are saying the direction of the SSE Composite is currently highly unpredictable. Well, gee, thanks!

Fat tails work in both directions.

Posted by: Eric Forgy on September 2, 2009 5:12 AM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

A one-off trial of their forecasting tells us very little. We need to have a series of forecasts to allow for calibration. This is where you penalise the forecaster’s predicted probality for an occurring event according to the proability, $p$, they assign it. There are various regimes, including $(1 - p^2)$ and $-log p$.

Predicting a event which occurs as certain earns you zero penalty, but if you ruled it out completely, you’re in big trouble, especially with the second penalty regime.

Posted by: David Corfield on September 2, 2009 8:49 AM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

By the way, the problem of evaluating performance is of real concern to me, so your comment is interesting. Any further thoughts or references would be greatly appreciated. Thanks!

Posted by: Eric Forgy on September 2, 2009 3:28 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

David will probably give more comprehensive references, so I’ll just note that assuming one is dealing with discretised events (ie, “was there an earthquake this hour” rather than “was there an earthquake a time instant x”), the story gets a bit more complicated because, almost by definition, interesting events are relatively infrequent so the huge mass of “event non-occurrence correctly predicted” terms drowns out the “event correctly predicted” terms, yet for applicability it’s important to include terms to stop the model generating “false alarms” constantly. The only real methodology I know of is to pick a (true positive rate,false positive rate) approximate pairing that fits your intended application and find which model gives best performance around that region on a ROC curve on some training data. (Which assumes you’ve got training data that’s representative of new data, etc, etc.) (I imagine Eric knows this, so the ref is more for completeness. I’d also be interested in references to more effective methodologies for choosing models.)

Posted by: bane on September 2, 2009 5:53 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

All I know is that this is a huge issue in machine learning, as you can see from this workshop.

Some measures for evaluation of performance are listed here.

Posted by: David Corfield on September 2, 2009 7:03 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

There is a better article about detecting early warning signs of a financial bubble or its bursting in the last issue of the journal Nature v461, 3 Sept, page 53- by Marten Scheffer et al. entitled “Early-warning signals for critical transitions”.

I would add to this two specific warning signs about the SSEC which is that there has been an increase in both borrowed money going into Chinese stocks and in the number of new individual investors opening stock brokerage accounts.

Also, I am supposed to write a new paper about the failures of quantitative finance for non-experts. Here are 7 principles that traditional quantitative finance models tend not to understand:

1) There has never been a 100% freely traded market, e.g., a market which is not unduly influenced by big government, big corporations, asymmetric information, etc. Thus, there cannot be a 100% efficient market, and the overall efficiency of any market keeps increasing or decreasing (for which there are many examples that I will cite in the paper).

2) There is not even the possibility of doing valid scientific experiments within macroeconomics or macrofinance.

3) Black swans (i.e., major disruptive events that are not expected) have actually occurred much more frequently than quant models have predicted.

4) Traditionally, both math and AI (artificial intelligence) have had a poor track record at modelling good common sense or good intuition.

5) Quant models fail to model the reality that over the short term tradable Western financial markets are driven mainly by two human drives which are fear and greed.

6) Countries, corporations and individuals can suddenly elect to default on their debt payments, thus negating the expected income returns generated by various quant approaches to FI (fixed income) markets.

7) Traditional quant models fail to detect the emergence of a herd-like behavior (i.e., groupthink) which causes asset prices to overshoot either to the upside or the downside.

Can you think of any other principles that quants tend not to understand that I should discuss in my paper? Thanks.

Posted by: Charlie Stromeyer on September 7, 2009 4:57 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

Hi! I’m in a rush and generally stressed out at the moment so I’ll leave some terse comments meant to provoke discussion. If you’re interested in my thoughts, I’m glad to dig a little deeper when time allows. I’m looking forward to seeing the paper. It’s a great topic close to my heart.

—–

A paper on the failures of quant finance would be grossly incomplete in light of the current crisis without a discussion of the failure of quantitative risk management.

Those failures were both technical and political.

Nassim Taleb is getting far more traction than is warranted so I hope your discussion of black swans does more than perpetuate the misinformation he’s propagating.

Another thing to keep in mind is that quantitative investment strategies have enjoyed and continue to enjoy high levels of success, so they are not all bad. There were certainly significant blowups in 2007 (who didn’t blow up in 2007-2008?), but overall (and since then) they’ve done well. My own strategies continue to perform well for example ;)

Posted by: Eric Forgy on September 7, 2009 5:44 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

Thanks, Eric. I will follow your advice because yesterday I ordered a copy of this book because it shows many examples of the failure of risk management, and thus using this book I might be able to uncover some generic and valid insights into why risk management fails and then what are some empirical or evidence-based remedies: The Failure of Risk Management by D.W. Hubbard.

Also, Taleb’s ideas are not rigorous or empirical enough to be of use over the long-term. There is also what is known as “guru overshoot” by which I mean that some follower of economics or finance makes one correct prediction about a major event and is then heralded as a “guru” even though he or she typically fails to then correctly predict future major turning points.

Posted by: Charlie Stromeyer on September 7, 2009 6:15 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

Here is a list of all my articles tagged with risk management.

In particular, this one is relevant I think:

The risks of risk management revisited

Here’s my favorite quote from March 16, 2006 when I was 15 months into my career change that turned out to be fairly accurate I think:

When I look out at the world, one of the major risks to the markets that I see is, ironically, risk management. I suspect that one of the primary employers of junior quants in the last 5 years has been in risk analytics (HHs, please correct me if I’m wrong). If there is any truth to that, it means there is literally an army of quants who have not lived through a business cycle building risk systems on markets that no one really understands, e.g. CDS/CDOs.

[snip]

If things are at all like what I have seen, then we’ve got a bunch of fairly clueless risk managers out there with an army of fairly green quants developing sophisticated risk models that are probably pretty useless in a crisis. Nonetheless, there seems to be this completely ludicrous false sense of security.

Across the boards, vols seem to be historically low which would mean that most VaR engines are saying “smooth sailing”. What happens if vol increases? Everyone’s VaR model is going to start sending out little red flags. Assets are going to start getting reallocated. Since everyone has almost identical VaR models, the signals will be pretty much identical at all firms. I know it is not an original argument, but this could easily lead to a negative feedback. A small red flag due to increased VaR could signal everyone to make very similar reallocations. If everyone does it at the same time, the market will obviously be affected. In essence, the impact of risk management could actually increase systemic risk in the markets and amplify vol movements.

Posted by: Eric Forgy on September 7, 2009 8:27 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

By the way, here are some thoughts of mine during a spike of quant bashing in the media back in February:

Disingenuous quant bashing

Posted by: Eric Forgy on September 7, 2009 7:57 PM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

Eric, here is one example of why risk managers failed more than the quants did:

In late 2006, I was one of very few people who downloaded a copy of the paper entitled “Measuring the Macroeconomic Risks Posed by Asset Price Booms” by Professor Stephen Cecchetti which was posted online October 2, 2006 and which shows that housing booms (within developed economies) lead to “outsized risks of very bad outcomes”.

This very good paper uses fairly simple statistical analysis of historical data, and thus risk managers should have been able to take its main finding into consideration.

Posted by: Charlie Stromeyer on September 9, 2009 11:56 AM | Permalink | Reply to this

Re: SSE Composite Index Bubble II

The stories I could tell…

Posted by: Eric Forgy on September 9, 2009 6:32 PM | Permalink | Reply to this

Post a New Comment