Thursday, August 12, 2004

Crude Cents: A Pun Built on a Homonym,

As well as a analysis of today's oil market. What is up with the oil market? Not what commentators think. If you want to understand markets, you have to understand that a market is more than the sum of its parts. If a market is to one degree or another truly efficient, then as a group it is pricing supply and demand using distributed processing.

Distributed processing is the same basic concept as those bright guys over at SETI.
The Search for Extra-Terrestrial Intelligence (SETI) is an arduous task. What to look for and how to look for it are extremely difficult questions themselves, and even assuming that we do know what signs of ETI would look like, the process of detecting those signals involves sifting through immense quantities of observational data. In the past, such work required prohibitively expensive supercomputers. Now, however, an emerging technology known as "distributed computing" may hold the key to a more cost-effective SETI by utilizing the spare CPU cycles of idle machines belonging to Internet volunteers. Even SETI skeptics have to agree that processing blocks of information from SETI projects is as least as good a use of idle CPU time as "doing nothing". Still, the inherent complexity of SETI suggests that even the dramatically improved efficiency and processing power that distributed computing makes possible cannot guarantee positive results.

A market takes advantage of the fact that the whole is more than the sum of the parts. Each buyer and seller is inputting information into the market, and the market as a whole synthesizes it. The problem is that the market participants don't necessarily understand their own contribution or the impact of the whole. This fact is hilighted in a recent book talking about the common wisdom of crowds.
If you ask a large enough group of diverse, independent people to make a prediction or estimate a probability, and then average those estimates, the errors of each of them makes in coming up with an answer will cancel themselves out. Each person's guess, you might say, has two components: information and error. Subtract the error, and you're left with the information.

The opposite of the wisdom of crowds is what happens when groups aren't diverse, or they start speculating based upon the perceived actions of others, or enact regulations to prevent dissenting viewpoints. The phenomena whether formal or informal results in "group-think". Group think is essentially the same sort of idea as lemmings or stampeding herds, only humans are as subject to it. If you look at a list of its characteristics here, you'll realize that market bubbles share many of these characteristics - especially the illusion of invulnerability.

If we are to talk about market efficiency and understand the impacts of regulation upon it, we must go beyond traditional economic orthodoxy. Traditional economic orthodoxy takes as an assumption that the market is already perfectly efficient and all variations in the price line from the demand-supply curves is just a random walk. Since then economists have evolved a bit, but still too much theory about market efficiency is so much received wisdom or framed in ways that do little to grant insight.

Markets are created by people participating in them. Markets are smarter than the people in them. Markets are also dumber than the people in them. Markets are a phenomena created by participant interactions and therefore scientifically should be subject to the same constraints that can be empirically observed in other group psychology studies. For instance, simple voting group consensus seems able to construct alphanumeric symbols but not draw a goat.
The Internet lets thousands of total strangers collaborate to produce a truly hivelike result. One intriguing example is "Typophile: The Smaller Picture," a project that let an anonymous crowd design a font. Kevan Davis, a British Web developer, created grids of pixels, 20 by 20 in size, one for each letter of the alphabet. He randomly dispersed black-and-white pixels in each. Then he put them online and let people vote on whether a particular pixel should be white or black. As thousands of people voted on each one, letters emerged, forming a democratic consensus of what the alphabet should look like.

Davis created animations that show each letter taking shape, and they're mesmerizing, a time-lapse movie of a collective mind at work. Another designer took the results and produced crisper-edged versions of each letter. The final result looks like a mildly punk version of Helvetica, with occasional flashes of creative weirdness, such as the jaunty serif on the foot of the letter "J."

Yet the process has its flaws. When the mob tried to draw a few simple pictures, it couldn't. Davis told it to draw a television, but the image never congealed. The group agreed that the tube should be represented by empty space, but it couldn't generate any other details. An attempt at drawing a face produced an even more shapeless mess. The only partially successful picture was a goat: At around 4,000 votes, it looked pretty goatlike, and at 5,000 votes the mob revised it to make the horns curvier. But after 7,000 votes the picture decayed into a random jumble of pixels, as if the group could no longer agree on what a goat should look like. Mobs, it seems, can't draw.

Obviously markets work in a more complicated way than this, but using these types of studies we can move beyond merely pontificating about markets. We can see from simply setting up and observing experiments like this that sometimes groups can come to consensus in collective activities, and sometimes consensus collapses. Groups are better at doing some things, like guessing weights, and worse at others like drawing TV sets. The market is not all knowing, it just knows different things.

Understanding all of this as context then, now let us look at the oil market. What is the oil market doing today? According the NYT it is ignoring the Saudi's.
Traders who had bid up oil futures at the start of the day shifted gears, and prices fell sharply.

"The Saudis are trying to calm the market now and said they're ready to provide the barrels needed," Lawrence J. Goldstein, president of the Petroleum Industry Research Foundation in New York, said. "That's a welcome comment."

But by the afternoon, prices reversed again, partly in reaction to new statistics from the Energy Department showing that inventories of crude oil in the United States fell unexpectedly last week. Gasoline supplies also fell. By the end of the day, any market effect from the Saudi announcement seemed to have been erased.

The International Energy Agency's report projected global oil demand at 82.2 million barrels a day this year and 84 million barrels a day in 2005, about 730,000 barrels more than previously forecast. The author of the report, Klaus Rehaag, said in a telephone interview that the new figures recognized that demand for oil had been underestimated.

Mr. Goldstein said of the report, "The I.E.A. has finally come around to the realization that global oil demand is strong and rising, fueled by a recovering global economy."

The thing you have to understand is that all of these market participants, they're well meaning but they're wrong. They're wrong because they haven't counter-factualized their own epistemology. The problem isn't rising demand. It's not tight supply. It's not risk premium by terrorists in the market. These are explanations but since they're not falsifiable predictions or empirical observations they're just so much wind.

The experts are not expert in what they are believed to be experts in. They are experts in operating the technical and profitable micro-aspects of the oil market. They should be respected for that. But when an outsider looks on, or a regulatory policy-maker, and asks "Why?" to either understand or understand how to alter the situation then expertise in technical process is mistaken for expertise in scientific understanding. This is why so many experts can spend years becoming experts and then so often come to grief in the policy fields.

Policy demands not technical process expertise but genuine scientific understanding of causation and application of first principles. This is emphasized by the competing story line from yet another NYT article by Floyd Norris on the lack of consensus on why the oil prices are rising:
Did George W. Bush rely too much on diplomacy when he planned the war in Iraq?

The diplomacy in question was not the effort to line up allies for the war, which ended with much of Europe on the sidelines and angry. Instead, it involved what initially appeared to be a diplomatic victory: the quiet promise obtained from Saudi Arabia to step up oil production if necessary to offset a temporary decline in Iraqi oil exports.

That strategy was a good one in the Iraq war run by the first President Bush. As Jeffrey R. Currie, the head of commodities research at Goldman Sachs in London, noted this week, the Saudis at the time were able to offset the temporary loss of exports from both Kuwait and Iraq. There was a brief spike in oil prices when that war began, but they retreated as rapidly as they climbed.

But this time the Saudis have not been able to come up with the oil. They claimed this week to have another 1.3 million barrels a day of available production, but there is widespread doubt they can produce that much now, or even after two new fields go online later this year.

Why not? The international energy business has been starved of major capital investment for two decades, since the price swoon of the early 1980's scared oil companies.

So the simple answer to why the market hasn't reacted to the Saudi announcement is quite simply to put it in poker terms, the market thinks that the Saudis are bluffing. Floyd Norris then goes on however to misexplain things himself, after having just counter-factualized the typical explanations.
Most oil market commentary still makes it sound as if high prices are a passing phenomenon. Each spike is attributed to the threat of a loss of Iraqi exports, or possible Saudi instability, or Yukos's problems in Russia, even though what is happening there seems unlikely to affect oil production, just to change who profits from it.

But the markets are smelling something different. Oil to be delivered next year now fetches $39 a barrel, and oil to be delivered in 10 years costs almost $35.

The problem is not a lack of oil in the world. The problem is getting the oil to refineries and then to market. The large undeveloped resources are in West Africa, around the Caspian Sea and in Siberia. Two of those areas have issues of political stability and the third has severe weather.

The trend now, Mr. Currie said, is to ''have new oil produced in West Africa, shipped to Asia to be refined, and the product then shipped to North America.''

If he is right, many arguments in Washington have been irrelevant. It does not make much difference whether oil is pumped from the Arctic National Wildlife Refuge in Alaska because a shortage of oil is not the biggest problem. ''The real problem is the shortage of infrastructure to obtain and deliver the commodity,'' Mr. Currie said. ''That seemed to go completely unnoticed until the last six months.'' Neither Europe nor the United States shows any indication of willingness to build new refineries.

Mr. Currie says the oil industry invested about $100 billion a year in the 1990's, a figure that has grown to $150 billion but needs to rise to perhaps $250 billion. Until that investment bears fruit, the world faces both higher prices and the possibility that supply interruptions could have severe effects.

The reality is becoming clear because of Iraq, but Iraq was not the cause, and diplomacy will not make it go away. [emphasis added]

What's happening is not quite clear. Part of the problem is that people are suckers for easy to understand answers. There is a temptation to think that because an explanation is more comprehenisble it must in fact be more likely. There is of course ideally no relation between these two aspects.

The easy explanation that the reporter Norris falls for is simply that it's the lack of investment in drilling and infrastructure that is to blame for the rising price of oil. On the other hand, there is an aspect of neural net learning - or in the case of the market neural network learning - that is important to discuss. The aspect is called irreducibility.

Irreducibility occurs when people started designing neural nets, and trying to get them to learn, and then realized you could not reverse engineer the information causation. What I mean by that is that they fed inputs into these neural nets, and the neural nets would synthesize this information. They did this among other things to produce cardio-analysis programs that were more efficient than any single physician in diagnosing heart problems - because they contained the inputs from many many physicians or their work.

However they couldn't reverse engineer these buggers to find or derive "first principles" of why things happened. This is often why experts often have no clue about why what they do works. If you ask them to do it, they will do it correctly. If you ask them an explaination, they will often give you an incorrect explanation. Moreover if asked to act upon their conjectures they will then fail.

The truth is that only a neural network that simulates another neural network can discover first principles. First you must develop a neural network that learns. Then you must completely virtually emulate that first neural network. The second neural network, in its own completely irreducible fashion, will then take apart and find the first principles of the first neural network. At this point I could inject something about Godel-Escher-Bach and Godel's Incompleteness Theorem but that would be a tangent.

Suffice it to say that this has important consequences for understanding why and how markets work, and how not to understand them. Floyd Norris is wrong, because he dislikes the simple explanations in the oil market and then provides his own simple explanation. The simple fact that as a collective neural network the market's determination of its outputs are irreducible. It does not know why it knows what it knows, and neither do any of its participants.

A while back a person at Crooked Timber brought up the notion of an "informational free lunch." They weren't completely wrong, but they were wrong because they didn't understand that there was entropy involved that made sure there was no such thing as a free lunch. How can the whole be more than the sum of the parts, and yet there be no such thing as a free lunch?

Because of the nature of consciousness is irreducible, or if you like from Godel's Incompleteness Theorem that within any system there are irreducible axioms that you cannot derive from within the system. This is entropy at work. The system cannot be used to derive the system. The whole is more than the sum of the parts, at the cost of you cannot break apart the whole and get more parts than you started with. The way you solve this in practice, is by taking another neutral network and using it to reverse engineer the process of the other one to discover first principles.

So it would take a market, or something as big and complex as a market, to logically reduce a market to first principles. However the second market is then irreducible, and it will be even more mysterious than the first. So you shall need a third group network or something equivalently complex to unravel the second. But there shall be an even more fiendishly difficult problem of understanding the third as the second. This creates an infinite cascading regression.

It also means that this line of inquiry is naturally bankrupt when it comes to markets. At this point surely my readers will be scratching their heads - those that have managed to hang in so far.

At this point it looks hopeless, except that we can solve the problem simply. What cannot be solved using an algorithim, can easily be solved using a heuristic. Heuristic problem solving is often called intuition. What it involves is using a neural net in order to partially reduce the complexity of the larger irreducible problem. We use this sort of problem solving every day. It's call gut instinct, heart, rules of thumb, etc or most commonly wisdom.

It is in fact because of the irreducible complexity of so many epiphenomena in life probably the most heavily used problem solving tools. The problem is that you can learn the process of learning heuristically, but like all neural nets you cannot reduce it to some simple maxim or folk saying. The very context that richly informs the neural net of when and how to follow a principle is precisely what is missing when you try to reduce it to a bumper sticker.

In fact, the majority of scientific problems are solved by what is called "physical insight" or "physical intuition". Scientists and physicists don't like to tell you this, but if you sit there and try to learn formal problem solving methods for approaching physics problems you'll always have a difficult time of it and it will be slow and very difficult and you won't get far. Real physicists use heuristics all the time to solve physics problems, day in and day out.

So what does this have to do with the oil markets? It means that guessing is the way we get most things right in life. The problem is that not all guesses are the same. It depends a lot on your experience, and your mental ability to simulate the complexity of a problem enough to reduce it so that you can develop a heuristic. This doesn't mean that education is doomed, but that we shouldn't be teaching what we think we're teaching and what we are teaching that works isn't what we think what works.

What works is to help people see the big picture, to help them learn to simulate things in their heads, and to give them enough experience and feedback so that they can develop their own heuristics. Too often learning is conceived of as transfering information about first principles from one head to another. It actually doesn't occur that way.

So what does this all have to do with why the oil markets are high? First of all, knowing all of this we should start by saying that the markets are pricing the long term price of oil higher. That means, correctly as noted by Floyd, that these price rises are not temporary.

Second we have to accept that we cannot, unless we imagine ourselves capable of mentally emulating the complexity of the entire oil network, understand exactly why the prices are higher. Third this does not have to stop us from developing sucessful heuristics for either predicting which way the oil market will move, or whether specific steps will impact the market going higher or lower - so long as we do not make the mistake of over-reaching for reductionism. The moment we say the market is going higher for "this or that" reason in identifying a reversible causation change of necessary and sufficient means we are making a mistake.

The market is taking it all in, and crunching all the numbers, and because of entropy and there being no free lunch we are not free to tease apart the exact why and whyfores.

What can be more constructively said is that the market is pricing the long term cost of oil at $35. What that is saying is that the risk premium is ten dollars at this point in time. I believe for reasons of group-think that this market is not efficiently pricing the long term risks, and therefore is lowballing the long term cost but that is a belief. I will not mistake it for causation barring further evidence such as a sharp increase in price to $60 per barrell or more because of security deterioration in the middle east.

Without such a hindsight confirmation of my guess, I cannot claim empirical evidence for my beliefs because of the irreducibility of the market. But in a world of unequal guessers, track record becomes king. The oldman is a good guesser about certain things and not about others. It's important to know which is which. For instance the oldman has a poor track record for predicting FOMC decisions, but he's got a good track record for oil prices and market watching.

As such, the question is whether or not there will be a sharp security deterioration that sends oil up to $60 per barrell and makes the long term price trendline $45. If that guess is right, what my heuristic is telling me, by testing $45 a barrell we are testing a price-line. If we break through that level and sustain it, then $45 a barrell will become the new normal - the new long term cost of oil in the world. That information comes from my market watching heuristics.

Furthermore I believe the question is going to be resolve within a year. I believe this because of yet a third heuristic, a market timing one, based upon watching price-trends and volatility. This is the one which I believe that if I work on it with Mickslam we can generate a model of sufficient complexity to extract the first principles from to create a formal algorithmic model to replicate that heuristic's efforts.


At November 10, 2005 at 6:37 AM, Blogger TadaDiscounts said...

Great site!

Please check out my blog- candles

Or your my be interested in my other site- candles where you can find candles


Post a Comment

<< Home