Oil-market watchers are struggling to reconcile the large estimated oversupply in the market with the much smaller build-up of reported inventories and narrowing contango in futures prices.

Some blame the barrel counters who compile official statistics on supply, demand and stocks. But the truth is that information on the world oil market is incomplete and it is easy for hundreds of millions of barrels of oil to disappear from the supply chain without being counted.

According to the three main statistical agencies, the global market has been oversupplied by between 1.5 million and 2.5 million barrels per day (bpd) since the start of the year. Stockpiles should have increased by between 200 million and 350 million barrels, according to the International Energy Agency, Opec and the US Energy Information Administration.

US crude stocks have indeed increased by around 100 million barrels since the start of the year while China’s stocks appear to have risen by between 50 and 100 million barrels. But that still leaves more than 100 million barrels that have simply vanished from the international statistical system.

These lost barrels show up under error terms such as “miscellaneous-to-balance” and “unaccounted for” in reports published by statistical agencies which could be overestimating supplies, underestimating demand and inventories, or a combination of all three — because all data on oil production, consumption and stocks, especially outside the US, is only a very rough approximation.

Once we accept that the numbers are fuzzy, it follows that any analysis and forecasts based upon them must be fuzzy too. The danger is that analysts try to over-interpret small changes in the published data without allowing for the enormous amount of uncertainty which shrouds them.

The “missing barrels” problem is not a new one: it was the hot topic during 1998 and 1999 when I first began writing about the oil industry.

“It is just over a year since the saga of the ‘missing barrels’ began,” the International Energy Agency wrote in June 1999. “Oil prices tumbled as a result of the Asian financial crisis, a mild winter, increasing production from Iraq and an OPEC quota increase in November 1997.”

The agency went on to wonder: “Millions of barrels of unneeded oil were placed in storage around the world; many of them have yet to reappear ... Did the barrels reported as produced, but that have not shown up in OECD stocks, ever really exist?”

Fast-forward six months and the agency was confident it had found the answer. Its December 1999 monthly report entitled “an un-fond farewell to the missing barrels” explained: “During the first half of 1998 a large amount of the excess supply in the oil market was unaccounted for ... There was strong disagreement at the time as to whether these ‘missing barrels’ were the result of statistical errors, or whether they represented a large increase in oil stored in non-OECD areas.

“Almost two years later the weight of evidence is that the missing barrels did exist and that they have now returned to the market. The return was triggered by the reversal in the shape of the forward price curve and the need for additional barrels following OPEC’s effective production limitation which began last March.”

The three major statistical agencies each publish lavishly detailed monthly reports on supply, demand and stocks for crude oil and the main refined products such as gasoline and diesel. The wealth of detail can fool the unwary reader into thinking that the global oil market is transparent and well-measured.

In fact, the market is extremely opaque and the data are mostly stale, incomplete and inaccurate. The best data are available for the US, where the Energy Information Administration administers mandatory surveys to oil refiners, importers and distributors and publishes results quickly and in comprehensive detail.

But European statistics are a patchwork of mostly aggregated numbers that appear too slowly to be useful for contemporary analysis, while much of the oil stored in the giant tank farms in the Bahamas, South Africa’s Saldanha Bay and third party storage facilities in Asia is essentially outside the statistical system.

Most Middle Eastern oil producers treat information about production, consumption and exports as a state secret and release only very limited data. Across the fast-growing fuel markets of Latin America and Asia, data on consumption is limited and out of date.

China, now the world’s second-largest oil consumer, presents a particular challenge. It is building a strategic petroleum reserve but, unlike the US, releases no data on how much crude is held at sites around the country and virtually no information on production, consumption or stocks. Analysts rely on data for refining runs and imports and exports of crude and products to estimate consumption.

There have been some improvements in data collection and publishing since the ‘missing barrels’ saga. Energy ministers agreed in April 2001 to launch what later became the Joint Organisations Data Initiative (JODI).

JODI promotes harmonised standards of data collection and publishes data on production, consumption, trade and stocks from about 100 countries. But the data is better on production than consumption and even then much of it is published with a delay.

Across the world, data on the consumption of refined fuels is almost always less comprehensive and timely than information about the production, shipping and refining of crude oil. The number of companies involved in the production and processing of crude oil is no more than a few thousand worldwide.

But the number of companies and individuals involved in the consumption of refined fuels runs into the millions. Once fuels leave the refinery, there is almost no information on how much consumers are using or how much they are holding in storage for their own use.

Errors in estimating demand are one of the most common problems in analysing the oil market. While data on production is often available, demand is always modelled until harder numbers become available months later, by which time the market has moved on.

Analysts deal with the lack of data on consumption and stocks held towards the end of the supply chain by assuming they change only very slowly even if prices change by a large amount. In practice, there is evidence consumers and distributors can and do change the amount of fuel they consume and store by a large amount in response to a big change in prices.

Traffic surveys and fuel tax collections show the amount of driving and fuel use responding to a large change in prices, even in the short term. And the volume of stocks held by fuel distributors (secondary stocks) and end-users (tertiary stocks) can also change.

In 1989, the US National Petroleum Council estimated distributors and consumers in the US held one-third of all gasoline stocks and 60 per cent of all diesel and heating oil.

If motorists choose to keep their fuel tanks topped up to 65 per cent rather than 60 per cent full because fuel is cheaper, or homeowners and businesses choose to fill their heating oil tanks earlier and more than usual, the resulting shift in secondary and tertiary stocks can be very large at global level.