London: “I cannot forecast to you the action of Russia. It is a riddle wrapped in a mystery inside an enigma,” Winston Churchill told his listeners in a radio broadcast in October 1939.

Much the same can be said about Saudi Arabia — one of the most compulsively secretive countries in the world at the start of the 21st century.

Almost nothing is known about how decisions are reached on political and economic reform, foreign policy and oil market strategy.

Outsiders are strongly discouraged from inquiring into matters of long-term policy and how decisions are made. In almost 20 years of writing about oil markets and the Middle East, I have not come across anyone who could consistently offer a deep insight into the government’s policymaking.

In the absence of hard information, plenty of theories have grown up around Saudi Arabia’s strategy in the oil market, most of which contain some truth but are often incomplete or misleading. The first myth is that there is a “grand bargain” in which Saudi Arabia provides a secure, reliable and affordable oil supply in exchange for a US security guarantee.

This myth conveys an essential truth but does not help much in understanding the complex relationship between the world’s largest oil exporter and its greatest military power. Saudi Arabia and the US have enjoyed a uniquely close relationship ever since the kingdom chose to sign a concession agreement with the Standard Oil Company of California in 1933 and King Abd Al Aziz met President Franklin Roosevelt on board on the USS Quincy in the Suez Canal in 1945.

More generally, the close military and strategic links have not translated into an agreement on oil prices and production: it is emphatically not the case that pricing policy is the result of discreet negotiations between Washington and Riyadh. To their chagrin, a succession of US presidents has discovered the limits of their influence over the Saudis when it comes to oil prices and production.

Diplomats and even some economists often assert Saudi Arabia upholds its part of the bargain, in part, by holding spare production capacity with which to meet disruptions in oil supplies from other producers. Only Saudi Arabia has the financial capability and the foresight to invest in spare capacity to help stabilise global oil prices.

The problem is that there is almost no evidence to support this claim. Since the kingdom’s exports peaked at almost 10 million barrels per day (BPD) in 1980, most of the spare capacity has been in the form of reduced output from older fields as new ones have come on-stream.

Most spare capacity appears to have been the result of past errors in forecasting oil demand and efforts to increase the amount of oil eventually recovered by lowering production rates from ageing fields like Ghawar to sustain reservoir energy and prevent water inundating the wells while bringing on new fields like Manifa.

 

Nearly all of the kingdom’s reported spare capacity has followed a downturn in prices and demand, notably during the 1980s and 1990s, which suggests capacity is the result of planning errors rather than deliberate policy. There is no evidence that Saudi Arabia has deliberately developed large new fields simply to allow them to be left idle “just in case” there is a supply interruption elsewhere in the world.

Saudi Arabia is often described as the oil market’s “swing producer”, a role which senior policymakers are said to dislike after the trauma of seeing exports shrivel from almost 10 million barrels per day in 1980 to less than three million in 1985.

But if Saudi Arabia once played that role, it does so now to a much smaller extent, if at all.

According to the BP Statistical Review of World Energy, Saudi Arabia’s share of world oil production has been remarkably stable at around 12-13 per cent and at about 30-35 per cent of Opec output since the mid-1990s.

Some observers suggested Saudi Arabia has stepped in to fill the supply gap left by sanctions on Iran and the turmoil which has cut output from Libya, Sudan and Syria. But Saudi exports have remained broadly stable since 2011, and are essentially unchanged since 2003-04. US shale oil, not Saudi Arabia, has filled the supply gap.

The kingdom is sometimes likened to a central bank managing the global oil market, adding or withdrawing supplies to control prices. But that vastly overstates the degree of influence, let alone control, which the kingdom can really exercise over the market.

In the short term, the Saudis, acting in concert with close allies Kuwait and the UAE or Opec as a whole, may exercise a mild restraining influence on price movements. But there is no evidence that Saudi Arabia, or Opec, has had a decisive impact on medium and long-term price trends. The big price movements of the last 30 years have all originated outside of it.

Opec members accounted for just over 50 per cent of world oil production in 1973. In recent years that share has been as low as 40 per cent.

 

The big movements in prices have been the result of production trends in countries outside Opec, whether the development of Alaskan, Russian, Chinese and North Sea oil in the late 1970s and through the 1980s, or the shale revolution in the US since 2008. “Unless it controlled the world’s entire production, Opec could not possibly maintain the new status quo forever,” one historian wrote of the difficulties in the mid-1980s.

 

Opec has never come anywhere near that degree of control since the mid-1970s. It can slow the rate at which prices move by adding or removing some barrels from the market, but there is no evidence that Saudi Arabia or Opec can choose the level of prices or guide the market to a particular level and keep it there.

The Saudis themselves seem aware of their limitations. In the last two decades, pricing and production policy appear to have been geared to maintaining market share rather than grander strategic aims which are the stuff of international relations specialists (such as bankrupting Iran and Russia, stifling the US shale revolution, or strengthening ties with the US).

The recent drop in Brent oil prices below $100 (Dh367.30) per barrel has encouraged much speculation about whether Saudi Arabia would respond by cutting production, either on its own or as part of a wider package of agreed reductions within Opec. Saudi officials have tried to squash the rumours in a series of meetings with customers and market analysts.

There is a widespread view Saudi Arabia might permit prices to trade below $100 per barrel, and perhaps even below $90 or $80, for an extended period to curb the amount of shale drilling in the US as well as investment in high cost production outside Opec such as deepwater off the coasts of Africa and Latin America.

In practice, the kingdom has little choice but to follow this course. Oil prices above $100 appear unsustainable because they incentivise too much growth in shale production as well as high-cost offshore drilling, and because they are encouraging too much conservation, efficiency and substitution.

If Saudi Arabia, with or without Opec support, cut its own output in a bid to keep prices high, it would be buying a temporary reprieve on prices but only at the expense of market share. With the shale boom continuing, and demand stagnant, any reprieve could only be temporary. In a few months or a year, Saudi Arabia and Opec would need to cut again, and again.

Ultimately, Saudi Arabia and Opec would end up with a combination of lower market share and lower prices, the worst of all outcomes, just as they did in 1985. The best strategy for the Saudis, indeed the only effective one, is to allow prices to fall until the market rebalances naturally, with slower growth in shale and bigger increases in demand.

Allowing prices to fall is not a matter of choice but necessity. If the Saudis, and Opec, choose to trim their output slightly in the months ahead, it would be an attempt to smooth the process of adjustment, not arrest it.