When Opec embarked on a market share policy in mid-2014 and oil prices fell sharply, speculation was rife that the action was directed against Iran, Russia and shale oil producers in the US where this source of supply had gained millions of barrels within a few years.

But the battle turned out solely to be against shale oil as Iran and Russia are cooperating with other producers inside and out of the Organisation of Petroleum-Exporting Countries (Opec) to stabilise the market. The policy met with partial success whereby shale oil production stabilised and started falling slowly as a result of reduced investment and drilling. But shale producers did not take all this lying down.

They reduced cost, improved efficiency and further developed drilling techniques to continue their activities even if it meant break-even or little profit. In the meantime, prices fell from over a $100 (Dh367.3) a barrel to below $30 in February 2016 and Opec started seeking other solutions to bring price recovery by abandoning market share dreams. Thus the Opec and non-Opec agreements of December 2016 and May 2017 to reduce production by about 1.8 million barrels a day (mbd).

The first agreement made prices recover to above $50 and shale oil producers in the US came back with vengeance. It looked like a vicious circle was setting in and analysts warned that prices were likely to stay between $50 and $60 at best. Even that is no longer the outlook and prices now are in the mid-$40s and more likely to go down than up due to the persistent growth of non-Opec supplies, especially from shale oil and the high level of oil stocks in consuming countries.

This is expected to continue into 2018 and there is already speculation that Opec and its non-Opec associates may have to extend its production restraints beyond the current term which will run to the end of March 2018.

The pain continues and the hard-sought market balance looks to be far from coming soon.

But it is not all doom and gloom. While shale oil production is on a high, there are signs that things are not going well and that shale producers may have shot themselves in the foot by quickly increasing production.

Two recent reports discuss the challenges facing shale producers at current market conditions.

In the Permian Basin, the largest tight oilfield producing 23 per cent of US production, the signs of strain are clear. While it is expected to produce 2.47 mbd next month, the increase over the June estimate is only 65,000 barrels a day. The legacy (natural) decline from existing wells is increasing and now stands at 150,000 barrels a day, which can only be compensated by relentless drilling and completion of new wells.

The report says than “even assuming no change in current depletion rates, the Permian loses 1 million b/d of output every seven months or so.” Also, “rig productivity in the Permian has been consistently declining since reaching a high of an average 704 b/d in August 2016. In July, a new rig in the basin will add only 602 b/d of production” — about half the productivity in other shale plays. Therefore, more rigs and “furious drilling” are required “just to maintain output at current levels”.

Yet US production is increasing and the IEA says about half the increase of non-Opec supplies forecast for 2018 (1.5 mbd) will come from US shale oil.

Another dent for shale oil is the withdrawal of hedge funds, estimated at $400 million and attributed to concerns over the fast increase of production, which “will undo the nascent recovery in the industry”. There are even subtle calls to these producers “not [to] go crazy” and undermine the market.

EOG Resources’ executive vice-president Billy Helms said the company will “moderate” production if needed. Perhaps they should implicitly join Opec and other producers in their effort to stabilise the market.

The accumulated debt of shale producers has impacted their share prices, where, according to Reuters, they “have, on average, dropped 18 per cent [this] year, compared with the broader S&P 500 energy sector’s 13 per cent fall.”

Of course it is not only the Permian as “Bloomberg reported that the Bakken would see a lot of wells fall below profitability with WTI at $45 per barrel”. Opec however, should also be aware of the huge number of wells that are drilled but uncompleted (DUC), which overhang the market and any substantial increase of oil prices is likely to bring these wells quickly into production.

If shale producers continue their current behaviour, the battle with Opec can only be settled with further falls in oil prices to the detriment of all. Can cooperation prevail at last?