I arrived in Berlin last week, hoping to see something rare: A country that is prosperous, well-governed and even happy, if only because it had just been crowned champion of the football World Cup.

Germany is all those things and Germans know they are doing well. They need only look around to notice that their economy is stronger and their government more effective than those of their neighbours. Their Chancellor, Angela Merkel, an indispensable ally of US President Barack Obama in crises from Ukraine to Iran, is “the uncrowned empress of Europe”, in the words of Josef Joffe, publisher of the weekly newspaper Die Zeit.

However, I also found a Germany that has been plunged back into angst over its place in the world, with the latest blow coming from the arrest of a mid-level official in the German intelligence service (identified only as “Markus R”.), who was pilfering secret documents for the CIA.

The message many Germans have taken from the latest spy scandal — coming less than a year after the revelation that the National Security Agency (NSA) had been intercepting Merkel’s cellphone calls — is that no matter how successful and reliable as allies they may be, they still do not get much respect from the US.

Documents leaked by former NSA contract worker Edward Snowden also revealed that although Germany was one of the agency’s top targets, the NSA gives Germany less access to its findings than it gives to France, the Netherlands or Norway.

As Merkel put it with cruel understatement: “This isn’t a trusting relationship.”

“Americans need to understand how deep the feeling of being humiliated has been,” Malte Lehming, an editor at the Berlin newspaper Der Tagesspiegel, told me. “It’s the feeling of being humiliated without the ability to retaliate ... And it reminds us, with a kind of shock, that Americans and Germans do not understand each other as well as we thought we did.”

“The substance of this little spy story is not serious,” agreed Karsten Voigt, a longtime foreign policy leader in Germany’s Social Democratic Party. “But underneath, it’s quite serious. What you’re seeing is two political cultures colliding.”

US spying is particularly galling to Germany because it knows the US does not spy on its closest English-speaking allies — Britain, Canada, Australia and New Zealand. That is because those countries have joined in an intelligence compact known as “Five Eyes” in which they share information and watch each others’ backs.

Germany has been offered membership in the “Five Eyes” club, but it has always found the entrance fee too high: A much bigger intelligence service that could operate on the same level as the CIA and Britain’s MI6.

So Germany is caught in an intelligence policy Catch-22: It wants to be an ally of the US, but it does not want to play by the same rules.

That is partly because almost 70 years after the end of the Second World War, Germans are still gun-shy.

“Germans don’t want a bigger role in the world,” said Voigt. “We still get nervous when other countries are unhappy with us.”

None of this threatens the core of the US-German alliance — not yet, anyway. Merkel has been a solid partner to Obama in supporting sanctions against Russia’s Vladimir Putin, despite Germany’s dependence on Russian oil and gas. “Putin is helping to keep us together,” a US diplomat in Berlin told me.

A poll released last week by the independent Pew Research Centre found that 51 per cent of Germans still say they have a high opinion of the US; that is a big drop from 64 per cent in 2009 (when the Obama presidency was new and fresh), but still well ahead of the 30 per cent recorded under George W. Bush in 2007.

However, another poll by German magazine Der Spiegel found that 57 per cent of Germans want their government to pursue a foreign policy more independent of the US.

The saddest thing is that this crisis did not need to happen. “The Americans have bungled this completely,” charged John C. Kornblum, who was US ambassador to Germany during the Clinton administration.

The problem, he said, is that after the terrorist attacks on September 11, 2001, the US intelligence community lifted most of the restraints it once observed on collecting intelligence on Germany — for example, on recruiting mid-level German officials like “Markus R” as spies.

“We never would have done this before 9/11,” Kornblum said. But the US intelligence community never subjected its post-9/11 projects to serious cost-benefit analysis, he said; the terrorist threat seemed to justify maximum measures, no matter what the cost.

So where do we go from here? Officials on both sides say they want to repair the relationship. Obama has telephoned Merkel; CIA Director John Brennan has offered to visit Berlin. However, the US has not offered what Germany really wants: A “no-spy” agreement that would exempt Germany from being a target.

The sensible thing for Washington to do would be to impose voluntary limits on its intelligence activities against one of its most valuable allies in Europe, especially when they have proved so counterproductive once exposed.

But the US also needs to learn a larger lesson: Alliances, even long-standing ones, need careful tending. They cannot be taken for granted.

— Los Angeles Times