London: A secret Metropolitan police report warned police chiefs that they needed to take tougher action to stop officers discriminating against black people, and that a failure to do so would threaten a breakdown in community confidence.

The report, obtained by the Guardian, warned top officers that innocent African-Caribbean people were being stopped too often by officers, who wrongly "racially stereotyped" them as criminals.

The report was by Brian Paddick, the Liberal Democrat candidate for London mayor, who was then a Met commander. He says his bosses ignored the warnings made in 2004 and buried the report.

For the past week allegations of racism in the ranks have dogged Scotland Yard's leadership. The Met has been dealing with a growing racism scandal, which was triggered when the Guardian revealed an audio recording of an officer racially abusing a man in east London.

On Thursday the Met said 20 police officers and one staff member were under investigation for alleged racist incidents, and that eight police officers and a civilian worker had been suspended. The allegations include assault, abuse and bullying.

In the decade since the damning Macpherson Report, 120 Metropolitan Police officers have been found guilty of racist behaviour, but just one has been dismissed, figures reveal.

Despite the nature of the offences, only 12 officers received written warnings and 21 received disciplinary sanctions, statistics obtained under the Freedom of Information Act show. Of the 21, eight had to pay a fine, six were forced to resign and there was the solitary dismissal.

The racism took place between 1999 and 2011 — the decade after the Macpherson Report branded the Met institutionally racist.

An inquiry by Channel 4 News, which uncovered the figures, found 79 of those found guilty were constables, 13 were sergeants, four were detective inspectors and four were detective chief inspectors or higher. Separate figures obtained by the Daily Mail show that 1,000 complaints of discrimination are made to forces across England and Wales every year.

But just one in 40 allegations, the vast majority of which are investigated by the forces themselves, leads to someone being punished.

In London alone, officers faced 1,339 allegations of racist behaviour over the past three years, with just 14 of the cases being upheld.

In many cases complaints are made by fellow employees after suffering derogatory remarks or overhearing unacceptable phrases.

Senior figures with close knowledge of the Met say it was a scandal waiting to happen. Tarek Gafour, a former assistant commissioner at Scotland Yard, said: "The leadership took their eye off the ball on racism for some time."

The Met is being threatened with two legal challenges over allegations that it discriminates in its use of section 60 stop and search, which does not require reasonable suspicion.

— Guardian News & Media Ltd