Cholesterol calculator ‘greatly overestimates risk’

Last week, America’s leading heart organisations released a sweeping new set of guidelines for lowering cholesterol, along with an online calculator meant to help doctors assess risks and treatment options.

But, in a major embarrassment to the health groups, the calculator appears to greatly overestimate risk, so much so that it could mistakenly suggest that millions more people are candidates for statin drugs.

The apparent problem prompted one leading cardiologist, a past president of the American College of Cardiology, to call for a halt to the implementation of the new guidelines.

“It’s stunning,” said the cardiologist, Dr Steven Nissen, chief of cardiovascular medicine at the Cleveland Clinic. “We need a pause to further evaluate this approach before it is implemented on a widespread basis.”

The controversy set off turmoil at the annual meeting of the American Heart Association, which started this weekend in Dallas. After an emergency session last week, the two heart organisations that published the guidelines - the American Heart Association and the American College of Cardiology - said that while the calculator was not perfect, it was a major step forward, and that patients and doctors should discuss treatment options rather than blindly following a calculator.

Dr Sidney Smith, the executive chairman of the guideline committee, said the associations would examine the flaws found in the calculator and determine if changes were needed. “We need to see if the concerns raised are substantive,” he said. “Do there need to be changes?”

The problems were identified by two Harvard Medical School professors whose findings will be published today in a commentary in The Lancet, a major medical journal. The professors, Dr Paul M. Ridker and Dr Nancy Cook, had pointed out the problems a year earlier when the National Institutes of Health’s National Heart, Lung, and Blood Institute, which originally was developing the guidelines, sent a draft to each professor independently to review. Both reported back that the calculator was not working among the populations it was tested it on by the guideline makers.

That was unfortunate because the committee thought the researchers had been given the professors’ responses, said Dr Donald Lloyd-Jones, co-chairman of the guidelines task force and chairman of the department of preventative medicine at Northwestern University.

Ridker and Cook saw the final guidelines and risk calculator last week, and saw that the problems remained.

Soon thereafter, members of the association and the college of cardiology held a hastily called closed-door meeting with Ridker, who directs the centre for cardiovascular disease prevention at Brigham and Women’s Hospital in Boston. He showed them his data and pointed out the problem.

Officials from the organisations struggled with how to respond. Are the risk calculators using current data?

Other experts said there has not been a real appreciation of the difficulties with this and other risk calculators.

“I don’t think people have a good idea of what needs to be done,” said Dr Michael Blaha, director of clinical research at the Ciccarone Centre for the Prevention of Heart Disease at Johns Hopkins University, who was not associated with forming the new guidelines.

Blaha said the problem might have stemmed from the fact that the calculator uses as reference points the results from various studies conducted more than a decade ago, when more people smoked and had strokes and heart attacks earlier in life.

For example, the guideline makers used data from studies in the 1990s to determine how various risk factors like cholesterol levels and blood pressure led to actual heart attacks and strokes over a decade of observation.

But people have changed in the past few decades, Blaha said. Among other things, there is no longer such a big gap between women’s risks and those of men at a given age. And people get heart attacks and strokes at older ages. “The cohorts were from a different era,” Blaha said.

Once upon a time

Ridker and Cook evaluated the risk calculator using three large studies that involved thousands of people and continued for at least a decade. They knew the subjects’ characteristics at the start - their ages, whether they smoked, their cholesterol levels, their blood pressures. Then they asked how many had heart attacks or strokes in the next 10 years and how many would the risk calculator predict.

The answer was that the calculator over-predicted risk by 75 to 150 per cent, depending on the population. A man whose risk was 4 per cent, for example, might show up as having an 8 per cent risk. With a 4 per cent risk, he would not warrant treatment - the guidelines say treatment is advised for those with at least a 7.5 per cent risk and that treatment can be considered for those whose risk is 5 per cent.

“Miscalibration to this extent should be reconciled and addressed before these new prediction models are widely implemented,” Ridker and Cook wrote in The Lancet.

“If real, such systematic overestimation of risk will lead to considerable over-prescription [of statins].”

In a response, Smith of the guidelines committee said the concerns raised by Cook and Ridker “merit attention”.

But, he continued, “a lot of people put a lot of thought into how can we identify people who can benefit from therapy”. Further, said Smith, who is also a professor of medicine at the University of North Carolina and a past president of the American Heart Association, “What we have come forward with represents the best efforts of people who have been working for five years.”

Asked to comment on the situation, some doctors said they worried that, with many people already leery of statins, the public would lose its trust in guidelines or the heart associations.

“We’re surrounded by a real disaster in terms of credibility,” said Dr Peter Libby, the chairman of the department of cardiovascular medicine at Brigham and Women’s Hospital.

What are patients and doctors to do?

There seemed to be no firm answers, except that those at the highest risk, like people who have had a heart attack or have diabetes, should take statins.

The guideline developers said they were not totally surprised by the problems with the calculator.

“We recognise a potential for overestimates, especially at the high end of risk,” said Dr David Goff, the dean of the University of Colorado School of Public Health and the co-chairman of the guidelines’ risk assessment working group.

Not long after receiving the assessments from Ridker and Cook, the National Heart, Lung and Blood Institute removed itself from the development of the guidelines, saying that was not its mission. The institute handed responsibility to the American Heart Association and the American College of Cardiology.

Michael Lauer, the director of the division of cardiovascular sciences at the institute, said that it had received many reviews and sent them to the other groups, together with the responses of the guidelines’ authors.

“We left it to them to take it through their vetting process,” Lauer said.

And the groups did make changes in response to Ridker’s and Cook’s comments, Lloyd-Jones said.

Some doctors who tested the calculator with hypothetical patients wondered if they should trust the results.

Nissen entered the figures for a 60-year-old African-American man with no risk factors - total cholesterol of 150, HDL (the good cholesterol) of 45, systolic blood pressure of 125 - who was not a diabetic or a smoker. He ended up with a 10-year risk of 7.5 per cent, meaning he should be taking cholesterol-lowering statins despite being in a seemingly low-risk group.

Nissen also calculated the figures for a healthy white man, age 60, and also got a risk factor of 7.5 per cent.

“Something is terribly wrong,” Nissen said. Using the calculator’s results, he said, “your average healthy Joe gets treated, virtually every African-American man over 65 gets treated.”