In the years that we have known each other, she has never asked me an opinion about her brother’s cancer, so when she finally asks if I will talk to him, I can’t help but say yes. I have heard about his trajectory over several months — how he has tried one chemotherapy after another, how he is becoming increasingly fatigued, and worryingly, how much time he has spent in hospital lately, fighting one or the other complication of treatment.

“I wish we knew what the future holds,” she says, overcome by emotion. My heart melts at her predicament. Her brother calls me. “The oncologist has put chemotherapy on hold while the lung specialist treats my pneumonia,” he says uncertainly. “That’s a good thing,” I reply.

“Does my chemo sound right to you?”

“I can assure you it’s what most oncologists would use.”

“I just can’t seem to hold my own — there’s always some problem.”

“I can see, and I am sorry.”

He goes through a charge sheet of ill-controlled symptoms, chief amongst them pain. The knowledgeable way in which he touches on targeted therapies, clinical trials and also his evident frailty tells me he is more educated than many other patients, yet he can’t seem to see that four lines of failed chemotherapy, each bout of weight loss and each unit of blood transfusion add up to a serious downturn in his prognosis that needs considering.

A discussion about his prognosis would be the entry point to a range of conversations about the futility of further aggressive chemotherapy, the value of palliative care and his wishes for care at the end of life but although he is receiving the best drugs on offer, what seems to be missing is an articulation of the goals of care at this last stage of his illness.

Having seen many patients die under a cloud of misinformation or worse, no information, I find the revelation upsetting but I recognise my bystander status. Also, I don’t know what serious news his oncologist might have shared. So in the end my growing concern manifests as a mild injunction: “Talk to your oncologist about the big picture — ask him to help you understand the future because it will help you make important decisions.”

His answer leaves me speechless: “He is not that kind of an oncologist.”

The authors of the influential Support study concluded as far back as 1995 that “the most fundamental choice that patients with incurable cancer face — the decision between life-extending therapy and comfort care — may be highly influenced by their understanding of their prognoses.”

Other studies have followed, including one in the New England Journal of Medicine showing that more than two thirds of patients with advanced, incurable, cancer mistakenly believed their chemotherapy was being given with curative intent. In the author’s blunt but memorable conclusion, “a focus on chemotherapy was the instrument that facilitated prognostic misunderstanding.”

In other words, when oncologists shift the conversation from one of prognosis to that of drug options and treatment schedules, there is a missed opportunity to provide holistic care.

That study changed my thinking, but as a new study in JAMA reveals, the vexed matter of oncologists and their discussion of prognosis shows little sign of being settled. In the study, 236 cancer patients were asked: “What do you believe the chances are that you will live for two years or more?”

Their 38 oncologists were asked: “What do you believe are the chances that this patient will live for two years or more?” Sixty-eight per cent of patients rated their two-year survival discordantly from their oncologist. A staggering 89 per cent of these patients were not aware that they and their oncologist held different opinions of their prognosis, and nearly all were more optimistic than their oncologist, despite the oncologist believing that a “completely thorough” discussion had occurred.

Most patients with an inaccurate estimation of their prognosis said that they desired palliative care input when they neared the end of life but if they didn’t appreciate their prognosis, they might easily miss the opportunity and keep receiving aggressive treatment.

I found this study elegant, brave and, to be honest, worthy of the headlines it will probably never make as long as a new drug discovery waits in the wings. The patients deserve credit for answering a confronting question about their mortality and discovering that they were mostly wrong. The oncologists deserve praise for participating in a study they probably expected to highlight a failing that they share with the wider profession.

In the wake of such studies, oncologists are left to rue many things, including poor communication, denial by the patient and collusion, where the oncologist and patient have an unspoken agreement to avoid certain topics. Tellingly, there is also evidence to suggest that patients perceive oncologists as better communicators when they deliver optimistic news. No one doubts that each side faces its own challenges to discussing prognosis and then, its consequences, but if medicine is to make good its promise of patient-centred care, the barriers must be broken.

Seven 7 million people die of cancer each year, the majority in the developing world. Unfortunately, some things in medicine change so slowly that many more will die while doctors debate the best way to deliver bad news. Where chemotherapy is unavailable and you need connections to get morphine, communicating prognosis seems an even lower priority. Elsewhere, in spite of the rhetoric, communication skills training is typically a low institutional priority. But to a great many oncologists around the world, getting communication right is becoming personally significant.

Most say they don’t deliberately withhold bad news or worse, fabricate good news, but if our patients claim they’re in the dark, we all have some soul-searching to do.

Patients themselves shouldn’t underestimate their power to nudge change. No doctor doubts the imperative to keep up with the science of medicine; a stronger individual and community expectation is required to finesse the art of medicine. Patients are entitled to honest, prognostic information that is delivered with sensitivity and empathy — it should be no less an expectation than informed consent about surgery or open disclosure of a medical error.

But patients who want to know the truth must be prepared to hear the truth, and not conflate bad news with an incompetent doctor, which is the fear that sets many doctors back when it comes to giving any kind of bad news. Importantly, patients should know that misconceptions regarding prognosis transcend the boundaries of age, gender, education and income. You can be educated, wealthy and conversant with your oncologist but still be unaware of your prognosis.

And if you happen to be non-white or non-English speaking, you are especially likely to be in the dark. The desire for a dignified and peaceful death is natural and universal but we know that there is substantial disparity in how people experience end of life care — this is something we can collectively change. Since we are just as likely as the next person to not know our prognosis, it must be our common mission to improve truth-telling in medicine.

— Guardian News & Media Ltd

Dr Ranjana Srivastava is an Australian oncologist, a Fulbright scholar and an award-winning author. Her latest book is After Cancer: A Guide to Living Well.