A group of neuroscientists believes it can communicate with "locked-in" coma patients

What if those coma patients thought to be beyond help could actually hear what we are saying? Could feel when pushed and prodded? Could respond to doctors, if only they could see the signs? A new group of neuroscientists thinks it has proved we can communicate with these “locked-in” patients – though not everyone believes them

"Imagine you wake up, locked inside a box," says Adrian Owen."It's a perfect fit, down to every last one of your fingers and toes. It's a strange box because you can listen to absolutely everything going on around you, yet your voice cannot be heard. In fact, the box fits so tightly around your face and lips that you can't speak, or make a noise. At first, this feels like a game. Then reality sets in. You see and hear your family lamenting your fate. You're too cold. Then too hot. You're always thirsty. The visits of your friends and family dwindle. Your partner moves on. And there's nothing you can do about it."

Owen and I are talking on Skype. I'm sitting in London while he's at the University of Western Ontario in Canada. Owen's reddish hair and close-cropped beard loom large on my screen as he becomes animated describing the torment of those with no voice: his patients.

People in a "vegetative state" are awake yet unaware. Their eyes can open and sometimes wander. They can smile, grasp another's hand, cry, groan or grunt. But they are indifferent to a hand clap, unable to see or understand speech. Their motions are not purposeful but reflexive. Their minds remain firmly shut. Still, when their eyelids flutter open, you are always left wondering if there's a glimmer of consciousness.

A decade ago, the answer would have been a bleak and emphatic no. Not any longer. Using brain scanners, Owen has found that some may be trapped inside their bodies yet able to think and feel to varying extents. The number of these cases has soared in recent decades, ironically, because doctors have steadily become better at saving patients with catastrophic injuries.

Today, trapped, damaged and diminished minds inhabit clinics and nursing homes worldwide – in Europe alone, the number of new coma cases is estimated to be 230,000 annually, of whom 30,000 will languish in a persistent vegetative state. They are some of the most tragic and expensive artefacts of modern intensive care.

Owen knows this only too well. In 1997, a close friend set off on her usual cycle to work. Anne had a weak spot on a blood vessel in her head, known as a brain aneurysm. Five minutes into her trip, the aneurysm burst and she crashed into a tree. She never regained consciousness.

The tragedy of Anne's accident would shape Owen's life. He began to wonder if there was a way to determine which of these patients were in an unconscious coma, which were conscious and which were somewhere in between?

That year, he had moved to the Medical Research Council's Cognition and Brain Sciences Unit in Cambridge, where researchers used various scanning techniques. One, positron emission tomography (PET), highlights different metabolic processes in the brain, such as oxygen and sugar use. Another, known as functional magnetic resonance imaging (fMRI), can reveal active centres in the brain by detecting the tiny surges in blood flow that take place as a mind whirrs. Owen wondered whether he could use these technologies to reach out to patients, like his friend, stuck between sensibility and oblivion.

Today, being alive is no longer linked to having a beating heart, explains Owen. If you are on a life-support machine, are you dead? Is a failure to sustain independent life a reasonable definition of death? No, otherwise we would all be "dead" in the nine months before birth.

The issue becomes murkier when we consider those trapped in the twilight worlds between normal life and death – from those who slip in and out of awareness, who are trapped in a "minimally conscious state", to those who are severely impaired in a vegetative state or a coma.

In the wake of the development of the artificial respirator during the 1950s in Denmark, pioneering work to categorise disorders of consciousness was carried out in the 1960s by the neurologist Fred Plum in New York and the neurosurgeon Bryan Jennett in Glasgow.

Plum coined the term "locked-in syndrome", in which a patient is aware and awake but cannot move or talk. With Plum, Jennett devised the Glasgow Coma Scale to rate the depth of coma, and Jennett followed up with the Glasgow Outcome Scale to weigh up the extent of recovery, from death to mild disability. Together they adopted the term "persistent vegetative state" for patients who, they wrote, "have periods of wakefulness when their eyes are open and move; their responsiveness is limited to primitive postural and reflex movements of the limbs, and they never speak". In 2002, Jennett was among a group of neurologists who chose the phrase "minimally conscious" to describe those who are sometimes awake and partly aware, who show erratic signs of consciousness so that at one time they might be able to follow a simple instruction and at another they might not.

Kate Bainbridge, a 26-year-old schoolteacher, lapsed into a coma three days after she came down with a flu-like illness. A few weeks after her infection had cleared, Kate awoke from the coma but was diagnosed as being in a vegetative state. Luckily, the intensive-care doctor responsible for her, David Menon, was also a principal investigator at the newly opened Wolfson Brain Imaging Centre in Cambridge, where Adrian Owen then worked.

Menon wondered whether elements of cognitive processing might be retained in patients in a vegetative state, and discussed with Owen how to use a brain scanner to detect them. In 1997, four months after she had been diagnosed as vegetative, Kate became the first patient in such a state to be studied by the Cambridge group. The results, published in 1998, were extraordinary. Her brain responses were indistinguishable from those of healthy volunteers; her scans revealed brain activity at the back of her brain, in a part which helps recognise faces. Kate became the first such patient in whom sophisticated brain imaging (in this case PET) revealed "covert cognition". Of course, whether that response was a reflex or a signal of consciousness was, at the time, a matter of debate.

The results were of huge significance not only for science but also for Kate and her parents. "The existence of preserved cognitive processing removed the nihilism that pervaded the management of such patients in general, and supported a decision to continue to treat Kate aggressively," recalls Menon.

Kate eventually surfaced from her ordeal, six months after the initial diagnosis. She described how she was indeed sometimes aware of herself and her surroundings. "They said I could not feel pain," she says. "They were so wrong."

Sometimes she'd cry out, but nurses thought it was nothing more than a reflex. Hospital staff had no idea how much she suffered in their care. Physiotherapy nurses never explained what they were doing to her. She was terrified when they removed mucus from her lungs. "I can't tell you how frightening it was, especially suction through the mouth," she has written. Her pain and despair became such that she tried to snuff out her life by holding her breath. "I could not stop my nose from breathing, so it did not work. My body did not seem to want to die."

Kate says her recovery was not so much like turning on a light but a gradual awakening. By then she had lost her job, her sense of smell and taste, and much of what might have been a normal future. Now back with her parents, Kate is still very disabled and needs a wheelchair. Yet, 12 years after her illness, she started to talk again and, though still angry about the way she was treated when she was at her most vulnerable, she remains grateful to those who helped her mind to escape.

She sent Owen a note:

"Dear Adrian, please use my case to show people how important the scans are. I want more people to know about them. I am a big fan of them now. I was unresponsive and looked hopeless, but the scan showed people I was in there. It was like magic, it found me." k

Nicholas Schiff is a neurologist at Weill Cornell Medical College in New York. His working life is a balancing act between putting the interests of his patients and their families first and keeping true to the science as he wrestles with disorders of consciousness. "There's a lot we don't know," he admits. "Frankly, I am wrong a lot of the time."

In 2005, Schiff applied his emerging understanding of the circuits of consciousness to Jim, a 38-year-old man who had been beaten and robbed and was left minimally conscious. Jim's eyes had mostly remained shut. He was unable to speak and could communicate only by a nod, or tiny eye or finger movements. His plight seemed hopeless. Eventually, Jim's mother gave a "do not resuscitate order" to doctors. Schiff thought differently.

Schiff had earlier scanned Jim with fMRI in 2001. His team had played subjects, including Jim, an audiotape in which a relative or loved one reminisced. In detailed fMRI scans, Jim had shown that, despite having a very underactive brain, he had preserved large-scale language networks. When he heard a story that meant something to him, his brain lit up. What, thought Schiff, if Jim's thalamus could be activated by deep brain stimulation?

A brain pacemaker was implanted into Jim. After its two electrodes delivered pulses of electricity to his thalamus, he was able to use words and gestures, respond reliably to requests, eat normally, drink from a cup, and carry out simple tasks such as brushing his hair. Schiff believes that once a brain re-engages with the world, it accelerates processes of repair. For the next six years, before Jim died of unrelated causes, he kept his mind above the minimally conscious state. "He could converse in short sentences reliably and consistently and make his wishes known," says Schiff. "He could chew and swallow and eat ice-cream and hang out. His family told us that they had him back." The case made the front page of the New York Times. "I prayed for a miracle," his mother told me at the time he was brought back. "The most important part is that he can say 'Mummy and Pop, I love you.' God bless those wonderful doctors. I still cry every time I see my son, but it is tears of joy."

In a forested campus south of Liège, Steven Laureys studies vegetative patients in research that dates back decades. Working there as part of the Cyclotron Research Centre in the 1990s, he was surprised when PET brain scans revealed that the patients could respond to a mention of their own name. Meanwhile, on the other side of the Atlantic, Nicholas Schiff was finding that partially working regions lay within catastrophically injured brains. What did it all mean?

At that time, doctors thought they already knew the answers: no patient in a persistent vegetative state was conscious. Medical practitioners, with the best intentions, thought it was perfectly acceptable to end the life of a vegetative patient by starvation and the withdrawal of water. This was the age of what Laureys calls "therapeutic nihilism".

What Owen, Laureys and Schiff were proposing was a rethink of some of the patients who were considered vegetative. A few of them could even be classed as being fully conscious and locked-in. The establishment was doggedly opposed. "The hostility we encountered [in the late 1990s] went well beyond simple scepticism," says Schiff. Looking back, Laureys pauses and smiles thinly: "Medical doctors do not like to be told they are wrong."

Then came 2006. Owen and Laureys were trying to find a reliable way to communicate with patients in a vegetative state, including Gillian. In July 2005, this 23-year-old had been crossing a road, chatting on her mobile phone. She was struck by two cars and diagnosed as vegetative.

Five months later, a strange piece of serendipity allowed Gillian to unlock her box. "I just had a hunch," says Owen. "I asked a healthy control [subject] to imagine playing tennis. Then I asked her to imagine walking through the rooms of her house." Imagining tennis activates part of the cortex called the supplementary motor area, involved in the mental simulation of movements. But imagining walking around the house activates the parahippocampal gyrus in the core of the brain, the posterior parietal lobe, and the lateral premotor cortex. So, if people were asked to imagine tennis for "yes" and walking around the house for "no", they could answer questions via fMRI.

Gazing into Gillian's "vegetative" brain with the brain scanner, he asked her to imagine the same things – and saw strikingly similar activation patterns to the healthy volunteers. It was an electric moment. Owen could read her mind.

Gillian's case, published in the journal Science in 2006, made front-page headlines around the world. The result provoked wonder and, of course, disbelief. "Broadly speaking, I received two types of email from my peers," says Owen. "'This is amazing – well done!' and 'How could you possibly say this woman is conscious?'"

As the old saw goes, extraordinary claims require extraordinary evidence. The sceptics suggested that it was wrong to make these "radical inferences" when there could be a more straightforward interpretation. Daniel Greenberg, a psychologist at the University of California, Los Angeles, suggested that, "the brain activity was unconsciously triggered by the last word of the instructions, which always referred to the item to be imagined".

Parashkev Nachev, a neurologist now at University College London, says he objected to Owen's 2006 paper not on grounds of implausibility or a flawed statistical analysis but because of "errors of inference". Although a conscious brain, when imagining tennis, triggers a certain pattern of activation, it does not necessarily mean the same pattern of activation signifies consciousness. The same brain area can be activated in many circumstances, Nachev says, with or without any conscious correlate. Moreover, he argues that Gillian was not offered a true choice to think about playing tennis. Just as a lack of response could be because of an inability to respond or a decision not to co- operate, a direct response to a simple instruction could be a conscious decision or a reflex.

What is needed is less philosophising and more data, says Owen. A follow-up study published in 2010 by Owen, Laureys and colleagues tested 54 patients with a clinical diagnosis of being in a vegetative state or a minimally conscious state; five responded in the same way as Gillian. Four were supposedly in a vegetative state at admission.

Owen, Schiff and Laureys have explored alternative explanations of what they observed and, for example, acknowledge that the brain areas they study when they interrogate patients can be activated in other ways. But the 2010 paper ruled out such automatic behaviours as an explanation, they say: the activations persist too long to signify anything other than intent. "You cannot communicate unconsciously – it is just not possible," says Owen. "We have won that argument".

Since Owen's 2006 Science paper, studies in Belgium, the UK, the US and Canada suggest that a significant proportion of patients who were classified as vegetative in recent years have been misdiagnosed – Owen estimates perhaps as many as 20 per cent. Schiff, who weighs up the extent of misdiagnosis a different way, goes further. Based on recent studies, he says around 40 per cent of patients thought to be vegetative are, when examined more closely, partly aware. Among this group of supposedly vegetative patients are those who are revealed by scanners to be able to communicate and should be diagnosed as locked-in, if they are fully conscious, or minimally conscious, if their abilities wax and wane.

There is anecdotal evidence that when contact is re-established with the occupant of a living box they are understandably morose, even suicidal. They have been ground down by frustration at their powerlessness, over the months, even years, it can take to recognise their plight. Yet the human spirit is resilient, so much so that they can become accustomed to life in this twilight state. In a survey of patients with locked-in syndrome, Laureys has found that when a line of communication is set up, the majority become acclimatised to their situation, even content (again, these insights took time to be accepted by the medical and scientific establishment – and even to be published in a scientific journal – reflecting the prevailing unease about the implications for hospitals and care homes).

The important question is detecting the extent to which such patients are conscious. Studies of large numbers of patients with brain injuries, and how they fare over the years, show that it makes a huge difference to the chance of recovery if a patient is minimally conscious rather than vegetative. The former have fragmentary understanding and awareness and may recover enough to return to work within a year or two.

Yet there are still surprises, such as the case of New York fireman Don Herbert, who awoke after a decade from a minimally conscious state caused by a severe brain injury suffered while fighting a fire in 1995. Schiff has used a technique called diffusion tensor imaging to show how a brain can rewire itself even decades after an injury – yet in the past year, even he has recommended withdrawing care from a man who had lain in a coma for eight weeks after a cardiac arrest. "I was wrong," he says. "This man is now back at work."

Parashkev Nachev has not changed his view since he first criticised Owen's work, and spelt out the basis of his unease in a more detailed paper published in 2010. "For every relative of a living PVS [persistent vegetative state] patient given (probably false) hope, another is burdened with the guilt of having acquiesced in the withdrawal of treatment from someone who – he has been led to believe – may have been more alive than it seemed," he says. "There are moral costs to false positives as well as to false negatives.

"I find the whole media circus surrounding the issue rather distasteful. The relatives of these patients are distressed enough as it is."

Laureys, Owen and Schiff spend a great deal of time with the families and understand these sensitivities only too well. Owen counters that, from his years of experience dealing with the families, they are grateful that doctors and scientists take an interest and are doing everything they can. "These patients have been short-changed over the years," he insists.

Owen is adamant that doctors have a moral duty to provide a correct diagnosis, even if the results cause guilt, unease or distress. "We must give every patient the best chance of an accurate diagnosis, so we can give them the appropriate care that goes along with that diagnosis."

Under the umbrella classification of "vegetative" lies a vast array of brain injuries and, as a result, even some of the most vocal critics accept that some vegetative patients are not as diminished as traditional measures suggest. Professor Lynne Turner-Stokes chairs a group for the Royal College of Physicians that is revising UK guidelines on "Prolonged Disorders of Consciousness". She remains unconvinced that the exceptional cases identified by Owen, Laureys and Schiff are particularly common or that enough has been done to establish brain scanners as a standard tool for routine diagnosis, particularly when the cost and convenience of these methods are taken into account. When it comes to extending these tests to all patients in a vegetative states as standard practice, "The evidence is just not there yet," she says.

But she stresses that she is simply being cautious, not sceptical, describing the work of Owen, Laureys and Schiff as "important and exciting". "We are only just beginning to scratch the surface," she says. "But I have no doubt [these techniques] will have a place, eventually, in the evaluation of patients."

Back on Skype, Owen smiles, considering whether to tell me what he is planning next. His partner, Jessica Grahn, also a neuroscientist, became pregnant at the start of 2013. What happens when consciousness winks on in the developing brain? He emails me a video of their unborn child, a montage of fMRI slices through their baby's head, as it twists and turns in Jessica's womb. "My colleagues have been doing fMRI on my wife's tummy every week for a few weeks now to see if we can activate the foetus's brain," he writes. "It is amazing."

Some names have been changed to protect identities. Adrian Owen's friend Anne remains in a vegetative state. Adrian Owen and Jessica Grahn's baby boy was born on 9 October 2013.

This is an edited version of an article published today by Mosaic (mosaicscience.com/story/mind-readers)