How tech can fill gaps in mental health care

As the clouds of mental illness gather, it can be difficult for patients to recognize their own symptoms and find necessary help to navigate storms like episodes of depression or schizophrenia.

With $1.2 million in new funding from the National Science Foundation, University of Maryland researchers are creating a computerized framework that could one day lead to a system capable of a mental weather forecast of sorts. It would meld language and speech analysis with machine learning and clinical expertise to help patients and mental health clinicians connect and head off crises while dealing with a sparsely resourced U.S. mental health care system.

“We’re addressing what has been called the ‘clinical white space’ in mental health care, when people are between appointments and their doctors have little ability to help monitor what’s happening with them,” said Philip Resnik, a professor of linguistics with a joint appointment in the University of Maryland Institute for Advanced Computer Studies (UMIACS) who is helping to lead the research.

The project was born with the help of a seed grant through the AI + Medicine for High Impact (AIM-HI) Challenge Awards, which bring together scholars at the University of Maryland, College Park (UMCP) with medical researchers at the University of Maryland, Baltimore (UMB) on major research initiatives that link artificial intelligence and medicine. Deanna Kelly, a professor of psychiatry at the University of Maryland School of Medicine, is another of the project’s leaders, as are electrical and computer engineering Professor Carol Espy-Wilson and computer science Assistant Professor John Dickerson, both at UMCP.

The new funding will help the research team pour their diverse expertise into a single framework, which would then be developed into a deployable system for testing in a clinical setting.

How would such a system work? Users might answer a series of questions about physical and emotional well-being, with the system employing artificial intelligence to analyze word choice and language use—Resnik’s area of focus in the project. It could also monitor the patient’s speech patterns, analyzing changes in the timing and degree of movement made by the lips and different parts of the tongue, and comparing it to a baseline sample taken from healthy control subjects or earlier when the participant was in remission, said Espy-Wilson, who has an appointment in the Institute for Systems Research.

People generally overlap neighboring sounds when speaking, beginning the next sound before finishing the previous one, a process called co-production. But someone suffering from depression, for instance, has simpler coordination, and their sounds don’t overlap to the same extent.

“You can't think as fast, you can't talk as fast when you’re depressed,” said Espy-Wilson. “And when you talk, you have more and longer pauses … You have to think more about what you want to say. The more depressed you are, the more of the psychomotor slowing you're going to have.”

While the final form of the system has yet to take shape, it could potentially live in an app on patients’ phones, and with their permission, automatically monitor their mental state and determine their level of need for clinical intervention, as well as what resources are available to help.

If the system simply directed streams of patients at already overloaded doctors or facilities with no open beds, it could potentially make things worse for everyone, said Dickerson, who has a joint appointment in UMIACS.

He’s adding his expertise to work that Resnik and Espy-Wilson have been pursuing for years, and taking on the central challenge—using an approach known in the machine learning field as the “multi-armed bandit” problem—of creating a system that can deploy limited clinical resources while simultaneously determining how to best meet a range of evolving patient needs. During development and testing, the AI system’s determinations will always be monitored by a human overseer, said Dickerson.

The World Health Organization estimated a decade ago that the cost of treating mental health issues between 2011 and 2030 would top $16 trillion worldwide, exceeding cardiovascular diseases. The stresses of the COVID-19 pandemic have exacerbated an already high level of need, and in some cases resulted in breakdown conditions for the system, said Kelly, director of the Maryland Psychiatric Research Center’s Treatment Research Program.

As the project develops, the technology could not only connect patients with a higher level of care to prevent worsening problems (avoiding costlier care), but also might help clinicians understand which patients don’t need hospitalization. Living in the community with necessary supports is often healthier than staying in a psychiatric facility—plus it’s cheaper and frees up a hospital bed for someone who needs it, she said.

“Serious mental illness makes up a large portion of health care costs here in the U.S. and around the world,” Kelly said. “Finding a way to assist clinicians in preventing relapses and keeping people well could dramatically improve people’s lives, as well as save money.”

—Thank you to Chris Carroll of Maryland Today for this article. Aadit Tambe M.Jour. ’22 also contributed.

About the funding
Espy-Wilson is the principal investigator on a four-year, $842,431 NSF “Smart and Connected Health” grant, Using Multi-Stage Learning to Prioritize Mental Health. Her co-PIs are Resnik and Dickerson.

Kelly is funded on a separate, related $307,565 NSF grant, Using Multi-Stage Learning to Prioritize Mental Health Risk Using Evidence from Speech and Text. Associate Professor Shuo Chen of UMSoM is Kelly’s co-PI.

Additional information
On June 8, 2021, Espy-Wilson gave a keynote address about this work at the 180th meeting of the Acoustical Society of America. | Learn more about Espy-Wilson’s research |

Published August 18, 2021