Project Details


Virtual human clinician helps with mental health diagnoses

Project URL:
Project Twitter: @USC

  • Health
  • Audiovisual
  • Physical Computing

When Canadian soldier Jody Mitic was sent to Afghanistan, he suffered a devastating landmine injury that severed both his feet. Undeterred, Mitic rebuilt his life and started a family and a business back in Canada. 

Like millions of soldiers around the world, Mitic carried a high risk of suffering Post Traumatic Stress Disorder (PTSD). PTSD can lead to depression and suicide. The US Veterans Administration says that veteran suicides have jumped 32% since 2001 and veterans between 18 and 29 years of age were six times more likely to take their own lives than civilians. 

PTSD is difficult to diagnose, which is why the team at the University of Southern California’s Institute for Creative Technologies (ICT) developed SimSensei – artificial intelligence with advanced emotional intelligence and facial recognition software called Multisense in the shape of a virtual clinician called ‘Ellie’.

Multisense automatically analyses over 60 facial expressions in real time and over the long term, recording a wide range of symptoms associated with depression, anxiety and PTSD. The results can then be reviewed by a human therapist to help inform treatment. 

ICT has now conducted sessions with more than 600 subjects, most of them soldiers returning from deployment. The results have been staggering: 88% of subjects agreed or strongly agreed that they were willing to share information with the completely automated Ellie. Many said they felt more comfortable speaking to a virtual human because they did not fear judgement, even when they knew researchers would review the results. 

When Mitic spoke to Ellie, he was able to recall a traumatic experience where he felt unable to help a civilian woman with a baby. Thankfully, his speech pattern, facial expression and tone showed that he was able to deal with the trauma and did not require treatment. 

SimSensei’s potential to improve mental healthcare for significantly more people is huge. Find out more at

Image courtesy of ResoluteSupportMedia

Last updated: 13th of September, 2016

Cookies on the Social Tech Guide

Social Tech Guide uses cookies to provide you with an enhanced user experience. To remove this message, please click continue.

Continue Learn more