What is it about?

This is an interpretable and ready-to-use pipeline for modelling engagement in human-robot interactions. In addition to predicting engagement, the pipeline provides deeper insights into a person's personality, attitude, and emotion. This innovative approach incorporates the Big Five personality traits, the Interpersonal Circumplex (IPC), and the Triandis Theory of Interpersonal Behavior (TIB).

Featured Image

Why is it important?

Prior work uses machine learning directly to predict engagement given multimodal data. This work uses theories from psychology -- the Big Five personality traits, the Interpersonal Circumplex, and Triandis' Theory of Interpersonal Behavior, to emulate how a human understands another's, emotion, personality, and engagement given multimodal cues. The results are thus more interpretable and explainable than those obtainable from standard "black box" ML models.


There is a lot of interest these days in "physics informed" machine learning. This work can be called "psychology informed" machine learning using computer vision and related data.

Shrisha Rao
International Institute of Information Technology - Bangalore

Read the Original

This page is a summary of: From multimodal features to behavioural inferences: A pipeline to model engagement in human-robot interactions, PLoS ONE, November 2023, PLOS,
DOI: 10.1371/journal.pone.0285749.
You can read the full text:

Open access logo



The following have contributed to this page