Event

ANNOUNCEMENT: Ph.D. Research Proposal Exam

Name: Thanushi Ruwanga Withanage

Committee:

Professor Carol Espy-Wilson (Chair)

Professor Shihab Shamma

Professor Ramani Duraiswami

Date/time: 04/21/2026, 3:30 PM 

Location: AVW 1146

Title:  Acoustic, Articulatory, and Facial Markers of Entrainment Toward Perceived Conversational Success in Spontaneous Speech Between Autistic and Non-autistic Individuals.

Abstract: 

Individuals naturally adapt their speaking patterns to align with their interlocutors, a phenomenon known as entrainment, which is closely associated with engagement, rapport, and interaction quality. While entrainment has been extensively studied in task-oriented settings, its role in naturalistic, non-task, and virtual conversations remains underexplored. In this work, we analyze a large corpus of spontaneous dyadic Zoom conversations and also a dyadic interaction dataset including autistic and non-autistic participants,  to investigate how multi modal conversational dynamics relate to perceived interaction quality. We extract features spanning turn-taking behavior, pause patterns, facial movements, articulatory data and acoustic cues such as pitch and intensity. Perceived conversational success is quantified using factor analysis of post-interaction self-report measures. Our results demonstrate that entrainment is reliably present in spontaneous speech and is positively associated with higher perceived conversational success. Specifically, high-success conversations are characterized by a greater number of turns with longer durations, shorter pauses, and stronger facial synchrony, whereas low-success conversations exhibit fewer, shorter turns and longer pauses.

Turn-level proximity and entrainment analyses revealed that pitch and speech intensity entrainment are significantly higher in high-success conversations compared to low-success conversations. In addition, across the 17 facial action units examined, synchrony was consistently greater in high-success conversation, particularly for those associated with smiling expressions. These findings suggest that synchrony in positive facial expressions, such as smiling, serves as a robust nonverbal marker of perceived conversational success. In contrast, synchrony in facial action units associated with negative affect contributes minimally to perceived success and, when present, tends to be higher in low-success conversations. Beyond acoustic and behavioral alignment, we focus on articulatory entrainment, the adaptation of vocal tract coordination during interaction, which remains largely underexplored in spontaneous dialogue, particularly among autistic speakers. We introduce a speaker-independent framework to quantify articulatory entrainment using acoustic-to-articulatory inversion and coordination complexity metrics. By analyzing temporal changes in articulatory coordination, we show that non-autistic dyads exhibit increasing coordination complexity and stronger entrainment over time, autistic dyads demonstrate moderate effects, and mixed-neurotype dyads show the least alignment. These patterns suggest that same-neurotype interactions facilitate greater interactional comfort and reduced articulatory effort as conversations progress, whereas mixed-neurotype interactions may involve sustained or increased articulatory effort, potentially reflecting adaptive strategies to maintain clarity and mutual intelligibility, consistent with Lindblom’s Hypo- and Hyper-articulation theory.

Importantly, greater articulatory entrainment is associated with higher perceived conversational success, with this relationship being more pronounced in same-neurotype interactions than in mixed-neurotype exchanges. These findings provide empirical support for the double empathy problem, highlighting that interactional differences arise from bidirectional mismatches rather than unilateral deficits. Overall, this proposal introduces a principled, multimodal framework for quantifying conversational alignment and identifies key behavioral and articulatory markers of successful interaction, offering insights into communication processes in both neurotypical and neurodiverse populations and informing the development of interventions to foster more effective and inclusive communication.