Summary: A new study explores the use of virtual reality and AI to improve public speaking skills and reduce anxiety. PresentationPro simulates realistic presentation settings with AI-driven audience avatars that provide real-time feedback. This approach provides a safe and reproducible practice environment, potentially benefiting a wide range of users beyond students.
Highlights:
- PresentationPro uses AI and VR to create realistic audience interactions.
- The system tracks physical signals and speech patterns to simulate real feedback.
- The project aims to help users build confidence and reduce anxiety when speaking in public.
Source: Singapore Management University
If you dread speaking in public, you’re not alone. This is a major social phobia, which can cause a state of anxiety that reduces otherwise articulate people to nervous incoherence.
A strong fear of public speaking is known as glossophobia. Academic studies estimate that it affects 20 percent of the population, but depending on the sample and methodology, that figure could be as high as 40 percent.
As American writer and humorist Mark Twain said: “There are two types of speakers: those who get nervous and those who lie. »
But help could be on the way. Kyong Jin Shim, associate professor of information systems at Singapore Management University (SMU), is leading a research project that explores the integration of virtual reality (VR) technology and AI to improve taking skills public speaking for students.
And while the research focuses specifically on evaluating the effectiveness of using this technology for developing university students’ public speaking skills, the methodology could have broader applications.
The project received a grant from the MOE’s Higher Education Research Fund (TRF) and the proposed solution is called PresentationPro.
“(Through headsets), presenters will see a three-dimensional virtual environment that mimics a real-life presentation environment, with a crowd of AI-driven avatars representing an audience,” says Professor Shim.
“These avatars will display behaviors typical of a live audience, such as nodding, making eye contact, showing various expressions, and providing real-time feedback to the presenter,” says Professor Shim.
In a high-tech update on practice makes perfect, PresentationPro aims to provide presenters a way to hone their public speaking skills without the logistical challenges of building a live audience for each student.
The team is collaborating with SMU’s Center for English Communication (CEC) to translate its “presentation” know-how and best practices into a digital platform, and potentially expand CEC’s communication coaching.
Avatar Triggers
VR content, including audience avatars, is generated through a combination of advanced computer graphics and AI algorithms. Making avatars responsive in real time is no easy feat.
“This is achieved through sophisticated AI programming that includes natural language processing (NLP) and behavior modeling. The system uses machine learning to analyze the presenter’s speech and body language, allowing avatars to respond realistically and in real time to verbal and non-verbal cues,” explains Professor Shim.
Working with SMU’s Center for Teaching Excellence (CTE), Professor Shim’s faculty team leveraged CTE’s expertise in classroom management and its wealth of knowledge about different types of behaviors that may manifest in classroom “presentation” scenarios. Student and faculty/instructor behaviors play a crucial role in engineering the behaviors of PresentationPro’s “audience avatar” using AI.
But can avatars interrupt the presenter?
“Yes, avatars can interrupt and ask questions, simulating dynamic interaction typical of a real audience. This capability is made possible by the integration of NLP and speech recognition technologies, allowing avatars to process spoken language and respond appropriately,” explains Professor Shim.
Presenters’ physical signals will also be monitored.
“In addition to heart rate tracking with Fitbits, the system uses VR headsets such as Meta Quest equipped with head and gaze tracking technology to monitor where the presenter is looking, such as whether they are avoiding contact visual while staring at his feet. Gesture tracking is also used to detect other physical behaviors like agitation,” says Professor Shim.
Verbal triggers for avatars are configured using a combination of speech recognition and sentiment analysis technologies.
“These triggers are calibrated to recognize various speech patterns and anomalies such as tics, stutters, or deviations from the script, which then prompt avatars to react in specific ways that mimic the response of a real audience,” explains Professor Shim.
Behavioral changes
Researchers generated digital twins, which are highly detailed digital replicas of human behaviors and interactions – just like individuals – ensuring a diverse and realistic audience simulation, mirroring a typical SMU classroom.
“Using different avatars helps avoid repetition and predictability of audience reactions, enhancing the realism of the virtual environment and mimicking a typical seminar or classroom,” explains Professor Shim.
“VR and AI can simulate realistic social interactions, which can help individuals practice and improve their public speaking skills in a low-risk environment. Repeated exposure and positive reinforcement through VR can reduce anxiety, build confidence and lead to behavioral changes.
“Improvements will be measured through both subjective assessments (participant and instructor feedback) and objective measures (performance data collected during virtual reality sessions and traditional in-person assessments). Comparisons will be made between the control and experimental groups to assess the effectiveness of the VR training,” explains Professor Shim.
Transformation tool
Interestingly, for a project focusing on behavior change, no psychologists were among the project’s expert investigators at the outset.
“The research team is mainly composed of specialists in educational technology, AI and public speaking, focusing on the technological and pedagogical aspects of the project,” explains Professor Shim.
“While psychologists play a crucial role in understanding and managing anxiety, the current scope of our project focuses on the development and integration of AI-based solutions for public speaking training in public.
“Nevertheless, we recognize the value of interdisciplinary collaboration and are very open to partnering with social science experts to improve our understanding of anxiety management.
“Such collaborations could lead to additional improvements to our VR system, ultimately enriching the learner experience by more effectively addressing public speaking anxiety.”
Professor Shim has since added SMU psychology assistant professor Andree Hartanto to the team to explore:
- Psychological mechanisms by which virtual reality can reduce glossophobia;
- Long-term impacts of VR training on public speaking anxiety; And
- Differential Effects of VR Training on Various Demographic Groups
Professor Shim’s journey into VR applications began in 2021 with a prototype designed to train new faculty at SMU.
“My personal experiences as a faculty member grappling with the challenges of adapting to a new cultural and academic environment have deeply influenced this initiative. During my first few years at SMU, I found it particularly intimidating to lecture in front of a seminar-style class of 45 students from diverse backgrounds,” she says.
“As I transitioned into a mentoring role for new faculty, I realized how beneficial immersive technologies like virtual reality could be in accelerating the onboarding process for new faculty.
“This technology allows them to practice their lessons in their own time and space, repeat sessions as needed, and eliminate the logistical challenges of scheduling real seminar rooms and real audiences.”
“Inspired by the potential of this first application, we decided to develop a similar VR system to improve students’ speaking skills. This project not only leverages my education-related research in collaboration with CTE, but also builds on our foundational work in virtual reality, aiming to provide a transformative educational tool for a wider audience,” says Professor Shim.
About this research news on AI, virtual reality and social anxiety
Author: Lijie Goh
Source: Singapore Management University
Contact: Lijie Goh – Singapore Management University
Picture: Image is credited to Neuroscience News