When conversing, people tend to adjust their speech on several different dimensions to correspond the speech of their conversational partner(s). This ubiquitous tendency is called speech entrainment. Evidence of entrainment in verbal communication between people has been demonstrated at the phonetic level, like pronunciation of words, speech intensity, speaking rate or voice quality. Speech entrainment has also been linked to positive social outcomes. Subjects who entrain their speech to that of their conversational partners are seen more socially attractive, intelligent and supportive.
Similar parallel is also applicable to communication between human and machine. Human-tomachine communication approaches and solutions focused on socially-aware communication are being developed in recent decades. Machine, which is capable to communicate with human using speech entrainment, may have stronger influence over his opinion than other machines. This influence can be used, for example, in a combat situation involving army, when the soldier has to make a decision based on information provided by artificially intelligent droid. Another possible uses involve marketing, on-call support centers or automated dialog advertising. However, relationship between speech entrainment and a level of trust is yet to be proven.
To investigate the existence of such a relationship, we designed an interactive game for collecting data concerning this phenomenon. We measure the level of player’s trust by evaluating decisions he made under a condition of risk, that he based on another party’s advice. Our goal is to evaluate whether the advisor’s speech adaptation has an impact on these decisions.
Our game has a concept of an adventure game. Player is confronted with series of tasks and decision-making points. The game features two virtual helpers who give advice to the player. The helpers should evoke machines and communicate with the player verbally. We use speech recognition and vocal synthesis module provided by the Slovak Academy of Sciences.
Helpers are visually identical; however, their voices differ. Their visual identity decreases the number of aspects that can affect player’s decision. Their position, colour or even the order of spoken advices can influence a player; we only use the voice difference. One of the voices is adapted to player’s voice. To adapt the helper’s voice to the target player the vocal module analyses player’s voice in real-time and derives and uses optimal parameters for helper’s speech synthesis (e.g., tempo and pitch). Player is encouraged to converse with these helpers. His speech is subsequently analysed using the integrated vocal module.
Before player decides, he is allowed to ask helpers for guidance. Afterwards, he is presented with two different pieces of advice, one from each helper delivered in their respective voices. These recommendations not only differ, but are often contradicting one another. Players are therefore forced to consider which advice to take, while enduring the risk of choosing incorrectly, which might place them at a disadvantage. Once the decision is made and acted upon, we can observe and record which advice player favours and therefore which helper is considered to be more trustworthy.
Our game closely monitors the player and collects all data about his decisions during each gameplay session. At its end we are capable of determining whether entrainment of speech had a real impact on the subject in context of a human-machine communication.
We created an extensible prototype of a game that will serve as a tool for collecting research data. Our test group will be given a survey to fill out, before proceeding any further with the experiment. Afterwards, a psychological profile for each test subject is created. Finally, in-game decisions can be analysed and the impact of speech entrainment evaluated.
— Developer team
Eduard Kuric, Daniela Hajdu, Lukáš Marták, Aleš Mäsiar, Lukáš Miškovský, Zorka Moravčíková, Filip Šandor
TraSpi in media:
Kúzelný svet inšpirovaný Harry Potterom testuje dôveru k robotom