A study published in Frontiers, aiming to assess the user experience of a digital mental health chatbot, has found the intervention to provide a positive user experience and an acceptable intervention with participants.
The study entitled ‘User experience with a parenting chatbot micro intervention’ by authors Guido A Entenberg; Gilly Dosovitsky; Shirin Aghakhani; Karin Mostovoy; N. Carre; Zendrea Marshall; D. Benfica; S. Mizrahi; Alanna Testerman; Alexandra Rousseau; Grace Lin; and Eduardo L Bunge sought to examine engagement with the solution, along with completion rates, net promoter score, acceptability and satisfaction.
The chatbot ‘parenting micro intervention’ was designed based on an initial module of the Incredible Years parenting programme, with the general objective of teaching parents to use positive attention and praise to stimulate positive behaviours in children. An AI software to provide mental health care was used, with the intervention aiming to teach five skills for effective use of praise: focusing on the behaviours to encourage, being specific, avoiding combining praise with criticism, showing enthusiasm when praising, and praising immediately after good behaviour.
As part of a randomised and controlled trial study, 89 parents were assigned to the chatbot micro intervention. The study was conducted on Facebook; whilst using the social media platform in their daily life, participants were presented with posts with a link to the chat feature to initiate the intervention along with an explanation on how to start the conversation with the chatbot.
A user experience questionnaire was provided to assess acceptability, including ease of use, comfort, lack of technical problems, interactivity and usefulness for everyday life. A further questionnaire was used to assess participants’ satisfaction. Completion rate was analysed using frequency and percentage of participants who completed each skill; engagement with the intervention was analysed using the average number of messages and characters sent.
Along with assessing the user experience, the researchers sought to compare the results with those of other mental health chatbots.
They found that 66.3 percent of participants completed the intervention, a rate “nearly equal” to another single-session, web-based, self-guided parenting intervention and “comparable to brief self-guided parenting studies utilising other technologies”. They noted that the completion rate was lower than that reported in human-supported digital parenting interventions, and lower when compared to a preventatively focused face-to-face discussion on how to manage disruptive behaviours.
The researchers noted that the completion rate was double that of another single-session parenting intervention delivered in Finland early in the pandemic. The authors of the Finnish study suggested that the high drop-out rate “may have been due to participants having several support channels”. As such, the researchers here have suggested that “digital parenting interventions may be most useful in countries such as Argentina, where parents have fewer support resources”.
In regards to the attrition rates, the first skill (focusing on behaviours to encourage) accounted for over half the drop-outs in total. The first skill “demanded more interactions than the subsequent skills,” the authors wrote, “suggesting that more agile modules favour adherence.”
They also suggested that participants often drop out in chatbot interventions within the first stages as they may not perceive the need for an intervention or feel that too much information is being requested of them.
On engagement, the study saw an average of 49.8 messages during a 15 minute period, which the authors noted was similar to other studies.
Participants are said to have provided a high satisfaction score and reported that they would be very likely to recommend the chatbot to other parents; similar results can be found in other mental health chatbot studies. The researchers said: “It is possible that the high satisfaction level of the current chatbot was associated with the high ease of use and low rate of technical problems. Chatbots that tend to repeat questions or do not understand the user’s intention have been reported as a cause of user annoyance) and time constraints are frequently reported by parents as barriers to therapy.”
They added that the short duration of the intervention “may have promoted greater satisfaction by adjusting to their needs and not asking only a few questions. Therefore, brief chatbot conversations may be well suited for parents struggling with their children’s behaviours.”
Concluding, the researchers acknowledged a need to develop “novel forms of parent training” and highlighted that parents found the intervention to be “acceptable, [had] a positive user experience, were engaged, and were highly satisfied with the chatbot…. Chatbot intervention may align well with the participants’ interests (especially during a time of need, such as the COVID-19 pandemic). Future studies should focus on the efficacy of the intervention by measuring if parents learn the skills, improve their parenting self-efficacy and decrease disruptive behaviour in their children.”
Citation: Entenberg GA, Dosovitsky G, Aghakhani S, Mostovoy K, Carre N, Marshall Z, Benfica D, Mizrahi S, Testerman A, Rousseau A, Lin G and Bunge EL (2023) User experience with a parenting chatbot micro intervention. Front. Digit. Health 4:989022. doi: 10.3389/fdgth.2022.989022
