Blending Human and Artificial Intelligence to Support Autistic Children’s Social Communication Skills

Article


Porayska-Pomsta, Kaśka, Keay-Bright, Wendy, Kossyvaki, Lila, Lemon, Oliver, Mademtzi, Marilena, Menzies, Rachel, Pain, Helen, Rajendran, Gnanathusharan, Waller, Annalu, Wass, S., Smith, Tim J., Alcorn, Alyssa M., Avramides, Katerina, Beale, Sandra, Bernardini, Sara, Foster, Mary Ellen, Frauenberger, Christopher, Good, Judith and Guldberg, Karen 2018. Blending Human and Artificial Intelligence to Support Autistic Children’s Social Communication Skills. ACM Transactions on Computer-Human Interaction. 25 (6), p. Art. 35. https://doi.org/10.1145/3271484
AuthorsPorayska-Pomsta, Kaśka, Keay-Bright, Wendy, Kossyvaki, Lila, Lemon, Oliver, Mademtzi, Marilena, Menzies, Rachel, Pain, Helen, Rajendran, Gnanathusharan, Waller, Annalu, Wass, S., Smith, Tim J., Alcorn, Alyssa M., Avramides, Katerina, Beale, Sandra, Bernardini, Sara, Foster, Mary Ellen, Frauenberger, Christopher, Good, Judith and Guldberg, Karen
Abstract

This article examines the educational efficacy of a learning environment in which children diagnosed with Autism Spectrum Conditions (ASC) engage in social interactions with an artificially intelligent (AI) virtual agent and where a human practitioner acts in support of the interactions. A multi-site intervention study in schools across the UK was conducted with 29 children with ASC and learning difficulties, aged 4--14 years old. For reasons related to data completeness and amount of exposure to the AI environment, data for 15 children was included in the analysis. The analysis revealed a significant increase in the proportion of social responses made by ASC children to human practitioners. The number of initiations made to human practitioners and to the virtual agent by the ASC children also increased numerically over the course of the sessions. However, due to large individual differences within the ASC group, this did not reach significance. Although no evidence of transfer to the real-world post-test was shown, anecdotal evidence of classroom transfer was reported. The work presented in this article offers an important contribution to the growing body of research in the context of AI technology design and use for autism intervention in real school contexts. Specifically, the work highlights key methodological challenges and opportunities in this area by leveraging interdisciplinary insights in a way that (i) bridges between educational interventions and intelligent technology design practices, (ii) considers the design of technology as well as the design of its use (context and procedures) on par with one another, and (iii) includes design contributions from different stakeholders, including children with and without ASC diagnosis, educational practitioners, and researchers.

JournalACM Transactions on Computer-Human Interaction
Journal citation25 (6), p. Art. 35
ISSN1073-0516
Year2018
PublisherAssociation for Computing Machinery (ACM)
Publisher's version
License
Supplemental file
Digital Object Identifier (DOI)https://doi.org/10.1145/3271484
Web address (URL)https://doi.org/10.1145/3271484
Publication dates
OnlineDec 2018
Publication process dates
Deposited15 May 2019
AcceptedAug 2018
AcceptedAug 2018
FunderEconomic and Social Research Council
Engineering and Physical Sciences Research Council
Economic and Social Research Council
Engineering and Physical Sciences Research Council
Copyright information© 2018 The authors
LicenseCC BY 4.0
Permalink -

https://repository.uel.ac.uk/item/8458v

Download files


Publisher's version
8026.pdf
License: CC BY 4.0


Supplemental file
  • 203
    total views
  • 372
    total downloads
  • 7
    views this month
  • 19
    downloads this month

Export as

Related outputs

Editorial perspective: Leaving the baby in the bathwater in neurodevelopmental research
Wass, S. and Jones, E. J. H. 2023. Editorial perspective: Leaving the baby in the bathwater in neurodevelopmental research. Journal of Child Psychology and Psychiatry. 64 (8), pp. 1256-1259. https://doi.org/10.1111/jcpp.13750
Sing to me, baby: Infants show neural tracking and rhythmic movements to live and dynamic maternal singing
Nguyen, T., Reisner, S., Luegner, A., Wass, S. V., Hoehl, S. and Markova, G. 2023. Sing to me, baby: Infants show neural tracking and rhythmic movements to live and dynamic maternal singing. Developmental Cognitive Neuroscience. 64 (Art. 101313). https://doi.org/10.1016/j.dcn.2023.101313
Form-function relationship in the amplitude and frequency modulations of infant - directed speech: A predictive processing perspective
Labendzki, P., Greenwood, E., Esposito, G., White, J., Viswanathan, N., Northrop, T., Perapoch Amado, M., Lancaster, K., Goupil, L. and Wass, S. 2023. Form-function relationship in the amplitude and frequency modulations of infant - directed speech: A predictive processing perspective. Lancaster Conference on Infant & Early Child Development. Lancaster UK 21 - 25 Aug 2023
The development of the relationship between auditory and visual neural sensitivity and autonomic arousal from 6 m to 12 m
Daubney, K., Suata, Z., Marriott Haresign, I., Thomas, M., Kushnerenko, E. and Wass, S. V. 2023. The development of the relationship between auditory and visual neural sensitivity and autonomic arousal from 6 m to 12 m. Developmental Cognitive Neuroscience. 63 (Art. 101289). https://doi.org/10.1016/j.dcn.2023.101289
The Sub-Second Dynamics of Spontaneous Mimicry: An Electromyography Study Tracking Infant Caregiver Dyads during Free Play
Viswanathan, N., Labendzki, P., Perapoch Amado, M., Ives, J., Greenwood, E., Northrop, T., de Klerk, C., Goupil, L. and Wass, S. 2022. The Sub-Second Dynamics of Spontaneous Mimicry: An Electromyography Study Tracking Infant Caregiver Dyads during Free Play. XXIII ICIS 2022 Developmental Cascades. Ottawa, Canada 07 - 10 Jul 2022
Assessing the Efficacy of Open-Source Solutions to Automated Facial Coding: A Methods-Comparison Study with EMG
Viswanathan, N., Labendzki, P., Perapoch Amado, M., Ives, J., Greenwood, E., Northrop, T., de Klerk, C., Goupil, L. and Wass, S. 2022. Assessing the Efficacy of Open-Source Solutions to Automated Facial Coding: A Methods-Comparison Study with EMG. Face2face: advancing the science of social interaction. 6-9 Carlton House Terrace, London SW1Y 5AG 04 Apr - 05 Aug 2022
DEEP: A dual EEG pipeline for developmental hyperscanning studies
Kayhan, E., Matthes, D., Marriott Haresign, I., Bánki, A., Michel, C., Langeloh, M, Wass, S. and Hoehl, S. 2022. DEEP: A dual EEG pipeline for developmental hyperscanning studies. Developmental Cognitive Neuroscience. 54 (Art. 101104). https://doi.org/10.1016/j.dcn.2022.101104
Oscillatory entrainment to our early social or physical environment and the emergence of volitional control
Wass, S., Perapoch Amado, M. and Ives, J. 2022. Oscillatory entrainment to our early social or physical environment and the emergence of volitional control. Developmental Cognitive Neuroscience. 54 (Art. 101102). https://doi.org/10.1016/j.dcn.2022.101102
Multimodal hyperscanning reveals that synchrony of body and mind are distinct in mother-child dyads
Reindl, V., Wass, S., Leong, V., Scharke, W., Wistuba, S., Wirth, C. L., Konrad, K. and Gerloff, C. 2022. Multimodal hyperscanning reveals that synchrony of body and mind are distinct in mother-child dyads. NeuroImage. 251 (Art. 118982). https://doi.org/10.1016/j.neuroimage.2022.118982
Infant Effortful Control Mediates Relations Between Nondirective Parenting and Internalising-Related Child Behaviours in an Autism-Enriched Infant Cohort
Smith, C. G., Jones, E. J. H., Wass, S., Pasco, G., Johnson, M. H., Charman, T., Wan, M. W. and The BASIS Team 2022. Infant Effortful Control Mediates Relations Between Nondirective Parenting and Internalising-Related Child Behaviours in an Autism-Enriched Infant Cohort. Journal of Autism and Developmental Disorders. 52, p. 3496–3511. https://doi.org/10.1007/s10803-021-05219-x
Automatic classification of ICA components from infant EEG using MARA
Marriott Haresign, I., Phillips, E., Whitehorn, M., Noreika, V., Jones, E. J. H., Leong, V. and Wass, S. 2021. Automatic classification of ICA components from infant EEG using MARA. Developmental Cognitive Neuroscience. 52 (Art. 101024). https://doi.org/10.1016/j.dcn.2021.101024
The origins of effortful control: How early development within arousal/regulatory systems influences attentional and affective control
Wass, S. 2021. The origins of effortful control: How early development within arousal/regulatory systems influences attentional and affective control. Developmental Review. 61 (Art. 100978). https://doi.org/10.1016/j.dr.2021.100978
What is the Difference between 'Good' and 'Bad' Stress? Understanding Possible Effects of Socio-economic Status on Learning, 2016-2018
Wass, S. 2021. What is the Difference between 'Good' and 'Bad' Stress? Understanding Possible Effects of Socio-economic Status on Learning, 2016-2018. UK Data Service. https://doi.org/10.5255/UKDA-SN-854793
Very preterm infants engage in an intervention to train their control of attention: results from the feasibility study of the Attention Control Training (ACT) randomised trial
Perra, O., Wass, S., McNulty, A, Sweet, D., Papageorgiou, K. A., Johnston, M., Bilello, D., Patterson, A. and Alderdice, F. 2021. Very preterm infants engage in an intervention to train their control of attention: results from the feasibility study of the Attention Control Training (ACT) randomised trial. Pilot and Feasibility Studies. 7 (Art. 66). https://doi.org/10.1186/s40814-021-00809-z
In Infancy, It’s the Extremes of Arousal That Are ‘Sticky’: Naturalistic Data Challenge Purely Homeostatic Approaches to Studying Self-Regulation
Wass, S., Smith, C. G., Clackson, K. and Mirza, F. U. 2020. In Infancy, It’s the Extremes of Arousal That Are ‘Sticky’: Naturalistic Data Challenge Purely Homeostatic Approaches to Studying Self-Regulation. Developmental Science. 24 (Art. e13059). https://doi.org/10.1111/desc.13059
Toward the Understanding of Topographical and Spectral Signatures of Infant Movement Artifacts in Naturalistic EEG
Georgieva, S., Lester, S., Noreika, V., Yilmaz, M. N., Wass, S. and Leong, V. 2020. Toward the Understanding of Topographical and Spectral Signatures of Infant Movement Artifacts in Naturalistic EEG. Frontiers in Neuroscience. 14 (Art. 352). https://doi.org/10.3389/fnins.2020.00352
Interpersonal Neural Entrainment during Early Social Interaction
Wass, S., Whitehorn, M., Marriott Haresign, I., Phillips, E. and Leong, V. 2020. Interpersonal Neural Entrainment during Early Social Interaction. Trends in Cognitive Sciences. 24 (4), pp. 329-342. https://doi.org/10.1016/j.tics.2020.01.006
Training attention control of very preterm infants: protocol for a feasibility study of the Attention Control Training (ACT)
Perra, O., Wass, S., McNulty, A., Sweet, D., Papageorgiou, K., Johnston, M., Patterson, A., Bilello, D. and Alderdice, F. 2020. Training attention control of very preterm infants: protocol for a feasibility study of the Attention Control Training (ACT). Pilot and Feasibility Studies. 6, p. Art. 17. https://doi.org/10.1186/s40814-020-0556-9
Research Review: Do parent ratings of infant negative emotionality and self‐regulation predict psychopathology in childhood and adolescence? A systematic review and meta‐analysis of prospective longitudinal studies
Kostyrka-Allchorne, K., Wass, S. and Sonuga‐Barke, E. J. S. 2019. Research Review: Do parent ratings of infant negative emotionality and self‐regulation predict psychopathology in childhood and adolescence? A systematic review and meta‐analysis of prospective longitudinal studies. Journal of Child Psychology and Psychiatry. 61 (4), pp. 401-416. https://doi.org/10.1111/jcpp.13144
Do helpful mothers help? Effects of maternal scaffolding and infant motivation on cognitive performance
Clackson, K., Wass, S., Georgieva, S., Brightman, L., Nutbrown, R., Almond, H., Bieluczyk, J., Carro, G., Rigby Dames, B. and Leong, V. 2019. Do helpful mothers help? Effects of maternal scaffolding and infant motivation on cognitive performance. Frontiers in Psychology. 10 (Art. 2661). https://doi.org/10.3389/fpsyg.2019.02661
14 challenges for conducting social neuroscience and longitudinal EEG research with infants
Noreika, V., Georgieva, S., Wass, S. and Leong, V. 2019. 14 challenges for conducting social neuroscience and longitudinal EEG research with infants. Infant Behavior and Development. 58 (Art. 101393). https://doi.org/10.1016/j.infbeh.2019.101393
Emotional valence modulates the topology of the parent-infant inter-brain network
Santamaria, L., Noreika, V., Georgieva, S., Clackson, K., Wass, S. and Leong, V. 2019. Emotional valence modulates the topology of the parent-infant inter-brain network. NeuroImage. 207 (Art. 116341). https://doi.org/10.1016/j.neuroimage.2019.116341
A psychophysiological investigation of the interplay between orienting and executive control during stimulus conflict: A heart rate variability study
Sørensen, L., Wass, S., Osnes, B., Schanche, E., Adolfsdottir, S., Svendsen, J. L., Visted, E., Eilertsen, T., Jensen, D. A., Nordby, H., Fasmer, O. B., Binder, P. E., Koenig, J. and Sonuga-Barke, E. 2019. A psychophysiological investigation of the interplay between orienting and executive control during stimulus conflict: A heart rate variability study. Physiology & Behavior. 211 (Art. 112657). https://doi.org/10.1016/j.physbeh.2019.112657
Influences of environmental stressors on autonomic function in 12-month-old infants: understanding early common pathways to atypical emotion regulation and cognitive performance
Wass, S., Smith, C. G., Daubney, K., Suata, Z. M., Clackson, K., Begum, A. and Mizra, F. U. 2019. Influences of environmental stressors on autonomic function in 12-month-old infants: understanding early common pathways to atypical emotion regulation and cognitive performance. Journal of Child Psychology and Psychiatry. 60 (12), pp. 1323-1333. https://doi.org/10.1111/jcpp.13084
Parents mimic and influence their infant’s autonomic state through dynamic affective state matching
Wass, S., Smith, C. G., Clackson, K., Gibb, C., Eitzenberger, J. and Mirza, F. U. 2019. Parents mimic and influence their infant’s autonomic state through dynamic affective state matching. Current Biology. 24 (14), pp. 2415-2422. https://doi.org/10.1016/j.cub.2019.06.016
Shorter spontaneous fixation durations in infants with later emerging autism
Wass, S., Jones, Emily J. H., Gliga, Teodora, Smith, Tim J., Charman, Tony and Johnson, Mark H. 2015. Shorter spontaneous fixation durations in infants with later emerging autism. Scientific Reports. 5 (1), p. Art. 8284. https://doi.org/10.1038/srep08284
Impaired Visual Search in Children with Rett Syndrome
Rose, Susan A., Wass, S., Jankowski, Jeffery J., Feldman, Judith F. and Djukic, Aleksandra 2018. Impaired Visual Search in Children with Rett Syndrome. Pediatric Neurology. 92, pp. 26-31. https://doi.org/10.1016/j.pediatrneurol.2018.10.002
Parental neural responsivity to infants’ visual attention: how mature brains influence immature brains during social interaction
Wass, S., Noreika, V., Georgieva, S., Clackson, K., Brightman, L., Nutbrown, R., Santamaria, L. and Leong, V. 2018. Parental neural responsivity to infants’ visual attention: how mature brains influence immature brains during social interaction. PLoS Biology. 16 (2), p. e2006328. https://doi.org/10.1371/journal.pbio.2006328
Attentional Shifting and Disengagement in Rett Syndrome
Rose, Susan A., Wass, S., Jankowski, Jeffery J., Feldman, Judith F. and Djukic, Aleksandra 2019. Attentional Shifting and Disengagement in Rett Syndrome. Neuropsychology. 33 (3), pp. 335-342. https://doi.org/10.1037/neu0000515
Speaker Gaze Increases Information Coupling Between Infant and Adult Brains
Leong, Victoria, Byrne, Elizabeth, Clackson, Kaili, Georgieva, Stanimira, Lam, Sarah and Wass, S. 2017. Speaker Gaze Increases Information Coupling Between Infant and Adult Brains. Proceedings of the National Academy of Sciences. 114 (50), pp. 13290-13295. https://doi.org/10.1073/pnas.1702493114
Toward a Neuroscientific Understanding of Play: A Dimensional Coding Framework for Analyzing Infant–Adult Play Patterns
Neale, Dave, Clackson, Kaili, Georgieva, Stanimira, Dedetas, Hatice, Scarpate, Melissa, Wass, S. and Leong, Victoria 2018. Toward a Neuroscientific Understanding of Play: A Dimensional Coding Framework for Analyzing Infant–Adult Play Patterns. Frontiers in Psychology. 9, p. Art. 273. https://doi.org/10.3389/fpsyg.2018.00273
New meanings of thin-skinned: The contrasting attentional profiles of typical 12-month-olds who show high, and low, stress reactivity.
Wass, S., de Barbaro, Kaya, Clackson, Kaili and Leong, Victoria 2018. New meanings of thin-skinned: The contrasting attentional profiles of typical 12-month-olds who show high, and low, stress reactivity. Developmental Psychology. 54 (5), pp. 816-828. https://doi.org/10.1037/dev0000428
Increases in Arousal are More Long-Lasting than Decreases in Arousal: On Homeostatic Failures During Emotion Regulation in Infancy
Wass, S., Clackson, Kaili and Leong, Vicky 2018. Increases in Arousal are More Long-Lasting than Decreases in Arousal: On Homeostatic Failures During Emotion Regulation in Infancy. Infancy. 23 (5), pp. 628-649. https://doi.org/10.1111/infa.12243
How orchids concentrate? The relationship between physiological stress reactivity and cognitive performance during infancy and early childhood
Wass, S. 2018. How orchids concentrate? The relationship between physiological stress reactivity and cognitive performance during infancy and early childhood. Neuroscience & Biobehavioral Reviews. 90, pp. 34-49. https://doi.org/10.1016/j.neubiorev.2018.03.029
Infants' visual sustained attention is higher during joint play than solo play: is this due to increased endogenous attention control or exogenous stimulus capture?
Wass, S., Clackson, Kaili, Georgieva, Stanimira D, Brightman, Laura, Nutbrown, Rebecca and Leong, Victoria 2018. Infants' visual sustained attention is higher during joint play than solo play: is this due to increased endogenous attention control or exogenous stimulus capture? Developmental Science. 21 (6), p. e12667. https://doi.org/10.1111/desc.12667
Elevated physiological arousal is associated with larger but more variable neural responses to small acoustic change in children during a passive auditory attention task
Wass, S., Daubney, K., Golan, J., Logan, F. and Kushnerenko, E. 2018. Elevated physiological arousal is associated with larger but more variable neural responses to small acoustic change in children during a passive auditory attention task. Developmental Cognitive Neuroscience. 37 (Art. 100612). https://doi.org/10.1016/j.dcn.2018.12.010
Erratum to: Attention training for infants at familial risk of ADHD (INTERSTAARS): study protocol for a randomised controlled trial
Goodwin, Amy, Salomone, Simona, Bolton, Patrick, Charman, Tony, Jones, Emily J.H., Mason, Luke, Pickles, Andrew, Robinson, Emily, Smith, Tim, Sonuga-Barke, Edmund J.S., Wass, S. and Johnson, Mark H. 2017. Erratum to: Attention training for infants at familial risk of ADHD (INTERSTAARS): study protocol for a randomised controlled trial. Trials. 18 (1), p. 419. https://doi.org/10.1186/s13063-017-2167-1
Changes in behaviour and salivary cortisol following targeted cognitive training in typical 12-month-old infants
Wass, S., Cook, Clare and Clackson, Kaili 2017. Changes in behaviour and salivary cortisol following targeted cognitive training in typical 12-month-old infants. Developmental Psychology. 53 (5), pp. 815-825. https://doi.org/10.1037/dev0000266
Sustained Attention in the Face of Distractors: A Study of Children with Rett Syndrome
Rose, Susan A., Wass, S., Jankowski, James J., Feldman, Judith F. and Djukic, Aleksandra 2017. Sustained Attention in the Face of Distractors: A Study of Children with Rett Syndrome. Neuropsychology. 31 (4), pp. 403-410. https://doi.org/10.1037/neu0000369
Stress reactivity speeds basic encoding processes in infants
de Barbaro, Kaya, Clackson, Kaili and Wass, S. 2016. Stress reactivity speeds basic encoding processes in infants. Developmental Psychobiology. 58 (5), pp. 546-555. https://doi.org/10.1002/dev.21399
Temporal dynamics of arousal and attention in 12-month-old infants
Wass, S., Clackson, K. and de Barbaro, K. 2016. Temporal dynamics of arousal and attention in 12-month-old infants. Developmental Psychobiology. 58 (5), pp. 623-639. https://doi.org/10.1002/dev.21406
Attention training for infants at familial risk of ADHD (INTERSTAARS): study protocol for a randomised controlled trial
Goodwin, Amy, Salomone, Simona, Bolton, Patrick, Charman, Tony, Jones, Emily J. H., Pickles, Andrew, Robinson, Emily, Smith, Tim, Sonuga-Barke, Edmund J. S., Wass, S. and Johnson, Mark H. 2016. Attention training for infants at familial risk of ADHD (INTERSTAARS): study protocol for a randomised controlled trial. Trials. 17 (1), p. 608. https://doi.org/10.1186/s13063-016-1727-0
Training Basic Visual Attention Leads to Changes in Responsiveness to Social‐Communicative Cues in 9‐Month‐Olds
Forssman, Linda and Wass, S. 2017. Training Basic Visual Attention Leads to Changes in Responsiveness to Social‐Communicative Cues in 9‐Month‐Olds. Child Development. 89 (3), pp. e199-e213. https://doi.org/10.1111/cdev.12812
Developmental Psychology: How Social Context Influences Infants’ Attention
Wass, S. and Leong, Victoria 2016. Developmental Psychology: How Social Context Influences Infants’ Attention. Current Biology. 26 (9), pp. R357-R359. https://doi.org/10.1016/j.cub.2016.03.051
Infant Attention Is Dynamically Modulated With Changing Arousal Levels
de Barbaro, Kaya, Clackson, Kaili and Wass, S. 2016. Infant Attention Is Dynamically Modulated With Changing Arousal Levels. Child Development. 88 (2), pp. 629-639. https://doi.org/10.1111/cdev.12689