First-person vision & AI
We use wearable cameras and computational methods to study family interactions in everyday settings and to develop new ways of observing behaviour and health.
What does observing in the first person mean?
Instead of observing an interaction only from the outside, the camera follows the perspective of someone taking part in it.
Wearable cameras make it possible to record parts of an interaction from the point of view of mothers, children or other caregivers. This helps observe aspects of everyday life that can be hard to recover later through questionnaires or interviews alone.
The aim is not to replace what people report. It is to combine different ways of observing the same experience.
What we want to understand in interactions
Attention and engagement
How mothers, children and caregivers direct attention to one another and to their surroundings during an interaction.
Responsiveness and synchrony
How responses between two people are organised over time, and how moments of greater or lesser attunement appear in the interaction.
Parenting practices
How different ways of responding, guiding, comforting or dealing with difficult situations appear in caregiver–child relationships.
Context and mental health
How interaction patterns may relate to mental health trajectories, family conditions and experiences of climate disasters.
Where does artificial intelligence come in?
AI is an analysis tool — not the answer of the research.
Audiovisual recordings produce a great deal of information. Computational methods can help organise signals in these videos and turn observable patterns into measures that can be analysed systematically.
These measures can then be combined with questionnaires, mental health information and longitudinal data to study relationships that would be hard to observe using a single source of information.
The goal is not to diagnose people from a video, but to produce research data about interactions.
Photos of Cohort participants modified with artificial intelligence. The markers on the faces illustrate the analysis and are not research results.
How does a video become data?
Record
Wearable cameras record interactions in situations defined by the research protocol.
Organise
Recordings are prepared and synchronised to allow systematic analysis of the interactions.
Code
Trained teams and computational methods turn observable aspects of the videos into research variables.
Integrate
This information can be related to data on mental health, parenting, social context and longitudinal trajectories.
Videos require special protection
Audiovisual recordings are sensitive data. Participation depends on specific consent, and files are transferred to protected storage with access restricted to the authorised team.
Video processing follows institutional procedures for security and participant protection.
An approach connected to different projects
2019 Rio Grande Cohort
The Cohort provides the longitudinal history that makes it possible to relate interaction patterns to mental health, development and context over time.
Explore the Cohort →Floods and mental health
This project makes it possible to study how experiences of climate disasters, adversity and displacement relate to interactions between mothers and children.
Explore the project →Visões do Cuidado
The international network extends this approach to different countries, life stages and care contexts.
Explore the network →Opportunities to collaborate
We are open to dialogue with researchers and institutions interested in observational methods, longitudinal studies, family interactions, mental health and computational analysis of audiovisual data.
Propose a collaboration