Browsing by Author "Hayashi, Misato"
Now showing 1 - 3 of 3
Results Per Page
Sort Options
- Automated audiovisual behavior recognition in wild primatesPublication . Bain, Max; Nagrani, Arsha; Schofield, Daniel; Berdugo, Sophie; Bessa, Joana; Owen, Jake; Hockings, Kimberley J.; Matsuzawa, Tetsuro; Hayashi, Misato; Biro, Dora; Carvalho, Susana; Zisserman, AndrewThe field of ethology seeks to understand animal behavior from both mechanistic and functional perspectives and to identify the various genetic, developmental, ecological, and social drivers of behavioral variation in the wild (1). It is increasingly becoming a data-rich science: Technological advances in data collection, including biologgers, camera traps, and audio recorders, now allow us to capture animal behavior in an unprecedented level of detail (2). In particular, large data archives including both audio and visual information have immense potential to measure individual- and population-level variation as well as ontogenetic and cultural changes in behavior that may span large temporal and spatial scales. However, this potential often goes untapped: The training and human effort required to process large volumes of video data continue to limit the scale and depth at which behavior can be analyzed. Automating the measurement of behavior can transform ethological research, open up large-scale video archives for detailed interrogation, and be a powerful tool to monitor and protect threatened species in the wild. With rapid advances in deep learning, the novel field of computational ethology is quickly emerging at the intersection of computer science, engineering, and biology, using computer vision algorithms to process large volumes of data (3).
- Automated face recognition using deep neural networks produces robust primate social networks and sociality measuresPublication . Schofield, Daniel P.; Albery, Gregory F.; Firth, Josh A.; Mielke, Alexander; Hayashi, Misato; Matsuzawa, Tetsuro; Biro, Dora; Carvalho, SusanaLongitudinal video archives of behaviour are crucial for examining how sociality shifts over the lifespan in wild animals. New approaches adopting computer vision technology hold serious potential to capture interactions and associations between individuals in video at large scale; however, such approaches need a priori validation, as methods of sampling and defining edges for social networks can substantially impact results.Here, we apply a deep learning face recognition model to generate association networks of wild chimpanzees using 17 years of a video archive from Bossou, Guinea. Using 7 million detections from 100 h of video footage, we examined how varying the size of fixed temporal windows (i.e. aggregation rates) for defining edges impact individual-level gregariousness scores.The highest and lowest aggregation rates produced divergent values, indicating that different rates of aggregation capture different association patterns. To avoid any potential bias from false positives and negatives from automated detection, an intermediate aggregation rate should be used to reduce error across multiple variables. Individual-level network-derived traits were highly repeatable, indicating strong inter-individual variation in association patterns across years and highlighting the reliability of the method to capture consistent individual-level patterns of sociality over time. We found no reliable effects of age and sex on social behaviour and despite a significant drop in population size over the study period, individual estimates of gregariousness remained stable over time.We believe that our automated framework will be of broad utility to ethology and conservation, enabling the investigation of animal social behaviour from video footage at large scale, low cost and high reproducibility. We explore the implications of our findings for understanding variation in sociality patterns in wild ape populations. Furthermore, we examine the trade-offs involved in using face recognition technology to generate social networks and sociality measures. Finally, we outline the steps for the broader deployment of this technology for analysis of large-scale datasets in ecology and evolution.
- Chimpanzee face recognition from videos in the wild using deep learningPublication . Schofield, Daniel; Nagrani, Arsha; Zisserman, Andrew; Hayashi, Misato; Matsuzawa, Tetsuro; Biro, Dora; Carvalho, SusanaVideo recording is now ubiquitous in the study of animal behavior, but its analysis on a large scale is prohibited by the time and resources needed to manually process large volumes of data. We present a deep convolutional neural network (CNN) approach that provides a fully automated pipeline for face detection, tracking, and recognition of wild chimpanzees from long-term video records. In a 14-year dataset yielding 10 million face images from 23 individuals over 50 hours of footage, we obtained an overall accuracy of 92.5% for identity recognition and 96.2% for sex recognition. Using the identified faces, we generated co-occurrence matrices to trace changes in the social network structure of an aging population. The tools we developed enable easy processing and annotation of video datasets, including those from other species. Such automated analysis unveils the future potential of large-scale longitudinal video archives to address fundamental questions in behavior and conservation.