Automated analysis of human motion using AI
In this project we use deep learning algorithms to design fully automated methods for analyzing human motion and classifying it into distinct patterns. One application of the methods developed in this project is automated choreographic analysis by identifying the distinct dance move patterns in specific genres such as ballet, Afrogenic dance, etc. Our results have been presented in a series of papers on this topic.
Description
Human movement has been studied in multiple disciplines, including health sciences (biomechanics, kinesiology, neurology, sports medicine) and the Arts (theater and dance, cultural studies) as well as their intersection (arts in medicine, dance therapy), resulting in a large but disparate assortment of multi-modal datasets, including video, skeletal motion capture, manual annotations, and clinical metadata.
Traditional data collection processes often include Laban movement analysis, a standardized form of human movement annotation that parameterizes the observed motion changes in a pre-defined 4-dimensional feature space (effort, space, shape, body timing/phrasing). Such analysis requires lengthy manual input from professionals who annotate the recorded data through a time-consuming “watch and pause” process, which is also prone to human errors.
In this project, we use AI to fully automate the annotation process involved in Laban analysis by training a Bayesian-layered (in the temporal domain) Convolutional Neural Network (CNN) through supervised learning on existing human motion datasets of video and skeletal sequences.
The trained model is beeing tested in Laban-annotating existing video and skeletal sequences and validated by arts in medicine practitioners and experts in Laban analysis. This AI-driven project will have a significant impact as it will enable automated classification and understanding of human motion across a spectrum of movement-centered disciplines, including clinical and telehealth settings, orthopedic centers, choreographic practice, and cross-cultural movement analysis.
