Max Planck Institute for Dynamics and Self-Organization -- Department for Nonlinear Dynamics and Network Dynamics Group
Personal tools
Log in

Automated Identification and Facial Expression Recognition of Primates in the Wild

Monitoring the behavior groups of animals is important in many different areas, such as cognitive neuroscience and animal welfare. While the recording of big datasets is easily feasible thanks to ever cheaper video recording devices and data storage costs, their analysis by experts is costly and time-consuming.

We are establishing an automated data analysis pipeline that detects the full body of primates as well as their faces on videos recordings, identifies the respective animals and predicts their facial expressions. In the respective stages we

1.) use a Single Shot MultiBox Detector (SSD) network for the face and body detection. The SSD is a deep learning method that uses a single deep neural network for a real-time (~30fps) detection of objects in images.
2.) identify facial landmarks for different primates and generate a hand-labeled facial landmark training set.
3.) train a facial landmark detector with an ensemble of regression trees method and use the facial landmarks to align the faces
4.). use a FaceNet network to identify the primates. FaceNet is a so-called siamese network, which is a type a neural network architecture that can learn how to distinguish between two inputs.
5.)  use a convolutional network with a VGGNet-type architecture to classify the facial expressions from the cropped (unaligned) faces