ESR 6

Face analysis and body language understanding from egocentric cameras

Hiring Institution

University of Catania

PhD enrollment

Doctoral School in Computer Science of the University of Catania

YOUR TEAM: You will work at the Department of Mathematics and Computer Science of the University of Catania.

GOAL: The objective of this project is to develop algorithms to enable a mobile egocentric vision system on board to an agent, e.g., a robot, to observe the user during a conversation and infer socially-relevant information which could help improve human-machine interaction. The developed algorithms can be useful to support the users during daily activities and to build short video summaries of the days (e.g., of a mother during pregnancy) to support further analysis by experts (e.g., doctors) for personalised health. Among the other sources of information, attention will also be paid to speech analysis. The algorithms will be able to infer age, gender and emotions of the user, as well as to estimate his pose and understand his body language. Among the others, information obtained by First Person Vision Systems worn by a user will be considered as external source data to make more accurate inferences.

ESR activities

  • Face analysis: this includes face detection, extraction of facial landmark points, face recognition, and estimation of soft biometrics i.e. as age and gender;
  • Body language understanding: this includes inferring the pose of user and building a biometric body signature (e.g., to identify user from his movements);
  • Speech analysis: this includes the identification of emotions and the semantic topic of conversation from speech analysis;

Expected Results:

  • The ESR fellow will study the state of the art of the algorithms related to project objectives. This will require the acquisition of basic knowledge of Computer Vision, ML and Data Mining. This will allow the production of a survey identifying the state of the art technologies, as well as directions for future research;
  • The ESR fellow will study the datasets already available to the public useful for the research and survey the characteristics needed to acquire new domain-specific data. All the data will be collected in a repository, which will be documented in order to be available for future research;
  • Techniques for face analysis will be investigated. Such techniques include algorithms to detect face, extract facial landmark points and perform face recognition. As a form of soft biometric, algorithms to estimate age and gender of the user will also be investigated;
  • The ESR fellow will investigate and develop algorithms for body language understanding. This includes techniques to infer the pose of the user and to build a biometric body signature useful to identify the user from the way he moves;
  • Speech analysis will also be investigated to identify emotions and infer the semantic topic of conversation;
  • Data fusion techniques will be investigated to consider external data sources (e.g., provided by a first person vision worn by a user) during the inferences;
  • A number of personal health applications will be exploited, with emphasis to Mother&Child care, healthy-living and assisted care.
    The research project will also produce high quality scientific publication in international conferences and journals.

Additional essential requirements

  • Master degree in Computer Science, Information Engineering (or equivalent). A degree with distinction (cum laude) is an advantage;
  • Prior knowledge in Computer Vision, Machine Learning and Deep Learning is an advantage;
  • Prior publications at international conferences or journals are desirable;
  • Ability to program in Python is an advantage;
  • Communication skill and team play are desirable.

Principal Investigator: Prof. Giovanni Maria Farinella, (UNICT)
Academic PhD Supervisor: Sebastiano Battiato (UNICT)
Academic PhD Co-Supervisor: Giovanni Maria Farinella (UNICT)
Industrial PhD Supervisor: Binyam Gebre (PHILIPS)

Main contact: Prof. Giovanni Maria Farinella
Email: gfarinella@dmi.unict.it