Digit recognition by lip-movements and time recursive Neural Networks

Title Digit recognition by lip-movements and time recursive Neural Networks
Summary The project aims to recognize digits by lip movements and neural networks
Prerequisites Good knowledge in Image Analysis and Computer Vision in 3D
Supervisor Josef Bigun, Kevin Hernandez-Diaz, Fernando Alonso-Fernandez
Level Master
Status Open

Generate PDF template

Liveness verification is an important issue in Biometrics. It corresponds to verifying that the signal , e.g. face-image, fingerprint, audio-speech, on which biometric recognition is based is authentic, coming from a live person, in contrast to a synthetic signal, including (dis)playing a video or speech from memory.

Also, audio utterances of words or digits when prompted for in public, such as trains, and busses, are not suitable to be used as passwords, for a variety of reasons. This includes overhearing by others, but also because speech is difficult to be used as biometric source, beside password information, for identification due to environment noise, e.g. vehicle noise. Accordingly, recognizing digits by lip-movements can be a way to mitigate the issues posed by speech in public, noisy, and crowded places.

The project aims to use lip-movements based digit recognition to contribute to verification of liveness as well as recognizing password utterences in public. It will be done by convolutional neural networks with short time memory and optical flow.