The traditional authentication systems that are provided by the existing companies are vulnerable So our idea is to develop an application that will increase the reliability over the traditional authentication system In traditional authentication system, there is only one mode of authentication which is through the verification of OTP Whereas in this system we provide authentication through multiple factors like speech recognition visual speech recognition speaker recognition

The project aims at creating a multi factor authentication system with the power of Computer Vision and Natural Language Processing Initially, a OTP is generated during the login Then we use Visual Speech Recognition where the lip movements are predicted by the model and verified against the OTP Then we use Speech Recognition where the frequency of the voice is captured and the corresponding text is generated by the model  and verified against the OTP Then we use Speaker Recognition where the frequency of the voice is captured and the corresponding output class of the speaker is generated by the model and verified against the corresponding speaker’s voice embedding

References

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 6000–6010.

J.-w. Jung, H.-S. Heo, J.-h. Kim, H.-j. Shim, and H.-J. Yu, “Rawnet: Advanced end-to-end deep neural network using raw waveforms for text-independent speaker verification,” Proc. Interspeech 2019, pp. 1268–1272, 2019

Assael, Y. M., Shillingford, B., Whiteson, S., & deFreitas, N. (2016). LipNet: Sentence-level Lipreading. arXiv preprint arXiv:1611.01599

Graves, A., Fernández, S., Gomez, F., Schmidhuber, J. (2006). Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning. pp. Orlando. 369–376

J. Deng, J. Guo, N. Xue and S. Zafeiriou, "ArcFace: Additive Angular Margin Loss for Deep Face Recognition," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 4685-4694, doi: 10.1109/CVPR.2019.00482

 Melekhov, J. Kannala and E. Rahtu, "Siamese network features for image matching," 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico, 2016, pp. 378-383, doi: 10.1109/ICPR.2016.7899663.