Utilizing Deep Learning in Smart Glass System to Assist the Blind and Visually Impaired

Authors

Asmaa A. Hekal, Modern University for Technology and InformationFollow
Mohamed S. Sharaf, Modern University for Technology and InformationFollow
Ahmed A. Sayed, Modern University for Technology and InformationFollow
Ibrahim R. Abdelrahman, Modern University for Technology and InformationFollow
Ahmed A. Salem, Modern University for Technology and InformationFollow
Ahmed M. Elhussieny, Modern University for Technology and InformationFollow
Saeed Y. Kouta, Modern University for Technology and InformationFollow
Eman S. Abass, Modern University for Technology and InformationFollow

Abstract

This paper presents a groundbreaking assistive technology designed to empower visually impaired individuals in their daily lives. With an estimated global population of 2.2 billion facing visual impairments, addressing the challenges they encounter is of paramount importance. The research introduces a comprehensive electronic device integrating advanced computer vision and deep learning techniques. The system incorporates real-time object detection, robust facial recognition, and precise currency denomination identification. Powered by a Raspberry Pi 4 Model B+ and an ESP32-CAM Development Board, the device offers users unparalleled environmental awareness. Utilizing YOLOv4-tiny for object detection and a hybrid face recognition model combining HaarCascades, Histogram of Oriented Gradients (HOG) features, and deep neural networks, the system ensures accurate object and facial recognition. The integration of a MobileNetV2-based model facilitates Egyptian currency detection. Moreover, the system features seamless speech-to-text and text-to-speech functionalities, enhancing user interaction. The device's hardware components, including ultrasonic sensors and vibration motors, provide haptic feedback, further augmenting the user experience. This paper not only presents a state-of-the-art technological solution but also discusses its future potential. Through continuous refinement, user feedback integration, and exploration of wearable applications, the proposed system holds promise in significantly improving the quality of life for visually impaired individuals, fostering independence, and facilitating greater societal inclusion.

Recommended Citation

Hekal, Asmaa A.; Sharaf, Mohamed S.; Sayed, Ahmed A.; Abdelrahman, Ibrahim R.; Salem, Ahmed A.; Elhussieny, Ahmed M.; Kouta, Saeed Y.; and Abass, Eman S. (2024) "Utilizing Deep Learning in Smart Glass System to Assist the Blind and Visually Impaired," Future Engineering Journal: Vol. 4: Iss. 2, Article 2.
Available at: https://digitalcommons.aaru.edu.jo/fej/vol4/iss2/2