Recent estimates from the World Health Organisation indicate that hundreds of millions of people worldwide live with visual impairment. These individuals face daily challenges in navigation, object recognition and access to information. In real-world environments, especially in rapidly growing urban areas, these challenges are amplified by complex, dynamic and often unpredictable conditions.
For many visually impaired individuals, simple tasks such as crossing a street, reading a label, or identifying currency can be difficult and stressful. Although significant progress has been made in assistive technologies over the past decade, many existing solutions remain expensive, dependent on internet connectivity, or poorly adapted to real-world environments, particularly in low-resource settings.
In his seminar, Iso Lomso fellow Achille Melingui, who is in his last residency, presented an update on his project, which aims to develop an AI-powered smart glasses system that provides real-time, offline assistance via embedded computer vision.
“The main objective is to bridge the gap between existing advanced technology in AI and real-time assistance for people, especially those in lower-resource settings,” he said.
“Think of a busy intersection in Africa. This is a very difficult environment for a blind person,” he explained. “What if a blind person could interact verbally with the environment and receive real-time, meaningful responses. Asking questions like: what is in front of me, is the path clear, what does the sign say, which banknote am I holding?”
The problem is not blindness but having accurate information at the right moment, Melingui said, adding: “The difficulty lies in transforming visual information into something that is immediately usable and actionable. A challenge our system aims to address.”
Melingui is with the Department of Electrical and Telecommunication Engineering at the University of Yaoundé 1, Cameroon. He, together with his team, is working on a prototype system that integrates object detection, text recognition, and currency identification, combined with voice interaction to deliver intuitive audio feedback to the user.
“Unlike most existing solutions, this system operates entirely offline on low-cost hardware, making it more accessible, reliable and suitable for deployment in environments with limited connectivity,” he said.
He explained that although traditional assistive methods have evolved from white canes and service dogs to solar-powered canes and hooples (a hoop-shaped cane used to navigate rough terrain), among others, they are still not ideal for every environment. The older systems – while still indispensable – have their limitations, while the more advanced technologies are expensive and dependent on external infrastructure – in particular, internet connectivity and, as a result, are less suitable for large-scale deployment, especially in resource-constrained environments.
He said most assistive systems rely on camera-based perception, often combined with additional sensors to enable tasks such as navigation, recognition, localisation and multimodal interaction. There is a dominance of computer vision and deep learning, multimodal AI systems and embedded AI. Offline low-cost solutions remain rare, and few systems combine the three main properties: real-time performance, offline operation and affordability.
“Our system uniquely combines offline AI, low cost, and real-world robustness. Our design requirement is an effective assistive system that translates visual information into simple, timely spoken guidance that the person can immediately use,” he added.
Very smart glasses
Melingui and his colleagues have therefore developed a fully offline, low-cost smart glasses prototype that integrates real-time object detection, text recognition, currency identification and voice-based interaction. Using on-device processing on compact hardware, the system translates visual information into clear, timely audio feedback, enabling hands-free interaction and immediate decision-making for users. The design philosophy follows a ‘See – Understand – Act’ framework, emphasising perception, interpretation and response.
The wearable setup comprises smart glasses, headphones, Wi-Fi without internet and Bluetooth connections, as well as Raspberry Pi for processing and AI inference. (Raspberry Pi is a small, single-board computer about the size of a credit card that was developed by the Raspberry Pi Foundation in the United Kingdom to promote computer-science education, which has since gained popularity for a wide range of applications.)
The system currently does navigation, object detection, face recognition, currency and voice, but Melingui emphasised it should be possible to add additional functions.
“The mode manager selects and activates the appropriate function based on user input – via voice commands or a physical button – the button ensures usability even in noisy environments and supports discreet use,” said Melingui. “It’s a single, wearable device that can see, interpret, read and speak in real time, which is designed for robustness in everyday environments and prioritises user-centred interaction to reduce cognitive overload.”
“The hope is to improve autonomy in daily life for visually impaired users through multimodal perception and user-centred design.”
The team is also hoping to expand the capability for enhanced indoor navigation, which Melingui aims to develop in his current residency. There are also plans to expand personalisation and to address the need for extended battery life.
The discussion delved into broader questions about assistive technologies, disability studies, and social inclusion, acknowledging the importance of avoiding ableist assumptions, respecting personal identity and involving visually impaired users directly in design, testing and refinement of technologies. Melingui also stressed that the system is positioned as an assistive tool, complementary to existing support rather than a replacement.
The project also highlights opportunities for collaborative research, student involvement and potential policy engagement, particularly around cost, accessibility and deployment in African and other low-resource contexts.
“We believe that embedded, offline AI systems can meaningfully improve autonomy and quality of life for visually impaired individuals while contributing to equitable, sustainable technological development,” said Melingui.
Article: Michelle Galloway
Photo: Ignus Dreyer, SCPS Photos