How does the device work?

The device works through a camera and an ultrasonic sensor. 

Using wifi and Bluetooth to communicate, the remote immediately establishes a connection with the Drishtimote application. Once connected, the remote continues to capture and send pictures frame-by-frame to the application in real-time over WiFi. 

When a user presses the remote’s push button, Bluetooth notifies the application about the push events and eventually, the application begins to apply ML to recognise the items in the particular frame. The application has a dataset containing information on about 100 daily objects such as a book, a bottle, a cup etc.

Once the objects are recognised, the user’s phone would speak the names of the items in front of him. The ultrasonic sensor measures the distance nearby and activates the buzzer to alarm the user if any such objects are within 35 cm. 

How is it different

Although there are existing products in the market that assist the blind, such as Braille generators and text readers, none come close to matching the capability of this model.