What you need to know
- Microsoft just released its Seeing AI app onto Android, expanding the app’s reach to an additional 3 billion people.
- Seeing AI uses artificial intelligence to help the blind and low vision community.
- The app can perform tasks such as identifying currency, reading handwriting, describing a scene, and guiding a person through a document.
Microsoft first worked on its Seeing AI app as a research project in 2017. It has since evolved into a fully featured app available to the public. Up until now, Seeing AI has only been available for iOS, but it is now available on Android as well. With over 3 billion Android users on the planet, the rollout greatly expands the app’s reach.
Seeing AI uses artificial intelligence to help the blind and low vision community. It can perform daily tasks such as guiding someone through a document or menu, recognizing currency notes, and identifying images. To use the app, you can either point its camera or take a photo.
Seeing AI supports 18 languages at the moment, but that number will increase to 36 in 2024.
The following features made their way to the iOS version of Seeing AI and are also on the newly released Android version:
- Richer Descriptions of photos: In addition to providing a brief summary of photos on the Scene channel, you can now tap ‘more info’ and a rich description will be generated, including far greater detail about what is in the image.
- Chat to your documents: After scanning a document, in addition to hearing it read aloud, you can also chat to Seeing AI to ask questions, such as about items on a menu, the price of an item on a receipt, or to summarize an article.
Microsoft also highlighted some of Seeing AI’s key features for those unfamiliar with the app:
- Short Text: Speaks text as soon as it appears in front of the camera.
- Documents: Provides audio guidance to capture a printed page, and reads the content aloud, along with its original formatting. Chat with Seeing AI to quickly find information.
- Products: Scans barcodes, using audio beeps to guide you; hear the name, and package information when available.
- Scenes: Hear a description of the scene captured and tap ‘more info’ to generate a rich description. Explore the photo by moving your finger over the screen to hear the location of different objects.
- People: Identify friends around you.
- Currency: Recognizes currency notes.
- Colors: Identifies the perceived color.
- Handwriting: Reads handwritten text like in greeting cards (available in a subset of languages).
- Light: Generates an audible tone corresponding to the brightness in the surroundings.
- Images in other apps: Share a photo with Seeing AI to recognize it.
The video below is from four years ago, but it shows the general concept of Seeing AI.
