Apple bringing Eye Tracking, Vocal Shortcuts, Music Haptics to iPhone and iPad

Apple on Wednesday announced several new accessibility features that are coming to the iPhone and iPad. The company said the features are “coming later this year,” which likely indicates that they are part of iOS 18 and iPadOS 18, which are expected to be previewed at the Apple Worldwide Developers Conference keynote on June 10.

“Each year, we break new ground when it comes to accessibility,” said Sarah Herrlinger, Apple’s senior director of Global Accessibility Policy and Initiatives, in a press release. “These new features will make an impact in the lives of a wide range of users, providing new ways to communicate, control their devices, and move through the world.”

Here is a summary of the upcoming features for the iPhone and/or iPad.

• Eye Tracking: gives the user the option to navigate the iPhone of iPad using just their eyes. It uses the front-facing camera and the on-device machine learning allows the data to be kept on the iPhone or iPad and not shared with Apple.
• Music Haptics: the iPhone’s Taptic Engine plays taps, textures, and refined vibrations that go along with the music that is playing. An API will be available for developers.
• Vocal Shortcuts: “custom utterances” can be used with Siri for interface shortcuts.
• Listen for Atypical Speech: an enhanced speech recognition feature that uses on-device machine learning to recognize a user’s speech patterns. It’s designed for users with conditions that affect speech.
• Vehicle Motion Cues: for people who read their device in a car, this feature is designed to reduce motion sickness. Dots appear on the screen and move with the vehicle to help reduce sensory conflict, which causes motion sickness.

Table of Contents

Toggle

CarPlay updates coming soon

CarPlay is also getting new features:

• Voice Control: navigate the CarPlay UI with voice commands.
• Sound Recognition: horn and siren alerts for deaf or hard of hearing drivers.
• Color Filters: for colorblind drivers on the CarPlay interface, which also includes bold and large text.

Apple

• For users who are blind or have low vision, VoiceOver will include new voices, a flexible Voice Rotor, custom volume control, and the ability to customize VoiceOver keyboard shortcuts on Mac.
• Magnifier will offer a new Reader Mode and the option to easily launch Detection Mode with the Action button.
• Braille users will get a new way to start and stay in Braille Screen Input for faster control and text editing; Japanese language availability for Braille Screen Input; support for multi-line braille with Dot Pad; and the option to choose different input and output tables.
• For users with low vision, Hover Typing shows larger text when typing in a text field, and in a user’s preferred font and color.
• For users at risk of losing their ability to speak, Personal Voice will be available in Mandarin Chinese. Users who have difficulty pronouncing or reading full sentences will be able to create a Personal Voice using shortened phrases.
• For users who are nonspeaking, Live Speech will include categories and simultaneous compatibility with Live Captions.
• For users with physical disabilities, Virtual Trackpad for AssistiveTouch allows users to control their device using a small region of the screen as a resizable trackpad.
• Switch Control will include the option to use the cameras in iPhone and iPad to recognize finger-tap gestures as switches.
• Voice Control will offer support for custom vocabularies and complex words.

Apple is expected to release iOS 18, iPadOS 18, macOS 15, and a major update to visionOS throughout the fall of 2024. The new operating systems will be previewed at WWDC during the keynote on June 10.