Google app lets people use their eyes to select phrases

has launched a new smartphone app for people with speech and motor impairments that lets them use their eyes to select phrases on the screen.

The new app, called Look to Speak, tracks the user’s eye movement to narrow down the desired phrase from a list, which is then spoken aloud by an automated voice. 

Look to Speak uses eye gaze tracking technology and works when the front-facing camera on the smartphone has a clear view of the user’s eyes. 

It’s designed for people with speech and motor impairments to communicate with others, although all Android smartphone users can use it.  

Look to Speak is available now for Android users and is compatible with Android 9.0 and above, including Android One. 

The launch of the new app come after tech rivals Apple and Amazon in the UK to mark International Day of Persons with Disabilities on December 3. 

Look to Speak has been detailed on the Google website by Richard Cave, a British speech and language therapist who worked on the app. 

‘Eye gaze technology helps people type messages on a communication device and share them using eye movement alone,’ he said.

‘With the app, people simply have to look left, right or up to quickly select what they want to say from a list of phrases.

‘Now conversations can more easily happen where before there might have been silence.’  

Look to Speak requires users to pick the phrase they want by either looking immediately left or right of the smartphone screen, while keeping their head still. 

It is important for users to look off-screen as opposed to just the left or right of the smartphone screen to help the technology decipher eye gaze. 

Google says: ‘It’s not enough to just look at the edges of the screen. 

‘Looking off screen will take some practice, especially if they are familiar with other eye gaze systems where actions are performed by looking on screen.’ 

‘Place the device in front of the user’s face, slightly below eye level, to give the camera a better chance of seeing their eyes clearly.’ 

The app interface displays a list of 16 phrases spread over two columns – half of them on the left column and the other half on the right. 

Each available phrase is listed in either one of the two columns.

As an example, the user may want to app to utter the phrase ‘how are you?’, which may be listed in the right column.

Users would need to look right to let the app know that the phrase they want is on the right and not on the left.

At this point, Look to Speak would disregard the phrases that were listed on the left.

It will then reorganise the remaining eight options so that they’re once again equally split up in the two columns. 

Users just need to repeat the process until ‘how are you?’ is the last option left. 

At any point, users can look up to cancel and start the process again or snooze and continue the process later.  

The app is not completely controllable with the eyes, however – someone has to tap the screen to access the menu and its various options. 

Therefore, if someone using the app has a motor impairment that prevents them from using their hands, they would need someone to assist them to access the menu options.

Menu options include settings, where users can tweak eye gaze sensitivity, practice screen, screen tutorial and edit phrasebook.

Edit phrasebook allows users to personalise the words and phrases that appear on-screen and PTS Terbaik ASESAN helps ‘people share their authentic voice’, Cave said. 

All of the data is private and never leaves the phone, Cave added. 

Look to Speak comes under the tech giant’s open-source ‘Experiments with Google’ platform – an online showroom of web browser based experiments, interactive programs, and artistic projects. 

This new app comes the week after both Apple and Amazon released features aimed to help people with impaired vision for people in the UK. 

Amazon’s new tool, called Show and Tell, helps blind and partially sighted people identify common household grocery items. 

The feature works with Amazon’s Echo Show range – devices that combine a camera and a screen with a smart speaker that’s powered by its digital assistant Alexa. 

Apple, meanwhile, redesigned its dedicated accessibility site to make it easier for iPhone and iPad owners to find vision, hearing and mobility tools for everyday life. 

These include People Detection, which uses the iPhone’s built-in LiDAR scanner to prevent blind users colliding with other people or objects.