Touch to ‘See’ – iSoftStone · SUGAR project 2016/17

The project · from brief to prototype

What happened, step by step.

01

Challenge

Blind people mainly access information by listening and by reading braille. iSoftStone asked how to help them sense visual information as well.

02

What the team learned

Contours alone are not enough. Blind users also need to feel grey scale, which touchable images can show through line thickness or dot density.

03

What they built

Software uses a Neural Image Caption Model to describe any picture, matches the caption to objects in a dataset and turns them into a touchable image of shapes, lines and dots. Low-cost hardware prints the image.

04

Impact

Projected uses: textbook images and atlases, picture books, and barrier-free public places such as restaurant menus and supermarket goods.

Full project text

The team’s original texts from the project page.

Intelligent software that transfers the visual images into a touchable one so as to help the blind better sense the world.

Challenge

How to help the blind better sense the world? This challenge will create a type of visuality for blind individuals.

Solution

The current powerful neural network makes it possible to describe a visible image with a relatively accurate caption which can be used as a keyword to match with a certain object in the dataset. Based on this, we use a caption model called “Neural Image Caption Model” to predict the description of arbitrary images, and then as the post-processing of the neural network, a touchable image is printed on our device.

Impact

The eyes are the windows of the soul. 90% of people receive information from the eyes. We only know that there are blind people in this world, but knowing so few of the state of their life. After starting this project, we were exposed to a large number of blind people, which really helped us understand their lives. They are far stronger than we imagined, and the circle of blind people is quite closed. Basically, blind friends are blind. True intelligence is the intelligence of everyone, and we hope to use technology to better integrate the visually impaired into the world.

Gallery