---
title: "Touch to ‘See’"
description: "Touch to ‘See’ (iSoftStone, 2016/17): AI captions any picture and prints it as a touchable image for blind people"
url: "https://www.sugar-network.org/en/projects/touch-to-see-2016-17/"
lang: "en"
---

# Touch to ‘See’

**AI captions any picture and prints it as a touchable image for blind people**

_Help blind to sense the world_

- **Corporate partner:** [iSoftStone](https://www.sugar-network.org/en/network/partners/isoftstone/) (https://www.isoftstone.com/en/)
- **Cohort:** [2016/17](https://www.sugar-network.org/en/projects/cohort/2016-17/)
- **Theme:** [Education & society](https://www.sugar-network.org/en/projects/themes/education-society/)
- **Type:** Digital service
- **Universities:** [University of Science and Technology of China](https://www.sugar-network.org/en/network/universities/ustc/)
- **Team:** 5 master students
- **Disciplines:** Automation, Computer Science, Nuclear Engineering, Optical Engineering, Science Communication

## Challenge

Blind people mainly access information by listening and by reading braille. iSoftStone asked how to help them sense visual information as well.

## What the team learned

Contours alone are not enough. Blind users also need to feel grey scale, which touchable images can show through line thickness or dot density.

## What they built

Software uses a Neural Image Caption Model to describe any picture, matches the caption to objects in a dataset and turns them into a touchable image of shapes, lines and dots. Low-cost hardware prints the image.

## Impact

Projected uses: textbook images and atlases, picture books, and barrier-free public places such as restaurant menus and supermarket goods.

## Full project text

Intelligent software that transfers the visual images into a touchable one so as to help the blind better sense the world.

### Challenge

How to help the blind better sense the world? This challenge will create a type of visuality for blind individuals.

### Solution

The current powerful neural network makes it possible to describe a visible image with a relatively accurate caption which can be used as a keyword to match with a certain object in the dataset. Based on this, we use a caption model called “Neural Image Caption Model” to predict the description of arbitrary images, and then as the post-processing of the neural network, a touchable image is printed on our device.

### Impact

The eyes are the windows of the soul. 90% of people receive information from the eyes. We only know that there are blind people in this world, but knowing so few of the state of their life. After starting this project, we were exposed to a large number of blind people, which really helped us understand their lives. They are far stronger than we imagined, and the circle of blind people is quite closed. Basically, blind friends are blind. True intelligence is the intelligence of everyone, and we hope to use technology to better integrate the visually impaired into the world.

## Who built it.

- Zhanchi Wang, Computer Science (University of Science and Technology of China)
- Hang Yang, Automation (University of Science and Technology of China)
- Chengyuan Bian, Science communication (University of Science and Technology of China)
- Xiangyang Qin, Nuclear Engineering (University of Science and Technology of China)
- Yanruo Yang, Optics engineering (University of Science and Technology of China)

## Sources

Summarised by the SUGAR editorial team from the team’s project page and SUGAR Expo booklet.

Open the 2016/17 booklet: https://drive.google.com/file/d/1TXg5U1NWDb_LJz7j9X2CHIyjHksh0QIk/view?usp=sharing