TapTapSee is a mobile camera application designed specifically for the blind and visually impaired iOS users, powered by the CloudSight.ai image recognition API. The app utilizes the device’s camera and VoiceOver functions to photograph objects and identify them out loud for the user.
In TapTapSee, the user double-taps the device’s screen to photograph any two or three dimensional object at any angle, and have it accurately analyzed and defined within seconds. The device’s VoiceOver then speaks the identification audibly to the user.
TapTapSee includes the following additional features: Repetition of the last image’s identification, ability to upload images from the camera roll, share identification via Twitter, Facebook, text or email, rotor reader, flash toggle, and the ability to save the identified image to the camera roll with the attached tag.
American Foundation for the Blind, 2014 Access Award Recipient
Royal National Institute for the Blind, App of the month
AppleVis iOS Hall of Fame
TapTapSee is available through the Apple iTunes Mobile App Store and the Google Play Store.
TapTapSee sources are shared under an MIT license on GitHub.
Best Picture Taking Practices
The following are tips and suggestions for taking quality pictures with TapTapSee:
- The camera on the iDevice is located in the top right corner behind the front facing screen of the phone when the device is in the upright position.
- It is advised to hold the iDevice 8-12 inches (20-30 centimeters) away from the object being photographed. This method will help ensure that the object is in the scope of the camera.
- TapTapSee has an autofocus notification to let the user know when the photographed object is in focus. For best practice, wait until the app beeps before taking a picture. The autofocus notification can be turned ON and OFF in the About menu.
- For best results, pictures snapped with TapTapSee should be taken in a well lit environment. The app also features an automatic flash, which can be turned ON and OFF in the About menu.
- The barcode on canned goods is almost always located to the left of the seam of the can where the two label ends meet and overlap. Other written information, such as brand, product name and info is usually across from the seam on the opposite side of the can. To get the best results when taking a picture of the label, be sure to keep the camera 8-12 inches (20-30 centimeters) away from the can.
First Time User Overview
- Before you begin using the application, you must first turn VoiceOver ON in your device. To turn VoiceOver ON go to Settings > General > Accessibility > VoiceOver. Then switch to ON.
- When you first open TapTapSee a Privacy Notice pops up that needs to be accepted for app use. Once it's accepted you're taken into the app, which consists of a camera and four buttons at the top of the screen - Repeat, Library, Share and About.
- To take a picture, double-tap on the screen and wait approximately seven to 10 seconds to receive an identification. The wait time may fluctuate depending on your network connection. The image is sent to the server where it is identified and sent back to the user. The VoiceOver in the device then speaks the identification to the user. Up to three images at a time can be identified.
TapTapSee gives users a general identification of any picture taken. However, if, for example, the user takes a picture of a can of soup and wants to know the name of the brand, the application will be able to read the label and return the identification with the brand name. Nevertheless, keep in mind that TapTapSee will only be able to recognize the object that is within the camera's scope and in focus. Lighting conditions are also important for the quality of the identification.
TapTapSee includes an autofocus alert that can be turned ON and OFF in the About menu. Once an object comes into focus, the app makes a sound to notify the user that it's in the optimal position to take a picture.
- Repeat Button
- Hear the last identification spoken aloud in case it was missed the first time.
- Library Button
- Access the device's Camera Roll in order to send images to TapTapSee for identification. To access this feature simply click on the Library button and proceed to select an image that you want to have identified from the Camera Roll (Note: If access to the Camera Roll is denied then you must change the privacy setting. Go to Settings > Privacy > Photos > TapTapSee. Toggle the TapTapSee switch ON.
- Share Button
- Share the image via Twitter, Facebook, Email, or Text. The Share button also includes the option for the user to save the image to the device's Camera Roll. The saved image will include the tag that was provided by TapTapSee.