Visual assistance systems and related methods

• US 9,307,073 B2
• Filed: 12/31/2013
• Issued: 04/05/2016
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: Sorenson Ip Holdings, LLC
• Original Assignee: Sorenson Communications Incorporated
• Inventors: Maxwell, Conrad Arlo, Shields, Richard
• Primary Examiner(s): Patel, Hemant
• Application Number:
  US14/145,589
• Publication Number:
  US 20150189071A1
• Time in Patent Office: 826 Days
• Field of Search: 379 37- 51
• US Class Current: 1/1
• CPC Class Codes: H04M 1/236 including keys on side or rear faces
• CPC Class Codes: H04M 1/72481 for visually impaired users
• CPC Class Codes: H04M 11/045 using recorded signals, e.g. speech
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04M 3/42068 where the identifier is used to access a profile
• CPC Class Codes: H04M 3/51 Centralised call answering arrangements requiring operator intervention , e.g. call or contact centers for telemarketing
• CPC Class Codes: H04M 3/5108 Secretarial services
• CPC Class Codes: H04M 3/5183 Call or contact centers with computer-telephony arrangements
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet

Abstract

Visual assistance systems and related methods are disclosed. A visual assistance system includes a visual assistance service configured to receive and display images from a user device on an electronic display for a live assistant. The visual assistance service is also configured to transmit assistant audio from the live assistant conveying information derived by the live assistant'"'"'s review of the images. A visual assistance system comprises a user device configured to capture and transmit the images to the visual assistance service. The user device is also configured to receive the assistant audio from the visual assistance service, and reproduce the assistant audio to a user of the user device. The user is registered with the visual assistance service as having at least a minimum level of visual disability. A method of operating the visual assistance system includes receiving the images from, and transmitting the assistant audio to the user device.

Claims

1. A visual assistance system, comprising:

a visual assistance service comprising a database identifying a plurality of user devices that are registered with the visual assistance service as being associated with a plurality of users having at least a minimum level of visual disability, the visual assistance service configured to;

receive a call from a user device of the plurality of user devices;

receive one or more video images or still images taken of an environment proximate to the user device;

display the one or more video images or still images on an electronic display for a live assistant;

transmit assistant audio to the user device, the assistant audio comprising the live assistant'"'"'s speech conveying information derived by the live assistant'"'"'s review of the one or more video images or still images;

receive sensor data from the user device,

determine an average number of steps per unit of distance traveled by the user based on accelerometer data from the sensor data;

combine the average number of steps per unit of distance traveled with GPS data from the sensor data to determine a number of steps needed for the user to reach a desired destination; and

display information relating to the sensor data on the electronic display for the live assistant.

2. The visual assistance system of claim 1, wherein the visual assistance service is configured to access at least one of a call history, voice messages, or text messages for each user device.

3. The visual assistance system of claim 1, wherein the visual assistance service is further configured to provide a call routing service for the live assistant of the visual assistance service to route the call to connect the user device with a third party.

4. The visual assistance system of claim 3, wherein the visual assistance service is configured to access a contacts list for each user device and route the call to a contact from the contacts list.

5. A method of operating a visual assistance system, comprising:

receiving a call from a user device by a visual assistance service, the user device being registered with the visual assistance service as being associated with a user having at least a minimum level of visual disability;

receiving one or more images from the user device at the visual assistance service during the call;

displaying the one or more images for a live assistant on an electronic display at the visual assistance service;

transmitting assistant audio comprising speech descriptive of the one or more images from the live assistant to the user device;

receiving one or more sensor signals comprising sensor data from the user device;

determining an average number of steps per unit of distance traveled by the user based on accelerometer data from the sensor data;

combining the average number of steps per unit of distance traveled with GPS data from the sensor data to determine a number of steps needed for the user to reach a desired destination; and

displaying the sensor data on the electronic display for the live assistant.

6. The method of claim 5, further comprising routing the call from the visual assistance service to connect the user device to a third party device during the call.

7. The method of claim 6, wherein routing the call from the visual assistance service includes connecting the user device to a contact from a contact list of the user device that is accessible by the live assistant through the visual assistance service.

8. The method of claim 6, wherein routing the call from the visual assistance service includes connecting the user device to an emergency service.

9. A visual assistance system, comprising:

a user device configured to;

initiate a call to a visual assistance service for a user that is registered with the visual assistance service as having at least a minimum level of visual disability;

capture one or more images;

transmit the one or more images to the visual assistance service;

receive assistant audio descriptive of the one or more images from a live assistant at the visual assistance service;

reproduce the assistant audio as audible sound to the user of the user device;

generate sensor data including at least accelerometer data and GPS data; and

transmit the sensor data to the visual assistance system, wherein at least one of the user device or the visual assistance system is configured to;

determine an average number of steps per unit of distance traveled by the user based on the accelerometer data; and

combine the average number of steps per unit of distance traveled with GPS data to determine a number of steps needed for the user to reach a desired destination.

10. The visual assistance system of claim 9, wherein the user device comprises one or more sensors configured to generate the sensor data, the one or more sensors selected from the list consisting of a global positioning system (GPS) device, a motion detector, an ultrasonic proximity sensor, and an accelerometer.

11. The visual assistance system of claim 10, wherein the user device is configured to provide an audible alert responsive to the user device detecting an obstacle based on the sensor data from the one or more sensors.

12. The visual assistance system of claim 9, wherein the user device comprises a standalone device comprising:

a control circuit;

a camera operably coupled to the control circuit and configured to capture the one or more images;

a communication element operably coupled to the control circuit and configured to transmit the one or more images to the visual assistance service, and receive the assistant audio from the visual assistance service; and

an audio output device operably coupled to the control circuit configured to convert the assistant audio to audible sound to be heard by the user.

13. The visual assistance system of claim 9, wherein the user device comprises a mobile communication device and one or more separate devices configured to communicate with the mobile communication device.

14. The visual assistance system of claim 13, wherein the one or more separate devices comprise one or more devices selected from the list consisting of an earphone, a pair of glasses, a watch, and a mobility cane.

15. The visual assistance system of claim 9, further comprising a dedicated call button separate from other input devices, wherein the dedicated call button is configured to initiate a call directly to the visual assistance service responsive to a user input without dialing a full phone number.

16. The visual assistance system of claim 15, wherein the dedicated call button is a textured call button to enable the visually-impaired user to locate the dedicated call button by touch.

17. The visual assistance system of claim 9, wherein the threshold level is based on acceleration data from the sensor data.

18. The visual assistance system of claim 9, wherein the user device is configured to maintain a contacts list, and enable the visual assistance service to have access to the contacts list for routing the call to contacts therein during the call.

19. The visual assistance system of claim 9, wherein the user device is configured to automatically initiate the call responsive to a detection of a condition outside of a threshold level based on sensor data recorded by the user device.