Methods, apparatus and systems for providing remote assistance for visually-impaired users

• US 9,836,996 B2
• Filed: 02/14/2017
• Issued: 12/05/2017
• Est. Priority Date: 09/03/2014
• Status: Active Grant

• Current Assignee: Aira Tech Corporation
• Original Assignee: Aira Tech Corporation
• Inventors: Kanuganti, Suman, Chang, Yuja, Bock, Lawrence
• Primary Examiner(s): WALKER, JARED T
• Application Number:
  US15/432,863
• Publication Number:
  US 20170162076A1
• Time in Patent Office: 294 Days
• Field of Search: 348 62
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G09B 21/001 Teaching or communicating with blind persons G09B21/02 - G09B21/06 take precedence
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays
• CPC Class Codes: G09B 21/006 using audible presentation of the information
• CPC Class Codes: H04M 1/72475 specially adapted for disabled users
• CPC Class Codes: H04N 21/25841 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; arrangements for identifying locations of receiving stations in broadcast systems H04H60/51; location of the user terminal in data switching networks H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/2743 Video hosting of uploaded data from client
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/8545 for generating interactive applications
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Techniques and systems are provided for assisting a user through a guidance mode activity. Such a user may be visually or otherwise impaired, or a user requiring help for other reasons. In some embodiments, a client device, held or worn by the user, may comprise a video capture device and several sensors, and may send video data and sensor data to a server. The server may comprise a processor and artificial intelligence. The server may send the video data and sensor data to an agent device. The agent device may provide content for display on an agent interface. An agent may view the agent interface, and assist the user in real time through audio instructions or other feedback.

Claims

1. A system for providing remote assistance for one or more visually-impaired users, the system comprising:

an agent device having one or more processors and a display, the one or more processors configured to;

receive video content including images of an environment in which a client device is located;

receive information indicating a visual impairment of a user of the client device;

identify a mask that simulates the visual impairment, the mask comprising a fixed component filter and a variable-component filter, wherein the variable-component filter is adjustable to simulate variations in the visual impairment;

apply the mask to the received video content to generate filtered video content representative of what the environment looks like to the user based on the visual impairment of the user; and

cause the filtered video content to be displayed on the display.

2. The system of claim 1, wherein the one or more processors are further configured to receive profile data of the user, wherein the profile data includes the information indicating the visual impairment of the user.

3. The system of claim1, wherein the variable-component filter is at least partially determined by a condition of a slider control object in an agent interface.

4. The system of claim 1, wherein the one or more processors are further configured to provide a mask selector on the display of the agent device that can be selected and unselected to display and cancel the display of the filtered video content.

5. The system of claim 1, wherein the visual impairment comprises one or more conditions, the one or more conditions including at least one or more of head injury, glaucoma, retinal detachment, hypertensive retinotherapy, cataract, age-related macular degeneration, diabetic retinopathy, retinal vascular occlusion, stroke overview, optic neuritis, chlorine poisoning, strabismus, intracranial hemorrhage, vertebrobasilar circulatory disorder, subarachnoid hemorrhage, brain aneurysm, Tay-Sachs disease, chemical poisoning, basal cell nevus syndrome, and Ito syndrome.

6. A computer-implemented method of providing remote assistance for one or more visually-impaired users, the method comprising:

receiving video content at an agent device, the video content including images of an environment in which a client device is located;

receiving information indicating a visual impairment of a user of the client device;

identifying a mask that simulates the visual impairment of the user, the mask comprising a fixed component filter and a variable-component filter, wherein the variable-component filter is adjustable to simulate variations in the visual impairment;

applying the mask to the received video content to generate filtered video content representative of what the environment looks like to the user based on the visual impairment of the user; and

providing the filtered video content for display on the agent device.

7. The method of claim 6, further comprising receiving profile data of the user, wherein the profile data includes the information indicating the visual impairment of the user.

8. The method of claim 6, wherein the variable-component filter is at least partially determined by a condition of a slider control object in an agent interface.

9. The method of claim 6, further comprising providing a mask selector on a display of the agent device that can be selected and unselected to display and cancel the display of the filtered video content.

10. The method of claim 6, wherein the visual impairment comprises one or more conditions, the one or more conditions including at least one or more of head injury, glaucoma, retinal detachment, hypertensive retinotherapy, cataract, age-related macular degeneration, diabetic retinopathy, retinal vascular occlusion, stroke overview, optic neuritis, chlorine poisoning, strabismus, intracranial hemorrhage, vertebrobasilar circulatory disorder, subarachnoid hemorrhage, brain aneurysm, Tay-Sachs disease, chemical poisoning, basal cell nevus syndrome, and Ito syndrome.

11. A computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising:

instructions that cause the one or more processors to receive video content at an agent device, the video content including images of an environment in which a client device is located;

instructions that cause the one or more processors to receive information indicating a visual impairment of a user of the client device;

instructions that cause the one or more processors to identify a mask that simulates the visual impairment of the user, the mask comprising a fixed component filter and a variable-component filter, wherein the variable-component filter is adjustable to simulate variations in the visual impairment;

instructions that cause the one or more processors to apply the mask to the video content to generate filtered video content representative of what the environment looks like to the user based on the visual impairment of the user; and

instructions that cause the one or more processors to provide the filtered video content for display on the agent device.

12. The computer-readable memory of claim 11, further comprising instructions that cause the one or more processors to receive profile data of the user, wherein the profile data includes the information indicating the visual impairment of the user.

13. The computer-readable memory of claim 11, wherein the variable-component filter is at least partially determined by a condition of a slider control object in an agent interface.

14. The computer-readable memory of claim 11, further comprising instructions that cause the one or more processors to provide a mask selector on a display of the agent device that can be selected and unselected to display and cancel the display of the filtered video content.

15. The computer-readable memory of claim 11, wherein the visual impairment comprises one or more conditions, the one or more conditions including at least one or more of head injury, glaucoma, retinal detachment, hypertensive retinotherapy, cataract, age-related macular degeneration, diabetic retinopathy, retinal vascular occlusion, stroke overview, optic neuritis, chlorine poisoning, strabismus, intracranial hemorrhage, vertebrobasilar circulatory disorder, subarachnoid hemorrhage, brain aneurysm, Tay-Sachs disease, chemical poisoning, basal cell nevus syndrome, and Ito syndrome.