COLOR ACCOMMODATION FOR ON-DEMAND ACCESSIBILITY

• US 20200135052A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Singh, Ashish, Adams, Justice, Jati, Arindam, Omote, Masanori
• Granted Patent: US 10,854,109 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted
• CPC Class Codes: G10L 25/30 using neural networks
• CPC Class Codes: H04N 9/643 Hue control means, e.g. flesh tone control

Abstract

Accommodation for color or visual impairments may be implemented by selective color substitution. A color accommodation module receives an image frame from a host system and generates a color-adapted version of the image frame. The color accommodation module may include a rule based filter that substitutes one or more colors within the image frame with one or more corresponding alternative colors.

Claims

1. A system for enhancing the accessibility of Audio Visual content, the system comprising:

a color accommodation module implemented in hardware, software, or a combination of hardware and software configured to receive an image frame from a host system and generate a color-adapted version of the image frame; and

an accessibility controller coupled to the color accommodation module wherein the accessibility controller is configured to provide the color adapted version of the image frame to the host system for display on a display screen and synchronize the output of a scene annotation module with one or more other neural network modules, wherein the scene annotation module is implemented in hardware, software, or a combination of hardware and software, and wherein the scene annotation module is configured to generate a text description of scene elements within the image frame.

2. The system of claim 1 wherein the color accommodation module includes a rule based filter that substitutes one or more colors within the image frame with one or more corresponding alternative colors.

3. The system of claim 2 wherein the one or more corresponding alternative colors are configured to enhance differences in color hues such that the colors are visible to a person with a vision impairment.

4. The system of claim 1 wherein the rule based filter is applied to one or more image frames in a video sequence and wherein the video sequence is less than a second in length.

5. The system of claim 1 wherein the image frame is a video game frame data.

6. (canceled)

7. The system of claim 16 wherein the one or more other neural network modules includes an acoustic effect annotation module implemented in hardware, software, or a combination of hardware and software configured classify primary acoustic effects occurring within an audio segment wherein the audio segment is synchronized to occur during presentation of the image frame.

8. A method for enhancing the accessibility of Audio Visual content, comprising:

receiving an image frame from a host system and generating a color-adapted version of the image frame with a color accommodation module;

providing the color adapted version of the image frame to the host system for display on a display screen; and

synchronizing the output of a scene annotation module with one or more other neural network modules using an accessibility controller coupled to the color accommodation module wherein the scene annotation module is configured to generate a text description of scene elements within the image frame.

9. The method of claim 8 wherein generating the color-adapted version of the image frame includes substituting one or more colors within the image frame with one or more corresponding alternative colors using a rule based filter.

10. The method of claim 9 wherein the one or more corresponding alternative colors are configured to enhance differences in color hues such that the colors are visible to a person with a vision impairment.

11. The method of claim 8 wherein the rule based filter is applied to one or more image frames in a video sequence and wherein the video sequence is less than a second in length.

12. The method of claim 8 wherein the image frame is a video game frame data.

14. The method of claim 8 wherein the one or more other neural network modules includes an acoustic effects annotation module configured classify primary acoustic effects occurring within an audio segment wherein the audio segment is synchronized to occur during presentation of the image frame.

13. (canceled)

15. A non-transitory computer-readable medium having computer readable instructions embodied therein, the instructions being configured upon execution to implement a method for enhancing the accessibility of Audio Visual content, the method comprising:

receiving an image frame from a host system and generating a color-adapted version of the image frame with a color accommodation module;

providing the color adapted version of the image frame to the host system for display on a display screen; and

synchronizing the output of a scene annotation module with one or more other neural network modules using an accessibility controller coupled to the color accommodation module wherein the scene annotation module is configured to generate a text description of scene elements within the image frame.