Modifying display region for people with loss of peripheral vision

• US 10,650,702 B2
• Filed: 07/10/2017
• Issued: 05/12/2020
• Est. Priority Date: 07/10/2017
• Status: Active Grant

• Current Assignee: Saturn Licensing LLC (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Candelore, Brant, Nejat, Mahyar, Shintani, Peter
• Primary Examiner(s): Volentine, Rebecca A
• Application Number:
  US15/645,617
• Publication Number:
  US 20190012931A1
• Time in Patent Office: 1,037 Days
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G09B 21/008 using visual presentation of the information for the partially sighted
• CPC Class Codes: H04N 21/4355 involving reformatting operations of additional data, e.g. HTML pages on a television screen optimising the visualization of content for information retrieval from the Internet G06F16/9577; adaptation of message content in packet-switching networks H04L51/066; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440245 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4858 for modifying screen layout parameters, e.g. fonts, size of the windows
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

The most active part of a video frame is magnified on a display to accommodate people with eye maladies such as glaucoma or perhaps retinitis pigmentosa. An area of interest in a video frame is identified, and that area is expanded or magnified to the center of the display, where people suffering from loss of peripheral vision can best see. Assuming the viewer sits a benchmark distance from the display, the active part of the image is expanded to fit in what can be seen in the center of the display without moving the head left to right and up and down. This may be done by the display recognizing where most of the action is taking place by means of with motion vectors and I-macroblocks, or by allowing the viewer to switch to various predetermined blocks on screen using a remote control.

Claims

1. A device comprising:

at least one computer memory that is not a transitory signal and that comprises instructions executable by at least one processor to;

identify at least one preferred region of a video display having plural regions, including at least one non-preferred region;

identify at least one video item of interest (VII), the VII comprising a type of video object in at least one video frame for presentation on the video display;

identify, in the at least one video frame for presentation on the video display, the VII in a frame location corresponding to the non-preferred region of the video display;

present the VII in the preferred region of the video display instead of in the non-preferred region of the video display corresponding to the frame location in which the VII is received;

wherein the VII is identified at least in part by;

accessing a data store of VII, the VII in the data store comprising at least two video object types selected from;

someone speaking, an object moving, an object in a particular background; and

responsive to identifying at least a first VII in the data store as being at least one of someone speaking, an object moving, an object in a particular background, select the first VII and present the first VII in the preferred region of the video display.

2. The device of claim 1, wherein the instructions are executable to:

enlarge the VII relative to a size of the VII received in the video frame to render an enlarged VII which is presented in the preferred region in lieu of presenting the VII received in the video frame in the non-preferred region.

3. The device of claim 1, wherein the instructions are executable to identify the at least one preferred region of a video display at least in part by:

receiving an image of a viewer;

executing image recognition on the image to render a result;

using the result to access a database having information useful in identifying the preferred region based on the result of the image recognition.

4. The device of claim 1, wherein the instructions are executable to identify the at least one preferred region of a video display at least in part by:

receiving input from at least one user interface (UI) indicating a type of visual impairment; and

correlating the type of visual impairment to the preferred region, wherein the UI comprises;

an audible or visual prompt to indicate an impairment;

at least a first audible or visual selector selectable to indicate a first vision impairment;

at least a second selector audible or visual selectable to indicate a second vision impairment, the first vision impairment being a different from the second vision impairment.

5. The device of claim 4, wherein the first vision impairment is indicated by the UI as being a central vision impairment and the second vision impairment is identified by the UI as being a peripheral vision impairment.

6. The device of claim 1, wherein the instructions are executable to identify the at least one preferred region of a video display at least in part by:

receiving input from at least one user interface (UI) indicating, by a selection on a region of the display, the preferred region.

7. The device of claim 1, wherein the instructions are executable to identify the at least one preferred region of a video display at least in part by:

sending to at least one server at least one identification (ID), the ID including an ID of the video display and/or an ID of a person; and

receiving back from the server indication of the preferred region responsive to the ID.

8. The device of claim 1, wherein the instructions are executable to identify, in at least one video frame for presentation on the video display, the VII at least in part by:

selecting a portion of a video frame as the VII responsive to the portion being characterized by one or more motion vectors satisfying a first test.

9. The device of claim 1, wherein the instructions are executable to identify, in at least one video frame for presentation on the video display, the VII at least in part by:

executing image recognition on at least one video frame to identify a VII.

10. The device of claim 1, wherein the preferred region is at a center of the video display.

11. The device of claim 1, wherein the preferred region does not include a peripheral region of the video display.

12. The device of claim 1, wherein the instructions are executable to:

identify a distance from the video display at which a viewer is located; and

based at least in part on the distance, present the VII in the preferred region of the video display.

13. The device of claim 1, wherein the instructions are executable to:

identify, in at least one video frame for presentation on the video display, the VII at least in part by;

selecting a portion of a video frame as the VII responsive to the portion being characterized by having a color histogram indicating a range of colors in the portion.

14. A device comprising:

at least one computer memory that is not a transitory signal and that comprises instructions executable by at least one processor to;

identify at least one preferred region of a video display having plural regions, including at least one non-preferred region;

identify at least one video item of interest (VII), the VII comprising a type of video object in at least one video frame for presentation on the video display;

identify, in the at least one video frame for presentation on the video display, the VII in a frame location corresponding to the non-preferred region of the video display;

present the VII in the preferred region of the video display instead of in the non-preferred region of the video display corresponding to the frame location in which the VII is received;

wherein the instructions are executable to receive the identification of the at least one VII at least in part by;

receiving input from at least one user interface (UI) indicating at least one VII, wherein the UI comprises;

a prompt to identify VII;

a list of VII from which one or more VII may be selected, the list including VII comprising someone speaking, an object moving, an object in a particular background;

an entry element to receive input to identify a VII from someone speaking, an object moving, an object in a particular background.

15. A method, comprising:

identifying that a viewer of a video display has a peripheral visual impairment;

accessing a data store of default items of interest, the default items of interest comprising at least two video object types selected from;

someone speaking, an object moving, an object in a particular background;

responsive to identifying that the viewer of the video display has a peripheral visual impairment, selecting a first one of the default items of interest on the basis that the first one of the default items of interest is one of;

someone speaking, an object moving, an object in a particular background; and

presenting, in a central region of the video display, at least the first one of the default items of interest received in a peripheral region of a video frame.

16. A system comprising:

at least one video display;

at least one processor to control the video display; and

storage with instructions accessible to the processor to cause the processor to;

identify that a video item of interest received in a peripheral region in a video frame is to be presented in a central region of the video display;

responsive to identifying the video item of interest in a peripheral region in a video frame, present the video item of interest (VII) in the central region of the video display, wherein the VII comprises a video object type and the instructions are executable to receive an identification of the at least one VII at least in part by;

receiving input from at least one user interface (UI) indicating at least one VII, wherein the UI comprises;

a prompt to identify VII;

a list of VII from which one or more VII may be selected, the list including VII comprising someone speaking, an object moving, an object in a particular background;

an entry element to receive input to identify a VII from someone speaking, an object moving, an object in a particular background.

17. The system of claim 16, wherein the instructions are executable to:

enlarge the VII relative to a size of the VII received in the video frame to render an enlarged VII which is presented in the central region in lieu of presenting the VII received in the video frame in the peripheral region.

18. The system of claim 16, wherein the instructions are executable to receive an identification of the at least one VII at least in part by:

accessing a data store of default VII, the default VII comprising at least two video object types selected from;

someone speaking, an object moving, an object in a particular background.

19. The system of claim 16, wherein the instructions are executable to identify, in at least one video frame for presentation on the video display, the VII at least in part by:

selecting a portion of a video frame as the VII responsive to the portion being characterized by one or more motion vectors satisfying a first test.

20. The system of claim 16, wherein the instructions are executable to identify, in at least one video frame for presentation on the video display, the VII at least in part by:

executing image recognition on at least one video frame to identify a VII.

21. The system of claim 16, wherein the instructions are executable to:

identify a distance from the video display at which a viewer is located; and

based at least in part on the distance and responsive to identifying the video item of interest in a peripheral region in a video frame, present the video item of interest (VII) in the central region of the video display.