Methods and Systems for Display Viewer Motion Compensation Based on User Image Data

• US 20080199049A1
• Filed: 11/30/2007
• Published: 08/21/2008
• Est. Priority Date: 02/21/2007
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Daly, Scott J.
• Granted Patent: US 7,903,166 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20004 Adaptive image processing
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 3/007 Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

Embodiments of the present invention comprise systems and methods for compensating for motion of a viewer relative to a display device.

Claims

1. A method for displaying a motion-compensated image, said method comprising:

a) capturing a first user image of a user of a display device with an imaging device;

b) identifying a user facial feature in said first user image;

c) calculating a first user position relative to said display device from said user facial feature;

d) capturing a second user image of said user of said display device with said imaging device;

e) identifying said user facial feature in said second user image;

f) calculating a second user position relative to said display device from said user facial feature;

g) calculating a user motion characteristic using said first user position and said second user position

h) receiving an input image; and

i) modifying said input image by shifting said image relative to said display in a direction related to said user motion characteristic, thereby creating a modified input image.

2. A method as described in claim 1 further comprising filtering said modified image to compensate for image blur due to display temporal response, thereby creating a filtered, modified input image.

3. A method as described in claim 1 further comprising modifying said input image with a zoom operation in response to said user motion characteristic indicating motion of said user'"'"'s head in the direction of said display device.

4. A method as described in claim 2 wherein said filtering comprises a motion-compensated spatial inverse filtering operation.

5. A method as described in claim 2 further comprising applying a soft aperture region to said filtered, modified input image.

6. A method as described in claim 1 further comprising expanding said input image to a size that is larger than the size of said display.

7. A method as described in claim 1 wherein said receiving a position comprises receiving coordinates for said user'"'"'s head and said display and calculating a relative position of said user'"'"'s head with respect to said display.

8. A method as described in claim 2 wherein said filtering is based on display temporal response parameters.

9. A method as described in claim 2 wherein said filtering is based on a global velocity of said image relative to said display.

10. A method as described in claim 5 wherein the extent of said soft aperture region varies with the magnitude of said user motion characteristic.

11. A method as described in claim 5 wherein said soft aperture region comprises matching a color in a soft aperture region to a display device color surrounding a display screen on said display device.

12. A system for displaying a motion-compensated image with a display device, said system comprising:

a) an imaging device for capturing user images of a user of a display device;

b) a facial feature or face indicator identifier for identifying a user facial feature or face indicator in said user images;

c) a user position calculator for calculating a user position relative to said display device using said user facial feature;

d) a motion characteristic calculator for calculating a user motion characteristic using a plurality of said user positions calculated with said user position calculator;

e) an image receiver for receiving an input image;

f) an image modifier for modifying said input image by shifting said image relative to said display in a direction related to said user motion characteristic, thereby creating a modified input image; and

g) a filter for filtering said modified image to compensate for image blur.

13. A system as described in claim 12 wherein said image modifier further modifies said input image with a zoom operation in response to a distance between said user'"'"'s head and said display as determined with said user motion characteristic.

14. A system as described in claim 12 wherein said filtering comprises a motion-compensated spatial inverse filtering operation.

15. A system as described in claim 12 further comprising a soft aperture processor for applying a soft aperture region to said input image.

16. A system as described in claim 12 wherein said filtering is based on a display temporal response.

17. A system as described in claim 12 wherein said filtering is based on a global velocity of said image relative to said display.

18. A system as described in claim 15 wherein the extent of said soft aperture region applied by said soft aperture processor varies with the magnitude of a change in position of a user'"'"'s head relative to said display.

19. A system as described in claim 15 wherein said soft aperture processor matches a color in a soft aperture region to a display case color under the ambient illumination ??.

20. A method for displaying a motion-compensated image, said method comprising:

a) capturing a first user image of a user of a display device with an imaging device;

b) identifying a user facial feature in said first user image;

c) calculating a first user position relative to said display device from said user facial feature;

d) capturing a second user image of said user of said display device with said imaging device;

e) identifying said user facial feature in said second user image;

f) calculating a second user position relative to said display device from said user facial feature;

g) calculating a user motion characteristic using said first user position and said second user position

h) receiving an input image;

i) modifying said input image by shifting said image relative to said display in a direction related to said user motion characteristic;

j) filtering said input image with a motion-compensated spatial inverse filtering process; and

k) creating a soft aperture region in said input image.