Methods and systems for display viewer motion compensation based on user image data

• US 7,903,166 B2
• Filed: 11/30/2007
• Issued: 03/08/2011
• Est. Priority Date: 02/21/2007
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Daly, Scott J.
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): HOGUE, DENNIS A
• Application Number:
  US11/948,959
• Publication Number:
  US 20080199049A1
• Time in Patent Office: 1,194 Days
• Field of Search: None
• US Class Current: 348/333.12
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20004 Adaptive image processing
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 3/007 Use of pixel shift techniques, e.g. by mechanical shift of the physical pixels or by optical shift of the perceived pixels
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

Embodiments of the present invention comprise systems and methods for compensating for motion of a viewer relative to a display device.

Claims

1. A method for displaying a motion-compensated image, said method comprising:

a) capturing a first user image of a user of a display device with an imaging device;

b) identifying a user facial feature in said first user image;

c) calculating a first user position relative to said display device from said user facial feature;

d) capturing a second user image of said user of said display device with said imaging device;

e) identifying said user facial feature in said second user image;

f) calculating a second user position relative to said display device from said user facial feature;

g) calculating a user motion characteristic using said first user position and said second user position

h) receiving an input image;

i) modifying said input image by shifting said image relative to said display device in a direction related to said user motion characteristic, thereby creating a modified input image;

j) applying a soft aperture region to said modified input image, wherein the extent of said soft aperture region varies with the magnitude of said user motion characteristic.

2. A method as described in claim 1 further comprising filtering said modified image to compensate for image blur due to display temporal response, thereby creating a filtered, modified input image.

3. A method as described in claim 2 wherein said filtering comprises a motion-compensated spatial inverse filtering operation.

4. A method as described in claim 2 wherein said filtering is based on display temporal response parameters.

5. A method as described in claim 2 wherein said filtering is based on a global velocity of said input image relative to said display.

6. A method as described in claim 1 further comprising modifying said input image with a zoom operation in response to said user motion characteristic indicating motion of said user'"'"'s head in the direction of said display device.

7. A method as described in claim 1 further comprising expanding said input image to a size that is larger than the size of said display.

8. A method as described in claim 1 wherein said soft aperture region comprises matching a color in a soft aperture region to a display device color surrounding a display screen on said display device.

9. A system for displaying a motion-compensated image with a display device, said system comprising:

a) an imaging device for capturing user images of a user of a display device ;

b) a facial feature or face indicator identifier for identifying a user facial feature or face indicator in said user images;

c) a user position calculator for calculating a user position relative to said display device using said user facial feature;

d) a motion characteristic calculator for calculating a user motion characteristic using a plurality of said user positions calculated with said user position calculator;

e) an image receiver for receiving an input image;

f) an image modifier for modifying said input image by shifting said image relative to said display in a direction related to said user motion characteristic, thereby creating a modified input image; and

g) a soft aperture processor for applying a soft aperture region to said input image, wherein the extent of said soft aperture region applied by said soft aperture processor varies with the magnitude of a change in position of a user'"'"'s head relative to said display.

10. A system as described in claim 9 wherein said image modifier further modifies said input image with a zoom operation in response to a distance between said user'"'"'s head and said display as determined with said user motion characteristic.

11. A system as described in claim 9 further comprising a filter for filtering said modified image to compensate for image blur.

12. A system as described in claim 11 wherein said filtering comprises a motion-compensated spatial inverse filtering operation.

13. A system as described in claim 11 wherein said filtering is based on a display temporal response.

14. A system as described in claim 11 wherein said filtering is based on a global velocity of said input image relative to said display.

15. A system as described in claim 9 wherein said soft aperture processor matches a color in a soft aperture region to a display case color under ambient illumination.