User and device movement based display compensation

• US 9,417,666 B2
• Filed: 10/19/2012
• Issued: 08/16/2016
• Est. Priority Date: 10/19/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Abraham, Robin, Fawcett, Andrew V.
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): PATEL, SHIVANG I
• Application Number:
  US13/655,753
• Publication Number:
  US 20140111550A1
• Time in Patent Office: 1,397 Days
• Field of Search: 345/648
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 3/20 Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

A direction and distance of movement of a display device as well as of a user of the display device are determined. Based on these determined directions and distances of movement, compensation to apply to content displayed on the display device to compensate for movement of the user with respect to the device is determined and applied to the content. A portion of the display device at which the user is looking can also be detected. The compensation is applied to the content only if applying the compensation would not result in the portion being positioned beyond the display device. If applying the compensation would result in the portion being positioned beyond the display device then appropriate corrective action is taken, such as not applying the compensation to the content.

Claims

1. A method comprising:

determining, by a display device, a direction of movement and a distance of movement of the display device;

determining, by the display device, a direction of movement and a distance of movement of a user of the display device;

determining, by the display device, compensation to apply to content displayed on the display device to compensate for movement of the user with respect to the display device, the compensation being based on the direction of movement of the display device, the distance of movement of the display device, the direction of movement of the user, and the distance of movement of the user;

detecting, by the display device, a portion of the display device at which the user is looking;

determining whether applying the compensation would result in the content displayed in the portion of the display device at which the user is looking being positioned beyond a perimeter of the display device;

responsive to determining that applying the compensation would result in the content displayed in the portion of the display device at which the user is looking being positioned beyond the perimeter of the display device, taking corrective action to prevent the content displayed in the portion of the display device at which the user is looking from being positioned beyond the perimeter of the display device; and

applying, by the display device, the compensation to the content displayed on the display device.

2. A method as recited in claim 1, the corrective action comprising not applying the compensation to the content displayed on the display device.

3. A method as recited in claim 1, the corrective action comprising applying only part of the compensation to the content displayed on the display device.

4. A method as recited in claim 1, the compensation comprising shifting the content in a plane approximately parallel to a plane of the display device.

5. A method as recited in claim 1, the compensation comprising zooming in or zooming out the content.

6. A method as recited in claim 1, the compensation comprising shifting the content in a plane approximately parallel to a plane of the display device and further comprising zooming in or zooming out the content.

7. A method as recited in claim 1, further comprising:

generating, by the display device, a device movement vector identifying the direction of movement and the distance of movement of the display device;

generating, by the display device, a user movement vector identifying the direction of movement and the distance of movement of the user;

generating, by the display device, a relative movement vector by subtracting the user movement vector from the device movement vector; and

using the relative movement vector to determine the compensation.

8. A method as recited in claim 7, the relative movement vector including an x dimension element in a plane of the display device and a y dimension element in the plane of the display device, and the using the relative movement vector to determine the compensation comprising determining an amount to shift the content in the x dimension as the opposite of a value of the x dimension element and determining an amount to shift the content in the y dimension as the opposite of a value of the y dimension element.

9. A method as recited in claim 1, further comprising determining the direction of movement and distance of movement of the user based on detection of a face of the user in images captured by a camera included in a device that houses the display device.

10. A method as recited in claim 9, further comprising determining the direction of movement and distance of movement of the device based on one or more accelerometers included in the device that houses the display device.

11. A method as recited in claim 1, the determining the direction of movement and the distance of movement of the display device performed at 10 millisecond intervals.

12. A method as recited in claim 1, the corrective action comprising reducing an amount of shifting to apply to the content.

13. A method as recited in claim 1, the corrective action comprising reducing an amount of zooming to apply to the content.

14. One or more computer-readable storage media having stored thereon multiple instructions that, when executed by one or more processors of a computing device, cause the one or more processors to:

determine compensation to apply to content displayed on a display device to compensate for movement of a user with respect to the display device, the compensation being based on a direction of movement of the display device, a distance of movement of the display device, a direction of movement of the user, and a distance of movement of the user;

detect a portion of the display device at which the user is looking;

determine whether applying the compensation to the content would result in the content displayed in the portion of the display device being positioned beyond a perimeter of the display device;

apply the compensation to the content displayed on the display device in response to determining that applying the compensation would not result in the content in the portion of the display device being positioned beyond the perimeter of the display device; and

take corrective action in response to determining that applying the compensation would result in the content in the portion of the display device being positioned beyond the perimeter of the display device.

15. One or more computer-readable storage media as recited in claim 14, the corrective action comprising not applying the compensation to the content displayed on the display device.

16. One or more computer-readable storage media as recited in claim 14, the compensation comprising shifting the content in a plane approximately parallel to a plane of the display device and further comprising zooming in or zooming out the content displayed on the display device.

17. One or more computer-readable storage media as recited in claim 14, wherein the multiple instructions further cause the one or more processors to:

generate a device movement vector identifying the direction of movement and the distance of movement of the display device;

generate a user movement vector identifying the direction of movement and the distance of movement of the user;

generate a relative movement vector by subtracting the user movement vector from the device movement vector; and

use the relative movement vector to determine the compensation.

18. One or more computer-readable storage media as recited in claim 17, the relative movement vector including an x dimension element in a plane of the display device and a y dimension element in the plane of the display device, and wherein to use the relative movement vector to determine the compensation is to determine an amount to shift the content in the x dimension as the opposite of a value of the x dimension element and determine an amount to shift the content in the y dimension as the opposite of a value of the y dimension element.

19. One or more computer-readable storage media as recited in claim 14, wherein the multiple instructions further cause the one or more processors to determine both the direction of movement of the user and distance of movement of the user based on detection of a face of the user in images captured by a camera included in the computing device, the computing device also housing the display device.

20. A computing device comprising:

one or more processors; and

one or more computer-readable storage media having stored thereon multiple instructions of a movement compensation system that, when executed by the one or more processors, cause the one or more processors to;

determine a direction of movement and a distance of movement of a display device;

determine a direction of movement and a distance of movement of a user of the display device;

determine compensation to apply to content displayed on the display device to compensate for movement of the user with respect to the display device, the compensation being based on the direction of movement of the user with respect to the display device, the distance of movement of the display device, the direction of movement of the user, and the distance of movement of the user;

detect a portion of the display device at which the user is looking;

apply, in response to a determination that applying the compensation would not result in content in the portion of the display device being positioned external to a perimeter of the display device, compensation to the content displayed on the display device; and

display the content without compensation in response to a determination that applying the compensation would result in content in the portion of the display device being positioned external to the perimeter of the display device.