Correcting for parallax in electronic displays

• US 8,884,928 B1
• Filed: 01/26/2012
• Issued: 11/11/2014
• Est. Priority Date: 01/26/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Baldwin, Leo B.
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): Acha, III, Josemarie G
• Application Number:
  US13/358,881
• Time in Patent Office: 1,020 Days
• Field of Search: None
• US Class Current: 345/178
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

The ability of a user to provide input when using a touch screen or other such element of a computing device can be improved by correcting for parallax errors. A camera of the computing device can determine the relative position of the user'"'"'s head or eyes with respect to the device, and determine a viewing perspective of the user with respect to content displayed on the touch screen. Using the perspective direction and information about the dimensions of the touch screen, the device can determine an amount and direction of parallax error, and can make adjustments to remove or minimize that error. The device can cause the position of the displayed graphical elements to shift and/or can adjust the mappings of the touch sensitive regions that correspond to the graphical elements. Such an approach enables the graphical elements and touch mappings to align from the current perspective of the user.

Claims

1. A computer-implemented method of correcting for parallax errors in a touch screen of a computing device, comprising:

capturing image information using a camera of the computing device;

analyzing the image information, using a processor of the computing device, to track a relative position of a head of a user with respect to the computing device; and

for a current relative position of the head;

determining, using the processor of the computing device, an amount of offset between a first region and a second region of a touch sensitive surface of the touch screen, wherein the first region is mapped by the computing device to a graphical element displayed on a display surface of the touch screen and the second region corresponds to a viewed position, from the current relative position of the head, of the graphical element at the touch sensitive surface;

updating a mapping of the graphical element from the first region to approximately the second region on the touch sensitive surface when the amount of offset exceeds an offset threshold, wherein the user is able to select the graphical element using the second region of the touch sensitive surface; and

updating a displayed location of the graphical element such that the viewed position of the graphical element through the touch sensitive surface corresponds to the first region in response to determining that the amount of offset is greater than a maximum amount of acceptable offset but less than the offset threshold, wherein the user is able to select the graphical element using the first region of the touch sensitive surface.

2. The computer-implemented method of claim 1, wherein the graphical element is positioned near a center point of the display surface.

3. The computer-implemented method of claim 1, further comprising:

determining additional mapped locations on the touch sensitive surface mapped to multiple display locations on the display surface; and

updating at least one of the mapped locations or the display locations based at least in part upon the relative position of the head, each mapped location and corresponding display location corresponding to one or more of a different amount or direction of parallax error for off-axis positions of the head.

4. The computer-implemented method of claim further comprising:

determining a relative position of eyes of the user with respect to the computing device using the image information, wherein the relative position of the eyes is further used to determine the amount of offset.

5. A computer-implemented method of displaying content on an electronic device, comprising:

displaying content on a touch screen of the electronic device, the touch screen having a first surface for displaying the content and a second surface enabling a user to provide touch input to the electronic device, the first surface being separated from the second surface;

determining a viewing perspective of the user with respect to the touch screen;

determining an amount of offset between a first region of the first surface and a second region of the second surface based on the viewing perspective, wherein the first region corresponds to a rendering position for at least one element of the content displayed on the first surface and the second region corresponds to a touch input location for the at least one element on the second surface;

updating the rendering position for the at least one element content on the first surface in response to determining that the amount of offset is greater than a specified amount of offset but less than an offset threshold; and

updating the touch input location for the at least one element on the second surface in response to determining that the amount of offset exceeds the offset threshold.

6. The computer-implemented method of claim 5, further comprising:

capturing image information to locate at least one feature of the user; and

analyzing the image information to determine a relative position of the at least one feature.

7. The computer-implemented method of claim 6, wherein the feature includes at least one of a head, eyes, or facial feature of the user.

8. The computer-implemented method of claim 6, wherein the at least one feature is located in the image information using at least one of an image recognition algorithm, a contour matching algorithm, a retro-reflection detecting algorithm, or a color matching algorithm.

9. The computer-implemented method of claim 5, wherein the touch screen provides at least one of capacitive, resistive, or optical touch detection.

10. The computer-implemented method of claim 5, further comprising:

determining a plurality of different amounts of offsets for different portions of the touch screen; and

updating the different portions of the touch screen by different amounts or in different directions corresponding to the different amounts of offsets.

11. The computer-implemented method of claim 5, further comprising:

determining a specified amount of illumination needed to locate a feature of a user for determining the viewing perspective of the user; and

activating an illumination element when a current amount of illumination is less than the specified amount of illumination.

12. The computer-implemented method of claim 11, wherein the illumination element produces visible light or infrared radiation.

13. The computer-implemented method of claim 5, further comprising:

updating a scale of the at least one element or the touch input location to account for foreshortening based on the viewing perspective.

14. A computing device, comprising:

a processor;

a camera;

a touch screen having a first surface for displaying content and a second surface enabling a user to provide touch input to the computing device, the first surface being separated from the second surface; and

a memory device including instructions that, upon being executed by the processor, cause the computing device to;

capture image information using the camera to locate at least one feature of the user;

determine a viewing perspective of the user based at least in part upon the located at least one feature;

determine an amount of offset between a first region of the first surface and a second region of the second surface based on the viewing perspective, wherein the first region corresponds to a rendering position for at least one element of the content displayed on the first surface and the second region corresponds to a touch input location for the at least one element on the second surface;

update the rendering position for the at least one element on the first surface in response to determining that the amount of offset is greater than a specified amount of offset but less than an offset threshold; and

update the touch input location for the at least one element on the second surface in response to determining that the amount of offset exceeds the offset threshold.

15. The computing device of claim 14, wherein the instructions, upon being executed, to cause the computing device to determine the viewing perspective include causing the computing device to:

analyze the image information to determine a relative position of the at least one feature.

16. The computing device of claim 14, wherein the instructions, upon being executed, further cause the computing device to:

determine a plurality of different amounts of offsets for different portions of the touch screen; and

update the different portions of the touch screens by different amounts or in different directions corresponding to the different amounts of offsets.

17. The computing device of claim 14, wherein the instructions, upon being executed, further cause the computing device to:

determine a specified amount of illumination needed to locate the at least one feature of the user; and

activate an illumination element when a current amount of illumination is less than the specified amount of illumination.

18. The computing device of claim 17, wherein the illumination element produces visible light or infrared radiation.

19. A non-transitory computer-readable storage medium storing instructions that, upon being executed by at least one processor of a computing device, cause the computing device to:

display content on a touch screen of the computing device, the touch screen having a first surface for displaying the content and a second surface enabling a user to provide touch input to the computing device, the first surface being separated from the second surface;

determine a position of a feature of a user with respect to the computing device;

determine a viewing perspective of the user based at least in part upon the position of the feature;

determine an amount of offset between a first region of the first surface and a second region of the second surface based on the viewing perspective, wherein the first region corresponds to a rendering position for at least one element of the content displayed on the first surface and the second region corresponds to a touch input location for the at least one element on the second surface;

update the rendering position for the at least one element on the first surface in response to determining that the amount of offset is greater than a specified amount of offset but less than an offset threshold; and

update the touch input location for the at least one element on the second surface in response to determining that the amount of offset exceeds the offset threshold.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions, upon being executed, further cause the computing device to:

determine a plurality of different amounts of offsets for different portions of the touch screen; and

update the different portions of the touch screens by different amounts or in different directions corresponding to the different amounts of offsets.

21. The non-transitory computer-readable storage medium of claim 19, wherein the position of the feature of the user is determined using at least one of an image recognition algorithm, a contour matching algorithm, a retro-reflection detecting algorithm, or a color matching algorithm.

22. The non-transitory computer-readable storage medium of claim 19, wherein the instructions, upon being executed, further cause the computing device to:

update a scale of the at least one element or the touch input location to account for foreshortening based on the viewing perspective.