Tilt-based look around effect image enhancement method

• US 9,007,404 B2
• Filed: 03/15/2013
• Issued: 04/14/2015
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Legend3D, Inc.
• Original Assignee: Legend3D, Inc.
• Inventors: DeJohn, Matt, Sandrew, Jared, Schad, Thomas, Jones, Scott
• Primary Examiner(s): Hicks, Charles V
• Application Number:
  US13/843,527
• Publication Number:
  US 20140267235A1
• Time in Patent Office: 760 Days
• Field of Search: 345/659
• US Class Current: 345/659
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking

Abstract

A tilt-based look around effect image enhancement method that enables two-dimensional images to be depth enhanced and displayed for example from a different point of view based on the tilt or orientation, and/or movement of the viewing device itself. Embodiments may further alter or otherwise utilize different parallax maps to apply depth to the image based on the display type, e.g., two-dimensional or stereoscopic display, and may dynamically utilize both algorithms on a display capable of both types of display. In addition, embodiments may display information foreign to the image when portions of the image are exposed during the look around effect, including advertisements, game information, hyperlinks or any other data not originally in the image.

Claims

1. A tilt-based look around effect image enhancement method comprising:

obtaining an orientation of a display with respect to a reference frame from an orientation element physically associated with the display;

obtaining a parallax map associated with an image wherein the parallax map comprises at least two different depth values associated with at least two different pixels of the image respectively;

moving at least one of the at least two pixels in the image to at least one new pixel location based on the orientation of the display with respect to the reference frame and based on the parallax map;

displaying the image on the display including the at least one pixel in the image in the at least one new pixel location; and

one or more of

altering values of the parallax map to provide to a two-dimensional display to exaggerate depth values using an equation comprising

zz=((h−

g)*(1−

power((1−

z)/(1−

h/(h−

g)),1/0.3))+g;

and,

altering values of the parallax map to provide to a stereoscopic display to exaggerate depth values using an equation comprising

zz=((h−

g)*z)+g;

wherein

g=minimum depth value associated with an input depth map associated with the image,

h=maximum depth value associated with the input depth map associated with the image,

z=input parallax map value, and

zz=output parallax map value.

2. The method of claim 1 further comprising:

wherein the obtaining of the orientation of the display with respect to the reference frame comprises obtaining an orientation vector from the orientation element.

3. The method of claim 1 further comprising:

wherein the obtaining of the orientation of the display with respect to the reference frame comprises obtaining at least one acceleration value or angular value or both from the orientation element.

4. The method of claim 1 further comprising:

wherein the obtaining of the orientation of the display with respect to the reference frame comprises obtaining the orientation with respect to a vector associated with gravity.

5. The method of claim 1 further comprising:

wherein the obtaining of the orientation of the display with respect to the reference frame comprises obtaining the orientation with respect to an initial reference frame of the display at a particular point in time.

6. The method of claim 1 further comprising:

modifying the frame of reference over time.

7. The method of claim 1 further comprising:

normalizing a depth map having depth values for the at least two different pixels of the image to create the parallax map.

8. The method of claim 1 further comprising:

altering values of the parallax map to exaggerate depth values.

9. The method of claim 1 further comprising:

altering values of the parallax map based on said display to provide said two-dimensional display or said stereoscopic display based on a display output type associated with the display depending on whether the display is currently configured to display two-dimensional images or stereoscopic images.

10. The method of claim 1 further comprising:

obtaining a percentage of full tilt based on the orientation vector with respect to the reference frame;

multiplying the percentage of full tilt by the parallax map to obtain a pixel-by-pixel amount of movement to utilize in moving the at least one pixel in the image.

11. The method of claim 1 further comprising:

obtaining a percentage of full tilt based on the orientation vector with respect to the reference frame over time;

multiplying the percentage of full tilt by the parallax map to obtain a pixel-by-pixel amount of movement to utilize in moving the at least one pixel in the image over time.

12. The method of claim 1 further comprising:

obtaining a percentage of full tilt based on the orientation vector with respect to the reference frame in at least two dimensions orthogonal from each other and parallel to a plane defined by the display;

calculating a movement direction based on the percentage in each of two the directions;

multiplying the percentage of full tilt by the parallax map in the movement direction to obtain a pixel-by-pixel amount of movement to utilize in moving the at least one pixel in the image.

13. The method of claim 1 further comprising:

obtaining a percentage of full tilt based on the orientation vector or movement with respect to the reference frame in two dimensions orthogonal from each other and parallel to a plane defined by the display and based on movement in a third dimension not parallel to the plane defined by the display;

calculating a movement direction based on the percentage in each of two the directions;

multiplying the percentage of full tilt by the parallax map in the movement direction to obtain a pixel-by-pixel amount of movement to utilize in moving the at least one pixel in the image; and

,

translating the image based on the movement with respect to the reference frame in two dimensions

or

translating the image based on the movement in the third dimension

or

both translating the image in the two dimensions and in the third dimension.

14. The method of claim 1 further comprising:

accepting an input on the display wherein the display is a touch display and wherein the input translates the image in a third dimension not parallel to the plane defined by the display when two finger inputs are pinched toward one another or separated from one another.

15. The method of claim 1 further comprising:

revealing information foreign to the image and hidden by foreground objects wherein the information includes an advertisement or game clue.

16. The method of claim 1 further comprising:

revealing information foreign to the image and hidden by foreground objects wherein the information includes a hyperlink.

17. The method of claim 1 further comprising:

compositing at least two images to form the image.

18. The method of claim 1 wherein the displaying the image comprises displaying the image on a two-dimensional display.

19. The method of claim 1 further comprising:

generating a left eye viewpoint image and a right eye viewpoint image and wherein the displaying the image comprises displaying the left eye viewpoint image and the right eye viewpoint image on a stereoscopic display.

20. The method of claim 1 further comprising:

generating an encoded three-dimensional image wherein the displaying the image comprises displaying the encoded image for three-dimensional viewing with or without three-dimensional viewing glasses.

21. A tilt-based look around effect image enhancement method comprising:

obtaining an orientation of a display with respect to a reference frame from an orientation element physically associated with the display;

obtaining a parallax map associated with an image wherein the parallax map comprises at least two different depth values associated with at least two different pixels of the image respectively;

moving at least one of the at least two pixels in the image to at least one new pixel location based on the orientation of the display with respect to the reference frame and based on the parallax map;

displaying the image on the display including the at least one pixel in the image in the at least one new pixel location; and

,

altering values of the parallax map to provide to a two-dimensional display to exaggerate depth values using an equation comprising

zz=((h−

g)*(1−

power((1−

z)/(1−

h/(h−

g)),1/0.3)))+g;

wherein

g=minimum depth value associated with an input depth map associated with the image,

h=maximum depth value associated with the input depth map associated with the image,

z=input parallax map value, and

zz=output parallax map value.

22. A tilt-based look around effect image enhancement method comprising:

obtaining an orientation of a display with respect to a reference frame from an orientation element physically associated with the display;

obtaining a parallax map associated with an image wherein the parallax map comprises at least two different depth values associated with at least two different pixels of the image respectively;

moving at least one of the at least two pixels in the image to at least one new pixel location based on the orientation of the display with respect to the reference frame and based on the parallax map;

displaying the image on the display including the at least one pixel in the image in the at least one new pixel location; and

,

altering values of the parallax map to provide to a stereoscopic display to exaggerate depth values using an equation comprising

zz=((h−

g)*z)+g;

wherein

g=minimum depth value associated with an input depth map associated with the image,

h=maximum depth value associated with the input depth map associated with the image,

z=input parallax map value, and

zz=output parallax map value.