System and method for rendering dynamic three-dimensional appearing imagery on a two-dimensional user interface

• US 9,423,873 B2
• Filed: 09/19/2014
• Issued: 08/23/2016
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: McCarthy, Brendan A.
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US14/491,191
• Publication Number:
  US 20150097826A1
• Time in Patent Office: 704 Days
• Field of Search: 345/419, 345/427, 345/619, 345/633, 702/188
• US Class Current: 1/1
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering

Abstract

System and method for rendering dynamic three-dimensional appearing imagery on a two-dimensional user interface screen of a portable computing device in dependence on a user'"'"'s view-point of the screen. The method includes processing, on a portable computing device, data defining a plurality of user view-points of a user interface screen of the portable computing device. The method next includes rendering a first image of a constructed scene on the user interface screen based on a first determined user'"'"'s view-point of the user interface screen of the portable computing device. The method then includes rendering a different image of the constructed scene on the user interface screen based on a subsequently determined user'"'"'s view-point of the user interface screen and thereby presenting the illusion of a three-dimensional image of the constructed scene on the user interface screen.

Claims

1. A method comprising:

at a mobile device with a camera and a display;

capturing an image of a user with the camera;

analyzing the captured image of the user;

determining, from the analysis of the captured image of the user, whether positions of the user'"'"'s eyes are determinable;

in accordance with a determination that the positions of the user'"'"'s eyes are determinable, determining a first viewpoint of the user relative to the display based on the positions of the user'"'"'s eyes;

in accordance with a determination that the positions of the user'"'"'s eyes are not determinable, determining the first viewpoint of the user relative to the display using an alternate signature target;

rendering a first image of a constructed scene on the display based on the first viewpoint relative to the display;

after determining the first viewpoint of the user relative to the display, determining a second viewpoint of the user relative to the display; and

rendering a different image of the constructed scene on the display based on the second viewpoint of the user relative to the display.

2. The method of claim 1, wherein the constructed scene is wirelessly received from a remote source prior to rendering the different images on the display.

3. The method of claim 1, wherein the alternate signature target includes the user'"'"'s nose, the user'"'"'s chin, and/or the user'"'"'s ears.

4. The method of claim 1, wherein the method includes:

determining whether no target is identified; and

,

in accordance with a determination that no target is identified;

ceasing to display a three-dimensional image on the display, and

rendering a two-dimensional image on the display.

5. The method of claim 1, wherein:

the mobile device includes at least one infrared light emitter; and

the method includes;

emitting, via the at least one infrared light emitter, infrared light aimed at the user; and

detecting, by the camera, infrared light reflected from the user.

6. The method of claim 5, wherein the analysis of the captured image includes analysis of at least one infrared image captured by the camera.

7. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a mobile device with a camera and a display, cause the device to:

capture an image of a user with the camera;

analyze the captured image of the user;

determine, from the analysis of the captured image of the user, whether positions of the user'"'"'s eyes are determinable;

in accordance with a determination that the positions of the user'"'"'s eyes are determinable, determine a first viewpoint of the user relative to the display based on the positions of the user'"'"'s eyes;

in accordance with a determination that the positions of the user'"'"'s eyes are not determinable, determine the first viewpoint of the user relative to the display using an alternate signature target;

render a first image of a constructed scene on the display based on the first viewpoint relative to the display;

after determining the first viewpoint of the user relative to the display, determine a second viewpoint of the user relative to the display; and

render a different image of the constructed scene on the display based on the second viewpoint of the user relative to the display.

8. The non-transitory computer readable storage medium of claim 7, wherein the constructed scene is wirelessly received from a remote source prior to rendering the different images on the display.

9. The non-transitory computer readable storage medium of claim 7, wherein the alternate signature target includes the user'"'"'s nose, the user'"'"'s chin, and/or the user'"'"'s ears.

10. The non-transitory computer readable storage medium of claim 7, wherein the mobile device includes at least one infrared light emitter;

and

wherein the one or more programs include instructions, which when executed by the mobile device, cause the device to;

determine whether no target is identified; and

,

in accordance with a determination that no target is identified;

cease to display a three-dimensional image on the display, and

render a two-dimensional image on the display.

11. The non-transitory computer readable storage medium of claim 7, wherein the one or more programs include instructions, which when executed by the mobile device, cause the device to:

emit, via at least one infrared light emitter, infrared light aimed at the user; and

detect, by the camera, infrared light reflected from the user.

12. The non-transitory computer readable storage medium of claim 11, wherein the analysis of the captured image includes analysis at least one infrared image captured by the camera.

13. An electronic device, comprising:

a camera;

a display;

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

capturing an image of a user with the camera;

analyzing the captured image of the user;

determining, from the analysis of the captured image of the user, whether positions of the user'"'"'s eyes are determinable;

in accordance with a determination that the positions of the user'"'"'s eyes are determinable, determining a first viewpoint of the user relative to the display based on the positions of the user'"'"'s eyes;

in accordance with a determination that the positions of the user'"'"'s eyes are not determinable, determining the first viewpoint of the user relative to the display using an alternate signature target;

rendering a first image of a constructed scene on the display-based on the first viewpoint relative to the display;

after determining the first viewpoint of the user relative to the display, determining a second viewpoint of the user relative to the display; and

rendering a different image of the constructed scene on the display based on the second viewpoint of the user relative to the display.

14. The electronic device of claim 13, wherein the constructed scene is wirelessly received from a remote source prior to rendering the different images on the display.

15. The electronic device of claim 13, wherein the alternate signature target includes the user'"'"'s nose, the user'"'"'s chin, and/or the user'"'"'s ears.

16. The electronic device of claim 13, wherein the one or more programs include instructions for:

determining whether no target is identified; and

,

in accordance with a determination that no target is identified;

ceasing to display a three-dimensional image on the display, and

rendering a two-dimensional image on the display.

17. The electronic device of claim 13, wherein:

the device includes at least one infrared light emitter; and

the one or more programs include instructions for;

emitting, via the at least one infrared light emitter, infrared light aimed at the user; and

detecting, by the camera, infrared light reflected from the user.

18. The electronic device of claim 17, wherein the analysis of the captured image includes analysis of at least one infrared image captured by the camera.