System and method for rendering dynamic three-dimensional appearing imagery on a two-dimensional user interface

• US 8,866,809 B2
• Filed: 09/30/2008
• Issued: 10/21/2014
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: McCarthy, Brendan A.
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US12/242,848
• Publication Number:
  US 20100079449A1
• Time in Patent Office: 2,212 Days
• Field of Search: 345/419, 345/427, 345/619, 345/633, 702/188
• US Class Current: 345/419
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering

Abstract

System and method for rendering dynamic three-dimensional appearing imagery on a two-dimensional user interface screen of a portable computing device in dependence on a user'"'"'s view-point of the screen. The method includes processing, on a portable computing device, data defining a plurality of user view-points of a user interface screen of the portable computing device. The method next includes rendering a first image of a constructed scene on the user interface screen based on a first determined user'"'"'s view-point of the user interface screen of the portable computing device. The method then includes rendering a different image of the constructed scene on the user interface screen based on a subsequently determined user'"'"'s view-point of the user interface screen and thereby presenting the illusion of a three-dimensional image of the constructed scene on the user interface screen.

Claims

1. A method comprising:

locating an initial position of a face based on visual detection, by a mobile device, of a user'"'"'s pair of eyes;

determining, based on the initial position, a first viewpoint relative to a screen of the mobile device;

rendering a first image of a constructed scene on the screen based on the first viewpoint;

after determining the initial position based on the visual detection, monitoring subsequent movement through input received from an accelerometer of the mobile device, to determine a distance of the face from the initial position;

determining, based on the distance of the face from the initial position and without further visual detection, a subsequent position of the face;

determining, based on the subsequent position, a second viewpoint relative to the screen; and

rendering a different image of the constructed scene on the screen based on the second viewpoint.

2. The method as recited in claim 1, wherein a camera located on the mobile device is used to perform the visual detection.

3. The method as recited in claim 2, wherein the camera is positioned on the mobile computing device at a fixed location relative to the screen.

4. The method as recited in claim 3, wherein the camera has infrared sensitivity for sensing infrared images.

5. The method as recited in claim 3, further comprising emitting infrared light from an infrared light emitter located proximate the camera.

6. The method as recited in claim 1, further comprising emitting infrared light from the mobile device away from the screen.

7. The method as recited in claim 1, wherein the mobile device is wireless enabled and the constructed scene is wirelessly received by the mobile device from a remote source.

8. The method as recited in claim 1, wherein locating the user'"'"'s pair of eyes comprises attempting to locate the user'"'"'s pair of eyes based on absorption patterns within light reflected off of a user'"'"'s face back to a camera of the mobile device following emission of the light from the mobile device toward the user'"'"'s face, and based on a lack of reflection of the light from pupils.

9. The method as recited in claim 1, further comprising, in response to failing to calculate a position of the user'"'"'s head based on input from the accelerometer, the mobile device entering a mode in which the mobile device attempts to re-locate a user'"'"'s head based on further visual detection.

10. A system comprising:

a module to locate an initial position of a face based on visual detection of a user'"'"'s pair of eyes;

a module to determine, based on the initial position, a first viewpoint relative to a screen of a mobile device;

a module to render a first image of a constructed scene on the screen based on the first viewpoint;

a module to monitor, after detecting the initial position based on the visual detection, subsequent movement through input received from an accelerometer of the mobile device, to determine a distance of the face from the initial position;

a module to determine, based on the distance of the face from the initial position and without further visual detection, a subsequent position of the face;

a mobile to determine, based on the subsequent position, a second viewpoint relative to the screen; and

a module to render a different image of the constructed scene on the screen based on second viewpoint.

11. The system as recited in claim 10, wherein a camera located on the mobile device is used to perform the visual detection.

12. The system as recited in claim 11, further comprising an infrared light emitter located proximate the camera on the portable computing device.

13. The system as recited in claim 10, wherein the camera is positioned on the mobile computing device at a fixed location relative to the screen.

14. The system as recited in claim 13, wherein the camera has infrared sensitivity for sensing infrared images.

15. The system as recited in claim 10, further comprising a module to emit infrared light from the mobile device away from the user interface screen.

16. The system as recited in claim 10, wherein the mobile device is wireless enabled and the constructed scene is wirelessly received by the mobile device from a remote source.

17. The system as recited in claim 10, wherein the module to locate the user'"'"'s pair of eyes comprises a module to attempt to locate the user'"'"'s pair of eyes based on absorption patterns within light reflected off of a user'"'"'s face back to a camera of the mobile device following emission of the light from the mobile device toward the user'"'"'s face, and based on a lack of reflection of the light from pupils.

18. The system as recited in claim 10, further comprising a module to cause the mobile device to enter a mode in which the mobile device attempts to re-locate, based on further visual detection, a user'"'"'s head in response to the mobile device failing to calculate the position of the user'"'"'s head.