Eye tracking enabling 3D viewing

• US 10,372,209 B2
• Filed: 09/27/2018
• Issued: 08/06/2019
• Est. Priority Date: 08/31/2010
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Cheng, Howard, Newman, Jr., William C.
• Primary Examiner(s): Nawaz, Talha M
• Application Number:
  US16/144,635
• Publication Number:
  US 20190025913A1
• Time in Patent Office: 313 Days
• Field of Search: None
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/366 using viewer tracking

Abstract

The exemplary illustrative non-limiting technology herein enables 3D viewing on conventional 2D displays such as home television sets by tracking a person'"'"'s viewpoint. Detecting a player'"'"'s viewpoint movement to change the viewing of the displayed object gives the illusion that the object is physically present in three-dimensional space. Viewpoint movement detection can provide collision-related game logic benefits such as allowing a player to dodge projectiles, giving a game character an ability to “see” the player when not behind line-of-sight obstacles, and other advantages.

Claims

1. A system for generating 3D images comprising:

at least one sensor configured to track a direction in which a user is looking;

a graphics engine operatively coupled to the at least one sensor, the graphics engine rendering 3D graphical images in real time response to the direction the at least one sensor senses;

the graphics engine modeling objects in real world coordinates and introducing parallax distortion to near field and far field objects to increase perception of parallax motion.

2. The system of claim 1 wherein said sensor tracks the user'"'"'s head orientation.

3. The system of claim 1 wherein the introduced parallax enhances the 3D viewing a displayed image plane.

4. The system of claim 1 wherein the graphics engine animates motion of said objects.

5. The system of claim 1 wherein said graphics engine images said object as part of an animated interactive display.

6. The system of claim 1 wherein said objects include a projectile the user is to dodge in the real world.

7. The system of claim 1 wherein said sensor tracks plural points on the user.

8. The system of claim 7 wherein said sensor tracks at least three points on the user.

9. The system of claim 1 wherein said sensor comprises an infrared sensor.

10. The system of claim 1 wherein the graphics engine performs 3D collision detection for portions of said objects that appear to extend beyond a viewing screen into the real world.

11. The system of claim 1 wherein said graphics engine changes the viewing frustrum in response to said sensor.

12. A system for providing 3D viewing comprising:

a tracking device that tracks a user'"'"'s eyes; and

at least one processor coupled to said tracking device, said processor structured to project of at least one virtual object onto an image plane in response to the tracking device, and to use parallax responsive to the tracking device to generate a sequence of images that cause the virtual object to appear to extend beyond the image plane into the real world.

13. The system of claim 12 wherein said tracking device tracks the user'"'"'s head through use of a marker.

14. The system of claim 12 wherein said processor introduces parallax to enhance parallax motion.

15. The system of claim 12 wherein said processor animates motion of said virtual object.

16. The system of claim 12 wherein said processor is adapted to cause display of said virtual object as part of an animated interactive application.

17. The system of claim 12 wherein said virtual object comprises a projectile that the user is to dodge in the real world.

18. The system of claim 12 wherein said tracking device comprises an infrared sensor.

19. The system of claim 12 wherein said processor is structured to perform 3D collision detection for portions of said virtual object that appear to extend into the real world.

20. The system of claim 12 wherein said processor is structured to change the viewing frustrum in response to said tracking device.