Eye tracking enabling 3D viewing

• US 10,114,455 B2
• Filed: 07/08/2015
• Issued: 10/30/2018
• Est. Priority Date: 08/31/2010
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Cheng, Howard, Newman, Jr., William C.
• Primary Examiner(s): Nawaz, Talha M
• Application Number:
  US14/794,335
• Publication Number:
  US 20150309571A1
• Time in Patent Office: 1,210 Days
• Field of Search: None
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/366 using viewer tracking

Abstract

The exemplary illustrative non-limiting technology herein enables 3D viewing on conventional 2D displays such as home television sets by tracking a person'"'"'s viewpoint. Detecting a player'"'"'s viewpoint movement to change the viewing of the displayed object gives the illusion that the object is physically present in three-dimensional space. Viewpoint movement detection can provide collision-related game logic benefits such as allowing a player to dodge projectiles, giving a game character an ability to “see” the player when not behind line-of-sight obstacles, and other advantages.

Claims

1. A system for displaying images on a display device, the system comprising:

a tracking device that tracks at least a portion of a user'"'"'s head in three dimensional (3D) space relative to the display device; and

at least one processor in communication with the tracking device, the at least one processor configured to;

determine a viewpoint of the user relative to the display device in response to said head tracking;

use said determined viewpoint to define a parallax for 3D viewing, from said determined viewpoint, at least one 3D virtual object on the same display device resulting in the virtual object appearing as if it occupies real world space;

change the parallax for 3D viewing the 3D virtual object on the display device in response to changes in the determined viewpoint; and

render for display on the same display device, images with said parallax.

2. The system of claim 1 wherein the display device comprises a 2D display.

3. The system of claim 1 wherein the display device comprises a stationary display.

4. The system of claim 1 wherein the at least one processor is further configured to display said virtual object as part of an animated 3D video game.

5. The system of claim 1 wherein said tracking device senses at least one infrared marker disposed proximate the user'"'"'s head.

6. The system of claim 1 wherein said tracking device is configured to track light-emitting markers worn on the user'"'"'s head.

7. The system of claim 1 wherein the at least one processor is configured to perform 3D collision detection for portions of said virtual object that appear to extend from the display device into the real world.

8. A system for providing three-dimensional (3D) display on a display device without requiring a user viewing the display device to wear 3D glasses or goggles, comprising:

an optical tracking device that tracks a portion of the user'"'"'s head relative to the display device; and

at least one processor configured to;

determine a user viewpoint relative to the display device at least in part in response to said tracking and to use said determined user viewpoint to at least in part define the projection of at least one virtual object onto an image plane,

introducing parallax for viewing the at least one virtual object on the same display device;

adjusting 3D perspective transformations applied to the 3D virtual object as the determined user viewpoint changes to generate new viewing perspectives; and

generate images that cause said projected at least one virtual object to be rendered for parallax display on said same display device resulting in the virtual object appearing from the user viewpoint as if it occupies real world space.

9. The system of claim 8 wherein the display device is stationary.

10. The system of claim 8 wherein the display device comprises a 2D display.

11. The system of claim 8 wherein said tracking device tracks aspects of the position or orientation or both of the user'"'"'s head relative to the display device through use of at least one marker on the user'"'"'s head.

12. The system of claim 8 wherein said processor animates motion of said virtual object.

13. The system of claim 8 wherein said processor is adapted to cause display of said virtual object as part of an animated 3D video game.

14. The system of claim 8 wherein said tracking device tracks at least one point on the user'"'"'s head.

15. The system of claim 8 wherein said tracking device tracks at least three points on the user'"'"'s head.

16. The system of claim 8 wherein said tracking device comprises an infrared tracking device.

17. The system of claim 8 wherein said processor is structured to perform 3D collision detection for portions of said virtual object that appear to extend from the display out into the real world.

18. The system of claim 8 wherein the processor is configured to increase parallax to increase perceived depth from the viewpoint.

19. The system of claim 18 wherein the processor places display objects relative to said parallax.