Method and apparatus for adjusting a view of a scene being displayed according to tracked head motion

• US 8,337,306 B2
• Filed: 12/23/2010
• Issued: 12/25/2012
• Est. Priority Date: 09/15/2003
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Larsen, Eric J., Deshpande, Hrishikesh R., Marks, Richard L.
• Primary Examiner(s): Suhol, Dmitry
• Assistant Examiner(s): JONES, MARCUS D
• Application Number:
  US12/977,727
• Publication Number:
  US 20110090149A1
• Time in Patent Office: 733 Days
• Field of Search: 463/36, 345/7, 345/8
• US Class Current: 463/36
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/5258 by dynamically adapting the position of the virtual camera to keep a game object or game character in its viewing frustum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6661 for changing the position of the virtual camera
• CPC Class Codes: A63F 2300/6676 by dedicated player input

Abstract

A method for controlling a view of a scene is provided. The method initiates with detecting an initial location of a control object. An initial view of the scene is displayed on a virtual window, the initial view defined by a view-frustum based on a projection of the initial location of the control object through outer edges of the virtual window. Movement of the control object to a new location is detected. An updated view of the scene is displayed on the virtual window, the updated view defined by an updated view-frustum based on a projection of the new location of the control object through the outer edges of the virtual window.

Claims

1. A method for processing interactive user control for a view of a scene displayed on a display device configured as a virtual window into the scene, comprising:

identifying a head of a user that is to interact with the scene;

displaying an initial view of the scene on the display device configured as the virtual window into the scene, the initial view of the scene comprising a view-frustum initially defined by a gaze projection of a location of the head through outer edges of the virtual window when the location of the head is substantially normal to about a center point of the virtual window;

tracking the identified head of the user during display of the scene, the tracking enabling detection of a change in location of the head of the user;

adjusting the view-frustum in accordance with the change in location of the head of the user, the adjusting of the view-frustum being in response to tracking a move in the location of the head away from normal relative to the center point of the virtual window, the adjusted view-frustum defined by an updated gaze projection of the changed location of the head through the outer edges of the virtual window, such that the view-frustum moves in a direction opposite to the move in the location of the head; and

displaying an updated view of the scene on the display device based on the adjusted view-frustum.

2. The method of claim 1, further comprising,

adjusting a scale of the scene according to a change in a distance of the head of the user from a depth capture device.

3. The method of claim 1, wherein the interaction with the scene by tracking movement of the head of the user is independent of user hand-held controls for interacting with the scene.

4. The method of claim 1, wherein the tracking includes,

identifying a search region within a frame of the user image data;

and

comparing values within the search region to template values of a stored initial frame of image data.

5. The method of claim 1, wherein the method operation of tracking the identified head of the user during display of the scene includes,

tracking a facial portion of the head;

and

matching image data associated with the facial portion to image data associated with a template of the facial portion.

6. The method of claim 1, wherein the adjusted view-frustum is such that a move in the location of the head towards a side of the virtual window provides for a wider angle of view through an opposite side of the virtual window.

7. A method for controlling a view of a scene, comprising:

detecting an initial location of a control object;

displaying an initial view of the scene on a display device configured as a virtual window into the scene, the initial view defined by a view-frustum based on a projection of the initial location of the control object through outer edges of the virtual window;

detecting movement of the control object to a new location;

displaying an updated view of the scene on the display device, the updated view defined by an updated view-frustum based on a projection of the new location of the control object through the outer edges of the virtual window.

8. The method of claim 7, wherein lateral movement of the control object in a given direction relative to the virtual window causes lateral movement of the updated view-frustum in the opposite direction.

9. The method of claim 7, wherein a change in distance of the control object from the virtual window causes a change in scale in the updated view.

10. The method of claim 7, wherein the updated view-frustum is such that a move in the location of the head towards a side of the virtual window provides for a wider angle of view through an opposite side of the virtual window.

11. The method of claim 7, wherein the control object is a head of a user.

12. The method of claim 11, wherein detecting the initial location includes detecting a facial region of the head of the user.

13. A method for displaying a view of a virtual environment on a display device, the method comprising:

identifying a control object in front of the display device;

detecting an initial location of the control object;

correlating the initial location of the control object to an initial virtual location of a virtual viewpoint in the virtual environment;

displaying an initial view of the virtual environment on the display device, the initial view of the virtual environment determined by a view frustum in the virtual environment defined by a projection of the virtual viewpoint through a virtual viewport in the virtual environment;

tracking a movement of the control object to a new location;

moving the virtual viewpoint to a new virtual location in accordance with the tracked movement of the control object, such that the movement of the virtual viewpoint relative to the virtual viewport is in a same relative direction as the movement of the control object relative to the display device;

displaying an updated view of the virtual environment on the display device, the updated view of the virtual environment determined by an updated view frustum in the virtual environment defined by an updated projection of the virtual viewpoint at the new virtual location through the virtual viewport.

14. The method of claim 13, wherein identifying the control object includes capturing an initial frame of image data including the control object, and identifying the control object within the initial frame of image data.

15. The method of claim 14, wherein detecting the initial location of the control object includes,

generating a template of the control object based on the captured initial frame of image data,

capturing successive frames of image data,

determining a search region within the successive frames of image data, and

searching within the search region for a match to the template of the control object.

16. The method of claim 15, wherein tracking the movement of the control object includes,

repeating the capturing successive frames of image data, determining a search region, and searching within the search region.

17. The method of claim 13, wherein the moving the virtual viewpoint in accordance with the tracked movement of the control object is such that the movement of the virtual viewpoint relative to the virtual viewport has a greater scale of movement in the virtual environment compared to the movement of the control object relative to the display device.

18. The method of claim 13, wherein the control object is a head of a user.