Hands-free interface to a virtual reality environment using head tracking

• US 6,009,210 A
• Filed: 03/05/1997
• Issued: 12/28/1999
• Est. Priority Date: 03/05/1997
• Status: Expired due to Term

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Digital Equipment Corporation (HP Inc.)
• Inventors: Kang, Sing Bing
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US08/811,674
• Time in Patent Office: 1,028 Days
• Field of Search: 345/419, 345/427, 345/8, 345/156-158, 345/7, 345/9, 348/36, 348/39, 600/595, 702/153, 702/152, 382/276, 382/154, 382/103, 382/104, 382/107
• US Class Current: 382/276
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6661 for changing the position of the virtual camera
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A hands-free navigation system for tracking a head and responsively adjusting the display of a virtual reality environment includes a camera, the camera employed to follow movements of the head, a computer system connected to the camera, the computer system including a memory, a central processing unit (CPU), a digitizer, the digitizer providing a current image of a face of the head, a face tracker, the face tracker including the capability of receiving a reference face image from the digitizer, of receiving the current face image from the digitizer, of determining a head translation and orientation, and of providing the head translation and orientation to a three dimensional virtual environment viewer connected to the computer system for display of the virtual environment at the desired viewpoint.

Claims

1. A hands-free navigation system for tracking a head and responsively adjusting the display of a virtual reality environment, the system comprising:

a three-dimensional environment viewer for display of the virtual reality environment;

a digitizer for providing two-dimensional images of a face of a head; and

a face tracker including;

means for receiving a reference face image from the digitizer;

means for receiving a current face image from the digitizer;

means for determining a head translation and orientation from the reference face image to the current face image; and

means for providing the head translation and orientation to the three-dimensional environment viewer, thereby displaying the virtual reality environment at a desired viewpoint.

2. The hands-free navigation system of claim 1 further comprising a camera for providing images to the digitizer, wherein the camera is mounted in front of and above the head and is positioned at a tilted angle to the head.

3. The hands-free navigation system of claim 1 wherein the means for receiving a reference face image comprises means for receiving a snapshot of the face in a neutral position.

4. The hands-free navigation system of claim 1 wherein the means for determining a head translation and orientation comprises:

means for warping the reference face image to minimize a difference between the warped reference face image and the current face image to provide a warping matrix transformation; and

means for decomposing the warping matrix transformation to yield the head translation and orientation.

5. The hands-free navigation system of claim 4 wherein the means for warping the reference face image comprises:

means for producing a two-dimensional motion matrix;

means for decomposing the two-dimensional motion matrix into a plurality of warping parameters, the warping parameters including;

a displacement variable;

a rotational angle about a viewing axis variable;

a zoom factor variable;

an aspect ratio variable;

a skew factor variable;

a pinch parameter variable in an x direction; and

a pinch parameter variable in a y direction;

means for approximating the two-dimensional motion matrix using an affine tracking to determine a limited head pose; and

means for tracking a center of an affine patch.

6. A method of hands-free navigation for tracking a head and responsively adjusting the display of a virtual reality environment, the method comprising the steps of:

providing a digitizer;

providing a face tracker;

receiving a reference face image in the face tracker from the digitizer;

receiving a current face image in the face tracker from the digitizer;

determining a head translation and orientation from the reference face image to the current face image;

providing the head translation and orientation to a three-dimensional environment viewer to display the virtual reality environment at a desired viewpoint; and

repeating the steps of receiving the current face image and providing the head translation and orientation to the three-dimensional environment viewer to continue to display the virtual reality environment at desired viewpoints.

7. The method of hands-free navigation according to claim 6 wherein the step of receiving the reference face image in the face tracker from the digitizer comprises the step of receiving an image of a face in a neutral position.

8. The method of hands-free navigation according to claim 6 wherein the step of determining the head translation and orientation comprises the steps of:

warping the reference face image to minimize a difference between the warped reference face image and the current face image to provide a warping matrix transformation; and

decomposing the warping matrix transformation to yield the head translation and orientation.

9. The method of hands-free navigation according to claim 8 wherein the step of warping the reference face image comprises the steps of:

producing a two-dimensional motion matrix;

decomposing the two-dimensional motion matrix into a plurality of warping parameters, the warping parameters including;

a displacement variable;

a rotational angle about a viewing axis variable;

a zoom factor variable;

an aspect ratio variable;

a skew factor variable;

a pinch parameter variable in an x direction; and

a pinch parameter variable in a y direction;

approximating the two-dimensional motion matrix using an affine tracking to determine a limited head pose; and

tracking a center of an affine patch.