Real-time head tracking system for computer games and other applications

• US 7,121,946 B2
• Filed: 06/29/2001
• Issued: 10/17/2006
• Est. Priority Date: 08/10/1998
• Status: Expired due to Term

• Current Assignee: Jolly Seven, Series 70 of Allied Security Trust I (Allied Security Trust)
• Original Assignee: Cybernet Systems Corporation
• Inventors: Paul, George V., Beach, Glenn J., Cohen, Charles J., Jacobus, Charles J.
• Primary Examiner(s): JONES, SCOTT E
• Application Number:
  US09/896,150
• Publication Number:
  US 20020037770A1
• Time in Patent Office: 1,936 Days
• Field of Search: 463/1, 463 30- 34, 463/36, 463/42, 463 48- 49, 701/300, 701/302, 706 17- 18, 706/21, 706/24, 706/62, 345418-419, 345429-431, 345/112, 345/121, 345/141, 345326-327, 345/501, 345/199, 345/22, 345949-954, 345/959, 382/103, 382/236, 382/226, 382/276, 348/42, 348/51, 348/169, 600587-595
• US Class Current: 463/36
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A real-time computer vision system tracks the head of a computer user to implement real-time control of games or other applications. The imaging hardware includes a color camera, frame grabber, and processor. The software consists of the low-level image grabbing software and a tracking algorithm. The system tracks objects based on the color, motion and/or shape of the object in the image. A color matching function is used to compute three measures of the target'"'"'s probable location based on the target color, shape and motion. The method then computes the most probable location of the target using a weighting technique. Once the system is running, a graphical user interface displays the live image from the color camera on the computer screen. The operator can then use the mouse to select a target for tracking. The system will then keep track of the moving target in the scene in real-time.

Claims

1. A method controlling a computer game, comprising the steps of:

imaging a sequence of scenes including the head of a user of the computer;

comparing visual characteristics from a portion of a scene to a center of said portion of a scene to determine movement of the user'"'"'s head within the scene wherein at least one of the visual characteristics is color;

providing a weighted average of color to compute the location of the user'"'"'s head based upon color alone; and

controlling the game in accordance with the movements.

2. The method of claim 1, wherein the visual characteristics include shape or location.

3. The method of claim 2, wherein the visual characteristics include a combination of static and dynamic characteristics.

4. The method of claim 3, further including the step of modeling of the dynamic characteristics to yield an estimate of head position.

5. The method of claim 1, further including the step of initiating head tracking through a graphical user interface.

6. The method of claim 5, wherein the graphical user interface provides a bounding box displayed in a screen to assist in targeting the user'"'"'s head.

7. The method of claim 2, further comprising the step of enabling a match in color despite the differences arising from lighting and shadows.

8. The method of claim 2, further comprising the step of enabling a match in color within a threshold of hue.

9. The method of claim 1, wherein the step of comparing the visual characteristics includes a comparison of pixels from scene to scene.

10. The method of claim 1, further including the step of determining if the user'"'"'s head was moved outside of the scene.

11. The method of claim 1, further including the step of segmenting a region defined by a predetermined closeness of color as an estimate of target shape.

12. The method of claim 1, further including the step of continuing to track the user'"'"'s head when moving in front of or behind a similarly colored object in the scene.