Tracking and gesture recognition system particularly suited to vehicular control applications

• US 7,050,606 B2
• Filed: 11/01/2001
• Issued: 05/23/2006
• Est. Priority Date: 08/10/1999
• Status: Expired due to Term

• Current Assignee: Jolly Seven, Series 70 of Allied Security Trust I (Allied Security Trust)
• Original Assignee: Cybernet Systems Corporation
• Inventors: Paul, George V., Beach, Glenn J., Cohen, Charles J., Jacobus, Charles J.
• Primary Examiner(s): Choobin, Barry
• Application Number:
  US10/004,058
• Publication Number:
  US 20020126876A1
• Time in Patent Office: 1,664 Days
• Field of Search: 382/103, 382/104, 382/108, 382/106, 382/107, 180/271, 701/45, 280/734, 280/735, 340/903, 348/169, 345/865, 345/158, 345/173, 356/3
• US Class Current: 382/104
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A system and method tracks the movements of a driver or passenger in a vehicle (ground, water, air, or other) and controls devices in accordance with position, motion, and/or body or hand gestures or movements. According to one embodiment, an operator or passenger uses the invention to control comfort or entertainment features such the heater, air conditioner, lights, mirror positions or the radio/CD player using hand gestures. An alternative embodiment facilitates the automatic adjustment of car seating restraints based on head position. Yet another embodiment is used to determine when to fire an airbag (and at what velocity or orientation) based on the position of a person in a vehicle seat. The invention may also be used to control systems outside of the vehicle. The on-board sensor system would be used to track the driver or passenger, but when the algorithms produce a command for a desired response, that response (or just position and gesture information) could be transmitted via various methods (wireless, light, whatever) to other systems outside the vehicle to control devices located outside the vehicle. For example, this would allow a person to use gestures inside the car to interact with a kiosk located outside of the car.

Claims

1. A method of controlling one or more vehicular-related devices, comprising the steps of:

tracking the position, velocity or acceleration of the head, body or other body parts of a vehicle driver or passenger;

determining the position or motion of the driver or passenger using gesture or behavior recognition; and

controlling a device associated with the operation, safety or comfort of the vehicle in accordance with the recognized position or motion.

2. The method of claim 1, wherein the recognized motion includes a hand or body gesture.

3. The method of claim 1, wherein the device is associated with entertainment.

4. The method of claim 1, wherein the device is seat restraints based on head position.

5. The method of claim 1, the device is an airbag.

6. The method of claim 5, further including the control of the velocity or orientation of the airbag based upon body position, velocity or acceleration.

7. The method of claim 1, further including the control of a device external to the vehicle.

8. A system for controlling one or more vehicular-related devices, comprising:

a device for sensing the position or motion of the head, body, or other body parts of a driver or passenger of a vehicle;

a tracking system for tracking the head, body, or other body parts;

a gesture/behavior recognition system for recognizing and identifying the person'"'"'s motions; and

a controller for controlling devices associated with the vehicle, whether under active or passive control by the vehicle occupant.

9. The system of claim 8, wherein the device for sensing is a video camera.

10. The system of claim 8, wherein the device is associated with entertainment.

11. The system of claim 8, including the adjustment of car seating restraints based on head position.

12. The system of claim 8, including the deployment of an airbag based on the position, velocity or acceleration of a person in a vehicle seat.

13. The system of claim 12, further including the control of the velocity or orientation of the airbag based upon body position, velocity or acceleration.

14. The system of claim 8, further including the control of a device external to the vehicle.