Visual tracking using depth data

• US 7,590,262 B2
• Filed: 04/21/2008
• Issued: 09/15/2009
• Est. Priority Date: 05/29/2003
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Nanda, Harsh, Fujimura, Kikuo
• Primary Examiner(s): Werner; Brian P
• Assistant Examiner(s): WANG, CLAIRE X
• Application Number:
  US12/106,788
• Publication Number:
  US 20080212836A1
• Time in Patent Office: 512 Days
• Field of Search: 382/103, 382/104, 382/154, 382/239, 382/106, 382/108, 382/181, 345/419, 345/420, 463/30
• US Class Current: 382/104
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

Real-time visual tracking using depth sensing camera technology, results in illumination-invariant tracking performance. Depth sensing (time-of-flight) cameras provide real-time depth and color images of the same scene. Depth windows regulate the tracked area by controlling shutter speed. A potential field is derived from the depth image data to provide edge information of the tracked target. A mathematically representable contour can model the tracked target. Based on the depth data, determining a best fit between the contour and the edge of the tracked target provides position information for tracking. Applications using depth sensor based visual tracking include head tracking, hand tracking, body-pose estimation, robotic command determination, and other human-computer interaction systems.

Claims

1. A human-computer interaction system for visually tracking human movement comprising:

an active depth sensor for capturing depth images of human body parts;

a processing unit coupled to the active depth sensor for receiving the depth images, the processing unit comprising;

an edge detection module configured to determine an edge of a target in the depth images;

a tracking module coupled to the edge detection module to determine a best fit of a mathematically representable contour with respect to one or more body parts to determine a position of the one or more body parts within the captured depth images at least by;

dividing the mathematically representable contour into a plurality of segments, each contour segment comprising an inner side corresponding to lower depth values and an outer side corresponding to higher depth values; and

matching an orientation of each contour segment against a corresponding edge segment of the determined edge in the depth image by comparing the depth values on each side of the determined edge segment and determining that the depth values on a side of the determined depth segment that overlaps with the inner side of the matched contour segment are lower than the depth values of a side of the determined edge segment overlapping with the outer side of the matched contour segment; and

a body-lean determination module coupled to the tracking module to receive the determined position of the one or more body parts and configured to calculate a distance of each of the one or more body parts with respect to the active sensor and to relate the distance of each body part with respect to other body parts to determine lean data including the position of the body parts with respect to each other.

2. The human-computer interaction system of claim 1, wherein the lean data is sent to a control logic configured to determine whether to deploy an air-bag in a vehicle.

3. The human-computer interaction system of claim 1, wherein the position of the one or more body parts is used to provide user control input to a software application.

4. The human-computer interaction system of claim 3, wherein the software application is a computer game.

5. A motor vehicle configured to track occupant body-lean information, the motor vehicle comprising:

an active depth sensor for capturing depth images of human body parts;

a processing unit coupled to the active depth sensor for receiving the depth images, the processing unit comprising;

an edge detection module configured to determine edges of a target in the depth images; and

a tracking module coupled to the edge detection module to determine a best fit of a mathematically representable contour with respect to one or more body parts to determine a position of the one or more body parts within the captured depth images at least by;

dividing the mathematically representable contour into a plurality of segments, each contour segment comprising an inner side corresponding to lower depth values and an outer side corresponding to higher depth values; and

matching an orientation of each contour segment against a corresponding edge segment of the determined edge in the depth image by comparing the depth values on each side of the determined edge segment and determining that the depth values on a side of the determined depth segment that overlaps with the inner side of the matched contour segment are lower than the depth values of a side of the determined edge segment overlapping with the outer side of the matched contour segment.

6. The motor vehicle of claim 5, further comprising a body-lean determination module coupled to the tracking module to receive the determined position of the one or more body parts and configured to calculate a distance of each of the one or more body parts with respect to the active sensor and to relate the distance of each body part with respect to other body parts to determine lean data including the position of the body parts with respect to each other.

7. The motor vehicle of claim 6, wherein the lean data is sent to a control logic configured to determine whether to deploy an air-bag in the motor vehicle.