Object classification in video images

• US 7,646,922 B2
• Filed: 12/30/2005
• Issued: 01/12/2010
• Est. Priority Date: 12/30/2005
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Curtner, Keith L., Bedros, Saad J., Au, Kwong Wing
• Primary Examiner(s): Alavi; Amir
• Application Number:
  US11/323,826
• Publication Number:
  US 20070154100A1
• Time in Patent Office: 1,474 Days
• Field of Search: 382/103, 382/107, 382/156, 382/159, 382/162, 382/164, 382/165, 382/167, 382/173, 382/181, 382/190, 382/195, 382/199, 382/224, 382/225, 382/226, 382/236, 345/589, 345600-604, 358/515, 358/518, 358/520, 358/523, 358/525, 358/530, 348/699, 348/E5.066, 348/E7.106, 375/240.02, 375/240.12, 375/240.13, 375/240.16
• US Class Current: 382/226
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A combination of inter-frame and intra-frame and sequential classification methods provide for robust and flexible object classification. Intra-frame classification determines the object type based on features extracted from a single image. Inter-frame classification determines object types based on the features extracted from several sequential frames. A sequential classification combines the results of the inter- and intra-frame classifications over a sequence of images.

Claims

1. A method of classifying objects from image frames, the method comprising:

using a computer to perform the elements of;

applying an intra-frame feature based object classifier to objects within each image frame to identify object classes for objects in the frame;

applying an inter-frame object classifier to persistent objects in sequence of frames to identify object classes for objects in multiple frames; and

applying a sequential classifier to multiple sequential frames to classify an object as a function of the classifications by the feature based object classifiers.

2. The method of claim 1 wherein the sequential classifier classifies an object if it is classified as the same in m of n successive frames.

3. The method of claim 2 wherein m is 2, and n is 3.

4. The method of claim 1 wherein the intra-frame classifier comprises a feature based classifier.

5. The method of claim 4 wherein the inter-frame classifier uses recurrent motion image features.

6. The method of claim 4 wherein the recurrent motion image features are selected from the group consisting of perspective and aspect.

7. The method of claim 1 and further comprising:

identifying objects in an image frame; and

tracking the identified objects in multiple sequential frames.

8. The method of claim 1 wherein objects are classified as person, vehicle or other.

9. The method of claim 8 and further comprising determining a dominant color of an object classified as a vehicle.

10. The method of claim 9 wherein determining a dominant color comprises:

converting a color space of segmented object pixels to a hue, saturation, and luminous (HSL) format;

assigning each color-transformed pixel a color on a palette; and

determining a dominant color.

11. The method of claim 8 and further comprising sub-classifying vehicles into cars, vans, sport utility vehicles and trucks.

12. The method of claim 1 wherein the feature based object classifier comprises multiple radial basis function neural net classifiers, each defining a boundary of known objects that is closed.

13. A method of classifying objects from image frames, the method comprising:

using a computer to perform the elements of;

applying an intra-frame feature based object classifier to objects in multiple sequential frames to identify object classes for objects in each frame;

an inter-frame feature based classifier that classifies objects as a function of features of the objects in multiple frames; and

applying a sequential classifier to multiple sequential frames to classify an object as a function of the classifications by the feature based object classifiers.

14. The method of claim 13 wherein the sequential classifier classifies an object if it is classified as the same in m of n successive frames.

15. The method of claim 14 wherein m is 2 and n is 3.

16. The method of claim 15 wherein the inter-frame classifier uses recurrent motion image features.