Method and apparatus for producing classifier training images

• US 20050196035A1
• Filed: 03/03/2004
• Published: 09/08/2005
• Est. Priority Date: 03/03/2004
• Status: Active Grant

• Current Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Original Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Inventors: Luo, Yun, Wallace, Jon K.
• Granted Patent: US 7,609,893 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A system (400) for generating training images for a pattern recognition classifier (54) from three-dimensional image data representing an output class comprises a model constructor (404) that constructs an image model in a standard coordinate domain from the three-dimensional image data. A model transformer (406) manipulates the image model in the standard coordinate domain to produce at least one manipulated image model. An image restorer (408) maps the at least one manipulated image model from the standard coordinate domain to a coordinate domain associated with an image source utilized during the run-time operation of the pattern recognition classifier.

Claims

1. A system for generating training images for a pattern recognition classifier from three-dimensional image data representing an output class, said system comprising:

a model constructor that constructs an image model in a standard coordinate domain from the three-dimensional image data;

a model transformer that manipulates the image model in the standard coordinate domain to produce at least one manipulated image model; and

an image restorer that maps the at least one manipulated image model from the standard coordinate domain to a coordinate domain associated with an image source utilized during the run-time operation of the pattern recognition classifier.

2. The system of claim 1 wherein three-dimensional image data is received from each of a plurality of cameras, each having an associated coordinate domain, and the model constructor maps at least a portion of the three-dimensional image data from each camera from a coordinate domain related to the camera to a standard coordinate domain to form an image model.

3. The system of claim 2 wherein the model constructor separates the three-dimensional image data from each camera into a portion representing an imaged subject and a background portion, the portion representing the imaged subject being mapped to the standard coordinate domain.

4. The system of claim 1 wherein the model transformer manipulates the image model by multiplying each of a plurality of vectors representing pixels within the image model by at least one transformation matrix.

5. The system of claim 1 wherein the model manipulator translates the image model along at least one coordinate axis associated with the standard domain to produce a manipulated image model.

6. The system of claim 1 wherein the model manipulator rotates the image model around at least one coordinate axis associated with the standard domain to produce a manipulated image model.

7. The system of claim 1 wherein the model manipulator scales the image model to produce a manipulated image model.

8. The system of claim 1 wherein the image restorer combines data associated with a given manipulated image model with desired background data to produce a training image.

9. The system of claim 8 wherein the desired background data represents one of a plurality of available image backgrounds, a given image background representing a desired environment for the system.

10. The system of claim 8 wherein the image restorer combines the data associated with a given manipulated image model and the desired background data with data associated with a second manipulated image model to produce a training image.

11. The system of claim 8 further comprising a feature extractor that extracts training data from the training image.

12. The system of claim 1 further comprising an image source that provides the three-dimensional image data as a three-dimensional surface represented in the standard coordinate domain.

13. The system of claim 12 wherein the image source includes a laser holography arrangement.

14. The system of claim 1 wherein the pattern recognition classifier includes at least one of a neural network and a support vector machine.

15. A system for generating training images for a pattern recognition classifier associated with a vehicle occupant safety system comprising:

a plurality of vision systems that image a subject to provide three-dimensional image data representing an output class;

a model constructor that maps at least a portion of the three-dimensional image data from each vision system from a coordinate domain related to the vision system to a vehicle coordinate domain associated with the vehicle interior to form an image model;

a model transformer that manipulates the image model in the vehicle coordinate domain to produce at least one manipulated image model; and

an image restorer that maps the at least one manipulated image model from the vehicle coordinate domain into a desired coordinate domain.

16. The system of claim 15 wherein the three-dimensional image data representing the output class includes a three-dimensional representation of a human adult.

17. The system of claim 15 wherein the three-dimensional image data representing the output class includes a three-dimensional representation of one of a rearward facing infant seat, and frontward facing child seat, and an empty seat.

18. The system of claim 15 wherein the three-dimensional image data representing the output class includes a three-dimensional representation of one of a briefcase, a handbag, and a shopping bag.

19. The system of claim 15 wherein the model manipulator translates the image model along at least one coordinate axis in the vehicle domain to produce a manipulated image model simulating a desired position of a vehicle seat.

20. The system of claim 15 wherein the model manipulator rotates the image model around at least one coordinate axis in the vehicle domain to produce a manipulated image model simulating a likely movement of a vehicle occupant.

21. The system of claim 15 wherein the model manipulator scales the image model to produce a manipulated image model simulating an occupant of a desired size.

31. The method of claim 21, further comprising merging data associated with the transformed image model with a desired background image.

22. An occupant classifier training method for use in vehicle restraining systems comprising the steps of:

producing an image model in a vehicle coordinate domain;

transforming the image model to simulate at least one desired property associated with a vehicle occupant;

mapping the transformed image model to a camera coordinate domain associated with the run-time operation of the classifier to create a training image; and

training a pattern recognition classifier using the training image.

23. The method of claim 22 wherein the step of producing the image model includes:

imaging a subject with a plurality of stereo cameras arranged at a plurality of different orientations to acquire base image data from each camera;

mapping at least a portion of the acquired base image data at each stereo camera from a coordinate domain associated with the camera to a vehicle coordinate domain; and

fusing the mapped image data from the plurality of stereo cameras to produce an image model representing the subject.

24. The method of claim 22 wherein the step of training the classifier includes classifying the training image and providing training data associated with the training image to the classifier if the classifier produces an incorrect classification.

25. The method of claim 22 wherein the steps of transforming the image model, mapping the transformed image model to the camera coordinate domain, and training the classifier are repeated iteratively until a termination event occurs.

26. The method of claim 25, wherein the termination event is the achievement of a desired level of classifier accuracy.

27. The method of claim 22 further comprising extracting feature data from the training image.

28. The method of claim 22, wherein transforming the image model includes applying a transformation matrix, comprising a plurality of coefficients, to the image model.

29. The method of claim 28, wherein the plurality of coefficients is selected randomly according to associated probability density functions.

30. The method of claim 28, wherein the plurality of coefficients are selected by an operator.