METHOD AND A DEVICE FOR TRAINING A POSE CLASSIFIER AND AN OBJECT CLASSIFIER, A METHOD AND A DEVICE FOR OBJECT DETECTION

• US 20130251246A1
• Filed: 01/16/2013
• Published: 09/26/2013
• Est. Priority Date: 03/21/2012
• Status: Abandoned Application

• Current Assignee: NEC China Co Ltd (NEC Corporation)
• Original Assignee: NEC China Co Ltd (NEC Corporation)
• Inventors: Tang, Shaopeng, Wang, Feng, Liu, Guoyi, Zhang, Hongming, Zeng, Wei
• Application Number:
  US13/743,010
• Publication Number:
  US 20130251246A1
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects

Abstract

A method and a device for training a pose classifier and an object classifier, and a method and a device for objection detection, relating to the field of image processing are provided. The object detection method includes acquiring input image samples; performing pose estimation processing on said input image sample according to said pose classifier; and performing object detection on the processed input image sample according to said pose classifier to acquire the location information of the object, wherein said object is an object with joints. Objects in different poses can be detected and therefore the object hit rate is increased.

Claims

1. A method for training a pose classifier, comprising:

acquiring a first training image sample set;

acquiring actual pose information of a specified number of training image samples in said first training image sample set; and

executing a regression training process according to said specified number of training image samples and the actual pose information thereof to generate a pose classifier.

2. The method according to claim 1, wherein said executing a regression training process according to said specified number of training image samples and the actual pose information thereof to generate a pose classifier comprises:

constructing a loss function, wherein an input of said loss function is said specified number of training image samples and the actual pose information thereof, an output of said loss function is a difference between the actual pose information and estimated pose information of said specified number of training image samples;

constructing a mapping function, wherein an input of said mapping function is said specified number of training image samples, an output of said mapping function is the estimated pose information of said specified number of training image samples; and

executing regression according to said specified number of training image samples and the actual pose information thereof, selecting a mapping function which minimizes an output value of said loss function as the pose classifier.

3. The method according to claim 2, wherein said loss function is a location difference between the actual pose information and the estimated pose information.

4. The method according to claim 2, wherein said loss function is a location difference and direction difference between the actual pose information and the estimated pose information.

5. A method for training an object classifier using the pose classifier generated by the method according to claim 1, wherein said object is an object with joints, said method comprising:

acquiring a second training image sample set;

performing pose estimation processing on a specified number of training image samples in said second training image sample set according to said pose classifier; and

executing training on the training image samples processed with said pose estimation to generate an object classifier.

6. The method according to claim 5, wherein said performing pose estimation processing on a specified number of training image samples in said second training image sample set according to said pose classifier comprises:

performing pose estimation on a specified number of training image samples in said second training image sample set according to said pose classifier to obtain the estimated pose information of said specified number of training image samples; and

constructing a plurality of training object bounding boxes for each object with joints according to the estimated pose information of said specified number of training image samples, performing normalization on said plurality of training object bounding boxes such that the training object bounding boxes of a same part of different objects are consistent in size and direction;

said executing training on the training image samples processed with said pose estimation further comprises;

executing training on said normalized training image samples.

7. The method according to claim 6, wherein after said obtaining the estimated pose information of said specified number of training image samples, the method further comprises:

displaying the estimated pose information of said specified number of training image samples.

8. The method according to claim 6, wherein after said performing normalization on said plurality of training object bounding boxes, the method further comprises:

displaying said plurality of normalized training object bounding boxes.

9. The method according to claim 5, wherein said estimated pose information includes location information of the structural feature points of the training object, said structural feature points of the training object comprising:

a head central point, a waist central point, a left foot central point, and a right foot central point;

said constructing a plurality of object bounding boxes for each object with joints according to the estimated pose information of said specified number of training image samples, performing normalization on said plurality of object bounding boxes comprises;

constructing three object bounding boxes for each object with joints by respectively taking a straight line between the head central point and the waist central point as a central axis, the straight line between the waist central point and the left foot central point as a central axis, and the straight line between the waist central point and the right foot central point as a central axis, rotating and resizing said three object bounding boxes;

wherein said structural feature points of the object are located in the corresponding object bounding boxes.

10. The method according to claim 5, wherein said estimated pose information includes location information of the structural feature points of the training object, said structural feature points of the training object comprising:

a head central point, a waist central point, a left knee central point, a right knee central point, a left foot central point, and a right foot central point;

said constructing a plurality of object bounding boxes for each object with joints according to the estimated pose information of said specified number of training image samples, performing normalization on said plurality of training object bounding boxes comprises;

constructing five object bounding boxes for each object with joints by respectively taking a straight line between the head central point and the waist central point as a central axis, the straight line between the waist central point and the left knee central point as a central axis, the straight line between the waist central point and the right knee central point as a central axis, the straight line between the waist central point and the left foot central point as a central axis, and the straight line between the waist central point and the right foot central point as a central axis, rotating and resizing said five object bounding boxes;

wherein said structural feature points of object are located in the corresponding object bounding boxes.

11. A method for object detection using the pose classifier generated by the method according to claim 1 and an object classifier wherein an object is an object with joints, comprising:

acquiring input image samples;

performing pose estimation processing on said input image samples according to said pose classifier; and

performing object detection on the processed input image samples according to said object classifier to acquire the location information of the object.

12. The method according to claim 11, wherein said performing pose estimation processing on said input image samples according to said pose classifier comprises:

performing pose estimation on said input image samples according to said pose classifier to obtain the estimated pose information of said input image samples; and

constructing a plurality of object bounding boxes for each object with joints according to the estimated pose information of said input image samples, performing normalization on said plurality of object bounding boxes such that the object bounding boxes of the same part of different objects are consistent in size and direction;

correspondingly, said performing object detection on the processed input image samples according to said object classifier comprises;

performing object detection on said normalized input image samples according to said object classifier.

13. The method according to claim 12, wherein after said obtaining the estimated pose information of said input image samples, further comprising:

displaying the estimated pose information of said input image samples.

14. The method according to claim 12, wherein after said performing normalization on the plurality of object bounding boxes, further comprising:

displaying said plurality of normalized object bounding boxes.

15. The method according to claim 12, wherein said estimated pose information includes location information of the structural feature points of an object, said structural feature points of the object comprise:

a head central point, a waist central point, a left foot central point, and a right foot central point;

said constructing a plurality of object bounding boxes for each object with joints according to the estimated pose information of said input image samples, performing normalization on said plurality of object bounding boxes comprising;

constructing three object bounding boxes for each object with joints by respectively taking a straight line between the head central point and the waist central point as a central axis, the straight line between the waist central point and the left foot central point as a central axis, and the straight line between the waist central point and the right foot central point as a central axis, rotating and resizing said three object bounding boxes;

wherein said structural feature points of object are located in the corresponding object bounding boxes.

16. The method according to claim 12, wherein said estimated pose information specifically includes location information of the structural feature points of an object, said structural feature points of the object comprise:

a head central point, a waist central point, a left knee central point, a right knee central point, a left foot central point, and a right foot central point;

said constructing a plurality of object bounding boxes for each object with joints according to the estimated pose information of said input image samples, performing normalization on said plurality of object bounding boxes comprising;

constructing five object bounding boxes for each object with joints by respectively taking a straight line between the head central point and the waist central point as a central axis, the straight line between the waist central point and the left knee central point as a central axis, the straight line between the waist central point and the right knee central point as a central axis, the straight line between the waist central point and the left foot central point as a central axis, and the straight line between the waist central point and the right foot central point as a central axis, rotating and resizing said five object bounding boxes;

wherein said structural feature points of said object are located in the corresponding object bounding boxes.

17. A device for training a pose classifier and stored in computer readable storage media, comprising:

a first acquisition module for acquiring a first training image sample set;

a second acquisition module for acquiring the actual pose information of a specified number of training image samples in said first training image sample set; and

a first training generation module for executing a regression training process according to said specified number of training image samples and the actual pose information thereof to generate a pose classifier.

18. The device according to claim 17, wherein said first training generation module comprises:

a first construction unit for constructing a loss function, wherein an input of said loss function is said specified number of training image samples and the actual pose information thereof, an output of said loss function is a difference between the actual pose information and the estimated pose information of said specified number of training image samples;

a second construction unit for constructing a mapping function, wherein an input of said mapping function is said specified number of training image samples, an output of said mapping function is the estimated pose information of said specified number of training image samples; and

a pose classifier acquisition unit for executing regression according to said specified number of training image samples and the actual pose information thereof, and for selecting the mapping function which minimizes an output value of said loss function as the pose classifier.

19. The device according to claim 18, wherein said loss function includes at least one of a location difference between the actual pose information and the estimated pose information or a location difference and direction difference between the actual pose information and the estimated pose information.

20. A device for training an object classifier using the pose classifier generated by the device according to claim 17, wherein said object is an object with joints, said device comprising:

a third acquisition module for acquiring a second training image sample set;

a first pose estimation module for performing pose estimation processing on a specified number of training image samples in said second training image sample set according to said pose classifier; and

a second training generation module for executing training on the training image samples processed with said pose estimation to generate an object classifier.

21. The device according to claim 20, wherein said first pose estimation module comprises:

a first pose estimation unit for performing pose estimation on a specified number of training image samples in said second training image sample set according to said pose classifier to obtain the estimated pose information of said specified number of training image samples; and

a first construction processing unit for constructing a plurality of training object bounding boxes for each object with joints according to the estimated pose information of said specified number of training image samples, performing normalization on said plurality of training object bounding boxes such that the training object bounding boxes of the same part of different objects are consistent in size and direction;

said second training generation module further comprising;

a training unit for executing training on said normalized training image samples.

22. The device according to claim 21, further comprising:

a first graphic user interface for displaying the estimated pose information of said specified number of training image samples after said obtaining the estimated pose information of said specified number of training image samples.

23. The device according to claim 21, further comprising:

a second graphic user interface for displaying said plurality of normalized training object bounding boxes after said performing normalization on said plurality of training object bounding boxes.

24. The device according to claim 21, wherein said estimated pose information specifically includes location information of the structural feature points of a training object, said structural feature points of the training object comprise:

a head central point, a waist central point, a left foot central point, and a right foot central point;

said first construction processing unit comprises;

a first construction sub-unit for constructing three object bounding boxes for each object with joints by respectively taking a straight line between the head central point and the waist central point as a central axis, the straight line between the waist central point and the left foot central point as a central axis, and the straight line between the waist central point and the right foot central point as a central axis, rotating and resizing said three object bounding boxes;

wherein said structural feature points of object are located in the corresponding object bounding boxes.

25. A device for object detection using the pose classifier generated by the device according to claim 17 and an object classifier wherein said object is an object with joints, said device comprising:

a fourth acquisition module for acquiring input image samples;

a second pose estimation module for performing pose estimation processing on said input image samples according to said pose classifier; and

a detection module for performing objects detection on processed input image samples according to said object classifier to acquire the location information of the object.

wherein said second pose estimation module comprises;

a second pose estimation unit for performing pose estimation on said input image samples according to said pose classifier to obtain the estimated pose information of said input image samples;

a second construction processing unit for constructing a plurality of object bounding boxes for each object with joints according to the estimated pose information of said input image samples, performing normalization on said plurality of object bounding boxes such that the training object bounding boxes of the same part of different objects are consistent in size and direction;

said detection module comprises;

a detection unit for performing object detection on said normalized input image samples according to said object classifier;

a third graphic user interface for displaying the estimated pose information of said input image samples after said obtaining the estimated pose information of said input image samples;

a fourth graphic user interface for displaying said plurality of normalized object bounding boxes after said performing normalization on the plurality of object bounding boxes;

said estimated pose information includes location information of the structural feature points of object, said structural feature points of object comprise;

a head central point, a waist central point, a left foot central point, and a right foot central point;

said second construction processing unit comprises;

a third construction sub-unit for constructing three object bounding boxes for each object with joints by respectively taking a straight line between the head central point and the waist central point as a central axis, the straight line between the waist central point and the left foot central point as a central axis, and the straight line between the waist central point and the right foot central point as a central axis, rotating and resizing said three object bounding boxes;

wherein said structural feature points of object are located in the corresponding object bounding boxes.