Adaptive scene dependent filters in online learning environments

• US 8,238,650 B2
• Filed: 12/13/2006
• Issued: 08/07/2012
• Est. Priority Date: 12/22/2005
• Status: Expired due to Fees

• Current Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Original Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Inventors: Gotting, Michael, Wersing, Heiko, Steil, Jochen J.
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): DRENNAN, BARRY T
• Application Number:
  US11/610,261
• Publication Number:
  US 20070147678A1
• Time in Patent Office: 2,064 Days
• Field of Search: 382/118, 382/155, 382/159, 382/160, 382/171, 382/173, 382/181, 382/190
• US Class Current: 382/155
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

A method and a system for determining an object segment in an electronic image. Preferably the method or system is sufficiently fast to allow real-time processing. A method for determining an object segment in an electronic image may comprise the steps of unsupervised learning of a multi-feature segmentation and of forming a relevance map. The method may further comprise the step of estimating the probability of a segment belonging to an object by the overlap of the segment and the relevance map in the electronic image.

Claims

1. A computer based method for determining an object segment of an object represented in an electronic image, comprising the steps of:

forming a multitude of binary maps obtained from a multitude of basic filter maps via unsupervised learning of a multi-feature segmentation, the step of unsupervised learning of a multi-feature segmentation further comprising the steps of;

forming training data vectors {right arrow over (m)}_(x,y) using basic filter maps (F_i);

obtaining codebook vectors {right arrow over (c)}^j from the training data vectors {right arrow over (m)}_(x,y) using a vector quantization network (VQ);

generating adaptive topographic activation maps (V^J) from the training data vectors {right arrow over (m)}_(x,y) and the codebook vectors {right arrow over (c)}^j, the adaptive topographic activation map (V^J) being scene dependent and computed as V^j_(x,y)=∥

{right arrow over (m)}_(x,y)−

{right arrow over (c)}^j∥

²;

forming a relevance map that serves as a prediction mask for a region around the object and computed as a superposition from a center map and a disparity map;

forming a selection of segments from the multitude of binary maps using the relevance map as a selection criterion; and

forming an object map based on the selection.

2. The method according to claim 1, further comprising the step of:

estimating a probability of a segment belonging to the object by the overlap of the segment and the relevance map in the electronic image.

3. The method according to claim 2, further comprising the step of forming a skin color mask or detecting skin color.

4. The method of claim 3, wherein an adaptive skin color segmentation further excludes skin color regions from a final mask.

5. The method of claim 1, wherein the step of generating adaptive topographic activation maps utilizes a standard vector quantization network VQ with a fixed number of training steps.

6. The method of claim 1, wherein the training data vectors {right arrow over (m)}(x,y) include the pixel positions (x,y) as features.

7. The method of claim 1, wherein each component of the training data vectors {right arrow over (m)}(x,y) are each normalized by their variance σ

(m_i)².

8. The method of claim 1, wherein each component of the training data vectors is weighted by an additional weighting factor (ξ

^I).

9. The method of claim 8, wherein the additional weighting factor (ξ

^I) is heuristically determined.

10. The method of claim 1, wherein the codebook vectors C^J are obtained by the steps of:

drawing a random (x,y)-position from an image;

generating the vector {right arrow over (m)}(x,y) at this position;

compute the minimal distance (d_min) of m(−

x,y) to all codebook vectors C^J in the current codebook; and

assigning a new codebook vector ({right arrow over (c)}^j).

11. The method of claim 10, wherein the new codebook vector ({right arrow over (c)}^j)is equal to {right arrow over (m)}_(x,y) if d_min is larger than a threshold value (d′

) and a new {right arrow over (m)}_(x,y) is drawn otherwise.

12. The method of claim 1, wherein the scene dependent adaptive topographic activation map (V^J) is binarised by the formula:

13. The method of claim 1, wherein the number of pixels (inPix) of the intersection between a relevance map based upon the relevance map and a binarised adaptive topographic activation map and the number of pixels (outPix) of the binarised adaptive topographic activation map without the relevance map (I_Rel) are used for selecting the suitable binary maps (B_i).

14. The method of claim 13, wherein a probability of a binary map (B_i) belonging to the object is estimated by the relative frequency outPix/inPix.

15. The method of claim 14, wherein a binary map (B_i) is included in an object map (I_final), if the relative frequency is larger than a given threshold.

16. The method of claim 15, wherein the object map (I_Final) is computed as the additive superposition of the selected binary map (B_i) and the skin color pixels are removed from the object map (I_Final=Σ

_iB_i−

I_Skin).

17. A computer software program product embodied on a non-transitory computer readable medium for determining an object segment of an object represented in an electronic image, the computer software program product comprising instructions, which, when executed, cause a processor to perform the steps of:

forming a multitude of binary maps obtained from a multitude of basic filter maps via unsupervised learning of a multi-feature segmentation, the step of unsupervised learning of a multi-feature segmentation further comprising the steps of;

forming training data vectors {right arrow over (m)}_(x,y) using basic filter maps (F_i);

obtaining codebook vectors {right arrow over (c)}^j from the training data vectors {right arrow over (m)}_(x,y) using a vector quantization network (VQ);

generating adaptive topographic activation maps (V^J) from the training data vectors {right arrow over (m)}_(x,y) and the codebook vectors {right arrow over (c)}^j, the adaptive topographic activation map (V^J) being scene dependent and computed as V^j_(x,y)=∥

{right arrow over (m)}_(x,y)−

{right arrow over (c)}^j∥

²;

forming a relevance map that serves as a prediction mask for a region around the object and computed as a superposition from a center map and a disparity map;

forming a selection of segments from the multitude of binary maps using the relevance map as a selection criterion; and

forming an object map based on the selection.