Apparatus and method for object recognition and for training object recognition model

• US 10,133,938 B2
• Filed: 09/15/2016
• Issued: 11/20/2018
• Est. Priority Date: 09/18/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Chang Hyun, Moon, Tae Sup, Lee, Dong Hwa, Choi, Hyun Jin, Choi, Ki Hwan
• Primary Examiner(s): Cunningham, Gregory F
• Application Number:
  US15/266,231
• Publication Number:
  US 20170083772A1
• Time in Patent Office: 796 Days
• Field of Search: 382103
• CPC Class Codes: B60C 9/00 Reinforcements or ply arrangement of pneumatic tyres inserts having reinforcing means B60C5/08; bead structure, e.g. turnup or overlap construction, B60C15/00
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06F 18/23213 with fixed number of clusters, e.g. K-means clustering
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

Disclosed are methods and apparatuses for object recognition using an artificial neural network. An object recognition apparatus includes an adjacent pixel setter configured to set the neighboring pixel pairs in the image frame, each neighboring pixel pair including first pixel and one or more second pixels adjacent to the first pixel, a labeler configured to label the first pixel using deep neural network-based model based on probability density function value of the neighboring pixel pairs, and an object recognizer configured to recognize an object based on the labeled first pixel.

Claims

1. An object recognition apparatus comprising:

a processor configured to;

set neighboring pixel pairs in an image frame, each neighboring pixel pair comprising a first pixel and one or more second pixels adjacent to the first pixel;

label the first pixel of each of the neighboring pixel pairs in the image frame using a deep neural network-based model based on a probability density function value of the neighboring pixel pairs; and

recognize an object based on the labeled first pixel of each of the neighboring pixel pairs of the image frame.

2. The object recognition apparatus of claim 1, wherein the processor comprises:

an adjacent pixel setter configured to set the neighboring pixel pairs in the image frame, the each neighboring pixel pair comprising the first pixel and the one or more second pixels adjacent to the first pixel;

a labeler configured to label the first pixel of each of the neighboring pixel pairs in the image frame using the deep neural network-based model based on the probability density function value of the neighboring pixel pairs; and

an object recognizer configured to recognize the object based on the labeled first pixel of each of the neighboring pixel pairs in the image frame.

3. The object recognition apparatus of claim 2, wherein the one or more second pixels are 8 pixels adjacent to the first pixel, and the neighboring pixel pairs are 8 pixel pairs, each comprising the first pixel and one of the 8 second pixels.

4. The object recognition apparatus of claim 2, wherein the adjacent pixel setter is further configured to set the neighboring pixel pairs on a k-layer according to a degree of proximity (k) between each pixel and the first pixel.

5. The object recognition apparatus of claim 4, wherein k is a constant.

6. The object recognition apparatus of claim 1, wherein the deep neural network-based model comprises any one or any combination of a convolutional neural network (CNN) model that uses spatial information and a recurrent deep neural network (RDNN) model that uses time information.

7. The object recognition apparatus of claim 2, wherein the pixel labeler is further configured to label the first pixel of each of the neighboring pixel pairs in the image frame using a pair-wise function, and the pair-wise function indicates a difference in entropy between pixels in the neighboring pixel pairs that occurs when a second pixel of the one or more second pixels is used to estimate a probability density function value of the first pixel.

8. The object recognition apparatus of claim 7, wherein the pixel labeler is further configured to calculate the pair-wise function using Kullback-Leibler divergence (KL-divergence).

9. The object recognition apparatus of claim 2, wherein the object recognizer is further configured to recognize first pixels of each of the neighboring pixel pairs in the image frame labeled with a same label as an object of a class.

10. The object recognition apparatus of claim 2, wherein the processor further comprises:

a clusterer configured to cluster first pixels of each of the neighboring pixel pairs in the image frame with a same label into a group, and to represent the group as a cluster box, wherein the cluster box indicates a boundary between different objects.

11. The object recognition apparatus of claim 1, further comprising a memory configured to store instructions, and wherein the processor is further configured to execute the instructions to set the neighboring pixel pairs in the image frame, the each neighboring pixel pair comprising the first pixel of each of the neighboring pixel pairs in the image frame and the one or more second pixels adjacent to the first pixel, label the first pixel of each of the neighboring pixel pairs in the image frame using the deep neural network-based model based on the probability density function value of the neighboring pixel pairs, and recognize the object based on the labeled first pixel of each of the neighboring pixel pairs in the image frame.

12. A processor-implemented object recognition method, the method comprising:

setting neighboring pixel pairs in an image frame, each neighboring pixel pair comprising a first pixel and one or more second pixels adjacent to the first pixel;

labeling the first of each of the neighboring pixel pairs in the image frame using a deep neural network-based model based on a probability density function value of the neighboring pixel pairs; and

recognizing an object based on the labeled first pixel of each of the neighboring pixel pairs in the image frame.

13. The object recognition method of claim 12, wherein the setting of the neighboring pixel pairs comprises setting the neighboring pixel pairs on a k-layer according to a degree of proximity (k) between each pixel and the first pixel.

14. The object recognition method of claim 12, wherein the first pixel is labeled using a pair-wise function, and the pair-wise function indicates a difference in entropy between pixels in the neighboring pixel pairs that occurs when a second pixel of the one or more second pixels is used to estimate a probability density function value of the first pixel of each of the neighboring pixel pairs in the image frame.

15. The object recognition method of claim 14, wherein the labeling of the first pixel of each of the neighboring pixel pairs in the image frame comprises calculating the pair-wise function using Kullback-Leibler divergence (KL-divergence).