APPARATUS FOR RECOGNIZING OBJECTS, APPARATUS FOR LEARNING CLASSIFICATION TREES, AND METHOD FOR OPERATING SAME

• US 20150023557A1
• Filed: 01/09/2013
• Published: 01/22/2015
• Est. Priority Date: 01/11/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yoo, Byung In, Choi, Chang Kyu, Han, Jae Joon, Lee, Chang Kyo
• Granted Patent: US 9,508,152 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/2134 based on separation criteria, e.g. independent component analysis
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06F 18/24765 Rule-based classification
• CPC Class Codes: G06T 15/06 Ray-tracing
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/40 Hidden part removal
• CPC Class Codes: G06T 17/005 Tree description, e.g. octree, quadtree
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20048 Transform domain processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/426 Graphical representations
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An object recognition system is provided. The object recognition system for recognizing an object may include an input unit to receive, as an input, a depth image representing an object to be analyzed, and a processing unit to recognize a visible object part and a hidden object part of the object, from the depth image, by using a classification tree. The object recognition system may include a classification tree learning apparatus to generate the classification tree.

Claims

1. An object recognition system, comprising:

an input unit configured to receive, as an input, a depth image representing an object to be analyzed; and

a processing unit configured to recognize a visible object part and a hidden object part of the object, from the depth image, using a classification tree.

2. The object recognition system of claim 1, further comprising:

a volume constructing unit configured to construct a volume of the object in a single data space, based on the recognized visible object part and the recognized hidden object part.

3. The object recognition system of claim 2, wherein the processing unit extracts additional information regarding the object, based on the volume.

4. The object recognition system of claim 3, wherein the additional information comprises information regarding at least one of a shape, a pose, a key joint, and a structure that are associated with the object.

5. The object recognition system of claim 2, wherein the volume constructing unit constructs the volume, based on a relative depth value stored in a leaf node of the classification tree, and

wherein the relative depth value indicates a difference value between a depth value of the recognized visible object part and a depth value of the recognized hidden object part.

6. The object recognition system of claim 1, wherein the processing unit applies the depth image to the classification tree,

wherein, when a current node of the classification tree is a split node, the processing unit reads a value of a feature and a threshold from the split node, applies the value of the feature and the threshold to a split function, and searches for one of a left child node and a right child node of the current node based on a result of the split function, and

wherein, when the current node is a leaf node, the processing unit reads, from the leaf node, a first histogram of the visible object part and a second histogram of the hidden object part, recognizes the visible object part from the depth image, based on the read first histogram, and recognizes the hidden object part from the depth image, based on the read second histogram.

7. The object recognition system of claim 6, wherein, when a result value of the split function does not meet the threshold, the processing unit searches for the left child node, and

wherein, when the result value meets the threshold, the processing unit searches for the right child node.

8. The object recognition system of claim 1, further comprising:

a size adjusting unit to adjust at least one size of a width and a height of an object model of the object.

9. The object recognition system of claim 1, wherein the classification tree comprises a probability value of the visible object part and a probability value of the hidden object part.

10. The object recognition system of claim 1, wherein the classification tree comprises a relative depth value associated with the visible object part and the hidden object part.

11. The object recognition system of claim 1, wherein the classification tree is represented using at least a portion of the hidden object part as a plurality of layers.

24. The object recognition system of claim 1, further comprising:

a classification tree obtaining unit configured to obtain the classification tree learned based upon consideration of modeled visible and hidden object parts for each of plural nodes of the classification tree.

25. The object recognition system of claim 24, wherein the classification tree obtaining unit generates the classification tree through the learning based upon consideration of visible and hidden object parts for each of plural nodes and obtains the generated classification tree to be available to the processing unit for the recognizing of the visible object part and the hidden object part.

26. The object recognition system of claim 25, further comprising:

a casting unit configured to cast rays towards a plurality of voxels of a three-dimensional (3D) object model of the object, using a virtual camera;

an image layer generating unit configured to generate a plurality of image layers sequentially every time the rays penetrate a surface of the 3D object model;

a capturing unit configured to capture a depth value and a voxel identifier (ID) of the surface, for each of the plurality of image layers, and to store the captured depth value and the captured voxel ID in each of the plurality of image layers; and

a training data generating unit configured to set an image layer with a minimum distance from the virtual camera to be visible object part data associated with the visible object part, to set the other image layers to be hidden object part data associated with the hidden object part, and to generate the training data.

27. The object recognition system of claim 1, wherein the processing unit recognizes the visible object part and the hidden object part of the object based upon a single depth image.

28. The object recognition system of claim 7, wherein the classification tree is represented using at least a portion of the hidden object part as a plurality of layers, with the processing unit reading from the leaf node plural hidden object histograms and recognizing at least one hidden object part from the depth image based on the read plural hidden object histograms.

29. The object recognition system of claim 28, wherein the plural hidden object histograms include at least one of a hidden object histogram representing muscles of a human body, a hidden object histogram representing a skeletal structure of the human body, a hidden object histogram representing internal organs of the human body, a hidden object histogram representing a cardiovascular system of the human body, and a hidden object histogram representing a nervous system for the human body.

12. A classification tree learning apparatus for generating a classification tree, the classification tree being configured for use in recognizing a visible object part and a hidden object part of an object to be analyzed, the classification tree learning apparatus comprising:

a learning unit configured to generate the classification tree, based on training data associated with the object.

13. The classification tree learning apparatus of claim 12, wherein the learning unit comprises:

a casting unit configured to cast rays towards a plurality of voxels of a three-dimensional (3D) object model of the object, using a virtual camera;

an image layer generating unit configured to generate a plurality of image layers sequentially every time the rays penetrate a surface of the 3D object model;

a capturing unit configured to capture a depth value and a voxel identifier (ID) of the surface, for each of the plurality of image layers, and to store the captured depth value and the captured voxel ID in each of the plurality of image layers; and

a training data generating unit configured to set an image layer with a minimum distance from the virtual camera to be visible object part data associated with the visible object part, to set the other image layers to be hidden object part data associated with the hidden object part, and to generate the training data.

14. The classification tree learning apparatus of claim 12, wherein the learning unit comprises:

a feature selecting unit configured to randomly select a single feature from among a plurality of feature sets;

a feature space transformation unit configured to transform visible object part data associated with the visible object part and hidden object part data associated with the hidden object part to a feature space, based on the selected feature;

a threshold selecting unit configured to randomly select a threshold from a range of a minimum value to a maximum value of the feature space;

a splitting unit configured to input, to a split function, the threshold, the feature space, and the visible object part data, to split the visible object part data into left visible object part data and right visible object part data, and to split the hidden object part data into left hidden object part data and right hidden object part data;

an information gain computing unit configured to compute an information gain for each of the left visible object part data, the right visible object part data, the left hidden object part data, and the right hidden object part data; and

a classification tree generating unit configured to store, in a current node of the classification tree, a value of the feature, the threshold, the left visible object part data, the right visible object part data, the left hidden object part data, and the right hidden object part data, and to generate the classification tree, when the information gain is within an optimum reference range.

15. The classification tree learning apparatus of claim 14, wherein the information gain computing unit computes a first intermediate information gain for the left visible object part data and the right visible object part data, computes a second intermediate information gain for the left hidden object part data and the right hidden object part data, and computes the information gain, based on the first intermediate information gain and the second intermediate information gain.

16. The classification tree learning apparatus of claim 14, wherein, when the information gain is beyond the optimum reference range, the classification tree generating unit controls an operation of each of the feature selecting unit, the feature space transformation unit, the threshold selecting unit, the split unit, and the information gain computing unit, so that a single feature is randomly reselected from among the plurality of feature sets, that a threshold is randomly reselected, and that learning of the current node of the classification tree is performed.

17. The classification tree learning apparatus of claim 14, wherein the classification tree generating unit determines whether the current node satisfies stopping criteria,

wherein, when the current node fails to satisfy the stopping criteria, the classification tree generating unit learns a left child node of the classification tree, by using, as inputs, the left visible object part data and the left hidden object part data, and learns a right child node of the classification tree, by using, as inputs, the right visible object part data and the right hidden object part data, and

wherein, when the current node satisfies the stopping criteria, the classification tree generating unit determines the current node to be a leaf node of the classification tree, and terminates learning of the classification tree.

18. The classification tree learning apparatus of claim 17, wherein, when the current node is determined to be the leaf node of the classification tree, the classification tree generating unit generates a first histogram representing a probability that each of a plurality of object parts of the object is to be determined to be the visible object part and generates a second histogram representing a probability that each of the plurality of object parts is to be determined to be the hidden object part, and the classification tree generating unit stores the first histogram and the second histogram in the current node.

19. The classification tree learning apparatus of claim 17, wherein, when the current node is determined to be the leaf node of the classification tree, the classification tree generating unit computes a relative depth value indicating a difference value between a depth value of the visible object part and a depth value of the hidden object part, and stores the computed relative depth value in the current node.

30. The classification tree learning apparatus of claim 12, wherein the learning unit is configured to generate the classification tree using training data associated with the object, based upon consideration of modeled visible and hidden object parts for each of plural nodes of the classification tree.

20. An object recognition method, the comprising:

receiving, as an input, a depth image representing an object to be analyzed; and

recognizing a visible object part and a hidden object part of the object, from the depth image, using a classification tree.

22. A non-transitory computer readable recording medium comprising computer readable code to control at least one processing device to implement the method of claim 20.

21. A classification tree learning method for generating a classification tree, the classification tree being configured for use by an object recognition system to recognize a visible object part and a hidden object part of an object to be analyzed from a depth image, the method comprising:

generating the classification tree, based on training data associated with the object.

23. A non-transitory computer readable recording medium comprising computer readable code to control at least one processing device to implement the method of claim 21.

31. An object recognition method, comprising:

obtaining a classification tree learned based upon consideration of modeled visible and hidden object parts for each of plural nodes of the classification tree; and

recognizing a visible object part and a hidden object part of an object based on a depth image of the object using the classification tree.

32. The object recognition method of claim 31, further comprising generating the classification tree through the learning based upon consideration of visible and hidden object parts for each of plural nodes of the classification tree.