Object detection approach using generative sparse, hierarchical networks with top-down and lateral connections for combining texture/color detection and shape/contour detection

• US 9,092,692 B2
• Filed: 09/13/2013
• Issued: 07/28/2015
• Est. Priority Date: 09/13/2012
• Status: Active Grant

• Current Assignee: TRIAD National Security, LLC
• Original Assignee: Los Alamos National Security LLC (Government of the United States of America)
• Inventors: Paiton, Dylan M., Kenyon, Garrett T., Brumby, Steven P., Schultz, Peter F., George, John S.
• Primary Examiner(s): Patel, Kanjibhai
• Application Number:
  US14/026,812
• Publication Number:
  US 20140072213A1
• Time in Patent Office: 683 Days
• Field of Search: 382/165, 382/199, 382/203, 382/284, 382/305, 382/312, 382/128
• US Class Current: 1/1
• CPC Class Codes: G06F 18/253 of extracted features
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06V 10/449 Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/806 of extracted features

Abstract

An approach to detecting objects in an image dataset may combine texture/color detection, shape/contour detection, and/or motion detection using sparse, generative, hierarchical models with lateral and top-down connections. A first independent representation of objects in an image dataset may be produced using a color/texture detection algorithm. A second independent representation of objects in the image dataset may be produced using a shape/contour detection algorithm. A third independent representation of objects in the image dataset may be produced using a motion detection algorithm. The first, second, and third independent representations may then be combined into a single coherent output using a combinatorial algorithm.

Claims

1. A computer-implemented method, comprising:

producing, by a computing system, a first independent representation of objects in an image dataset using a color/texture detection algorithm;

producing, by the computing system, a second independent representation of objects in the image dataset using a shape/contour detection algorithm; and

combining, by the computing system, the first and second independent representations into a single coherent output using a combinatorial algorithm, wherein the first independent representation of objects and the second independent representation of objects are independent from one another,

wherein the combining of the first and second independent representations into the single coherent output using the combinatorial algorithm comprises;

formatting, by the computing system, the first representation and the second representation as points in a matrix a size of a video frame of the image dataset;

forming, by the computing system, clusters based on hits that are within a predetermined proximity and of a same label; and

finding, by the computing system, a minimum volume enclosing ellipsoid to get an appropriate size and orientation of a bounding box.

2. The computer-implemented method of claim 1, wherein the color/texture detection algorithm is configured to learn a sparse overcomplete feature dictionary for the image dataset.

3. The computer-implemented method of claim 1, wherein the shape/contour detection algorithm is configured to take into account lateral interactions based on object-distractor difference (ODD) kernels.

4. The computer-implemented method of claim 1, wherein the first independent representation comprises rasterized PNG images and pixel values represent different categories.

5. The computer-implemented method of claim 1, wherein the second independent representation comprises neuron activity files.

6. The computer-implemented method of claim 1, wherein, for each frame in the image dataset, the color/texture detection algorithm gives a single output and the shape/contour detection algorithm gives an output for each object of interest.

7. The computer-implemented method of claim 1, further comprising:

producing, by the computing system, a third independent representation of objects in the image dataset using a motion detection algorithm, wherein

the combining further comprises combining the third independent representation with the first and second independent representation into the single coherent output using the combinatorial algorithm.

8. The computer-implemented method of claim 7, wherein the motion detection algorithm is configured to derive spatio-temporal filter properties for V1 neurons from their synaptic input weights, tuned such that a center frequency of a temporal band)(ω

_t0) divided by a center frequency of a spatial frequency band (ω

_x0) gives a velocity matched by a V1 cell.

9. The computer-implemented method of claim 8, wherein the spatial filter is implemented using a Gaussian-shaped excitatory connection surrounded by inhibitory Gaussian shaped flanks.

10. The computer-implemented method of claim 7, wherein an elliptical Gaussian-shaped excitatory connection located at x′

=0 and y′

=0 is surrounded by inhibitory Gaussian shaped flanks shifted by an amount x₀ and y₀ and centered about an instant in time denoted by t′

is given by;

11. The computer-implemented method of claim 10, wherein a center of the elliptical Gaussian is shifted in time and space according to:

x′

=x cos θ

−

t sin θ

;

t′

=t cos θ

+x sin θ

.

12. The computer-implemented method of claim 7, further comprising:

using, by the computing system, motion detections to modify confidence values.

13. The computer-implemented method of claim 1, further comprising:

using, by the computing system, background labels from the color/texture detection algorithm to modify confidence values.

14. A computer program embodied on a non-transitory computer-readable medium, the program configured to cause at least one processor to:

produce a first independent representation of objects in an image dataset using a color/texture detection algorithm;

produce a second independent representation of objects in the image dataset using a shape/contour detection algorithm;

produce a third independent representation of objects in the image dataset using a motion detection algorithm; and

combine the first, second, and third independent representations into a single coherent output using a combinatorial algorithm, wherein the first independent representation of objects, the second independent representation of objects, and the third independent representation of objects are independent from one another,

wherein the combining of the first, second, and third independent representations into the single coherent output using the combinatorial algorithm comprises;

formatting, by the computing system, the first, second, and third representations as points in a matrix a size of a video frame of the image dataset;

forming, by the computing system, clusters based on hits that are within a predetermined proximity and of a same label; and

finding, by the computing system, a minimum volume enclosing ellipsoid to get an appropriate size and orientation of a bounding box.

15. The computer program of claim 14, wherein, for each frame in the image dataset, the color/texture detection algorithm gives a single output and the shape/contour detection algorithm gives an output for each object of interest.

16. The computer program of claim 14, wherein the motion detection algorithm is configured to derive spatio-temporal filter properties for V1 neurons from their synaptic input weights, tuned such that a center frequency of a temporal band (ω

_t0) divided by a center frequency of a spatial frequency band (ω

_x0) gives a velocity matched by a V1 cell.

17. An apparatus, comprising:

memory storing computer program instructions; and

at least one processor configured to execute the computer program instructions, the at least one processor configured to;

combine a first independent representation of objects in an image dataset produced by a color/texture detection algorithm and a second independent representation of objects in the image dataset produced by a shape/contour detection algorithm into a single coherent output using a combinatorial algorithm, wherein the first independent representation of objects and the second independent representation of objects are independent from one another,

wherein the combining of the first and second independent representations into the single coherent output using the combinatorial algorithm comprises;

formatting, by the computing system, the first representation and the second representation as points in a matrix a size of a video frame of the image dataset;

forming, by the computing system, clusters based on hits that are within a predetermined proximity and of a same label; and

finding, by the computing system, a minimum volume enclosing ellipsoid to get an appropriate size and orientation of a bounding box.

18. The apparatus of claim 17, wherein the at least one processor is further configured to produce a third independent representation of objects in the image dataset using a motion detection algorithm, and the combining further comprises combining the third independent representation with the first and second independent representation into the single coherent output using the combinatorial algorithm.