APPARATUS AND METHODS FOR SALIENCY DETECTION BASED ON COLOR OCCURRENCE ANALYSIS

• US 20160086052A1
• Filed: 03/03/2015
• Published: 03/24/2016
• Est. Priority Date: 09/19/2014
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Piekniewski, Filip, Richert, Micah, Fisher, Dimitry
• Granted Patent: US 9,870,617 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/451 with interaction between the filter responses, e.g. cortical complex cells
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: H04N 9/7973 by dividing the luminance or colour component signal samples or frequency bands among a plurality of recording channels H04N9/804, H04N9/825 take precedence
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement
• CPC Class Codes: Y10S 901/47 : Optical

Abstract

Apparatus and methods for detecting and utilizing saliency in digital images. In one implementation, salient objects may be detected based on analysis of pixel characteristics. Least frequently occurring pixel values may be deemed as salient. Pixel values in an image may be compared to a reference. Color distance may be determined based on a difference between reference color and pixel color. Individual image channels may be scaled when determining saliency in a multi-channel image. Areas of high saliency may be analyzed to determine object position, shape, and/or color. Multiple saliency maps may be additively or multiplicative combined in order to improve detection performance (e.g., reduce number of false positives). Methodologies described herein may enable robust tracking of objects utilizing fewer determination resources. Efficient implementation of the methods described below may allow them to be used for example on board a robot (or autonomous vehicle) or a mobile determining platform.

Claims

1. A non-transitory computer-readable storage medium having instructions embodied thereon, the instructions being executable by a processing apparatus to configure a process enabled to track an object in a plurality of images each comprised of pixels by at least:

evaluation of a pixel chromaticity distribution for pixels of a given image of the plurality of images;

based at least on the evaluation, identification of a portion of pixels of the given image; and

provision of positions corresponding respectively to individual pixels of the portion to the process;

wherein;

the portion of pixels corresponds to a representation of the object in the given image; and

the provision of the positions enables the process to determine a location of the object in a subsequent image of the plurality of images.

2. The non-transitory computer-readable storage medium of claim 1, wherein:

the chromaticity distribution is configured to characterize frequency of occurrence of pixel chromaticity within the given image; and

the identification of the portion of pixels is based at least in part on a frequency of occurrence of pixel chromaticity breaching a threshold.

3. The non-transitory computer-readable storage medium of claim 2, wherein identification of the portion of pixels of the given image comprises:

(i) determination of a spatial distribution of a difference measure between chromaticity of pixels of the given image and a reference chromaticity; and

(ii) comparison of the difference measure at a pixel location to a level.

4. The non-transitory computer-readable storage medium of claim 3, wherein the reference chromaticity is determined based at least on evaluation of the frequency of occurrence of pixel chromaticity.

5. The non-transitory computer-readable storage medium of claim 4, wherein:

the reference chromaticity is determined based at least on a prevailing chromaticity of pixels satisfying a condition, and

the condition comprises frequency of occurrence of pixels chromaticity being below the threshold.

6. The non-transitory computer-readable storage medium of claim 2, wherein:

the chromaticity distribution evaluation comprises determination of a histogram of pixel chromaticity occurrence within the given image; and

the configuration of the process further includes producing a back projected map comprising a plurality of entries, a value of a given entry at a given location being assigned a value corresponding to a chromaticity occurrence of a corresponding pixel at the given location within the given image.

7. The non-transitory computer-readable storage medium of claim 6, wherein the portion corresponds to a pixel chromaticity occurrence being in the range comprising 0.1% to 5%, inclusive.

8. The non-transitory computer-readable storage medium of claim 6, wherein:

the pixel chromaticity is characterized by first and second chromaticity channel; and

the histogram comprises a first plurality of bins characterizing the first chromaticity channel, and a second plurality of bins characterizing the second chromaticity channel.

9. The non-transitory computer-readable storage medium of claim 6, wherein:

number of bins of the first plurality the pixel chromaticity is characterized by first and second chromaticity channels; and

the histogram comprises a first plurality of bins characterizing the first chromaticity channel, and a second plurality of bins characterizing the second chromaticity channel.

10. The non-transitory computer-readable storage medium of claim 9, wherein:

values of the first and the second chromaticity channel are characterized by an 8-bit value; and

a number of at least one of the first plurality of bins and/or the second plurality of bins is selected from a range comprising 2 to 8 bins, inclusive.

11. The non-transitory computer-readable storage medium of claim 1, wherein the evaluation of the pixel chromaticity distribution comprises determination of a pixel chromaticity distribution model characterized by at least two parameters.

12. The non-transitory computer-readable storage medium of claim 1, wherein the evaluation of the pixel chromaticity distribution comprises a determination of a pixel chromaticity distribution model, the determination of the model based at least on analysis of a portion of the plurality of images preceding the given image.

13. The non-transitory computer-readable storage medium of claim 1, wherein the provision of the positions is configured to reduce a number of occurrences of false positives associated with detection of the object by the process in the another image, as compared to detection of the object by the process in the another image in the absence of the provision of the positions.

14. The non-transitory computer-readable storage medium of claim 1, wherein:

the identification of the portion is further configured based at least on determination of a low-pass filtered version of the difference measure; and

the configuration of the process further includes determination of a shape parameter associated with a portion of the low-pass filtered version.

15. The non-transitory computer-readable storage medium of claim 14, wherein the portion of the low-pass filtered version is identified based at least on a greatest contiguous number of pixels.

16. The non-transitory computer-readable storage medium of claim 14, wherein the portion of the low-pass filtered version is identified based at least on a greatest cumulative saliency value, the value determined by at least accumulating a plurality of saliency values.

17. The non-transitory computer-readable storage medium of claim 14, wherein the shape parameter comprises a circularity determined based at least on:

determination of one or more contours associated with one or more contiguous pixel groups within the low-pass filtered version;

for individual ones of the one or more contours, determination of a smallest enclosing circle; and

determination of the circularity based at least on a ratio between an area of individual ones of the one or more contours and an area of the smallest enclosing circle.

18. The non-transitory computer-readable storage medium of claim 1, wherein:

the object comprises a target for a robotic apparatus; and

the object tracking is configured to cause provision of a control command to the robotic apparatus to cause the robotic apparatus to approach the object.

19. The non-transitory computer-readable storage medium of claim 1, wherein:

the object comprises a target object of a given color; and

the object tracking is configured to enable a robotic device to pick up the target object from a plurality of objects characterized by a plurality of colors, individual ones of the plurality of objects being characterized by a color of the plurality of colors.

20. A computerized image processing apparatus comprising:

a processing apparatus;

an interface configured to obtain an image, the image comprising a plurality of pixels; and

a computer-readable storage medium having instructions embodied thereon, the instructions being executable by the processing apparatus and configured to, upon execution;

determine at least one statistic of a color parameter associated with individual ones of the plurality of pixels;

evaluate the at least one statistic to determine an occurrence of one or more reference colors in a given area of the image; and

cause decrease of a parameter configured to characterize a likelihood of presence of an object in the given area.