Image processing including adjoin feature based object detection, and/or bilateral symmetric object segmentation

• US 10,074,034 B2
• Filed: 06/14/2013
• Issued: 09/11/2018
• Est. Priority Date: 06/14/2013
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Wang, Tao, Fu, Lianrui, Li, Qiang, Li, Jianguo, Chen, Yurong, Peng, Ya-ti, Zhang, Yimin, Yao, Anbang
• Primary Examiner(s): Kholdebarin, Iman K
• Assistant Examiner(s): Tran, Mai
• Application Number:
  US14/124,233
• Publication Number:
  US 20140369608A1
• Time in Patent Office: 1,915 Days
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 10/757 Matching configurations of points or features

Abstract

Apparatuses, methods and storage medium associated with processing an image are disclosed herein. In embodiments, a method for processing one or more images may include generating a plurality of pairs of keypoint features for a pair of images. Each pair of keypoint features may include a keypoint feature from each image. Further, for each pair of keypoint features, corresponding adjoin features may be generated. Additionally, for each pair of keypoint features, whether the adjoin features are similar may be determined. Whether the pair of images have at least one similar object may also be determined, based at least in part on a result of the determination of similarity between the corresponding adjoin features. Other embodiments may be described and claimed.

Claims

1. A method for processing one or more images, comprising:

generating, by a computing device, a plurality of pairs of keypoint features, with each pair of keypoint features including a first keypoint feature from a first of the one or more images, and a second keypoint feature from a second of the one or more images;

generating, by the computing device, for each pair of keypoint features, a first plurality of adjoin features for the first keypoint feature of the keypoint feature pair, and a corresponding second plurality of adjoin features for the second keypoint feature of the keypoint feature pair, wherein the first plurality of adjoin features for the first keypoint feature are immediately adjacent to and circularly surround the first keypoint feature, and the second plurality of adjoin features for the second keypoint feature are immediately adjacent to and circularly surround the second keypoint feature;

determining, by the computing device, for each pair of keypoint features, whether an immediately adjacent adjoin feature of the first keypoint feature of the keypoint feature pair is similar to the corresponding immediately adjacent adjoin feature of the second keypoint feature of the keypoint feature pair; and

determining, by the computing device, whether the first and the second image have at least one similar object, based at least in part on a result of the determination of similarity between the corresponding immediately adjacent and circularly surrounding adjoin features of the first and second keypoint features of the keypoint feature pairs;

wherein the first and second plurality of immediately adjacent and circularly surrounding adjoin features are generated based at least in part on an adjoin feature order; and

wherein given a first keypoint feature of s_i defined by at least [X_i, r_i, θ

_i,]^T, an adjoin feature order of K, the K immediately adjacent and circularly surrounding adjoin features of s_i comprises s_i*^k defined by at least [X*^k_i, r_i, θ

_i,]^T, where X*^k_i is defined by at least [X_i, r_i and α

_k], where α

_k=θ

_i+2π

k/K, k=1,2, . . . K−

1, r_i and θ

_i are point size and orientation of the keypoint feature s_i.

2. The method of claim 1, wherein generating a plurality of pairs of keypoint features comprises:

identifying a first plurality of keypoint features of the first image;

identifying a second plurality of keypoint features of the second image; and

pairing the first and second plurality of keypoint features of the first and second images.

3. The method of claim 1, wherein determining whether the first and second images have at least one similar object comprises:

generating a set of similar keypoint feature pairs, wherein the keypoint features of a similar keypoint feature pair are determined to be similar based at least in part on a result of the determination of similarity between the corresponding immediately adjacent and circularly surrounding adjoin features of the keypoint features of the similar keypoint feature pairs; and

outputting the set of similar keypoint feature pairs.

4. The method of claim 3, further comprising highlighting, by the computing device, at least an object in the first image based at least in part on some of the keypoint features of the first image that are members of the set of similar keypoint feature pairs.

5. The method of claim 4, wherein highlighting comprises bounding the object with a geometric primitive.

6. The method of claim 1, wherein the first and second plurality of immediately adjacent and circularly surrounding adjoin features are generated based at least in part on an adjoin feature order;

and wherein given a first keypoint feature of s_i=[X_i, r_i, θ

_i, f(X_i, r_i, θ

_i)]^T, an adjoin feature order of K, the K immediately adjacent and circularly surrounding adjoin features of s_i comprises s_i*^k=[X_i*^k,r_i,θ

_i,f(X_i*^k,r_i,θ

)]^T where X*^k_i, =X_i+2r_i[cos(α

_k), sin(α

_k)]^T, α

_k=θ

_i+2π

k/K, k=1,2, . . . K−

1, r_i and θ

_i are point size and orientation of the keypoint feature s_i, and f(X*^k_i, r_i, θ

_i) comprises local spatial relationships of keypoint feature s_i.

7. At least one non-transitory storage medium comprising a plurality of instructions configured to cause an apparatus, in response to execution of the instructions by an apparatus, to perform processing of a plurality of images, wherein performance of the processing includes:

generation of a plurality of pairs of keypoint features, with each pair of keypoint features including a first keypoint feature from a first of the one or more images, and a second keypoint feature from a second of the one or more images;

for each pair of keypoint features, generation of a first plurality of adjoin features for the first keypoint feature of the keypoint feature pair, and a corresponding second plurality of adjoin features for the second keypoint feature of the keypoint feature pair, wherein the first plurality of adjoin features for the first keypoint feature are immediately adjacent and circularly surrounding the first keypoint feature, and the second plurality of adjoin features for the second keypoint feature are immediately adjacent and circularly surrounding the second keypoint feature;

for each pair of keypoint features, determination of whether an adjoin feature of the first keypoint feature of the keypoint feature pair is similar to the corresponding adjoin feature of the second keypoint feature of the keypoint feature pair; and

determination of whether the first and the second image have at least one similar object, based at least in part on a result of the determination of similarity between the corresponding immediately adjacent and circularly surrounding adjoin features of the first and second keypoint features of the keypoint feature pairs;

wherein the first and second plurality of immediately adjacent and circularly surrounding adjoin features are generated based at least in part on an adjoin feature order; and

wherein given a first keypoint feature of s_i defined by at least [X_i, r_i, θ

_i,]^T, an adjoin feature order of K, the K immediately adjacent and circularly surrounding adjoin features of s_i comprises s_i*^k defined by at least [X*^k_i, r_i, θ

_i,]^T, where X*^k_i is defined by at least [X_i, r_i and α

_k], where α

_k=θ

_i+2π

k/K, k=1,2, . . .K−

1, r_i and θ

_i are point size and orientation of the keypoint feature s_i.

8. The storage medium of claim 7, wherein generation of a plurality of pairs of keypoint features comprises:

identification of a first plurality of keypoint features of the first image;

identification of a second plurality of keypoint features of the second image; and

performance of pairing of the first and second plurality of keypoint features of the first and second images.

9. The storage medium of claim 7, wherein determination of whether the first and the second image have at least one similar object comprises:

generation of a set of similar keypoint feature pairs, wherein the keypoint features of a similar keypoint feature pair are determined to be similar based at least in part on a result of the determination of similarity between the corresponding immediately adjacent and circularly surrounding adjoin features of the keypoint features of the similar keypoint feature pairs; and

output of the set of similar keypoint feature pairs.

10. The storage medium of claim 9, wherein process one or more images further includes highlight at least an object in the first image based at least in part on some of the keypoint features of the first image that are members of the set of similar keypoint feature pairs.

11. The storage medium of claim 10, wherein highlight at least an object comprises bound the object with a geometric primitive.

12. The at least one non-transitory storage medium of claim 7, wherein the first and second plurality of adjoin features are generated based at least in part on an adjoin feature order;

and wherein given a first keypoint feature of s_i=[X_i, r_i, θ

_i, f(X_i, r_i, θ

_i)]^T, an adjoin feature order of K, the K immediately adjacent and circularly surrounding adjoin features of s_i comprises s_i*^k=[X_i*^k,r_i,θ

_i,f(X_i*^k,r_i,θ

)]^T where X*^k_i,=X_i+2r_i[cos(α

_k), sin(α

_k)]^T, α

_k=θ

_i+2π

k/K, k=1,2, . . . K−

1, r_i and θ

_i are point size and orientation of the keypoint feature s_i, and f(X*^k_i, r_i, θ

_i) comprises local spatial relationships of keypoint feature s_i.

13. An apparatus for processing one or more images, comprising:

one or more processors; and

a storage medium coupled to the one or more processors, and having a plurality of modules configured to cause the apparatus, in response to operation by the one or more processors, to perform processing of a plurality of images, wherein performance of the processing includes;

generation of a plurality of pairs of keypoint features, with each pair of keypoint features including a first keypoint feature from a first of the one or more images, and a second keypoint feature from a second of the one or more images;

for each pair of keypoint features, generation of a first plurality of adjoin features for the first keypoint feature of the keypoint feature pair, and a corresponding second plurality of adjoin features for the second keypoint feature of the keypoint feature pair, wherein the first plurality of adjoin features for the first keypoint feature are immediately adjacent and circularly surrounding the first keypoint feature, and the second plurality of adjoin features for the second keypoint feature are immediately adjacent and circularly surrounding the second keypoint feature;

for each pair of keypoint features, determination of whether an immediately adjacent and circularly surrounding adjoin feature of the first keypoint feature of the keypoint feature pair is similar to the corresponding immediately adjacent and circularly surrounding adjoin feature of the second keypoint feature of the keypoint feature pair; and

determination of whether the first and the second image have at least one similar object, based at least in part on a result of the determination of similarity between the corresponding immediately adjacent and circularly surrounding adjoin features of the first and second keypoint features of the keypoint feature pairs;

wherein the first and second plurality of immediately adjacent and circularly surrounding adjoin features are generated based at least in part on an adjoin feature order; and

wherein given a first keypoint feature of si defined by at least [X_i, r_i, θ

_i,]^T, an adjoin feature order of K, the K immediately adjacent and circularly surrounding adjoin features of s_i comprises s_i*^k defined by at least [X*^k_i, r_i, θ

_i,]^T, where X*^k_i is defined by at least [X_i, r_i and α

_k], where α

_k=θ

_i+2π

k/K, k=1,2, . . . K−

1, r_i and θ

_i are point size and orientation of the keypoint feature s_i.

14. The apparatus of claim 13, wherein performance of processing one or more images further includes highlight at least an object in the first image based at least in part on some of the keypoint features of the first image that are members of a set of similar keypoint feature pairs.

15. The apparatus of claim 13, wherein the apparatus is a selected one of a set-top box, a smartphone, a computing tablet, an ultrabook, a laptop computer, a desktop computer, a server computer, a game console, or a media player.

16. The apparatus of claim 13, wherein the first and second plurality of adjoin features are generated based at least in part on an adjoin feature order;

and wherein given a first keypoint feature of s_i=[X_i, r_i, θ

_i, f(X_i, r_i, θ

_i)]^T, an adjoin feature order of K, the K immediately adjacent and circularly surrounding adjoin features of s_i comprises s_i*^k=[X_i*^k,r_i,θ

_i,f(X_i*^k,r_i,θ

)]^T where X*^k_i,=X_i+2r_i[cos(α

_k), sin(α

_k)]^T, α

_k=θ

_i+2π

k/K, k=1,2, . . . K−

1, r_i and θ

_i are point size and orientation of the keypoint feature s_i, and f(X*^k_i, r_i, θ

_i) comprises local spatial relationships of keypoint feature s_i.