Keypoint descriptor generation by complex wavelet analysis

• US 9,691,003 B2
• Filed: 08/31/2015
• Issued: 06/27/2017
• Est. Priority Date: 11/05/2008
• Status: Active Grant

• Current Assignee: Zebra Technologies Corporation
• Original Assignee: Cortexica Vision Systems Limited (Zebra Technologies Corporation)
• Inventors: Bharath, Anil Anthony, Ng Sing Kwong, Jeffrey
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US14/841,442
• Publication Number:
  US 20150371106A1
• Time in Patent Office: 666 Days
• Field of Search: None
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

The present application relates to a method of generating a keypoint descriptor for identifying an object in an image or a sequence of images, the keypoint descriptor being substantially invariant to a transformation of the object in the image. The present application further relates to a method of identifying an object in an image using a keypoint descriptor; and processing apparatus and computer program products for implementing a method of the present application.

Claims

1. A method of generating a keypoint descriptor for identifying an object in an image, wherein the keypoint descriptor is substantially invariant to a transformation of the object in the image, the method comprising:

receiving object data indicative of an object for identification in an image, the image having an image plane associated therewith;

generating, by processing the received object data, at least one basis function representing a feature having undergone at least one transformation, using transformations that are out of the image plane to recognize objects from multiple views;

modifying a prototype wavelet function based on the at least one basis function to generate a plurality of modified wavelet functions;

comparing the plurality of modified wavelet functions with the at least one basis function;

selecting a modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function; and

generating the keypoint descriptor by processing at least one of an input image or an input orientation field according to the selected modified wavelet function to.

2. The method of claim 1, wherein the keypoint descriptor is configured to identify an object in a single image.

3. The method of claim 2, wherein the object is a two-dimensional or three-dimensional object in a two-dimensional image.

4. The method of claim 1, wherein the prototype wavelet function and the plurality of modified wavelet functions comprise a radial distance component and an azimuthal angle component.

5. The method of claim 4, wherein the prototype wavelet function and the plurality of modified wavelet functions have the formula:

Ψ

_k(d,θ

)=d<sup>nk−

1</sup>e<sup>−

α

kd</sup>(n_k−

α

_kd)·

(cos(m_kθ

)+q_x sin(m_kθ

))

where d<sup>nk−

1</sup> e<sup>−

α

kd</sup> (n_k−

α

_kd) is the radial distance component and (cos(m_kθ

)+q_x sin(m_kθ

)) is the azimuthal angle component, and where the term q_x denotes an imaginary unit.

6. The method of claim 1, wherein the object is a three dimensional object in a three dimensional image and the keypoint descriptor is configured to identify an object in a series of images spaced over time.

7. The method of claim 6, wherein the prototype wavelet function and the plurality of modified wavelet functions comprise a radial distance component, an azimuthal angle component and a polar angle component.

8. The method of claim 7, wherein the prototype wavelet function and the plurality of modified wavelet functions have the formula:

Ψ

_k(d,θ

,φ

)=d<sup>nk−

1</sup>e<sup>−

α

kd</sup>(n_k−

α

_kd)·

(cos(m_kθ

)₊q_x sin(m_kθ

))·

(cos(l_kφ

)+q_y sin(l_kφ

))

where the term d<sup>nk−

1</sup> e<sup>−

α

kd</sup> (n_k−

α

_kd) is the radial distance component with radial distance, d, the term (cos(m_kθ

)+q_x sin(m_kθ

)) is the azimuthal angle component with azimuthal angle θ

, and the term (cos(l_kφ

)+q_y sin(l_kφ

)) is the polar angle component with polar angle φ

, and where the terms q_x and q_y denote unit quaternions and obey the rules q_x²=q_y²=q_z²=−

1, and q_x q_y=q_z.

9. The method of claim 4, wherein the prototype wavelet function and the modified wavelet functions comprise a non-linear weighting of the radial distance component.

10. The method of claim 9, wherein the non-linear weighting represents a radial decay.

11. The method of claim 1, wherein the prototype wavelet function and the modified wavelet functions are quaternion wavelet functions, a keypoint is arranged to identify an object in an image patch, the transformation comprises a rotation, the transformation comprises a translation, the at least one basis function is an eigenfunction, the object is a two-dimensional object in a temporal sequence of images or a three-dimensional object in three-dimensional volumetric data, or the modifying of the prototype wavelet function based on the at least one basis function to generate the plurality of modified wavelet functions is performed in accordance with the formula:

12. The method of claim 1, wherein the comparing of the plurality of modified wavelet functions with the at least one basis function includes matching at least one of the modified wavelet functions with the at least one basis function, and the selecting of the modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function is selected according to matching criteria.

13. The method of claim 12, wherein the processing of the object data includes generating a plurality of basis functions, the comparing of the plurality of modified wavelet functions with the at least one basis function includes matching a number of the plurality of basis functions with at least one of the modified wavelet functions, and the selecting of the modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function includes selecting a modified wavelet function for each of the number of the plurality of basis functions according to matching criteria.

14. The method of claim 13, wherein the keypoint descriptor is generated when selecting a modified wavelet function of the plurality of modified wavelet functions based on the selected wavelet functions.

15. A system for generating a keypoint descriptor, comprising:

a data store comprising executable software;

a processor in data communication with the data store, the processor configured to execute the software and cause a computing device to;

receive object data representing an object for identification in an image;

process the object data to generate at least one basis function representing a feature having undergone at least one transformation or a transformation sequence across several consecutive frames, using transformations that are out of a plane of the image to recognize objects from multiple views;

modify a prototype wavelet function based on the at least one basis function to generate a plurality of modified wavelet functions;

compare the plurality of modified wavelet functions with the at least one basis function;

select a modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function; and

process an input image or input orientation field according to the selected modified wavelet function to generate a keypoint descriptor.

17. The system of claim 15, wherein the processor is further configured to execute the executable software and cause the system to process image data representing at least part of the image using the generated keypoint descriptor.

16. A non-transitory computer-readable storage medium having computer readable instructions stored thereon, wherein, when executed, cause the computerized device to perform a method of generating a keypoint descriptor for identifying an object in an image or in a sequence of images, wherein keypoint descriptor is substantially invariant to a transformation of the object in the image, the method comprising:

receiving object data representing an object for identification in an image;

processing the object data to generate at least one basis function representing a feature having undergone at least one transformation or a transformation sequence across several consecutive frames, using transformations that are out of a plane of the image to recognize objects from multiple views;

modifying a prototype wavelet function based on the at least one basis function to generate a plurality of modified wavelet functions;

comparing the plurality of modified wavelet functions with the at least one basis function;

selecting a modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function; and

processing an input image or input orientation field according to the selected modified wavelet function to generate the keypoint descriptor.

18. The non-transitory computer-readable storage medium of claim 16, wherein the method further comprises including processing image data representing at least part of the image with a keypoint descriptor.