Keypoint descriptor generation by complex wavelet analysis

• US 9,152,879 B2
• Filed: 04/09/2014
• Issued: 10/06/2015
• Est. Priority Date: 11/05/2008
• Status: Active Grant

• Current Assignee: Zebra Technologies Corporation
• Original Assignee: Cortexica Vision Systems Limited (Zebra Technologies Corporation)
• Inventors: Bharath, Anil Anthony, Ng Sing Kwong, Jeffrey
• Primary Examiner(s): LE, BRIAN Q
• Application Number:
  US14/249,295
• Publication Number:
  US 20140219567A1
• Time in Patent Office: 545 Days
• US Class Current: 1/1
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

The present application relates to a method of generating a keypoint descriptor for identifying an object in an image or a sequence of images, the keypoint descriptor being substantially invariant to a transformation of the object in the image. The method includes receiving object data representing an object for identification in an image; processing the object data to generate at least one basis function representing a feature having undergone at least one transformation or a transformation sequence across several consecutive frames, optionally using transformations that are out of a plane of the image to recognize objects from multiple views; modifying a prototype wavelet function based on the at least one basis function to generate a plurality of modified wavelet functions; comparing the plurality of modified wavelet functions with the at least one basis function; selecting a modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function; and processing an input image or input orientation field according to the selected modified wavelet function to generate the keypoint descriptor. The present application further relates to a method of identifying an object in an image using a keypoint descriptor; and processing apparatus and computer program products for implementing a method of the present application.

Claims

1. A method of generating a keypoint descriptor for identifying an object in an image, wherein the keypoint descriptor is substantially invariant to a transformation of the object in the image, the method comprising:

receiving object data indicative of an object for identification in an image, the image having an image plane associated therewith;

generating, by processing the received object data, at least one basis function representing a feature having undergone at least one transformation, using transformations that are out of the image plane to recognize objects from multiple views;

modifying a prototype wavelet function based on the at least one basis function to generate a plurality of modified wavelet functions;

comparing the plurality of modified wavelet functions with the at least one basis function;

selecting a modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function; and

generating the keypoint descriptor by processing at least one of an input image or an input orientation field according to the selected modified wavelet function to,

wherein the object is a three-dimensional object in a three-dimensional image and the keypoint descriptor is configured to identify an object in a series of images spaced over time, and

wherein the prototype wavelet function and the plurality of modified wavelet functions comprise a radial distance component, and azimuthal angle component and a polar angle component, the prototype wavelet function and the plurality of modified wavelet functions having the formula;

Ψ

_k(d,θ

,φ

)=d<sup>nk−

1</sup>e<sup>−

α

kd</sup>(n_k−

α

_kd)·

(cos(m_kθ

)+q_x sin(m_kθ

))·

(cos(l_kφ

)+q_y sin(l_kφ

))

where the term d<sup>nk−

1</sup>e<sup>−

α

kd</sup>(n_k−

α

_kd) is the radial distance weighting component with radial distance, d, the term (cos(m_kθ

)+q_xsin(m_kθ

)) is the azimuthal angle component with azimuthal angle θ

, and the term (cos(l_kφ

)+q_ysin(l_kφ

)) is the polar angle component with polar angle Ψ

, and where the terms q_x and q_y denote unit quaternions and obey the rules q_x²=q_y²=q_z²=−

1, and q_xq_y=q_z.

2. The method of claim 1, wherein the keypoint descriptor is configured to identify an object in a single image.

3. The method of claim 2, wherein the object is a two-dimensional or three-dimensional object in a two-dimensional image.

4. The method of claim 1, wherein the prototype wavelet function and the plurality of modified wavelet functions comprise a radial distance component and an azimuthal angle component.

5. The method of claim 4, wherein the prototype wavelet function and the modified wavelet functions comprise a non-linear weighting of the radial distance component.

6. The method of claim 5, wherein the non-linear weighting represents a radial decay.

7. The method of claim 1, wherein the comparing of the plurality of modified wavelet functions with the at least one basis function includes matching at least one of the modified wavelet functions with the at least one basis function, and the selecting of the modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function is selected according to matching criteria.

8. The method of claim 7, wherein the processing of the object data includes generating a plurality of basis functions, the comparing of the plurality of modified wavelet functions with the at least one basis function includes matching a number of the plurality of basis functions with at least one of the modified wavelet functions, and the selecting of the modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function includes selecting a modified wavelet function for each of the number of the plurality of basis functions according to matching criteria.

9. The method of claim 8, wherein the keypoint descriptor is generated when selecting a modified wavelet function of the plurality of modified wavelet functions based on the selected wavelet functions.

10. A system for generating a keypoint descriptor, comprising:

a data store comprising executable software;

a processor in data communication with the data store, the processor configured to execute the software and cause a computing device to;

receive object data representing an object for identification in an image;

process the object data to generate at least one basis function representing a feature having undergone at least one transformation or a transformation sequence across several consecutive frames, optionally using transformations that are out of a plane of the image to recognize objects from multiple views;

modify a prototype wavelet function based on the at least one basis function to generate a plurality of modified wavelet functions;

compare the plurality of modified wavelet functions with the at least one basis function;

select a modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function; and

process an input image or input orientation field according to the selected modified wavelet function to generate a keypoint descriptor,

wherein the object is a three-dimensional object in a three-dimensional image and the keypoint descriptor is configured to identify an object in a series of images spaced over time, and

wherein the prototype wavelet function and the plurality of modified wavelet function comprise a radial distance component, and azimuthal angle component and a polar angle component, the prototype wavelet function and the plurality of modified wavelet functions having the formula;

Ψ

_k(d,θ

,φ

)=d<sup>nk−

1</sup>e<sup>−

α

kd</sup>(n_k−

α

_kd)·

(cos(m_kθ

)+q_x sin(m_kθ

))·

(cos(l_kφ

)+q_y sin(l_kφ

))

where the term d<sup>nk−

1</sup>e<sup>−

α

kd</sup>(n_k−

α

_kd) is the radial distance weighting component with radial distance, d, the term (cos(m_kθ

)+q_x sin(m_kθ

)) is the azimuthal angle component with azimuthal angle θ

, and the term (cos(l_kφ

)+q_y sin(l_kφ

)) is the polar angle component with polar angle Ψ

, and where the terms q_x and q_y denote unit quaternions and obey the rules q_x²=q_y²=q_z²=−

1, and q_xq_y=q_z.

11. The system of claim 10, wherein the processor is further configured to execute the executable software and cause the system to process image data representing at least part of the image using the generated keypoint descriptor.

12. A non-transitory computer-readable storage medium having computer readable instructions stored thereon, wherein, when executed, cause the computerized device to perform a method of generating a keypoint descriptor for identifying an object in an image or in a sequence of images, wherein keypoint descriptor is substantially invariant to a transformation of the object in the image, the method comprising:

receiving object data representing an object for identification in an image;

processing the object data to generate at least one basis function representing a feature having undergone at least one transformation or a transformation sequence across several consecutive frames, optionally using transformations that are out of a plane of the image to recognize objects from multiple views;

modifying a prototype wavelet function based on the at least one basis function to generate a plurality of modified wavelet functions;

comparing the plurality of modified wavelet functions with the at least one basis function;

selecting a modified wavelet function of the plurality of modified wavelet functions based on the comparison of the plurality of modified wavelet functions with the at least one basis function; and

processing an input image or input orientation field according to the selected modified wavelet function to generate the keypoint descriptor,

wherein the object is a three-dimensional object in a three-dimensional image and the keypoint descriptor is configured to identify an object in a series of images spaced over time, and

wherein the prototype wavelet function and the plurality of modified wavelet functions comprise a radial distance component, and azimuthal angle component and a polar angle component, the prototype wavelet function and the plurality of modified wavelet functions having the formula;

Ψ

_k(d,θ

,φ

)=d<sup>nk−

1</sup>e<sup>−

α

kd</sup>(n_k−

α

_kd)·

(cos(m_kθ

)+q_x sin(m_kθ

))·

(cos(l_kφ

)+q_y sin(l_kφ

))

where the term d<sup>nk−

1</sup>e<sup>−

α

kd</sup>(n_k−

α

_kd) is the radial distance weighting component with radial distance, d, the term (cos(m_kθ

)+q_x sin(m_kθ

)) is the azimuthal angle component with azimuthal angle θ

, and the term (cos(l_kφ

)+q_y sin(l_kφ

)) is the polar angle component with polar angle Ψ

, and where the terms q_x and q_y denote unit quaternions and obey the rules q_x²=q_y²=q_z²=−

1, and q_xq_y=q_z.

13. The non-transitory computer-readable storage medium of claim 12, wherein the method further comprises including processing image data representing at least part of the image with a keypoint descriptor.