Signal detection, recognition and tracking with feature vector transforms

• US 10,339,664 B2
• Filed: 12/21/2017
• Issued: 07/02/2019
• Est. Priority Date: 10/27/2014
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rhoads, Geoffrey B.
• Primary Examiner(s): Azima, Shaghayegh
• Application Number:
  US15/850,019
• Publication Number:
  US 20180182116A1
• Time in Patent Office: 558 Days
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/579 from motion

Abstract

A method for obtaining object geometry in which image frames of a scene (e.g., video frames from a user passing a smartphone camera over an object) are transformed into dense feature vectors, and feature vectors are processed to obtain geometry of an object in the scene. The object geometry is determined from the feature vectors. Feature vector transforms are leveraged in a signal processing method for object identification (e.g., using machine learning classification), digital watermark or bar code reading and image recognition.

Claims

1. A method of object recognition comprising:

receiving a sequence of images captured of a scene by an image sensor;

using a hardware processor of a computer system, performing a feature vector transform on plural images in the sequence of the images to produce N-dimensional feature vector per pixel of the plural images, the feature vector transform producing for each pixel in an array of pixels, a first vector component corresponding to plural comparisons between a center pixel and pixels at plural directions around the center pixel for a first scale, and second vector component corresponding to plural comparisons between the center pixel and pixels at plural directions around the center pixel for a second scale,

wherein N is a number of dimensions of the N-dimensional feature vector;

wherein the plural comparisons at the first and second scales comprise quantized differences, and the quantized differences are encoded in a first data structure representing magnitude and direction of the quantized differences at each of the first and second scales; and

using a hardware processor of a computer system, deriving a second data structure characterizing geometry of an object in the scene from N-dimensional feature vectors represented using the first data structure, obtaining a pixel patch geometrically registered to the geometry of the object, and identifying the object by processing the registered pixel patch with a digital watermark reader to extract an identifier, a barcode reader to extract an identifier, or a trained classifier to identify the object according to training images for the object.

2. The method of claim 1 further comprising:

applying a filter to determine magnitude and direction of pixel relationships between a sample value at the center pixel and sample values at pixels located along rings at the first and second scales.

3. The method of claim 2 wherein applying the filter comprises applying a filter kernel that produces ring-arc vectors.

4. The method of claim 3 wherein the ring-arc vectors comprise arcs, and an arc represents locations along a ring where the quantized differences are the same along a ring.

5. The method of claim 1 wherein the feature vector transform is configurable to adjust radii from the center pixel to a ring to select the first and second scales and spacing of pixels along rings at the first and second scales.

6. A non-transitory, computer readable medium, on which is stored instructions, which when executed by a processor, perform steps comprising:

receiving a sequence of images captured of a scene by an image sensor;

performing a feature vector transform on plural images in the sequence of the images to produce N-dimensional feature vector per pixel of the plural images, the feature vector transform producing for each pixel in an array of pixels, a first vector component corresponding to plural comparisons between a center pixel and pixels at plural directions around the center pixel for a first scale, and second vector component corresponding to plural comparisons between the center pixel and pixels at plural directions around the center pixel for a second scale,

wherein N is a number of dimensions of the N-dimensional feature vector;

wherein the plural comparisons at the first and second scales comprise quantized differences, and the quantized differences are encoded in a first data structure representing magnitude and direction of the quantized differences at each of the first and second scales; and

deriving a second data structure characterizing geometry of an object in the scene from N-dimensional feature vectors represented using the first data structure, obtaining a pixel patch geometrically registered to the geometry of the object, and

identifying the object by processing the registered pixel patch with a digital watermark reader to extract an identifier, a barcode reader to extract an identifier, or a trained classifier to identify the object according to training images for the object.

7. The non-transitory, computer readable medium of claim 6, further comprising instructions, which when executed by the processor, perform an act of:

applying a filter to determine magnitude and direction of pixel relationships between a sample value at the center pixel and sample values at pixels located along rings at the first and second scales.

8. The non-transitory, computer readable medium of claim 7 wherein applying the filter comprises applying a filter kernel that produces ring-arc vectors.

9. The non-transitory, computer readable medium of claim 8 wherein the ring-arc vectors comprise arcs, and an arc represents locations along a ring where the quantized differences are the same along a ring.

10. The non-transitory, computer readable medium of claim 6 wherein the feature vector transform is configurable to adjust radii from the center pixel to a ring to select the first and second scales and spacing of pixels along rings at the first and second scales.

11. A system comprising:

an image sensor for capturing a sequence of images captured of a scene by an image sensor;

a hardware processor configured with instructions to perform a feature vector transform on plural images in the sequence of the images to produce N-dimensional feature vector per pixel of the plural images, the feature vector transform producing for each pixel in an array of pixels, a first vector component corresponding to plural comparisons between a center pixel and pixels at plural directions around the center pixel for a first scale, and second vector component corresponding to plural comparisons between the center pixel and pixels at plural directions around the center pixel for a second scale,

wherein N is a number of dimensions of the N-dimensional feature vector;

wherein the plural comparisons at the first and second scales comprise quantized differences, and the quantized differences are encoded in a first data structure representing magnitude and direction of the quantized differences at each of the first and second scales; and

a hardware processor configured with instructions to characterize geometry of an object in the scene from N-dimensional feature vectors represented using the first data structure;

a hardware processor configured with instructions to register a pixel patch to the geometry of the object; and

a hardware processor configured with instructions to identify the registered pixel patch based on a digital watermark, barcode or trained classifier that has been trained on training images for the object.

12. The system of claim 11 comprising a filter configured to execute the feature vector transform by determining magnitude and direction of pixel relationships between a sample value at the center pixel and sample values at pixels located along rings at the first and second scales.

13. The system of claim 12 wherein the filter comprises a filter kernel that produces ring-arc vectors when applied to patches of pixels.

14. The system of claim 13 wherein the ring-arc vectors comprise arcs, and an arc represents locations along a ring where the quantized differences are the same along a ring.

15. The system of claim 11 wherein the feature vector transform is configurable to adjust radii from the center pixel to a ring to select the first and second scales and spacing of pixels along rings at the first and second scales.