System and method for image comparison based on hyperplanes similarity

• US 10,452,958 B2
• Filed: 10/06/2017
• Issued: 10/22/2019
• Est. Priority Date: 10/06/2017
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Jones, Michael
• Primary Examiner(s): Bezuayehu, Solomon G
• Application Number:
  US15/726,415
• Publication Number:
  US 20190108423A1
• Time in Patent Office: 746 Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/2451 linear, e.g. hyperplane
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An image processing system includes a memory to store data indicative of benchmark feature vectors of benchmark images, an input interface to accept data indicative of a first feature vector of a first image and a second feature vector of a second image, and an output interface to render a similarity value between the first and the second images. The system includes a processor to determine the similarity using a first hyperplane separating the benchmark feature vectors from the first feature vector and a second hyperplane separating the benchmark feature vectors from the second feature vector. The processor determines a first normal vector to the first hyperplane as the difference between the first feature vector and the mean of the benchmark feature vectors. The processor determines an offset for the first hyperplane as the average of the maximum inner product of the benchmark feature vectors with the first normal vector and the inner product of the first feature vector with the first normal vector. The processor determines the similarity value as a function of a sum of a signed distance of the second feature vector to the first hyperplane and a signed distance of the first feature vector to the second hyperplane.

Claims

1. An image processing system, comprising:

a memory to store data indicative of benchmark feature vectors of benchmark images;

an input interface to accept data indicative of a first feature vector of a first image and a second feature vector of a second image, wherein the benchmark images do not include the first image and the second image;

a processor to determine a similarity value between the first and the second images using a first hyperplane separating the benchmark feature vectors from the first feature vector and a second hyperplane separating the benchmark feature vectors from the second feature vector, wherein the processor is configured to

determine a first normal vector to the first hyperplane as the difference between the first feature vector and the mean of the benchmark feature vectors;

determine an offset for the first hyperplane as the average of the maximum inner product of the benchmark feature vectors with the first normal vector and the inner product of the first feature vector with the first normal vector; and

determine the similarity value as a function of a sum of a signed distance of the second feature vector to the first hyperplane and a signed distance of the first feature vector to the second hyperplane; and

an output interface to render the similarity value.

2. The system of claim 1, wherein the processor is further configured to

determine a second normal vector to the second hyperplane as the difference between the second feature vector and the mean of the benchmark feature vectors;

determine an offset for the second hyperplane as the average of the maximum inner product of the benchmark feature vectors with the second normal vector and the inner product of the second feature vector with the second normal vector; and

determine the second hyperplane intersecting the second normal vector at the offset from the second feature vector.

3. The system of claim 1, wherein the second hyperplane is predetermined using a linear support vector machine (SVM) solver.

4. The system of claim 1, wherein the input interface accepts data indicative of a first set of feature vectors of a first set of images and a second set of feature vectors of a second set of images, wherein the benchmark images do not include the first set of images and the second set of images, wherein the processor

determines the first normal vector to the first hyperplane as the difference between the mean of the first set of feature vectors and the mean of the benchmark feature vectors;

determine the offset for the first hyperplane as the average of the maximum inner product of the benchmark feature vectors with the first normal vector and the minimum inner product of the feature vectors from the first set of feature vectors with the first normal vector;

determines the second normal vector to the second hyperplane as the difference between the mean of the second set of feature vectors and the mean of the benchmark feature vectors;

determine the offset for the second hyperplane as the average of the maximum inner product of the benchmark feature vectors with the second normal vector and the minimum inner product of the feature vectors from the second set of feature vectors with the second normal vector.

5. The system of claim 1, wherein the processor uses a neural network to process the first image to produce the first feature vector.

6. The system of claim 5, wherein the processor uses the neural network to process the second image to produce the second feature vector.

7. The system of claim 6, wherein the processor uses the neural network to process the benchmark images to produce the benchmark feature vectors.

8. The system of claim 1, wherein the first image, the second image, and the benchmark images are images of faces, thereby making the system a face recognition system.

9. The system of claim 1, further comprising:

a device operated based on the similarity value.

14. The method of claim 1, wherein the benchmark feature vectors, the first set of feature vectors, and the second set of feature vectors are produced by a neural network.

10. An image processing method, wherein the method uses a processor coupled to a memory storing data indicative of benchmark feature vectors of benchmark images, wherein the processor is coupled with stored instructions implementing the method, wherein the instructions, when executed by the processor carry out at least some steps of the method, comprising:

accepting data indicative of a first set of feature vectors of a first set of images and a second set of feature vectors of a second set of images, wherein the benchmark images do not include the first set of images and the second set of images;

comparing a first hyperplane separating the benchmark feature vectors from the first set of feature vectors with a second hyperplane separating the benchmark feature vectors from the second set of feature vectors to produce a similarity value between the first and the second set of images, wherein the comparing comprises

determining a first normal vector to the first hyperplane as difference between the mean of the first set of feature vectors and the mean of the benchmark feature vectors;

determining an offset for the first hyperplane as an average of the maximum inner product of the benchmark feature vectors with the first normal vector and the minimum inner product of the feature vectors from the first set of feature vectors with the first normal vector; and

determining the similarity value as a function of an average signed distance of the all feature vectors in the second set of feature vectors to the first hyperplane; and

rendering the similarity value.

11. The method of claim 10, wherein the first set of images includes only one image.

12. The method of claim 10, further comprising:

determining a second normal vector to the second hyperplane as difference between the mean of the second set of feature vectors and the mean of the benchmark feature vectors;

determining an offset for the second hyperplane as an average of the maximum inner product of the benchmark feature vectors with the second normal vector and the minimum inner product of the feature vectors from the second set of feature vectors with the second normal vector.

13. The method of claim 12, wherein the second set of images includes only one image.

15. A non-transitory computer readable storage medium embodied thereon a program executable by a processor for performing a method, the method comprising:

accepting data indicative of a first set of feature vectors of a first set of images, a second set of feature vectors of a second set of images, and benchmark feature vectors of benchmark images, wherein the benchmark images do not include the first set of images and the second set of images;

comparing a first hyperplane separating the benchmark feature vectors from the first set of feature vectors with a second hyperplane separating the benchmark feature vectors from the second set of feature vectors to produce a similarity value between the first and the second set of images; and

rendering the similarity value, wherein the comparing comprises

determining a first normal vector to the first hyperplane as difference between the mean of the first set of feature vectors and the mean of the benchmark feature vectors;

determining an offset for the first hyperplane as an average of the maximum inner product of the benchmark feature vectors with the first normal vector and the minimum inner product of the feature vectors from the first set of feature vectors with the first normal vector; and

determining the similarity value as a function of an average signed distance of the all feature vectors in the second set of feature vectors to the first hyperplane.