Image recognition verification

• US 10,318,576 B2
• Filed: 06/10/2015
• Issued: 06/11/2019
• Est. Priority Date: 12/12/2013
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Jaber, Mustafa, Song, Bing, Sudol, Jeremi
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US14/736,222
• Publication Number:
  US 20150278224A1
• Time in Patent Office: 1,462 Days
• Field of Search: None
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06T 2207/10024 Color image

Abstract

Systems and methods of verifying the results of an initial image recognition process are presented. A verification engine can receive a set of candidate images corresponding to the results of an image recognition process performed on a captured query image. The verification engine can determine an appropriate verification technique to apply to the images of the candidate set, and classify, re-rank or otherwise re-organize the candidate set such that the best match from the candidate set is confirmed as a proper match.

Claims

1. A method of image recognition verification for a candidate image of a plurality of candidate images returned in response to an image recognition search, the method comprising:

processing, by a verification engine, at least a portion of the candidate image according to at least one color space image processing technique, wherein the processing includes selecting at least one color channel as a prioritized image verification color channel based on a relative size of the at least one color channel in comparison to at least one other color channel within a color space;

selecting, by the verification engine, at least one matching technique from two or more matching techniques based on the at least one color space image processing technique and one or more characteristics of the candidate image, wherein selecting includes accessing information representing the two or more matching techniques that relates to image characteristics comprising the one or more characteristics, and wherein the one or more characteristics comprise at least one of;

an object class, a product class, a product family, a cluster of products, color information, shape information, text information, a logo, a product history, a time, a location, an orientation, a context, a position, a user, a license agreement, an image attribute, a frame rate, a hierarchy, a priority, an ontology, and metadata; and

comparing, by the verification engine, the candidate image processed according to the at least one color space image processing technique with a query image of the image recognition search according to the at least one matching technique, wherein the comparing includes weighting match results based on the prioritized image verification color channel.

2. The method of claim 1, wherein the at least one color space image processing technique includes selecting a color channel of a multiple channel color space as an image verification color channel.

3. The method of claim 2, wherein the multiple channel color space is a two-channel color space.

4. The method of claim 3, wherein the multiple channel color space is an rg chromaticity space.

5. The method of claim 2, wherein the multiple channel color space is a three-channel color space.

6. The method of claim 5, wherein the multiple channel color space is one of an RGB, HSV or YUV color space.

7. The method of claim 2, wherein the at least one color space image processing technique includes selecting a plurality of color channels of the multiple channel color space as image verification color channels.

8. The method of claim 7, wherein the plurality of color channels selected are equal in size.

9. The method of claim 8, wherein the plurality of color channels selected have an equal number of pixels.

10. The method of claim 7, wherein the plurality of color channels selected are not equal in size.

11. The method of claim 10, wherein the plurality of color channels selected do not have an equal number of pixels.

12. The method of claim 7, further comprising selecting a prioritized order for the plurality of color channels selected as image verification color channels.

13. The method of claim 1, wherein the at least one color space image processing technique includes processing one of interlaced color channel coding and non-interlaced color channel coding from the candidate image.

14. The method of claim 13, wherein the at least one color space image processing technique includes determining an offset and a skip count for each color channel of the candidate image.

15. The method of claim 1, further comprising processing at least a portion of the query image according to the at least one color space image processing technique prior to comparing the candidate image with the query image.

16. The method of claim 15, further comprising generating a match score or confidence score for the candidate image based on the comparing of the candidate image with the query image processed according to the at least one color space image processing technique.

17. The method of claim 16, further comprising one of ranking or re-ranking the candidate image with respect to at least one other candidate image of the plurality of candidate images based on the match score or confidence score.

18. A system for image recognition verification for a candidate image of a plurality of candidate images returned in response to an image recognition search, the system comprising:

a processor;

a memory device; and

a verification engine executable on the processor according to software instructions stored in the memory device, the verification engine being configured to;

process at least a portion of the candidate image according to at least one color space image processing technique, wherein the processing includes selecting at least one color channel as a prioritized image verification color channel based on a relative size of the at least one color channel in comparison to at least one other color channel within a color space;

select at least one matching technique from two or more matching techniques based on the at least one color space image processing technique and one or more characteristics of the candidate image, wherein selecting includes accessing information representing the two or more matching techniques that relates to image characteristics comprising the one or more characteristics, and wherein the one or more characteristics comprise at least one of;

an object class, a product class, a product family, a cluster of products, color information, shape information, text information, a logo, a product history, a time, a location, an orientation, a context, a position, a user, a license agreement, an image attribute, a frame rate, a hierarchy, a priority, an ontology, and metadata; and

compare the candidate image processed according to the at least one color space image processing technique with a query image of the image recognition search according to the at least one matching technique, wherein the comparing includes weighting match results based on the prioritized image verification color channel.

19. A computer program product embedded in a non-transitory computer readable medium comprising instructions executable by a computer processor for image recognition verification for a candidate image of a plurality of candidate images returned in response to an image recognition search, the instructions being executable by a computer processor to execute processing comprising:

processing at least a portion of the candidate image according to at least one color space image processing technique, wherein the processing includes selecting at least one color channel as a prioritized image verification color channel based on a relative size of the at least one color channel in comparison to at least one other color channel within a color space;

selecting at least one matching technique from two or more matching techniques based on the at least one color space image processing technique and one or more characteristics of the candidate image, wherein selecting includes accessing information representing the two or more matching techniques that relates to image characteristics comprising the one or more characteristics, and wherein the one or more characteristics comprise at least one of;

an object class, a product class, a product family, a cluster of products, color information, shape information, text information, a logo, a product history, a time, a location, an orientation, a context, a position, a user, a license agreement, an image attribute, a frame rate, a hierarchy, a priority, an ontology, and metadata; and

comparing the candidate image processed according to the at least one color space image processing technique with a query image of the image recognition search according to the at least one matching technique, wherein the comparing includes weighting match results based on the prioritized image verification color channel.