Composite image detection

• US 10,062,173 B1
• Filed: 12/09/2015
• Issued: 08/28/2018
• Est. Priority Date: 12/09/2015
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Padfield, Dirk Ryan
• Primary Examiner(s): Krasnic, Bernard
• Application Number:
  US14/964,252
• Time in Patent Office: 993 Days
• Field of Search: None
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06V 10/762 using clustering, e.g. of similar faces in social networks
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 30/1914 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries
• CPC Class Codes: G06V 30/248 involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q" G06V30/242 takes precedence

Abstract

Features are disclosed for processing composite images. Composite images may be received that include a common item such as a t-shirt with different graphics overlaid on the item. Features for detecting such composite images by comparing shape and color features of an uploaded image to previously detected composite images are described. Composite images including the common item may be grouped into clusters. The clustered images can then be processed as a group such as to separate the graphics from the underlying image and to make authorization determinations for inclusion in an online catalog system.

Claims

1. A device comprising:

a computer-readable memory storing specific computer-executable instructions; and

a processor in communication with the computer-readable memory, wherein the processor executes the specific computer-executable instructions to cause the device to at least;

receive a feature signature for a known composite image;

receive a prior authorization determination for the known composite image, the prior authorization determination indicating that the known composite image may be transmitted in response to a search request received from another electronic device by a catalog system;

receive an input image to be authorized for transmission via the catalog system, the input image comprising a background image region, an underlying image region, and an overlaid image region being overlaid upon at least a portion of the underlying image region;

remove the background image region from the input image;

generate an input image feature signature indicative of a shape of an item shown in the underlying image region;

determine that the input image feature signature matches the feature signature; and

upon determining that the input feature signature matches the feature signature, authorize the input image for transmission via the catalog system based at least in part on the prior authorization determination for the known composite image.

2. The device of claim 1, wherein the input image feature signature is generated from a segmented image, the segmented image comprising, for each pixel, one of a pixel-on value or a pixel-off value, and wherein the processor executes the specific computer-executable instructions to at least:

divide the segmented image into sectors; and

for each sector of the segmented image, generate a count of pixel-on values within the sector, wherein the input image feature signature includes a vector of elements, wherein each element corresponds to a sector and the value for the element corresponds to the count of pixel-on values within the sector.

3. The device of claim 2, wherein the processor executes the specific computer-executable instructions to divide the segmented image by causing the device to at least:

identify a center point of the segmented image; and

form a sector by extending two lines from the center point, the two lines separated by a predetermined angle.

4. The device of claim 1, wherein the processor executes the specific computer-executable instructions to further cause the device to at least:

generate an indication of a color of the item shown in the underlying image region, wherein the input image feature signature includes a vector of elements, wherein an element corresponds to item color and the value for the element is the indication of the color of the item shown in the underlying image region.

5. The device of claim 4, wherein the processor executes the specific computer-executable instructions to further cause the device to generate the indication of the color of the item shown in the underlying image region by causing the device to at least:

extract a contour around the underlying image region; and

generate a color value for the underlying image region using color channel information for pixels within the contour, wherein the color value is used as the value for the element corresponding to the item color.

6. The device of claim 1,

wherein the known composite image is included in a cluster of composite images;

wherein the processor executes the specific computer-executable instructions to further cause the device to at least determine that a size of the cluster of composite images satisfies an auto-approve threshold; and

wherein authorizing the input image is based at least in part on the prior authorization determination for the known composite image and determining that the size of the cluster of composite images satisfies the auto-approve threshold.

7. The device of claim 1, wherein the processor executes the specific computer-executable instructions to further cause the device to at least:

identify a set of images in which each image of the set of images has a composite detection result including an identifier for the known composite image;

transmit the set of images to an authorization server;

receive an authorization determination for an image included in the set of images; and

apply the authorization determination to at least one other image in the set of images.

8. The device of claim 1, wherein the processor further executes the specific computer-executable instructions to further cause the device to at least:

identify a set of images in which each image of the set of images has a feature signature matches the feature signature for the known composite image; and

generate a reconstructed underlying image using pixel color values from the set of images and the known composite image, wherein the reconstructed underlying image does not include a graphic overlay included in the overlaid image region.

9. The device of claim 8, wherein the processor further executes the specific computer-executable instructions to further cause the device to at least:

generate, on a pixel-by-pixel basis, for each pixel location, a mode color value of pixels at a respective pixel location in the set of images,

wherein the pixel color values include the mode color value of the pixels at the respective pixel location in the set of images.

10. The device of claim 8, wherein the processor further executes the specific computer-executable instructions to further cause the device to at least:

extract the overlaid image region from the input image based at least in part on the reconstructed underlying image;

identify, via an optical character recognition system, a word or phrase contained in the overlaid image region extracted from the input image; and

determine that the word or phrase is not included in a list of prohibited words or phrases, wherein authorizing the input image is based at least in part on the prior authorization determination for the known composite image and determining that the word or phrase is not included in the list of prohibited words or phrases.

11. A computer-implemented method comprising:

under control of a computing device executing specific computer-executable instructions,

receiving a feature signature for a known composite image;

receiving a prior authorization determination for the known composite image, the prior authorization determination indicating that the known composite image may be transmitted in response to a search request received from another electronic device by a catalog system;

receiving an input image to be authorized for transmission via the catalog system, the input image comprising an overlaid image region, an underlying image region showing an item upon which the overlaid image region is placed, and a background image region;

removing the background image region from the input image;

generating an input image feature signature indicative of a shape of the item shown in the underlying image region;

determining that the input image feature signature matches the feature signature; and

upon determining that the input feature signature matches the feature signature, authorizing the input image for transmission via the catalog system based at least in part on the prior authorization determination for the known composite image.

12. The computer-implemented method of claim 11, further comprising:

generating a segmented image, the segmented image comprising, for each pixel, one of a pixel-on value or a pixel-off value;

dividing the segmented image into sectors; and

for each sector of the segmented image, generating a count of pixel-on values within the sector, wherein the input image feature signature includes a vector of elements, wherein each element corresponds to a sector and the value for the element corresponds to the count of pixel-on values within the sector.

13. The computer-implemented method of claim 12, further comprising:

generating an indication of a color of the item shown in the underlying image region, wherein the input image feature signature includes a vector of elements, wherein an element corresponds to item color and the value for the element is the indication of the color of the item shown in the underlying image region.

14. The computer-implemented method of claim 13, wherein generating the indication of the color of the item shown in the underlying image region comprises:

extracting a contour around the underlying image region; and

generating a color value for the underlying image region using color channel information for pixels within the contour, wherein the color value is used as the value for the element corresponding to the item color.

15. The computer-implemented method of claim 13, further comprising:

identifying a hashing function for combining the indication of shape and the indication of color; and

applying the hashing function to combine the indication of shape and the indication of color to generate the feature signature.

16. The computer-implemented method of claim 11,

wherein the known composite image is included in a cluster of composite images;

wherein the computer-implemented method further comprises determining that a size of the cluster of composite images satisfies an auto-approve threshold; and

wherein authorizing the input image is based at least in part on the prior authorization determination for the known composite image and determining that the size of the cluster of composite images satisfies the auto-approve threshold.

17. The computer-implemented method of claim 11, further comprising:

identifying a set of images in which each image of the set of images has a composite detection result including an identifier for the known composite image;

transmitting the set of images to an authorization server;

receiving an authorization determination for an image included in the set of images; and

applying the authorization determination to at least one other image in the set of images.

18. The computer-implemented method of claim 11, further comprising:

identifying a set of images in which each image of the set of images has a feature signature matches the feature signature for the known composite image; and

generating a reconstructed underlying image using pixel color values from the set of images and the known composite image, wherein the reconstructed underlying image does not include a graphic overlay included in the overlaid image region.

19. The computer-implemented method of claim 18, further comprising:

extracting the overlaid image region from the input image based at least in part on the reconstructed underlying image;

identifying, via an optical character recognition system, a word or phrase contained in the overlaid image region extracted from the input image; and

determining that the word or phrase is not included in a list of prohibited words or phrases, wherein authorizing the input image is based at least in part on the prior authorization determination for the known composite image and determining that the word or phrase is not included in the list of prohibited words or phrases.

20. The computer-implemented method of claim 11, further comprising determining that a first number of pixels included in the input image corresponds to a second number of pixels included in the known composite image, wherein authorizing the input image is based at least in part on the prior authorization determination for the known composite image and determining that the first number of pixels included in the input image corresponds to the second number of pixels included in the known composite image.