Object recognition based photo filters

• US 9,754,355 B2
• Filed: 01/09/2015
• Issued: 09/05/2017
• Est. Priority Date: 01/09/2015
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Chang, Sheldon, Samaranayake, Chamal, Sehn, Timothy Michael, Yan, Rong
• Primary Examiner(s): CONNER, SEAN M
• Application Number:
  US14/593,065
• Publication Number:
  US 20160203586A1
• Time in Patent Office: 970 Days
• Field of Search: None
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/538 Presentation of query results
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56

Abstract

Systems and methods for generating and distributing photo filters are described. A photo filter publication application receives filter data and object criteria and generates a photo filter based on the filter data. The photo filter is associated with satisfaction of the object criteria. A photo filter engine then identifies that a client device has taken a photograph. The photo filter engine then provides the photo filter to the client device based on the photograph including an object that satisfies the object criteria. The photo filter may then be displayed as an option on a user interface of the client device. The object criteria may include associations between an object and a source of image data, for example, a brand of a merchant in which case the associated photo filter may include images associated with the brand of the merchant.

Claims

1. A system comprising:

a processor; and

a memory coupled to the processor, the memory including instructions which, when executed by the processor, cause the system to;

access filter data and object criteria,

generate a photo filter using the filter data;

store data specifying an association between the photo filter and the object criteria;

detect that a client device of the server has taken a photograph;

identify an object in the photograph;

based on the identified object satisfying the object criteria associated with the photo filter in the stored data, provide the photo filter to the client device;

store photo filters provided to the client device in a photo filter collection associated with the client device;

determine that the photo filter collection includes more than a specified number of stored photo filters of a specified type, and

provide a new photo filter to the client device in response to the determination.

2. The system of claim 1, wherein the object criteria includes an association between an object and a media property or a work of art and the associated photo filter includes images associated with the media property or work of art.

3. The system of claim 1, wherein the object criteria includes an association between an object and a brand of a merchant and the associated photo filter includes images associated with the brand of the merchant.

4. The system of claim 1, wherein the object criteria includes an association between an object and a discount associated with a type of the object at a local merchant establishment and the associated photo filter includes images comprising information about the discount and the merchant establishment.

5. The system of claim 1, wherein the object criteria includes an association between an object and educational information about the object and the associated photo filter includes images comprising the educational information.

6. The system of claim 1, wherein the object criteria includes recognizing an object as an animal and the associated photo filter includes images associated with a safari.

7. The system of claim 1, wherein the object criteria includes recognizing an object as a human shoulder and the associated photo filter includes images associated with pets perched on the human shoulder.

8. The system of claim 1, further comprising instructions which, when executed by the processor, cause the system to receive the filter data and the object criteria from a user.

9. The system of claim 8, wherein the photo filter comprises a plurality of photo filters, the system further comprising instructions which, when executed by the processor, cause the system to:

generate a ranking of the plurality of photo filters associated with the object criteria based on specified photo filter priority criteria; and

provide a specified number of the plurality of photo filters to the client device according to the ranking of the plurality of photo filters.

10. The system of claim 9, wherein the photo filter priority criteria includes criteria based on a filter creation date, a filter type or a user ranking of the filter.

11. The system of claim 1, further comprising instructions which, when executed by the processor, cause the system to:

receive first filter data from a first merchant and second filter data from a second merchant;

receive first object criteria from the first merchant, and second object criteria from the second merchant;

identify object criteria common to both the first object criteria and the second object criteria;

receive a first bid amount from the first merchant and a second bid amount from the second merchant;

identify a largest bid amount of the first and second bid amounts;

generate a merchant photo filter using the first or second filter data based on the identified largest bid amount;

associate the merchant photo filter with the common object criteria; and

supply the merchant photo filter to the client device based on the identified object satisfying the common object criteria.

12. The system of claim 11, wherein the common object criteria includes a type of object for which the first merchant and the second merchant sell branded products of the same type.

13. The system of claim 1, further comprising instructions which, when executed by the processor, cause the system to:

generate a count of objects, of a specified object type, that have been identified in photographs taken by the client device and adjust a content of the photo filter in response to the count reaching a specified threshold value.

14. The system of claim 1, further comprising instructions which, when executed by the processor, cause the system to:

receive video data and object criteria;

generate a video based on the video data and the photograph;

associate the video with the object criteria; and

based on the identified object satisfying the object criteria, provide the video to the client device.

19. The system of claim 1, wherein the specified type comprises a photo filter associated with specified object criteria.

20. The system of claim 10, wherein:

the specified object criteria comprise object criteria associated with at least one menu item at a merchant establishment; and

the new photo filter provides information regarding a discount associated with the at least one menu item at the merchant establishment.

15. A method implemented by a server including at least one processor and comprising:

accessing filter data and object criteria;

generating a photo filter with the filter data;

storing data specifying an association between the photo filter and the object criteria;

detecting that a client device of the server has taken a photograph;

identifying an object in the photograph;

providing the photo filter to the client device based on the identified object satisfying the object criteria associated with the photo filter;

providing an option to display the provided photo filter on a user interface of the client device;

storing photo filters provided to the client device in a photo filter collection associated with the client device;

determining that the photo filter collection includes more than a specified number of stored photo filters of a specified type, and

providing a new photo filter to the client device in response to the determining.

16. The method of claim 15, further comprising:

receiving first filter data and first object criteria from a first merchant and second filter data and second object criteria from a second merchant;

identifying common object criteria between the first object criteria and the second object criteria;

receiving a first bid amount from the first merchant and a second bid amount from the merchant;

identifying a largest bid amount of the first and second bid amounts;

generating a merchant photo filter with the first or second filter data based on the identified largest bid amount

associating the merchant photo filter with the common object criteria,

supplying the merchant photo filter to the client device based on the identified object satisfying the common object criteria.

17. The method of claim 16, wherein the common object criteria includes a type of object for which the first merchant and the second merchant sell branded products of the same type.

18. A non-transitory computer-readable storage medium storing a set of instructions that, when executed by a processor of a machine, cause the machine to perform operations comprising:

accessing filter data and object criteria;

generating a photo filter with the filter data;

storing data specifying an association between the photo filter and the object criteria;

determining that a client device of the server has taken a photograph;

identifying an object in the photograph;

providing the photo filter to the client device based on the identified object satisfying the object criteria associated with the photo filter; and

providing an option to display the provided photo filter on a user interface of the client device;

storing photo filters provided to the client device in a photo filter collection associated with the client device;

determining that the photo filter collection includes more than a specified number of stored photo filters of a specified type, and

providing a new photo filter to the client device in response to the determining.