OBJECT RECOGNITION BASED PHOTO FILTERS

• US 20160203586A1
• Filed: 01/09/2015
• Published: 07/14/2016
• Est. Priority Date: 01/09/2015
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Samaranayake, Chamal, Chang, Sheldon, Sehn, Timothy Michael, Yan, Rong
• Granted Patent: US 9,754,355 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/538 Presentation of query results
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56

Abstract

Systems and methods for generating and distributing photo filters are described. A photo filter publication application receives filter data and object criteria and generates a photo filter based on the filter data. The photo filter is associated with satisfaction of the object criteria. A photo filter engine then identifies that a client device has taken a photograph. The photo filter engine then provides the photo filter to the client device based on the photograph including an object that satisfies the object criteria. The photo filter may then be displayed as an option on a user interface of the client device. The object criteria may include associations between an object and a source of image data, for example, a brand of a merchant in which case the associated photo filter may include images associated with the brand of the merchant.

Claims

1. A server comprising:

a photo filter publication module comprising one or more processors and configured to;

access filter data and object criteria,

generate at least one photo filter using the filter data, and

associate the at least one generated photo filter with the object criteria; and

a photo filter engine comprising one or more processors and configured to;

identify determine that a client device of the server has taken a photograph,

identify at least one object in the photograph; and

based on at least one of the identified objects satisfying the object criteria associated with the at least one generated photo filter, provide at least one of the generated photo filters to the client device.

2. The server of claim 1, wherein the photo filter engine further comprises an object recognition module configured to identify objects in the photograph and compare each object against the object criteria.

3. The server of claim 2, wherein the object criteria includes an association between an object and a media property or a work of art and the at least one associated photo filter includes images associated with the media property or work of art.

4. The server of claim 2, wherein the object criteria includes an association between an object and a brand of a merchant and the at least one associated photo filter includes images associated with the brand of the merchant.

5. The server of claim 2, wherein the object criteria includes an association between an object and a discount associated with a type of the object at a local merchant establishment and the at least one associated photo filter includes images comprising information about the discount and the merchant establishment.

6. The server of claim 2, wherein the object criteria includes an association between an object and educational information about the object and the at least one associated photo filter includes images comprising the educational information.

7. The server of claim 2, wherein the object criteria includes recognizing an object as an animal and the at least one associated photo filter includes images associated with a safari.

8. The server of claim 2, wherein the object criteria includes recognizing an object as a human shoulder and the at least one associated photo filter includes images associated with pets perched on the human shoulder.

9. The server of claim 2, wherein the photo filter publication module comprises a user-based filter data upload module configured to receive the filter data from a user and a user-based object criteria upload module configured to receive the object criteria from the user.

10. The server of claim 9, wherein:

the at least one generated photo filter comprises a plurality of photo filters;

the photo filter engine further comprises a photo filter priority module configured to generate a ranking of the plurality of photo filters associated with object criteria based on specified photo filter priority criteria; and

the photo filter engine is configured provide a specified number of the plurality of photo filters to the client device according to the ranking of the plurality of photo filters.

11. The server of claim 10, wherein the photo filter priority criteria includes criteria based on a filter creation date, a filter type or a user ranking of the filter.

12. The server of claim 2, wherein the photo filter publication module comprises:

a merchant-based filter data upload module configured to receive first filter data from a first merchant and a second filter data from a second merchant;

a merchant-based object criteria upload module configured to receive first object criteria from the first merchant, and second object criteria from the second merchant, and to identify object criteria common to both the first object criteria and the second object criteria;

a merchant-based bidding module configured to receive a first bid amount from the first merchant and a second bid amount from the second merchant, and to identify a largest bid amount of the first and second bid amounts;

wherein the photo filter publication module is configured to generate a merchant photo filter using the first or second filter data based on the identified largest bid amount and associate the merchant photo filter with the common object criteria; and

wherein the photo filter engine is configured to supply the merchant photo filter to the client device based on at least one of the identified objects satisfying the common object criteria.

13. The server of claim 12, wherein the common object criteria includes a type of object for which the first merchant and the second merchant sell branded products of the same type.

14. The server of claim 1, wherein the photo filter engine further comprises a collection module configured to store photo filters previously provided to the client device in a photo filter collection associated with the client device and provide a new photo filter to the client device in response to the photo filter collection including a specified number of stored photo filters of a specified type.

15. The server of claim 1, wherein the photo filter engine further comprises a count module configured to generate a count of objects, of a specified object type, that have been identified in photographs taken by the client device and adjust a content of at least one of the generated photo filters in response to the count reaching a specified threshold value.

16. The server of claim 1, wherein the photo filter publication module further comprises a video-creation module configured to:

receive video data and object criteria,

generate a video based on the video data and the photograph, and

associate the generated video with the object criteria; and

the photo filter engine configured to, based on at least one of the identified objects satisfying the object criteria, provide the generated video to the client device.

17. A method implemented by a server including at least one processor and comprising:

accessing filter data and object criteria;

generating at least one photo filter with the filter data;

associating the at least one generated photo filter with the object criteria;

determining that a client device of the server has taken a photograph;

identifying at least one object in the photograph;

providing at least one of the generated photo filters to the client device based on at least one the identified objects satisfying the object criteria associated with the at least one generated photo filter; and

providing an option to display the at least one provided photo filter on a user interface of the client device.

18. The method of claim 17, further comprising:

receiving first filter data and first object criteria from a first merchant and second filter data and second object criteria from a second merchant;

identifying common object criteria between the first object criteria and the second object criteria;

receiving a first bid amount from the first merchant and a second bid amount from the merchant;

identifying a largest bid amount of the first and second bid amounts;

generating a merchant photo filter with the first or second filter data based on the identified largest bid amount

associating the merchant photo filter with the common object criteria,

supplying the merchant photo filter to the client device based on at least one of the identified objects that satisfying the common object criteria.

19. The method of claim 18, wherein the common object criteria includes a type of object for which the first merchant and the second merchant sell branded products of the same type.

20. A non-transitory computer-readable storage medium storing a set of instructions that, when executed by a processor of a machine, cause the machine to perform operations comprising:

accessing filter data and object criteria;

generating at least one photo filter with the filter data;

associating the at least one generated photo filter with the object criteria;

determining that a client device of the server has taken a photograph;

identifying at least one object in the photograph;

providing at least one of the generated photo filters to the client device based on at least one of the identified objects satisfying the object criteria associated with the at least one generated photo filter; and

providing an option to display the at least one provided photo filter on a user interface of the client device.