Geo-location-based image filters

• US 10,157,449 B1
• Filed: 07/27/2017
• Issued: 12/18/2018
• Est. Priority Date: 01/09/2015
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Chang, Sheldon, Samaranayake, Chamal, Sehn, Timothy Michael, Yan, Rong
• Primary Examiner(s): Conner, Sean M
• Application Number:
  US15/661,966
• Time in Patent Office: 509 Days
• Field of Search: None
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/538 Presentation of query results
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56

Abstract

Systems and methods for generating and distributing photo filters are described. A photo filter publication application receives filter data and object criteria and generates a photo filter based on the filter data. The photo filter is associated with satisfaction of the object criteria. A photo filter engine then identifies that a client device has taken a photograph. The photo filter engine then provides the photo filter to the client device based on the photograph including an object that satisfies the object criteria. The photo filter may then be displayed as an option on a user interface of the client device. The object criteria may include associations between an object and a source of image data, for example, a brand of a merchant in which case the associated photo filter may include images associated with the brand of the merchant.

Claims

1. A system comprising:

a processor; and

memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising;

detecting that a client device in communication with the system has captured an image;

identifying an object in the image;

identifying a location of the object in the image;

identifying an image overlay associated with the identified location and having object criteria satisfied by the identified object;

providing the identified image overlay to the client device;

storing the provided image overlay in a collection of image overlays associated with the client device;

determining that the collection of image overlays contains more than a specified number of stored image overlays of a specified type; and

in response to determining that the collection of image overlays contains more than the specified number of stored image overlays of the specified type, providing a new image overlay to the client device.

2. The system of claim 1, wherein the object in the image is in a fixed location, and wherein identifying the location of the object in the image includes determining the fixed location.

3. The system of claim 2, wherein identifying the location of the object in the image includes:

determining a location of the client device relative to the fixed location of the object;

determining a perspective of the client device relative to the object based on the location of the client device and the fixed location of the object.

4. The system of claim 3, wherein identifying the image overlay is based on the perspective of the client device relative to the object.

5. The system of claim 3, wherein determining the location of the client device is based on geo-location information for the client device.

6. The system of claim 5, wherein the geo-location information for the client device is contained in metadata associated with a message transmitted from the client device to the system.

7. The system of claim 1, wherein identifying the object in the image includes determining the object is a monument.

8. The system of claim 7, wherein the identified image overlay includes viewable information associated with the monument.

9. The system of claim 1, wherein providing the identified image overlay to the client device includes attaching the identified image overlay to an ephemeral message accessible for a predetermined duration of time.

10. The system of claim 9, wherein providing the identified image overlay to the client device further includes:

receiving a request from the client device to access the ephemeral message;

in response to the request, causing the ephemeral message to be displayed on the client device; and

ceasing display of, and access to, the ephemeral message in response to expiration of the predetermined duration of time.

11. The system of claim 1, wherein identifying the object in the image is based on one or more of:

edge matching, grey-scale matching, large model bases, and gradient matching.

12. The system of claim 1, wherein the memory further stores instructions for causing the system to generate an image overlay for transforming the image into a video based on identifying the object in the image.

13. The system of claim 12, wherein transforming the image into the video includes adding motion to one or more of:

the identified object in the image, and another object in the image.

14. The system of claim 1, wherein the object criteria for the identified image overlay includes a number of object types.

15. The system of claim 14, wherein the identified object satisfying the object criteria includes the number of object types in a plurality of images captured by the client device meeting or exceeding a predetermined threshold.

16. The system of claim 1, wherein the memory further stores instructions for causing the system to perform operations comprising:

accessing filter data and object criteria;

generating an image overlay based on the filter data; and

storing data specifying an association between the image overlay and the object criteria.

17. The system of claim 16, wherein the generated image overlay is added to a collection of image overlays from which the identified image overlay is selected.

18. The system of claim 16, wherein the filter data and object criteria are received from the client device.

19. A computer-implemented method comprising:

detecting, by a computer system, that a client device in communication with the computer system has captured an image;

identifying, by the computer system, an object in the image;

identifying, by the computer system, a location of the object in the image;

identifying, by the computer system, an image overlay associated with the identified location and having object criteria satisfied by the identified object;

providing, by the computer system, the identified image overlay to the client device;

storing, by the computer system, the provided image overlay in a collection of image overlays associated with the client device;

determining, by the computer system, that the collection of image overlays contains more than a specified number of stored image overlays of a specified type; and

in response to determining that the collection of image overlays contains more than the specified number of stored image overlays of the specified type, providing, by the computer system, a new image overlay to the client device.

20. A non-transitory computer-readable medium storing instructions that, when executed by a computer system, cause the computer system to perform operations comprising:

detecting that a client device in communication with the system has captured an image;

identifying an object in the image;

identifying a location of the object in the image;

identifying an image overlay associated with the identified location and having object criteria satisfied by the identified object;

providing the identified image overlay to the client device;

storing the provided image overlay in a collection of image overlays associated with the client device;

determining that the collection of image overlays contains more than a specified number of stored image overlays of a specified type; and

in response to determining that the collection of image overlays contains more than the specified number of stored image overlays of the specified type, providing a new image overlay to the client device.