Selective identification and order of image modifiers

• US 10,623,666 B2
• Filed: 11/07/2017
• Issued: 04/14/2020
• Est. Priority Date: 11/07/2016
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Charlton, Ebony James, Evans, Michael John, Hare, Samuel Edward, McPhee, Andrew James, Murphy, Robert Cornelius, Pilipski, Eitan
• Primary Examiner(s): Hannett, James M
• Application Number:
  US15/806,021
• Publication Number:
  US 20180131878A1
• Time in Patent Office: 889 Days
• Field of Search: None
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Systems, devices, media, and methods are presented for presentation of modified objects within a video stream. The systems and methods receive a selection at a user interface of a computing device and determine a modifier context based at least in part on the selection and a position within the user interface. The systems and methods identify at least one set of modifiers based on the modifier context. The systems and methods determine an order for the set of modifiers based on the modifier context and cause presentation of modifier icons for the set of modifiers within the user interface.

Claims

1. A method, comprising:

receiving a selection at a user interface of a computing device, the user interface comprising a field of view of an image capture device in communication with the computing device, the selection indicating a position within the user interface;

determining a modifier context based at least in part on one or more characteristics of the selection and the position within the user interface, the one or more characteristics of the selection comprising sensor data detected by the computing device contemporaneously to receiving the selection;

identifying at least one set of image modifiers based on the modifier context, the set of image modifiers comprising a modifier icon for each image modifier of the set of image modifiers and at least one modifier category for each image modifier of the set of image modifiers;

receiving an image modifier for incorporation into the set of image modifiers, the image modifier associated with a modifier category indicating a context indicator triggering identification of the image modifier, the context indicator being one or more of an object of interest and sensor data;

determining an order for the set of image modifiers based on the modifier context; and

causing presentation of the modifier icons for the set of image modifiers within the user interface.

2. The method of claim 1, further comprising:

in response to receiving the selection, generating a representation graphic indicating one or more of the position of the selection within the user interface and an object of interest identified within the field of view of the image capture device by the selection, the representation graphic including a mesh overlaid on the object of interest; and

determining the modifier context based on the selection, the position of the selection, and the object of interest identified by the selection.

3. The method of claim 1, wherein the selection indicates a position within the field of view of the image capture device depicted within the user interface, and the method further comprising:

determining one or more characteristics of one or more objects of interest depicted within the field of view of the image capture device;

determining an identification for each object of interest of the one or more objects of interest depicted within the field of view, the identifications based on the one or more characteristics; and

determining the modifier context based on the selection, the position of the selection within the user interface, and the identifications for the one or more objects of interest.

4. The method of claim 1, further comprising:

identifying one or more context indicators; and

determining the modifier context based on the selection, the position of the selection, and the one or more context indicators.

5. The method of claim 4, wherein the one or more context indicators is selected from a group consisting of a geolocation of the computing device;

a time of day;

a set of weather data;

a face depicted within the field of view;

a plurality of faces depicted within the field of view;

a specified person of a set of predetermined people depicted within the field of view;

a natural feature tracking identification;

a representative symbol identifying an entity, an event, or a product;

a sponsorship;

an orientation of the computing device;

a movement of the computing device;

an object type depicted within the field of view;

an unlock code applied to a specified modifier;

an image capture device position relative to the computing device;

a modifier popularity score; and

combinations thereof.

6. The method of claim 1, wherein the set of modifiers is a first set of modifiers, and the method further comprising:

determining a set of identifications for a set of objects of interest depicted within the field of view of the image capture device, each identification corresponding to an object of interest of the set of objects of interest;

identifying the first set of modifiers based on the modifier context and at least one identification of the set of identifications, the first set of modifiers having a first compatibility value indicating a compatibility of the first set of modifiers to at least one of the objects of interest of the set of objects of interest;

identifying a second set of modifiers based on the modifier context and at least one identification of the set of identifications, the second set of modifiers having a second compatibility value indicating a compatibility of the second set of modifiers to at least one of the objects of interest of the set of objects of interest;

determining an order for the first set of modifiers and the second set of modifiers, the order determined based on the modifier context and at least a portion of the identifications of the set of objects of interest; and

causing presentation of at least a portion of a first set of modifier icons and at least a portion of a second set of modifier icons, each modifier icon of the first set of modifier icons corresponding to a modifier of the first set of modifiers and each modifier icon of the second set of modifier icons corresponding to a modifier of the second set of modifiers.

7. The method of claim 6, wherein the second set of modifiers has a third compatibility value indicating a compatibility of the second set of modifiers with the at least one object of interest determined to be compatible with the first set of modifiers, the third compatibility value being less than the first compatibility value.

8. The method of claim 1, wherein the at least one modifier category comprises a first modifier category and a second modifier category, the first modifier category indicating a primary context indicator associated with the modifier, the second modifier category indicating a secondary context indicator associated with the modifier, the received image modifier being identified and displayed upon detection of one or more of the first modifier category and the second modifier category.

9. The method of claim 1, wherein the selection of at the user interface is a first selection, the method further comprising:

receiving a second selection of a modifier icon presented within the user interface;

determining a modifier category associated with the received image modifier represented by the modifier icon;

determining an object category associated with an object of interest used at least in part to determine the modifier context;

determining the modifier category corresponds to the object category; and

applying the modifier associated with the modifier icon indicated by the second selection.

10. The method of claim 9, wherein the selection at the user interface is a first selection, the method further comprising:

receiving a second selection of a modifier icon presented within the user interface;

determining a modifier category associated with the modifier represented by the modifier icon;

determining a first object category associated with a first object of interest and a second object category associated with a second object of interest, the first object of interest used at least in part to determine the modifier context;

determining the modifier category corresponds to the second object category; and

generating a representation graphic indicating the second object of interest identified within the field of view of the image capture device by the first selection, the representation graphic including a mesh overlaid on the second object of interest.

11. A device comprising:

one or more processors; and

a processor-readable storage device coupled to the one or more processors, the processor-readable storage device storing processor executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

receiving a selection at a user interface of a computing device, the user interface comprising a field of view of an image capture device in communication with the computing device, the selection indicating a position within the user interface;

determining a modifier context based at least in part on one or more characteristics of the selection and the position within the user interface, the one or more characteristics of the selection comprising sensor data detected by the computing device contemporaneously to receiving the selection;

identifying at least one set of image modifiers based on the modifier context, the set of image modifiers comprising a modifier icon for each image modifier of the set of image modifiers and at least one modifier category for each image modifier of the set of image modifiers;

receiving an image modifier for incorporation into the set of image modifiers, the image modifier associated with a modifier category indicating a context indicator triggering identification of the image modifier, the context indicator being one or more of an object of interest and sensor data;

determining an order for the set of modifiers based on the modifier context; and

causing presentation of the modifier icons for the set of modifiers within the user interface.

12. The device of claim 11, wherein the operations further comprise:

in response to receiving the selection, generating a representation graphic indicating one or more of the position of the selection within the user interface and an object of interest identified within the field of view of the image capture device by the selection, the representation graphic including a mesh overlaid on the object of interest; and

determining the modifier context based on the selection, the position of the selection, and the object of interest identified by the selection.

13. The device of claim 11, wherein the selection indicates a position within the field of view of the image capture device depicted within the user interface, and the operations further comprising:

determining one or more characteristics of one or more objects of interest depicted within the field of view of the image capture device;

determining an identification for each object of interest of the one or more objects of interest depicted within the field of view, the identifications based on the one or more characteristics; and

determining the modifier context based on the selection, the position of the selection within the user interface, and the identifications for the one or more objects of interest.

14. The device of claim 11, wherein the set of modifiers is a first set of modifiers, and the operations further comprising:

determining a set of identifications for a set of objects of interest depicted within the field of view of the image capture device, each identification corresponding to an object of interest of the set of objects of interest;

identifying the first set of modifiers based on the modifier context and at least one identification of the set of identifications, the first set of modifiers having a first compatibility value indicating a compatibility of the first set of modifiers to at least one of the objects of interest of the set of objects of interest;

identifying a second set of modifiers based on the modifier context and at least one identification of the set of identifications, the second set of modifiers having a second compatibility value indicating a compatibility of the second set of modifiers to at least one of the objects of interest of the set of objects of interest;

determining an order for the first set of modifiers and the second set of modifiers, the order determined based on the modifier context and at least a portion of the identifications of the set of objects of interest; and

causing presentation of at least a portion of a first set of modifier icons and at least a portion of a second set of modifier icons, each modifier icon of the first set of modifier icons corresponding to a modifier of the first set of modifiers and each modifier icon of the second set of modifier icons corresponding to a modifier of the second set of modifiers.

15. A processor-readable storage device storing processor executable instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

receiving a selection at a user interface of a computing device, the user interface comprising a field of view of an image capture device in communication with the computing device, the selection indicating a position within the user interface;

determining a modifier context based at least in part on one or more characteristics of the selection and the position within the user interface, the one or more characteristics of the selection comprising sensor data detected by the computing device contemporaneously to receiving the selection;

identifying at least one set of image modifiers based on the modifier context, the set of image modifiers comprising a modifier icon for each image modifier of the set of image modifiers and at least one modifier category for each image modifier of the set of image modifiers;

receiving an image modifier for incorporation into the set of image modifiers, the image modifier associated with a modifier category indicating a context indicator triggering identification of the image modifier, the context indicator being one or more of an object of interest and sensor data;

determining an order for the set of modifiers based on the modifier context; and

causing presentation of the modifier icons for the set of modifiers within the user interface.

16. The processor-readable storage device of claim 15, wherein the operations further comprise:

in response to receiving the selection, generating a representation graphic indicating one or more of the position of the selection within the user interface and an object of interest identified within the field of view of the image capture device by the selection, the representation graphic including a mesh overlaid on the object of interest; and

determining the modifier context based on the selection, the position of the selection, and the object of interest identified by the selection.

17. The processor-readable storage device of claim 15, wherein the selection indicates a position within the field of view of the image capture device depicted within the user interface, and the operations further comprising:

determining one or more characteristics of one or more objects of interest depicted within the field of view of the image capture device;

determining an identification for each object of interest of the one or more objects of interest depicted within the field of view, the identifications based on the one or more characteristics; and

determining the modifier context based on the selection, the position of the selection within the user interface, and the identifications for the one or more objects of interest.

18. The processor-readable storage device of claim 15, wherein the set of modifiers is a first set of modifiers, and the operations further comprising:

determining a set of identifications for a set of objects of interest depicted within the field of view of the image capture device, each identification corresponding to an object of interest of the set of objects of interest;

identifying the first set of modifiers based on the modifier context and at least one identification of the set of identifications, the first set of modifiers having a first compatibility value indicating a compatibility of the first set of modifiers to at least one of the objects of interest of the set of objects of interest;

identifying a second set of modifiers based on the modifier context and at least one identification of the set of identifications, the second set of modifiers having a second compatibility value indicating a compatibility of the second set of modifiers to at least one of the objects of interest of the set of objects of interest;

determining an order for the first set of modifiers and the second set of modifiers, the order determined based on the modifier context and at least a portion of the identifications of the set of objects of interest; and

causing presentation of at least a portion of a first set of modifier icons and at least a portion of a second set of modifier icons, each modifier icon of the first set of modifier icons corresponding to a modifier of the first set of modifiers and each modifier icon of the second set of modifier icons corresponding to a modifier of the second set of modifiers.

19. The processor-readable storage device of claim 15, wherein the selection of at the user interface is a first selection, the operations further comprise:

receiving a second selection of a modifier icon presented within the user interface;

determining a modifier category associated with the modifier represented by the modifier icon;

determining an object category associated with an object of interest used at least in part to determine the modifier context;

determining the modifier category corresponds to the object category; and

applying the modifier associated with the modifier icon indicated by the second selection.

20. The processor-readable storage device of claim 15, wherein the operations further comprise:

identifying one or more context indicators; and

determining the modifier context based on the selection, the position of the selection, and the one or more context indicators.