Generating moments

• US 10,318,574 B1
• Filed: 03/16/2015
• Issued: 06/11/2019
• Est. Priority Date: 03/16/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Bonechi, Marco, Lauritzen, Christopher
• Primary Examiner(s): Bashore, William L
• Assistant Examiner(s): Parcher, Daniel W
• Application Number:
  US14/659,283
• Time in Patent Office: 1,548 Days
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/447 Temporal browsing, e.g. timeline
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Implementations generally relate to generating moments. In some implementations, a method includes generating a moment, where the moment includes one or more visual content items. The method further includes determining one or more other types of content items in response to the generating of the moment. The method further includes associating the one or more other types of content items with the moment.

Claims

1. A computer-implemented method comprising:

generating a moment, wherein the moment includes visual content items, wherein (1) the visual content items include a first image of a first person captured by a first lens of a camera and a second image of a second person captured by a second lens of the camera, (2) the first lens and the second lens are on opposite sides of the camera, and (3) the first image is captured substantially simultaneously to when a command is received from the first person to capture the second image;

determining two or more other types of content items in response to the generating of the moment, wherein the two or more other types of content items comprise (a) at least one audio content item that was created during a time interval associated with a time of capture of the visual content items of the moment and (b) a temperature content item that is determined, by one or more processors, based on an object in the second image being associated with a temperature;

associating the two or more other types of content items with the moment;

rendering a first user interface that enables user identification of the moment;

upon receipt of the user identification of the moment, accessing a first set of content items associated with the moment in a moment data structure, wherein the first set of content items includes the visual content items and the one or more other types of content items;

determining a first set of content item types associated with the first set of content items; and

dynamically configuring and rendering a second user interface for presentation of the first set of content items, wherein;

the configuring of the second user interface is based on the first set of content item types, the second user interface includes the first image of the first person and the second image of the second person, and two or more types of content items are played concurrently in the second user interface.

2. The method of claim 1, wherein the one or more other types of content items include a news story of an event that occurred at a time the moment was generated.

3. The method of claim 1, wherein the temperature content item includes an identification of a cold temperature based on the object being associated with the cold temperature.

4. The method of claim 1, wherein:

the one or more other types of content items include a location associated with the moment; and

the second user interface includes a map of the location associated with the moment.

5. The method of claim 1, wherein the one or more other types of content items comprise geographic information, and wherein the geographic information is associated with a geographic region where at least one of the visual content items of the moment was captured.

6. The method of claim 1, wherein the determining of the at least one audio content item starts when a preview of one or more of the visual content items is detected.

7. The method of claim 1, wherein the first set of content item types comprises a photo content item type, a video content item type, and an audio content item type, and wherein dynamically configuring the second user interface comprises:

displaying one or more of an image viewer and a video player on the second user interface; and

activating an audio player concurrently with presentation of the second user interface.

8. The method of claim 1, wherein the generating of the moment is triggered by a detection of a photo capture process associated with one or more of the visual content items by a device with the camera and a display screen, and wherein the detection of the photo capture process comprises detecting a preview of one or more of the visual content items based on the device framing an object in the display screen.

9. The method of claim 8, further comprising indexing the moment based on one or more objects detected by one or more sensors of the device, wherein the one or more objects include one or more of:

a person, a sound, geographic information, and time, and wherein the first user interface enables user identification of the moment based on searches based on one or more of the objects.

10. The method of claim 1, wherein the two or more types of content items include a photo content item type and a video content item type, and wherein the second user interface includes a photo viewer and a video player separate from the photo viewer.

11. The method of claim 10, wherein the two or more types of content items further include an audio content item type, the method further comprising:

activating an audio player concurrently with the rendering of the second user interface.

12. A non-transitory computer-readable storage medium carrying program instructions thereon, the instructions when executed by one or more processors cause the one or more processors to perform operations comprising:

generating a moment, wherein the moment includes visual content items and wherein (1) the generating of the moment is triggered by a detection of a photo capture process associated with one or more of the visual content items by a device, (2) the visual content items include a first image of a first person captured by a first lens of the device and a second image of a second person captured by a second lens of the device, and (3) the first lens and the second lens are on opposite sides of the device;

determining two or more other types of content items in response to the generating of the moment, wherein the two or more other types of content items comprise (a) at least one audio content item that was created during a time interval associated with a time of capture of the visual content items of the moment and (b) a temperature content item that is determined, by one or more processors, based on an object in the second image being associated with a temperature;

associating the two or more other types of content items with the moment;

indexing the moment based on one or more objects detected by one or more sensors of the device;

rendering a first user interface that enables user identification of the moment based on searches based on one or more of the objects;

upon receipt of the user identification of the moment, accessing a first set of content items associated with the moment in a moment data structure, wherein the first set of content items includes the visual content items and the one or more other types of content items;

determining a first set of content item types associated with the first set of content items; and

dynamically configuring and rendering a second user interface for presentation of the first set of content items, wherein the configuring of the second user interface is based on the first set of content item types and the second user interface includes the first image of the first person and the second image of the second person.

13. The computer-readable storage medium of claim 12, wherein the detection of the photo capture process comprises detecting a preview of one or more of the visual content items.

14. The computer-readable storage medium of claim 12, wherein:

the one or more other types of content items include a location associated with the moment; and

the second user interface includes a map of the location associated with the moment.

15. The computer-readable storage medium of claim 12, wherein the one or more other types of content items comprise geographic information, and wherein the geographic information is associated with a geographic region where at least one of the visual content items of the moment was captured.

16. The computer-readable storage medium of claim 12, wherein the determining of the at least one audio content item starts when a preview of one or more of the visual content items is detected.

17. A system comprising:

one or more processors; and

logic encoded in one or more tangible media for execution by the one or more processors and when executed operable to perform operations comprising;

generating a moment, wherein the moment includes visual content items, wherein (1) the visual content items include a first image of a first person captured by a first lens of a camera and a second image of a second person captured by a second lens of the camera, (2) the first lens and the second lens are on opposite sides of the camera, and (3) the first image is captured substantially simultaneously to when a command is received from the first person to capture the second image;

determining two or more other types of content items in response to the generating of the moment, wherein the two or more other types of content items comprise (a) at least one audio content item that was created during a time interval associated with a time of capture of the visual content items of the moment and (b) a temperature content item that is automatically determined based on an object in the second image being associated with a temperature;

associating the one or more other types of content items with the moment;

rendering a first user interface that enables user identification of the moment;

upon receipt of the user identification of the moment, accessing a first set of content items associated with the moment in a moment data structure, wherein the first set of content items includes the visual content items and the one or more other types of content items;

determining a first set of content item types associated with the first set of content items; and

dynamically configuring and rendering a second user interface for presentation of the first set of content items, wherein the configuring of the second user interface is based on the first set of content item types, the second user interface includes the first image of the first person and the second image of the second person, and two or more types of content items are played concurrently in the second user interface.

18. The system of claim 17, wherein the generating of the moment is triggered by a detection of a photo capture process associated with one or more of the visual content items.

19. The system of claim 17, wherein:

the one or more other types of content items include a location associated with the moment; and

the second user interface includes a map of the location associated with the moment.

20. The system of claim 17, wherein the one or more other types of content items comprise geographic information, and wherein the geographic information is associated with a geographic region where at least one of the visual content items of the moment was captured.