Generation of smart tags to locate elements of content

• US 8,589,402 B1
• Filed: 08/21/2008
• Issued: 11/19/2013
• Est. Priority Date: 08/21/2008
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Iampietro, Michael, Darbari, Abhinav, P B, Ramesh, Yadav, Ram Narain, Singh, Arvinder
• Primary Examiner(s): SOMERS, MARC S
• Application Number:
  US12/196,127
• Time in Patent Office: 1,916 Days
• Field of Search: 707/736, 707/741, 707/752, 707/758, 707/914, 707/915
• US Class Current: 707/741
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information

Abstract

Generating smart tags that allow a user to locate any portion of image content without viewing the image content is disclosed. Image-based processing is performed on image content to find an event of interest that is an occurrence captured by the image content. Thus, metadata is derived from analyzing the image content. The metadata is then analyzed. Different types of characteristics associated with portions of the image content as indicated by the metadata are detected. Responsive to this, tags are created, and different types of tags are applied to the portions of image content to categorize the portions into classes. Thus, a tag is associated with each portion of the image content including the event of interest. The tag describes a characteristic of that portion of the image content. Display of the different types of tags is initiated for selective viewing of the portions of the image content.

Claims

1. A method comprising:

receiving, by a processor, an input describing an event of interest, wherein the event of interest comprises at least one quality parameter associated with a specified range of values for the quality parameter and at least one of a specified person or a specified object;

responsive to receiving the input describing the event of interest, analyzing, by the processor, a plurality of images depicting a motion with respect to the specified person or the specified object to determine which of the plurality of images include the event of interest, wherein analyzing the plurality of images comprises;

for each of the plurality of images, generating a respective value for the at least one quality parameter based on a comparison of at least one image of the plurality of images with at least another image of the plurality of images, and

identifying a subset of images of the plurality of images depicting the motion for which the respective value is within the specified range; and

based on analyzing the plurality of images, generating, by the processor, a respective tag for each image of the subset of images including the event of interest.

2. The method of claim 1, wherein receiving the input describing the event of interest comprises the input identifying the event of interest in a first image and wherein analyzing the plurality of images comprises determining that the event of interest is in a second image different from the first image.

3. The method of claim 1, wherein the event of interest further comprises one or more physical characteristics of the least one of the specified person or the specified object.

4. The method of claim 1, wherein the event of interest further comprises a position in a respective image of the least one of the specified person or the specified object.

5. The method of claim 1, wherein the event of interest further comprises at least one additional quality parameter different from the at least one quality parameter and wherein analyzing the plurality of images comprises determining which of the plurality of images includes image content having an additional respective value for the at least one additional quality parameter that is within an additional specified range.

6. The method of claim 5, wherein the wherein the at least one quality parameter and the at least one additional quality parameter each comprises at least one of panning, zooming, camera tilting, focus, shakiness of camera, brightness, or contrast.

7. The method of claim 1, wherein the plurality of images comprises video content.

8. The method of claim 7, wherein generating the respective tag for each image of the subset of images comprises generating a first respective tag for a first subset of images in the video content having the event of interest and generating a second respective tag for a second subset of images in the video content, wherein one or more intervening images in the video content between the first subset of images and the second subset of images lacks the event of interest and further comprising:

receiving, by the processor, additional input comprising a command to initiate display the video content having the event of interest;

initiating display of the first subset of images and the second subset of images consecutively by omitting the one or more intervening images from display.

9. The method of claim 1, wherein generating the respective tag for each image of the subset of images comprises, for each image of the subset of images, generating a tag for each portion of the image having image content corresponding to the event of interest.

10. The method of claim 1, further comprising receiving, by the processor, search parameters specifying retrieval of one or more images associated with a tag indicating that the one or more images includes the event of interest;

and

in response to receiving the search parameters, initiating, by the processor, display of the one or more images.

11. The method of claim 10, wherein each tag includes data identifying a respective point in time associated with each of the one or more images and wherein initiating display of the one or more images comprises:

initiating display of a timeline associated with the one or more images; and

populating the timeline with one or more tags corresponding to the one or more images to indicate relative points in time associated with the one or more images including the event of interest.

12. The method of claim 1, further comprising selecting the plurality of images for analysis based on at least some of the plurality of images corresponding to audio data associated with the event of interest.

13. The method of claim 1, further comprising determining that the plurality of images depicts the motion with respect to the specified person or the specified object.

14. A non-transitory computer-readable medium having instructions stored thereon for processing data information, such that the instructions, when carried out by a processing device, enable the processing device to perform operations of:

receiving an input describing an event of interest, wherein the event of interest comprises at least one quality parameter associated with a specified range of values for the quality parameter and at least one of a specified person or a specified object;

responsive to receiving the input describing the event of interest, analyzing a plurality of images depicting a motion with respect to the specified person or the specified object to determine which of the plurality of images include the event of interest identified by the input, wherein analyzing the plurality of images comprises;

for each of the plurality of images, generating a respective value for the at least one quality parameter based on a comparison of at least one image of the plurality of images with at least another image of the plurality of images, and

identifying a subset of images of the plurality of images depicting the motion for which the respective value is within the specified range; and

based on analyzing the plurality of images, generating a respective tag for each image of the subset of images including the event of interest.

15. The non-transitory computer-readable medium of claim 14, wherein receiving the input identifying the event of interest comprises the input identifying the event of interest in a first image and wherein analyzing the plurality of images comprises determining that the event of interest is in a second image different from the first image.

16. The non-transitory computer-readable medium of claim 14, wherein the event of interest further comprises one or more physical characteristics of the least one of the specified person or the specified object.

17. The non-transitory computer-readable medium of claim 14, wherein the event of interest further comprises a position in a respective image of the least one of the specified person or the specified object.

18. The non-transitory computer-readable medium of claim 14, further comprising instructions that enable the processing device to perform additional operations comprising:

receiving search parameters specifying retrieval of one or more images associated with a tag indicating that the one or more images includes the event of interest; and

in response to receiving the search parameters;

initiating display of a timeline associated with the one or more images;

populating the timeline with one or more tags corresponding to the one or more images to indicate relative points in time associated with the one or more images including the event of interest; and

initiating display of the one or more images associated with the timeline.

19. The non-transitory computer-readable medium of claim 18, wherein the event of interest further comprises at least one additional quality parameter different from the at least one quality parameter and wherein analyzing the plurality of images comprises determining which of the plurality of images includes image content having an additional respective value for the at least one additional quality parameter that is within an additional specified range.

20. The non-transitory computer-readable medium of claim 19, wherein the wherein the at least one quality parameter and the at least one additional quality parameter each comprises at least one of panning, zooming, camera tilting, focus, shakiness of camera, brightness, or contrast.

21. The non-transitory computer-readable medium of claim 14, wherein the input comprises audiovisual content and wherein analyzing the plurality of images comprises, for each image of the plurality of images, determining whether image content of the image includes at least part of the audiovisual content.

22. The non-transitory computer-readable medium of claim 14, wherein analyzing the plurality of images comprises, for each image of the plurality of images, analyzing a coloration of pixels in the image.

23. A computer system comprising:

a processor;

a memory unit that stores instructions associated with an application executed by the processor;

a display; and

an interconnect coupling the processor, the memory unit, and the display, enabling the computer system to execute the application and perform operations of;

receiving an input describing an event of interest, wherein the event of interest comprises at least one quality parameter associated with a specified range of values for the quality parameter and at least one of a specified person or a specified object;

responsive to receiving the input describing the event of interest, analyzing a plurality of images depicting a motion with respect to the specified person or the specified object to determine which of the plurality of images include the event of interest, wherein analyzing the plurality of images comprises;

for each of the plurality of images, generating a respective value for the at least one quality parameter based on a comparison of at least one image of the plurality of images with at least another image of the plurality of images, and

identifying a subset of images of the plurality of images depicting the motion for which the respective value is within the specified range; and

based on analyzing the plurality of images, generating a respective tag for each image of the subset of images including the event of interest.

24. The computer system of claim 23, wherein the event of interest further comprises one or more physical characteristics of the least one of the specified person or the specified object.

25. The computer system of claim 24, wherein the event of interest comprises at least one person and the one or more physical characteristics comprises, for the least one person, include one or more of hair color or eye color.

26. The computer system of claim 23, wherein the event of interest comprises a first person or object and a second person or object and wherein generating the respective tag for each image of the subset of images comprises generating a first tag of a first tag type corresponding to the first person or object and generating a second tag of a second tag type corresponding to the second person or object.

27. The computer system of claim 23, wherein the event of interest further comprises a first parameter and a second parameter and wherein analyzing the plurality of images to determine which of the plurality of images include the event of interest comprises determining that a respective image has a first respective value for the first parameter in a first range and a second respective value for the second parameter in a second range.

28. The computer system of claim 23, wherein the event of interest further comprises at least one additional quality parameter different from the at least one quality parameter and wherein analyzing the plurality of images comprises determining which of the plurality of images includes image content having an additional respective value for the at least one additional quality parameter that is within an additional specified range.