Facilitating computer-assisted tagging of object instances in digital images

• US 7,587,101 B1
• Filed: 02/28/2005
• Issued: 09/08/2009
• Est. Priority Date: 02/28/2005
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Bourdev, Lubomir B.
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US11/069,692
• Time in Patent Office: 1,653 Days
• Field of Search: 382/132, 382/181, 382/205, 382/291, 382/107, 382/275, 382/305, 382/312, 707/6, 707/100
• US Class Current: 382/291
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06V 10/987 with the intervention of an operator
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

One embodiment of the present invention provides a system that facilitates computer-assisted tagging of objects in a digital image. During operation, the system receives locations for one or more objects-of-interest in the digital image. Next, the system determines likelihoods of specific tags being assigned to the objects-of-interest. The system then automatically assigns tentative tags to the objects-of-interest based on the determined likelihoods. Next, the system displays the assignments of tentative tags to a user, and receives corrections to the assignments, if any, from the user.

Claims

1. A method for facilitating computer-assisted tagging of objects in a digital image, comprising:

using a computer to perform;

receiving locations for one or more objects-of-interest in the digital image;

automatically determining likelihoods of specific tags being assigned to the objects-of-interest;

automatically assigning tentative tags to the objects-of-interest based on the determined likelihoods;

displaying, on the computer, the assignments of tentative tags to a user; and

receiving corrections to the assignments from the user.

2. The method of claim 1, wherein the digital image includes a set of related digital images.

3. The method of claim 1, wherein the digital image comprises a portion of a video image sequence.

4. The method of claim 1, wherein receiving the locations for the one or more objects-of-interest involves:

receiving the digital image; and

performing an object detection operation on the digital image to detect the objects-of-interest and to determine the locations of the objects-of-interest.

5. The method of claim 1, wherein automatically assigning the tentative tags to the objects-of-interest involves assigning the tentative tags based on information about similar assignments from related digital images.

6. The method of claim 1, wherein automatically assigning the tentative tags to the objects-of-interest involves assigning the tentative tags based on time stamps associated with the digital image and with related digital images.

7. The method of claim 1, wherein automatically assigning the tentative tags to the objects-of-interest involves assigning the tentative tags based on information about the similarity of areas in proximity to the objects in related digital images.

8. The method of claim 1, wherein automatically assigning the tentative tags to the objects-of-interest involves assigning the tentative tags based on probabilities that associated objects will appear together within a digital image.

9. The method of claim 1, wherein automatically assigning the tentative tags to the objects-of-interest involves using information about the orientation, position and scale of a given object within the digital image to determine a probability that the given object will be tagged.

10. The method of claim 1, wherein displaying the assignments of the tentative tags to the objects-of-interest involves displaying the tentative tags in proximity to the associated objects in the digital image.

11. The method of claim 1, wherein displaying the assignments of the tentative tags to the objects-of-interest involves displaying the objects grouped by tentative tag assignments.

12. The method of claim 1, wherein displaying the assignments of the tentative tags to the objects-of-interest involves displaying a set of most likely tentative tags for each object.

13. The method of claim 1, wherein in addition to automatically assigning tentative tags to the objects-of-interest, the method further comprises allowing the user to manually assign tags to objects.

14. The method of claim 1, wherein the method further comprises ceasing to display objects which have been tagged (or have had tag assignments confirmed) by the user.

15. The method of claim 1, further comprising using the assigned tags to facilitate subsequent operations involving the digital image and/or a set of related digital images.

16. A computer-readable storage device storing instructions that when executed by a computer cause the computer to perform a method for facilitating computer-assisted tagging of objects in a digital image, the method comprising:

receiving locations for one or more objects-of-interest in the digital image;

automatically determining likelihoods of specific tags being assigned to the objects-of-interest;

automatically assigning tentative tags to the objects-of-interest based on the determined likelihoods;

displaying the assignments of tentative tags to a user; and

receiving corrections to the assignments from the user.

17. The computer-readable storage device of claim 16, wherein the digital image includes a set of related digital images.

18. The computer-readable storage device of claim 16, wherein the digital image comprises a portion of a video image sequence.

19. The computer-readable storage device of claim 16, wherein receiving the locations for the one or more objects-of-interest involves:

receiving the digital image; and

performing an object detection operation on the digital image to detect the objects-of-interest and to determine the locations of the objects-of-interest.

20. The computer-readable storage device of claim 16, wherein automatically assigning the tentative tags to the objects-of-interest involves assigning the tentative tags based on information about similar assignments from related digital images.

21. The computer-readable storage device of claim 16, wherein automatically assigning the tentative tags to the objects-of-interest involves assigning the tentative tags based on time stamps associated with the digital image and with related digital images.

22. The computer-readable storage device of claim 16, wherein automatically assigning the tentative tags to the objects-of-interest involves assigning the tentative tags based on information about the similarity of areas in proximity to the objects in related digital images.

23. The computer-readable storage device of claim 16, wherein automatically assigning the tentative tags to the objects-of-interest involves assigning the tentative tags based on probabilities that associated objects will appear together within a digital image.

24. The computer-readable storage device of claim 16, wherein automatically assigning the tentative tags to objects-of-interest involves using information about the position, orientation and scale of a given object within the digital image to determine a probability that the given object will be tagged.

25. The computer-readable storage device of claim 16, wherein displaying the assignments of the tentative tags to the objects-of-interest involves displaying the tentative tags in proximity to the associated objects in the digital image.

26. The computer-readable storage device of claim 16, wherein displaying the assignments of the tentative tags to the objects-of-interest involves displaying the objects grouped by tentative tag assignments.

27. The computer-readable storage device of claim 16, wherein displaying the assignments of the tentative tags to the objects-of-interest involves displaying a set of most likely tentative tags for each object.

28. The computer-readable storage device of claim 16, wherein in addition to automatically assigning tentative tags to the objects-of-interest, the method further comprises allowing the user to manually assign tags to objects.

29. The computer-readable storage device of claim 16, wherein the method further comprises ceasing to display objects which have been tagged (or have had tag assignments confirmed) by the user.

30. The computer-readable storage device of claim 16, wherein the method further comprises using the assigned tags to facilitate subsequent operations involving the digital image and/or a set of related digital images.

31. An apparatus that facilitates computer-assisted tagging of objects in a digital image, comprising:

a receiving mechanism configured to receive locations for one or more objects-of-interest in the digital image;

a determination mechanism configured to automatically determine likelihoods of specific tags being assigned to the objects-of-interest;

an assignment mechanism configured to automatically assign tentative tags to the objects-of-interest based on the determined likelihoods;

a display mechanism configured to display the assignments of tentative tags to a user; and

a correction mechanism configured to receive corrections to the assignments, if any, from the user.