Method and system for learning an attention model for an image

• US 20060112031A1
• Filed: 10/12/2004
• Published: 05/25/2006
• Est. Priority Date: 10/12/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ma, Wei-Ying, Xie, Xing
• Granted Patent: US 7,562,056 B2
• Time in Patent Office: Days
• US Class Current: 706/16
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 10/248 by interactive preprocessing or interactive shape modelling, e.g. feature points assigned by a user
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]

Abstract

A system for learning an attention model for an image based on user navigation actions while viewing the image is provided. An attention model learning system generates an initial attention model based on static features derived from the image. The learning system logs the actions of users as they view the image. The learning system identifies from the actions of the users those areas of the image that may be of user interest: After the learning system identifies areas that may be of user interest, the learning system attempts to identify attention objects based on those identified areas. After identifying the user attention objects, the learning system updates the attention model for the image to reflect the additional information provided by the user attention objects.

Claims

1. A method for learning an attention model for an image, the method comprising:

receiving indications of actions of a user while viewing the image; and

generating the attention model based on the actions of the user.

2. The method of claim 1 including generating an initial attention model based on static features of the image and wherein the generating of the attention model based on the actions of the user includes updating the attention model.

3. The method of claim 1 wherein the indications compose a browse log of actions of the user collected while the user viewed the image.

4. The method of claim 1 wherein the attention model includes attention objects and the generating of the attention model includes:

generating a user interest map of areas of the image of user interest as determined from the actions of the user;

creating attention objects of the image based on the user interest map; and

updating attention objects of the attention model based on the created attention objects.

5. The method of claim 4 wherein the user interest map is generated by identifying image zones of the image in which the actions indicate user interest.

6. The method of claim 5 wherein an image zone has attributes that include a time first viewed, a number of views, a display ratio, and a time of action.

7. The method of claim 5 wherein an image zone has an attribute that represents a number of interesting actions relating to the image zone.

8. The method of claim 5 wherein an image zone has an attribute that represents a display ratio.

9. The method of claim 5 wherein an image zone has an attribute that represents time of action.

10. The method of claim 4 wherein the creating of attention objects includes:

identifying primary attention objects based on viewing duration;

identifying secondary attention objects derived from the primary attention objects that represent a larger attention object; and

modifying the identified primary attention objects based on the identified secondary attention objects.

11. The method of claim 4 wherein the updating of attention objects of the attention model includes:

when a new attention object is similar to an existing attention object, modifying attributes of the existing attention object based on attributes of the new attention object; and

when a new attention object is not similar to an existing attention object, adding the new attention object to the attention model.

12. The method of claim 4 wherein:

the generating of a user interest map includes identifying image zones of the image in which the actions indicate user interest;

the creating of attention objects includes identifying primary attention objects based on viewing duration, identifying secondary attention objects derived from the primary attention objects that represent a larger attention object, and modifying the identified primary attention objects based on the identified secondary attention objects; and

the updating of attention objects of the attention model includes when a new attention object is similar to an existing attention object, modifying attributes of the existing attention object based on attributes of the new attention object, and when a new attention object is not similar to an existing attention object, adding the new attention object to the attention model.

13. A method in a computer system for identifying user interest in an image, the method comprising:

receiving indications of actions of a user while viewing the image;

dividing the image into image zones;

identifying actions of the user that indicate user interest; and

for identified actions,

determining an image zone associated with the action; and

indicating that the determined image zone is of user interest.

14. The method of claim 13 wherein when an identified action is associated with multiple image zones, indicating that the multiple image zones are of user interest.

15. The method of claim 13 wherein the indicating that the determined image zone is of user interest includes setting an interest score.

16. The method of claim 15 wherein the interest score is a weighted viewing time.

17. The method of claim 16 wherein the viewing time is weighted based on distance of image zone from focus point of the identified action.

18. The method of claim 13 wherein indicating that the determined image zone is of user interest includes setting a display ratio for the image zone.

19. The method of claim 13 wherein indicating that the determined image zone is of user interest includes setting a number of different viewings of the image zone.

20. The method of claim 13 wherein indicating that the determined image zone is of user interest includes setting a first time of viewing the image zone.

21. A method in a computer system for identifying attention objects of an image, the method comprising:

receiving indications of actions of a user while viewing the image;

identifying actions of the user that indicate user interest; and

defining areas associated with the identified actions as attention objects of the image.

22. The method of claim 21 wherein user interest is based on time between actions.

23. The method of claim 21 wherein the defining includes selecting areas as attention objects based on time the user spent viewing the areas.

24. The method of claim 23 including setting an attention value of an attention object based on time the user spent viewing the area.

25. The method of claim 21 wherein the defining includes growing an area based on time the user spent viewing adjacent areas.

26. The method of claim 21 wherein the identifying of attention objects is based on actions of multiple users.

27. A computer-readable medium containing instructions for controlling a computer system to learn an attention model for an image, by a method comprising:

generating an attention model based on static features of the image;

receiving indications of actions of users while viewing the image;

updating the attention model based on the actions of the users; and

providing the updated attention model for use in displaying the image.

28. The computer-readable medium of claim 27 wherein the indications compose a browse log of actions of the user collected while the user viewed the image.

29. The computer-readable medium of claim 27 wherein the attention model includes attention objects and the updating of the attention model includes:

generating a user interest map of areas of the image of user interest as determined by the actions of the user;

creating attention objects of the image based on the user interest map; and

updating attention objects of the attention model based on the created attention objects.

30. The computer-readable medium of claim 29 wherein the user interest map is generated by identifying image zones of the image in which the actions indicate user interest.

31. The computer-readable medium of claim 30 wherein an image zone has attributes that include a time first viewed, a number of views, a display ratio, and a time viewed.

32. The computer-readable medium of claim 30 wherein an image zone has an attribute that represents a number of interesting actions relating to the image zone.

33. The computer-readable medium of claim 30 wherein an image zone has an attribute that represents a display ratio.

34. The computer-readable medium of claim 30 wherein an image zone has an attribute that represents time viewed.

35. The computer-readable medium of claim 29 wherein the creating of attention objects includes:

identifying primary attention objects based on time viewed;

identifying secondary attention objects derived from the primary attention objects that represent a larger attention object; and

modifying the identified primary attention objects based on the identified secondary attention objects.

36. The computer-readable medium of claim 29 wherein the updating of attention objects of the attention model includes:

when a new attention object is similar to an existing attention object, modifying attributes of the existing attention object based on attributes of the new attention object; and

when a new attention object is not similar to an existing attention object, adding the new attention object to the attention model.