Method and system for providing advertisement based on gaze of user

• US 10,235,693 B2
• Filed: 09/24/2013
• Issued: 03/19/2019
• Est. Priority Date: 09/24/2012
• Status: Active Grant

• Current Assignee: Naver Corporation
• Original Assignee: Naver Corporation
• Inventors: Byun, Eunja, Yu, Sukku
• Primary Examiner(s): Bramer, John Van
• Application Number:
  US14/035,054
• Publication Number:
  US 20140089097A1
• Time in Patent Office: 2,002 Days
• Field of Search: 705 141- 1473
• CPC Class Codes: G06Q 30/0267 Wireless devices

Abstract

Methods and systems of modifying content displayed by a mobile device are provided. A region of interest (ROI) of a user is recognized. The ROI may be associated with content being displayed on a display of the mobile device. A time duration of interest based on the ROI is determined. The displayed content is modified based on the ROI, the time duration of interest, and a content type of the displayed content.

Claims

1. A method of modifying content displayed by a mobile device, the method comprising:

displaying, on a display of the mobile device, a user interface for interacting with a camera of the mobile device, the content being displayed in response to a user event associated with the user interface;

recognizing, by a processor of the mobile device via the camera, a direction of a gaze of a user with respect to the content, the direction of the gaze of the user being determined by the processor by recognizing eyes of the user from an image acquired by using the camera;

recognizing, by the processor via the camera of the mobile device, a region of interest (ROI) of the user with respect to the content by

dividing region of the display where the content is displayed into a plurality of areas,

determining a gaze area of the user on the display based on input from the camera of the mobile device and the recognizing the direction of the gaze of the user,

associating the gaze area with one of the plurality of areas of the region based on a location of the gaze area on the display, a detected size of pupils and/or corneas of eyes of the user, and a time duration of the direction of the gaze of the user being directed at the gaze area, the gaze area and the one of the plurality of areas may be different, and

recognizing the one of the plurality of areas as the ROI;

determining, by the mobile device, a time duration of interest based on an amount of time for which the direction of the gaze of the user is directed at the ROI; and

modifying, by the mobile device, the content displayed in the ROI on the display based on the time duration of interest and a content type of the content displayed in the ROI.

2. The method of claim 1 further comprising:

at least one of,

modifying at least one of a format, a form, a size, and a position of the content displayed on the display based on at least one of the content type and a position of the content displayed in the ROI, and

providing additional content associated with the content type when the time duration of interest is greater than or equal to a set amount of time.

3. The method of claim 1, further comprising:

altering the content displayed on the display when the ROI is changed in response to the modifying.

4. The method of claim 3, wherein the altering comprises:

at least one of,

restoring the content displayed on the display to an original state,

providing an action corresponding to the changed ROI, and

terminating display of the content displayed on the display.

5. The method of claim 1, wherein the content displayed on the display includes a user interface for interacting with the user of the mobile device, the user interface providing content to the mobile device in response to a user event.

6. The method of claim 5, wherein

the content on the display includes an advertisement, and

the method further comprises:

charging an advertising cost associated with displaying the advertisement to an advertiser; and

charging an additional advertising cost based on modifying the advertisement to the advertiser.

7. A system for modifying displayed content, the system comprising:

a server including a mobile application providing module configured to provide a mobile application to a mobile device, and

the mobile application is for controlling the mobile device to,

display content on a display of the mobile device, a user interface for interacting with a camera of the mobile device, the content being displayed in response to a user event associated with the user interface,

recognize, by a processor of the server via the camera of the mobile device, a direction of a gaze of a user with respect to the content, the direction of the gaze of the user being determined by the processor by recognizing eyes of the user from an image acquired by using the camera,

recognize, by the processor via the camera of the mobile device, a region of interest (ROI) of the user with respect to the content by,

dividing region of the display where the content is displayed into a plurality of areas,

determining a gaze area of the user on the display based on input from the camera of the mobile device and the recognizing the direction of the gaze of the user,

associating the gaze area with one of the plurality of areas of the region based on a location of the gaze area on the display, a detected size of pupils and/or corneas of eyes of the user, and a time duration of the direction of the gaze of the user being directed at the gaze area, the gaze area and the one of the plurality of areas may be different, and

recognizing the one of the plurality of areas as the ROI,

determine a time duration of interest based on an amount of time for which the direction of the gaze of the user is directed at the ROI, and

modify the content displayed in the ROI on the display based on the time duration of interest, and a content type of the content displayed in the ROI.

8. The system of claim 7, wherein the mobile application is for controlling the mobile device to

modify at least one of a format, a form, a size, and a position of a content displayed on the display based on at least one of the content type and a position of the content displayed in the ROI, or

provide additional content associated with the content type when the time duration of interest is greater than or equal to a set amount of time.

9. The system of claim 7, wherein the mobile application is for controlling the mobile device to alter the content displayed on the display when the ROI is changed in response to modifying the content displayed on the display.

10. The system of claim 9, wherein in altering the content displayed on the display, the mobile application is configured to control the mobile device to

restore the modified content to an original state,

provide an action corresponding to the changed ROI, or

terminate displaying of the modified content.

11. The system of claim 7, wherein

the mobile application is configured to control the mobile device to display, on the display of the mobile device, an advertisement, and

the display includes a user interface for interacting with the user of the mobile device such that the user interface provides content to the mobile device in response to a user event.

12. The system of claim 11, wherein

the content displayed on the display includes the advertisement, and

the system for modifying displayed content is configured to

charge an advertising cost associated with displaying the advertisement to an advertiser, and

charge an additional advertising cost to the advertiser based on modifying the content displayed on the display.

13. A non-transitory computer-readable storage medium including computer readable instructions that, when executed by a computer system, cause the computer system to,

displaying, on a display of a mobile device, a user interface for interacting with a camera of the mobile device, a content being displayed in response to a user event associated with the user interface,

recognize, by a processor of a mobile device via the camera of the mobile device, a direction of a gaze of a user with respect to the content, the direction of the gaze of the user being determined by the processor by recognizing eyes of the user from an image acquired by using the camera,

recognize, by the processor via the camera of the mobile device, a region of interest (ROI) of the user with respect to the content by,

dividing region of the display where the content is displayed into a plurality of areas,

determining a gaze area of the user on the display based on input from the camera of the mobile device and the recognizing the direction of the gaze of the user,

associating the gaze area with one of the plurality of areas of the region based on a location of the gaze area on the display, a detected size of pupils and/or corneas of eyes of the user, and a time duration of the direction of the gaze of the user being directed at the gaze area, the gaze area and the one of the plurality of areas may be different, and

recognizing the one of the plurality of areas as the ROI,

determine a time duration of interest based on an amount of time for which the direction of the gaze of the user is directed at the ROI, and

modify the content displayed in the ROI on the display based on the time duration of interest, and a content type of the content displayed in the ROI.

14. A method of modifying content displayed by a mobile device, the method comprising:

displaying, on a display of the mobile device, a user interface for interacting with a gaze tracking device associated with the mobile device, the content being displayed in response to a user event associated with the user interface;

recognizing, by the gaze tracking device associated with the mobile device, a direction of a gaze of a user with respect to the content, the direction of the gaze of the user being determined by the gaze tracking device by recognizing eyes of the user from an image acquired by the gaze tracking device;

recognizing, by the gaze tracking device associated with the mobile device, a region of interest (ROI) of the user with respect to the content by,

dividing region of the display where the content is displayed into a plurality of areas,

determining a gaze area of the user on the display based on input from the gaze tracking device of the mobile device and the recognizing the direction of the gaze of the user,

associating the gaze area with one of the plurality of areas of the region based on a location of the gaze area on the display, a detected size of pupils and/or corneas of eyes of the user, and a time duration of the direction of the gaze of the user being directed at the gaze area, the gaze area and the one of the plurality of areas may be different, and

recognizing the one of the plurality of areas as the ROI;

determining, by the mobile device, a time duration of interest based on an amount of time for which the direction of the gaze of the user is directed at the ROI; and

modifying, by the mobile device, the content displayed in the ROI on the mobile device based on the time duration of interest, and the content displayed in the ROI.

15. The method of claim 14, wherein the content displayed on the mobile device includes a plurality of areas and the modifying modifies the content displayed on the mobile device according to the gaze detecting device detecting the ROI within at least one of the plurality of areas.

16. The method of claim 14, wherein the modifying modifies the content displayed on the mobile device according to a position of the ROI.

17. The method of claim 14, wherein the modifying modifies the content displayed on the mobile device according to a changing position of the ROI.