APPARATUS AND METHOD FOR DETECTING A DRIVER'S INTEREST IN AN ADVERTISEMENT BY TRACKING DRIVER EYE GAZE

• US 20150006278A1
• Filed: 06/30/2014
• Published: 01/01/2015
• Est. Priority Date: 06/28/2013
• Status: Abandoned Application

• Current Assignee: Harman International Industries Incorporated (Samsung Group)
• Original Assignee: Harman International Industries Incorporated (Samsung Group)
• Inventors: DI CENSO, Davide, MARTI, Stefan, JUNEJA, Ajay
• Application Number:
  US14/319,338
• Publication Number:
  US 20150006278A1
• Time in Patent Office: Days
• US Class Current: 705/14.43
• CPC Class Codes: G06Q 30/0244 Optimization
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness

Abstract

A controller for providing advertisements to a vehicle or a wearable housing, and a computer readable medium, when executed by one or more processors, performs an operation to provide an audio advertisement to the vehicle or wearable housing. A first signal input receives a first camera signal, a second signal input receives a second camera signal, and at least one signal output transmits to at least one acoustic transducer, which provides the audio advertisement to the user. The computer logic that may be arranged within the controller determines whether the direction of the captured images of the advertisements and the direction of the user'"'"'s eye gaze correspond to one another, and, if so, the computer logic outputs the audio advertisement to the audio transducer.

Claims

1. A controller for providing audio information, the controller comprising:

a first signal input configured to receive a first camera signal that indicates a direction in which a user is looking;

a second signal input configured to receive a second camera signal that includes captured images of one or more visual information;

a signal output configured to drive at least one acoustic transducer; and

computer logic programmed to;

determine a direction to each of the captured images of the one or more visual information;

determine whether the indicated direction in which the user is looking corresponds to the determined direction of the captured image of one of the one or more visual information; and

upon determining that the indicated direction in which the user is looking corresponds to one of the one or more visual information;

determine a context of the one of the one or more visual information; and

output to the signal output an audio information related to the context of the one of the one or more visual information.

2. The controller of claim 1, wherein a first camera is connected to the first signal input, wherein at least one second camera is connected to the second signal input, wherein at least one acoustic transducer is connected to the signal output, wherein the first camera, the at least one second camera, the at least one acoustic transducer, and the computer logic are arranged in a passenger vehicle, wherein the first camera module is arranged in a passenger compartment of the passenger vehicle to determine a direction in which the user is looking, and wherein the at least one second camera is arranged on the vehicle in an outward-facing arrangement.

3. The controller of claim 1, further comprising a data transceiver, wherein, upon determining that the user is looking at one of the one or more visual information, the computer logic retrieves the audio information from a remote database through the data transceiver.

4. The controller of claim 1, wherein the first camera detects an eye gaze direction of the user, and wherein the indicated direction in which the user is looking is the detected eye gaze direction.

5. The controller of claim 1, wherein the first camera detects a head orientation of the user, including a direction in which the head of the user is facing, and wherein the indicated direction in which the user is looking is the detected direction in which the head of the user is facing.

6. The controller of claim 1, wherein the computer logic further determines whether the user is interested in one of the one or more visual information by at least one of:

determining that the user has looked in the direction of the one of the one or more visual information for at least a predetermined amount of time;

determining that the user has looked in the direction of the one of the one or more visual information more than a predetermined number of times; and

determining that the user has looked in the direction of the one of the one or more visual information for a total cumulative amount of time that exceeds a predetermined amount,

receiving an input signal from a user interface (e.g., physical button, icon on a digital interface, etc.);

and wherein the computer logic outputs the audio information upon determining that the user is interested in the one of the one or more visual information.

7. The controller of claim 1, wherein the at least one acoustic transducer comprises an audio speaker arranged in a vehicle.

8. A wearable controller for providing audio information, the controller comprising:

a first signal input configured to receive a first camera signal that indicates an eye gaze direction;

a second signal input configured to receive a second camera signal that includes captured images of one or more visual information;

a signal output configured to drive at least one acoustic transducer; and

computer logic programmed to;

determine a direction to each of the captured images of the one or more visual information;

determine whether the indicated eye gaze direction corresponds to the determined direction of the captured image of one of the one or more visual information; and

upon determining that the indicated eye gaze direction corresponds to one of the one or more visual information;

determine a context of the one of the one or more visual information; and

output to the signal output an audio information related to the context of the one of the one or more visual information,

wherein a first camera that provides the first camera signal, the at least one second camera that provides the second camera signal, and the at least one acoustic transducer are arranged in at least one wearable housing.

9. The controller of claim 8, wherein the first camera is arranged on a head-mounted device that is wearable.

10. The controller of claim 8, wherein the at least one acoustic transducer is arranged in headphones, and wherein the at least one second camera is arranged in a housing for the headphones.

11. The controller of claim 10, wherein the computer logic is arranged in the housing for the headphones.

12. The controller of claim 10, wherein the computer logic is arranged in a smart phone.

13. The controller of claim 8, further comprising a data transceiver, wherein, upon determining that the indicated eye gaze direction corresponds to the determined direction of the captured image of one of the one or more visual information, the data transceiver transmits the captured image of the one of the one or more visual information to a remote computer system and receives the audio information from the remote computer system.

14. A computer readable medium comprising a program which, when executed by one or more processors, performs an operation comprising:

determining a direction in which a user is looking;

determining locations for a plurality of visual information relative to the user;

determining whether the user is looking in a direction corresponding to one of the plurality of visual information;

determining a context of the one of the plurality of visual information; and

outputting an audio information related to the context of the one of the plurality of visual information to the user.

15. The computer readable medium of claim 14, wherein determining a direction in which the user is looking comprises determining a direction of eye gaze of the user relative to a reference direction, wherein determining locations for the plurality of visual information relative to the user comprises determining at least one direction to each of the plurality of visual information relative to the reference direction, and wherein determining whether the user is looking in a direction corresponding to one of the plurality of visual information comprises determining whether the direction of eye gaze is within a predefined threshold of the at least one direction of the one of the plurality of visual information.

16. The computer readable medium of claim 15, wherein determining at least one direction to each of the plurality of visual information comprises determining a first direction relative to the reference direction that corresponds to a first boundary of each of the plurality of visual information and determining a second direction relative to the reference direction that corresponds to a second boundary of each of the plurality of visual information, wherein the second boundary is opposite the first boundary, and wherein determining whether the direction of eye gaze is within a predefined threshold of the at least one direction of the one of the plurality of visual information comprises whether the determined eye gaze direction is between the first direction and the second direction of the one of the plurality of visual information.

17. The computer readable medium of claim 16, wherein determining at least one direction to each of the plurality of visual information further comprises determining a third direction relative to the reference direction that corresponds to a third boundary of each of the plurality of visual information and determining a fourth direction relative to the reference direction that corresponds to a fourth boundary of each of the plurality of visual information, wherein the third boundary is orthogonal to the first boundary, wherein the fourth boundary is opposite the third boundary, and wherein determining whether the direction of eye gaze is within a predefined threshold of the at least one direction of the one of the plurality of visual information comprises whether the determined eye gaze direction is between the third direction and the fourth direction of the one of the plurality of visual information.

18. The computer readable medium of claim 14, wherein determining a context of the one of the plurality of visual information comprises:

receiving an image of the one of the plurality of visual information;

comparing the received image to a plurality of images in a database, wherein each image in the database is associated with a context; and

upon matching an image from the database to the received image, associating the context of the matched image from the database with the one of the plurality of visual information.

19. The computer readable medium of claim 14, wherein determining a context of the one of the plurality of visual information comprises:

determining a geolocation of the one of the one or more visual information;

querying a database that comprises a plurality of georeferenced visual information, wherein each georeferenced visual information is associated with a context; and

upon matching the determined geolocation with a georeferenced visual information, associating the context of the matched georeferenced advertisement with the one of the plurality of visual information.