Multiple Viewpoint Image Capture of a Display User

• US 20150070481A1
• Filed: 09/06/2013
• Published: 03/12/2015
• Est. Priority Date: 09/06/2013
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: S., Arvind, Jothiswaran, Gangatharan
• Granted Patent: US 10,108,258 B2
• Time in Patent Office: Days
• US Class Current: 348/78
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Embodiments of the invention describe apparatuses, systems and methods for capturing, selecting and analyzing image data of a user of a computing device display from multiple viewpoints. Embodiments of the invention capture or receive data comprising a plurality of images from an image capture system, wherein the plurality of images comprises images of a user of a computing system from multiple viewpoints. A gaze direction of the user with respect to the display is determined based, at least in part, on the plurality of images. Embodiments further determine a target area on the computing device display based, at least in part, on the determined gaze direction of the user, and execute an application operation in response to determining the target area of the user'"'"'s gaze.

Claims

1. An apparatus comprising:

a plurality of image sensors to capture a plurality of images of a user of the display of the computing device from a plurality of viewpoints included on the display of the computing device; and

a deflector comprising;

a first side to deflect light away from each of the viewpoints included on the display towards the plurality of image sensors; and

a second transmissive side, opposite the first side, to allow light from the display to pass through to the user.

2. The apparatus of claim 1, further comprising:

logic to;

determine a gaze direction of the user with respect to the display based, at least in part, on the plurality of images;

determine a target area of the user'"'"'s gaze on the display based, at least in part, on the determined gaze direction of the user; and

execute an application operation in response to determining the target area of the user'"'"'s gaze.

3. The apparatus of claim 2, wherein the logic to determine the target area of the user'"'"'s gaze on the display by identifying an eye of the user from each of the plurality of images, and determining the gaze direction of the user based, at least in part, on iris eccentricity of the identified eye of the user in each of the plurality of the images.

4. The apparatus of claim 1, wherein the first side of the deflector comprises a Fresnel prism.

5. The apparatus of claim 4, further comprising:

a Fresnel lens disposed over the plurality of image sensors.

6. The apparatus of claim 1, wherein the plurality of image sensors and the plurality of viewpoints are equal to a number of pixels of the display of the computing device.

7. A system comprising:

a display of a computing device;

a plurality of image sensors disposed next to the display of the computing device to capture a plurality of images of a user of the display;

a deflector comprising;

a first side to deflect light away from the display towards the plurality of image sensors; and

a second transmissive side, opposite the first side, to allow light from the display to pass through to the user; and

logic to;

determine a gaze direction of the user with respect to the display based, at least in part, on the plurality of images;

determine a target area of the user'"'"'s gaze on the display based, at least in part, on the determined gaze direction of the user; and

execute an application operation in response to determining the target area of the user'"'"'s gaze.

8. The system of claim 7, wherein determining the target area of the user'"'"'s gaze on the display includes:

identifying an eye of the user from each of the plurality of images; and

determining the gaze direction of the user based, at least in part, on iris eccentricity of the identified eye of the user in each of the plurality of the images.

9. The system of claim 7, wherein the plurality of image sensors are disposed below the display of the computing device.

10. The system of claim 7, wherein the plurality of image sensors and the defector are included in a bezel surrounding the display of the computing device.

11. The system of claim 7, wherein the display of the computing device comprises a tabletop display surface.

12. The system of claim 7, wherein the first side of the deflector comprises a Fresnel prism.

13. The system of claim 12, further comprising:

a Fresnel lens disposed over the plurality of image sensors.

14. The system of claim 7, wherein the plurality of images captured by the plurality of imaging sensors comprises real-time video data, the application comprises a streaming video application, and the application operation comprises selecting the real-time video data corresponding to the target area of the user'"'"'s gaze as an output of the streaming video application.

15. The system of claim 7, wherein the application operation executed in response to determining the target area of the user'"'"'s gaze comprises a user input/output (I/O) operation.

16. The system of claim 15, wherein the application is to display a plurality of click-thru icons, and the user I/O operation comprises selecting one of the click-thru icons.

17. The system of claim 7, wherein the plurality of image sensors and the plurality of viewpoints are equal to a number of pixels of the display of the computing device.

18. The system of claim 7, wherein the deflector comprises a plurality of regions, the user of the computing device comprises a plurality of users, and the logic to further:

assign one or more of the regions of the deflector to each of the plurality of users.

19. The system of claim 7, further comprising:

an antenna; and

radio frequency circuitry coupled to the antenna to receive signal data to be processed by the system.

20. An article of manufacture comprising a computer-readable non-transitory storage medium having instructions stored thereon to cause a processor to perform operations including:

receiving data comprising a plurality of images from an image capture system, wherein the plurality of images comprises images of a user of a computing system captured from a plurality of viewpoints;

determining a gaze direction of the user with respect to the display based, at least in part, on the plurality of images;

determining a target area on the display based, at least in part, on the determined gaze direction of the user; and

executing an application operation in response to determining the target area of the user'"'"'s gaze.

21. The article of manufacture of claim 20, wherein determining the target area of the user'"'"'s gaze on the display includes:

identifying an eye of the user from each of the plurality of images; and

determining the gaze direction of the user based, at least in part, on iris eccentricity of the identified eye of the user in each of the plurality of images.

22. The article of manufacture of claim 21, wherein the plurality of images captured by the plurality of imaging sensors comprises real-time video data, the application comprises a streaming video application, and the application operation comprises selecting the real-time video data corresponding to the target area of the user'"'"'s gaze as an output of the streaming video application.

23. The article of manufacture of claim 20, wherein the application operation executed in response to determining the target area of the user'"'"'s gaze comprises a user input/output (I/O) operation.

24. The article of manufacture of claim 20, wherein the deflector comprises a plurality of regions, the user of the computing device comprises a plurality of users, and the operations further include:

assigning one or more of the regions of the deflector to each of the plurality of users.