SYSTEMS AND METHODS FOR 3-D TARGET LOCATION

• US 20100322479A1
• Filed: 06/17/2010
• Published: 12/23/2010
• Est. Priority Date: 06/17/2009
• Status: Active Grant

• Current Assignee: Eyegaze, Inc.
• Original Assignee: LC Technologies Incorporated
• Inventors: Cleveland, Dixon
• Granted Patent: US 8,320,623 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/57 Control of contrast or brightness
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A target is imaged in a three-dimensional real space using two or more video cameras. A three-dimensional image space combined from two video cameras of the two or more video cameras is displayed to a user using a stereoscopic display. A right eye and a left eye of the user are imaged as the user is observing the target in the stereoscopic video display, a right gaze line of the right eye and a left gaze line of the left eye are calculated in the three-dimensional image space, and a gazepoint in the three-dimensional image space is calculated as the intersection of the right gaze line and the left gaze line using a binocular eyetracker. A real target location is determined by translating the gazepoint in the three-dimensional image space to the real target location in the three-dimensional real space from the locations and the positions of the two video cameras using a processor.

Claims

1. A system for remotely determining the location of a target in a three-dimensional real space, comprising:

two or more video cameras that image a target in a three-dimensional real space;

a stereoscopic display that displays a three-dimensional image space combined from two video cameras of the two or more video cameras to a user;

a binocular eyetracker that images a right eye and a left eye of the user as the user is observing the target in the stereoscopic video display, calculates a right gaze line of the right eye and a left gaze line of the left eye in the three-dimensional image space, and calculates a gazepoint in the three-dimensional image space as an intersection of the right gaze line and the left gaze line; and

a processor that is in communication with the two or more video cameras, the stereoscopic display, and the binocular eyetracker and that determines a real target location by translating the gazepoint in the three-dimensional image space to the real target location in the three-dimensional real space from locations and positions of the two video cameras.

2. The system of claim 1, further comprising an actuator connected to the processor and at least one video camera of the two video cameras that controls a relative distance between the two video cameras.

3. The system of claim 2, wherein the processor instructs the actuator to increase or decrease the relative distance between the two cameras to determine the real target location at longer or shorter ranges respectively.

4. The system of claim 1, wherein the processor selects the two video cameras of the two or more video cameras based on the relative distance between the two video cameras.

5. The system of claim 4, wherein the processor selects two video cameras with larger or smaller relative distances to determine the real target location at longer or shorter ranges respectively.

6. The system of claim 1, wherein the processor calculates a velocity of the target from two or more real target positions determined over time.

7. The system of claim 1, wherein the processor calculates an acceleration of the target from three or more real target positions determined over time.

8. The system of claim 1, wherein the processor calculates a direction of the target from two or more real target positions determined over time.

9. The system of claim 1, further comprising a switch that is activated by the user to designate a target identified by the gazepoint to a client application.

10. The system of claim 1, further comprising a microphone that receives a keyword from the user to designate a target identified by the gazepoint to a client application.

18. The system of claim 1, further comprising designating a target identified by the gazepoint to a client application when a switch is activated by the user using the processor.

19. The method of claim 1, further comprising designating a target identified by the gazepoint to a client application when a microphone receives a keyword from the user using the processor.

11. A method for remotely determining the location of a target in a three-dimensional real space, comprising:

imaging a target in a three-dimensional real space using two or more video cameras;

displaying a three-dimensional image space combined from two video cameras of the two or more video cameras to a user using a stereoscopic display;

imaging a right eye and a left eye of the user as the user is observing the target in the stereoscopic video display, calculating a right gaze line of the right eye and a left gaze line of the left eye in the three-dimensional image space, and calculating a gazepoint in the three-dimensional image space as an intersection of the right gaze line and the left gaze line using a binocular eyetracker; and

determining a real target location by translating the gazepoint in the three-dimensional image space to the real target location in the three-dimensional real space from locations and positions of the two video cameras using a processor.

12. The method of claim 11, further comprising instructing an actuator to increase or decrease the relative distance between the two cameras to determine the real target location at longer or shorter ranges respectively using the processor.

13. The method of claim 11, further comprising selecting the two video cameras of the two or more video cameras based on the relative distance between the two video cameras using the processor.

14. The method of claim 13, selecting the two video cameras of the two or more video cameras comprises selecting two video cameras with larger or smaller relative distances to determine the real target location at longer or shorter ranges respectively using the processor.

15. The method of claim 11, further comprising calculating a velocity of the target from two or more real target positions determined over time using the processor.

16. The method of claim 11, further comprising calculating an acceleration of the target from three or more real target positions determined over time using the processor.

17. The method of claim 11, further comprising calculating a direction of the target from two or more real target positions determined over time using the processor.

20. A computer program product, comprising a tangible computer-readable storage medium whose contents include a program with instructions being executed on a processor so as to perform a method for determining the location of a target in a three-dimensional real space, the method comprising:

providing a system, wherein the system comprises distinct software modules, and wherein the distinct software modules comprise an imaging/display module, an eye tracking module, and a target location module;

imaging a target in a three-dimensional real space with two or more video cameras using the imaging/display module;

displaying a three-dimensional image space combined from two video cameras of the two or more video cameras to a user on a stereoscopic display using the imaging/display module;

imaging a right eye and a left eye of the user with a binocular eyetracker as the user is observing the target in the stereoscopic video display, calculating a right gaze line of the right eye and a left gaze line of the left eye in the three-dimensional image space, and calculating a gazepoint in the three-dimensional image space as an intersection of the right gaze line and the left gaze line using the eye tracking module; and

determining a real target location by translating the gazepoint in the three-dimensional image space to the real target location in the three-dimensional real space from locations and positions of the two video cameras using the target location module.