Method and apparatus for three-dimensional input entry

• US 6,198,485 B1
• Filed: 07/29/1998
• Issued: 03/06/2001
• Est. Priority Date: 07/29/1998
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Hirt, Ernie R., Korta, Jean C., Mack, Walter J.
• Primary Examiner(s): Powell, Mark R.
• Assistant Examiner(s): HARRISON, CHANTE E
• Application Number:
  US09/123,965
• Time in Patent Office: 951 Days
• Field of Search: 345/127, 345/156-158, 345/161, 345/419, 345/7, 345/473, 395/119
• US Class Current: 345/419
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

The present invention is directed to a method and apparatus for providing three-dimensional (3-D) input data to a computer. A marker is moved in a 3-D work place. The marker has at least one unique feature distinctive from other objects in the 3-D work place. The movement of the marker in the 3-D work place is sensed by at least two sensors in a stereo imaging geometry to produce stereo images containing the marker. The stereo images are processed to produce marker images. The 3-D coordinates of the marker are computed from the marker images to provide the 3-D input data to the computer.

Claims

1. A method comprising:

moving a marker in a work place, the marker having at least one unique feature distinctive from other objects in the work place;

sensing movement of the marker in the work place by at least two sensors in a stereo imaging geometry using the at least one unique feature of the marker to produce stereo images containing the marker;

processing the stereo images to produce marker images; and

computing 3-D coordinates of the marker from the marker images to produce a three-dimensional (3-D) input data using a point of intersection from the corresponding stereo images, the 3-D input data being combined with an entry from a device to indicate an input action.

2. The method of claim 1 further comprising:

illuminating the marker by a light source.

3. The method of claim 1 wherein computing the 3-D coordinates includes:

determining image points corresponding to the marker images;

determining lines connecting the image points and the at least two sensors through corresponding images planes in the stereo imaging geometry; and

determining a point of intersection between the lines, the point of intersection providing the 3-D coordinates.

4. The method of claim 3 wherein determining image points comprises:

detecting an object representing the marker in the marker images;

determining an object point corresponding to the object.

5. The method of claim 4 wherein the object point includes a centroid point, an edge point, and a high curvature point.

6. The method of claim 1 wherein the at least one unique feature includes a predetermined color, a predetermined shape, a predetermined size, and a predetermined relative location.

7. The method of claim 1 wherein the marker is mounted on a finger of a users'"'"' hand.

8. The method of claim 1 wherein the work place is enclosed in an input unit.

9. The method of claim 1 wherein the sensors are cameras.

10. The method of claim 1 wherein the marker includes user'"'"'s eyes, user'"'"'s facial expressions, and user'"'"'s head.

30. The method of claim 1 wherein the device is one of a keyboard, a mouse, a voice recognition system, and a heat sensing element.

31. The method of claim 30 wherein the entry is generated by one of clicking a mouse button, moving a mouse on a mouse pad, holding down a control key on a keyboard, and uttering a command word.

11. An apparatus comprising:

at least two sensors to sense movement of a marker in a stereo imaging geometry to produce stereo images containing the marker using at least one unique feature of the marker distinctive from other objects in a work place; and

a processing element coupled to the at least two sensors to process the stereo images, the processing element computing 3-D coordinates of the marker from marker images produced by the stereo images to produce the 3-D input data using a point of intersection from the corresponding stereo images, the 3-D input data being combined with an entry from a device to indicate an input action.

12. The apparatus of claim 11 further comprising:

a light source to illuminate the marker.

13. The apparatus of claim 11 wherein the processing element computes the 3-D coordinates by

determining image points corresponding to the marker images;

determining lines connecting the image points and the at least two sensors through corresponding images planes in the stereo imaging geometry; and

determining a point of intersection between the lines, the point of intersection providing the 3-D coordinates.

14. The apparatus of claim 13 wherein the processing element further

detects an object representing the marker in the marker images;

and

determines an object point corresponding to the object.

15. The apparatus of claim 14 wherein the object point includes a centroid point, an edge point, and a high curvature point.

16. The apparatus of claim 11 wherein the at least one unique feature includes a predetermined color, a predetermined shape, a predetermined size, and a predetermined relative location.

17. The apparatus of claim 11 wherein the marker is mounted on a finger of a users'"'"' hand.

18. The apparatus of claim 11 wherein the work place is enclosed in an input unit.

19. The apparatus of claim 11 wherein the sensors are cameras.

20. The apparatus of claim 11 wherein the marker includes user'"'"'s eyes, user'"'"'s facial expressions, and user'"'"'s head.

32. The apparatus of claim 11 wherein the device is one of a keyboard, a mouse, a voice recognition system, and a heat sensing element.

33. The apparatus of claim 32 wherein the entry is generated by one of clicking a mouse button, moving a mouse on a mouse pad, holding down a control key on a keyboard, and uttering a command word.

21. A system comprising:

a computer to provide display data corresponding to three-dimensional (3-D) input data entered by a user; and

an input unit coupled to the computer to produce the 3-D input data, the input unit including;

at least two sensors to sense movement of a marker in a stereo imaging geometry to produce stereo images containing the marker using at least one unique feature of the marker distinctive from other objects in a work place, and

a processing element coupled to the at least two sensors to process the stereo images, the processing element computing 3-D coordinates of the marker from marker images produced by the stereo images to provide the 3-D input data to a computer using a point of intersection from the corresponding stereo images, the 3-D input data being combined with an entry from a device to indicate an input action.

22. The system of claim 21 wherein the input unit further comprises:

a light source to illuminate the marker.

23. The system of claim 21 wherein the processing element computes the 3-D coordinates by

determining image points corresponding to the marker images;

determining lines connecting the image points and the at least two sensors through corresponding images planes in the stereo imaging geometry; and

determining a point of intersection between the lines, the point of intersection providing the 3-D coordinates.

24. The system of claim 23 wherein the processing element further

detects an object representing the marker in the marker images;

and

determines an object point corresponding to the object.

25. The system of claim 24 wherein the object point includes a centroid point, an edge point, and a high curvature point.

26. The system of claim 21 wherein the at least one unique feature includes a predetermined color, a predetermined shape, a predetermined size, and a predetermined relative location.

27. The system of claim 21 wherein the marker is mounted on a finger of a users'"'"' hand.

28. The system of claim 21 wherein the marker includes user'"'"'s eyes, user'"'"'s facial expressions, and user'"'"'s head.

29. The system of claim 21 further comprises an entry device coupled to the computer to provide an input entry entered by the user, the input entry being combined with the 3-D input data to define an input action.

34. The system of claim 21 wherein the device is one of a keyboard, a mouse, a voice recognition system, and a heat sensing element.

35. The system of claim 34 wherein the entry is generated by one of clicking a mouse button, moving a mouse on a mouse pad, holding down a control key on a keyboard, and uttering a command word.