More useful man machine interfaces and applications

• US 6,750,848 B1
• Filed: 11/03/1999
• Issued: 06/15/2004
• Est. Priority Date: 11/09/1998
• Status: Expired due to Term

• Current Assignee: Gesture Technology Partners, LLC
• Original Assignee: Timothy R. Pryor
• Inventors: Pryor, Timothy R.
• Primary Examiner(s): Nguyen, Chanh
• Assistant Examiner(s): Bell, Paul A.
• Application Number:
  US09/433,297
• Time in Patent Office: 1,686 Days
• Field of Search: 345/156, 345/157, 345/158, 345/168, 345/700, 345/719, 345/863, 340/407.2, 341/22, 400/719
• US Class Current: 345/168
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0386 for light pen

Abstract

The invention is a continuation of earlier applications aimed at providing affordable methods and apparatus for inputting position, attitude(orientation) or other object characteristic data to computers for the purpose of controlling the display thereof. Preferred embodiments utilize electro-optical sensors such as TV cameras, to input data from objects and fingers, and/or other body parts of the user, to laptop, hand held and other computers used for Computer Aided learning, Gaming, 3D graphics, internet commerce and other applications.

Claims

1. A method for entering data into a computer, comprising the steps of:

providing a generally planar member containing a keyboard associated with the computer, the member having a TV camera incorporated therein for viewing above the keyboard but precluded from viewing the keyboard,

providing a display to display data generated or controlled by the computer,

utilizing the TV camera to acquire an image of a portion of a user or an object associated with the user located above the keyboard,

determining from the image, the position of the portion of the user and/or the object, and

from the position, determining data to be entered into the computer.

2. A method for entering data into a computer as claimed in claim 1, further including the step of determining orientation from the position.

3. A method for entering data into a computer as claimed in claim 2, further including the step of determining a point on the display which is being pointed at by the user.

4. A method for entering data into a computer as claimed in claim 1, wherein a second TV camera is used in conjunction with the first-mentioned TV camera.

5. A method for entering data into a computer as claimed in claim 4, wherein the first-mentioned and second TV cameras form a stereo pair.

6. A method for entering data into a computer as claimed in claim 5:

wherein the first-mentioned and second TV cameras forming the stereo pair are both incorporated in the member on left and right sides thereof; and

further including the step of viewing a space above the keyboard with the stereo pair of TV cameras.

7. A method for entering data into a computer as claimed in claim 4, wherein the second TV camera is located in a housing of the display.

8. A method for entering data into a computer as claimed in claim 4, wherein the second camera is also incorporated in the member and is spaced from the first TV camera.

9. A method for entering data into a computer as claimed in claim 1, wherein the determining step includes analyzing colors or shapes in the image.

10. A method for entering data into a computer as claimed in claim 1, wherein a datum on the user portion is used to aid in the determining step.

11. A method for entering data into a computer as claimed in claim 1, wherein the computer is a laptop computer with an integrated display.

12. A method for entering data into a computer, comprising the steps of:

providing a generally planar member containing a keyboard associated with the computer, the member having a TV camera incorporated therein for viewing above the keyboard but precluded from viewing the keyboard,

providing a display to display data generated or controlled by the computer,

utilizing the TV camera to acquire an image of a portion of a user or an object associated with the user located above the keyboard,

determining from the image, the orientation of the user portion and/or the object, and

from the orientation, determining data to be entered into the computer.

13. A method for entering data into a computer as claimed in claim 12, further including the step of determining position of at least one point on the user or object.

14. A method for entering data into a computer as claimed in claim 12, further including the step of determining a point on the display which is being pointed at by the user.

15. A method for entering data into a computer as claimed in claim 12, wherein a second TV camera is are used in conjunction with the first-mentioned TV camera.

16. A method for entering data into a computer as claimed in claim 15, wherein the first-mentioned aid second TV cameras form a stereo pair.

17. A method for entering data into a computer as claimed in claim 16:

wherein the first-mentioned and second TV cameras forming the stereo pair are both incorporated in the member on left and right sides thereof; and

further including the step of viewing a space above the keyboard with the stereo pair of TV cameras.

18. A method for entering data into a computer as claimed in claim 15, wherein the second TV camera is located in a housing of the display.

19. A method for entering data into a computer as claimed in claim 12, wherein the determining step includes analyzing colors or shapes in the image.

20. A method for entering data into a computer as claimed in claim 12, wherein a datum on the user portion is used to aid in the determining step.

21. A method for entering data into a computer as claimed in claim 12, wherein the computer is a laptop computer with an integrated display.

22. A method for entering data into a computer as claimed in claim 12, wherein the second camera is also incorporated in the member and is spaced from the first TV camera.