Unconstrained pointing interface for natural human interaction with a display-based computer system

• US 5,793,361 A
• Filed: 12/09/1996
• Issued: 08/11/1998
• Est. Priority Date: 06/09/1994
• Status: Expired due to Term

• Current Assignee: Corporation for National Research Initiatives
• Original Assignee: Corporation for National Research Initiatives
• Inventors: Cafarella, John H., Kahn, Robert E.
• Primary Examiner(s): Chow, Dennis-Doon
• Application Number:
  US08/762,531
• Time in Patent Office: 610 Days
• Field of Search: 348/383, 348/36, 348/39, 348/734, 345/156, 345/157, 345/158, 345/172, 345/179, 345/180, 345/1, 345/2, 341/22, 178/18, 178/19
• US Class Current: 345/179
• CPC Class Codes: G06F 3/03542 Light pens for emitting or receiving light
• CPC Class Codes: G06F 3/037 using the raster scan of a cathode-ray tube [CRT] for detecting the position of the member, e.g. light pens cooperating with CRT monitors
• CPC Class Codes: G06F 3/0386 for light pen

Abstract

An unconstrained pointing interface for natural human interaction with a display-based computer system includes a hand-manipulated pointer configured to permit it to be (1) aimed by line of sight to point at any arbitrary position within the user'"'"'s field of view, including positions beyond the user'"'"'s natural reach, and (2) swept along any arbitrary path and at any arbitrary speed within the user'"'"'s natural capability, A detector detects emanations from at least some positions at which the pointer may be aimed; and circuitry identifies the locations of positions at which the user is aiming the pointer. The interface enables the user to point to a position within his field of view and beyond his reach, move his hand along any arbitrary path and at any arbitrary speed within the user'"'"'s natural capability to point to another position, and so on, and to have the circuitry automatically identify the locations of the series of positions to which he has pointed.

Claims

1. Apparatus which enables a user to communicate with an image appearing on a display surface associated with the computer system, the display surface being characterized by predetermined time-varying light emanations from the surface, the time variations of the emanations being sufficient to enable a determination of a localized area on the surface which is the source of portions of the light emanations, the interface comprising

a hand-held pointer comprising

a detector which detects the time-varying light emanations received from a localized area at which the pointer may be aimed,

the detector including at least three sensors arranged to receive respectively different portions of the time-varying light emanations, and

circuitry which identifies the localized area at which the user is aiming the pointer, based on the time-varying emanations which are received from the localized area and which are sensed at the respective sensors of the pointers, the localized area being identified using sums and/or differences between outputs of pairs of the sensors as discriminators for indicating the localized area along two intersecting axes.

2. The apparatus of claim 1 further comprising

an event mechanism which enables the user to generate events associated with a localized area at which he is aiming the pointer.

3. The apparatus of claim 2 wherein the event mechanism is part of or attached to the pointer.

4. The apparatus of claim 2 wherein

the events generated by the event mechanism are represented as event signals delivered to the circuitry.

5. The apparatus of claim 4 wherein

the event signals are carried by wire.

6. The apparatus of claim 4 wherein

the event signals are carried by electromagnetic radiation.

7. The apparatus of claim 4 further comprising a wireless communication channel for carrying the monopulse signal.

8. The apparatus of claim 4 wherein the event signals are carried on a coded channel.

9. The apparatus of claim 4 wherein the event signals are carried on a spread spectrum channel.

10. The apparatus of claim 1 in which

the circuitry identifies the localized area within a coordinate system, of each localized area in a series of localized areas for each of the users.

11. The apparatus of claim 1 wherein

the emanations differ at different positions, and

the circuitry identifies the location of each localized area in a series of localized areas based on differences in the emanations.

12. The apparatus of claim 4 wherein the differences comprise differences in time of occurrence.

13. The apparatus of claim 5 wherein the differences in time of occurrence are associated with raster scanning of the positions.

14. The apparatus of claim 1 wherein the emanations comprise electromagnetic radiation.

15. The apparatus of claim 14 wherein the emanations comprise visible light or non-visible radiation.

16. The apparatus of claim 1 wherein the emanations originate at the localized area.

17. The apparatus of claim 1 wherein the emanations are reflected from the localized area.

18. The apparatus of claim 17 wherein the reflections are of scanned emanations.

19. The apparatus of claim 18 wherein the scanned emanations are directed to at least one passive everyday object.

20. The apparatus of claim 1 wherein the circuitry is part of or attached to the pointer.

21. The interface of claim 1 further comprising a communication channel from the circuitry which identifies the localized area at which the user is aiming the pointer, to the pointer, and wherein the pointer includes an indicator for conveying information to a user based on data received via the communication link.

22. The apparatus of claim 1 wherein the circuitry generates a monopulse position signal as a result of a monopulse processing technique and the monopulse position signal carries information about the timing of the emanations.

23. The apparatus of claim 22 wherein the circuitry derives a timing signal from the monopulse position signal.

34. The apparatus of claim 1 or 24 wherein

the circuitry generates a monopulse position signal as a result of a monopulse processing technique.

35. The apparatus of claim 34 further comprising a wireless communication channel for carrying the monopulse position signal.

36. The apparatus of claim 35 wherein the communication channel comprises a coded channel.

38. The apparatus of claim 1 or 24 wherein there are two or more of said pointers.

24. Apparatus for enabling a user to communicate with a computer system, the apparatus comprising

a group of display devices,

a pointer including a beam source, and

circuitry for identifying to which of the display devices the pointer is being aimed, the circuitry comprising at least three sensors located at positions within the group, the identification of the display device to which the pointer is being aimed being based on sums and/or differences between outputs of pairs of the sensors as discriminators.

25. The apparatus of claim 24 wherein each of said display devices comprises a pixel image display device.

26. The apparatus of claim 25 wherein each of the pixel image display devices comprises a raster scanned display.

27. The apparatus of claim 25 wherein each of the pixel image display devices comprises a static display and a means for imposing a pseudo-raster scan on said static displays.

28. The apparatus of claim 27 wherein the pseudo-raster scan comprises altering the appearance of successive lines shown on each of the static displays.

29. The apparatus of claim 24 wherein

the display devices are raster scanned using respective timing signals which are synchronized among the displays display devices.

30. The apparatus of claim 24 wherein each of the display devices comprises a separately scanned image display surface, said image display surfaces being arranged in the group.

31. The apparatus of claim 24 wherein each of the display devices comprises a separately scanned image display surface, each of said image display surfaces being rectangular and the sensors being located at the corners of the display surfaces.

32. The apparatus of claim 24 wherein the circuitry generates information associated with the rotational orientation of the pointer.

33. The interface of claim 24, wherein each of said display devices comprises a separately scanned image display surface on an everyday object.

37. The apparatus of claim 37 wherein the channel comprises a spread-spectrum channel.