Unconstrained pointing interface for natural human interaction with a display-based computer system

• US 6,404,416 B1
• Filed: 02/25/1998
• Issued: 06/11/2002
• Est. Priority Date: 06/09/1994
• Status: Expired due to Term

• Current Assignee: Corporation for National Research Initiatives
• Original Assignee: Corporation for National Research Initiatives
• Inventors: Cafarella, John H., Kahn, Robert E.
• Primary Examiner(s): Chow, Dennis-Doon
• Application Number:
  US09/030,646
• Time in Patent Office: 1,567 Days
• Field of Search: 345/179, 345/180, 345/156, 345/157, 345/158, 345/172, 345/1, 345/2, 348/383, 348/36, 348/39, 341/22
• US Class Current: 345/158
• CPC Class Codes: G06F 3/03542 Light pens for emitting or receiving light
• CPC Class Codes: G06F 3/037 using the raster scan of a cathode-ray tube [CRT] for detecting the position of the member, e.g. light pens cooperating with CRT monitors
• CPC Class Codes: G06F 3/0386 for light pen

Abstract

An unconstrained pointing interface for natural human interaction with a display-based computer system includes a hand-manipulated pointer configured to permit it to be (1) aimed by line of sight to point at any arbitrary position within the user'"'"'s field of view, including positions beyond the user'"'"'s natural reach, and (2) swept along any arbitrary path and at any arbitrary speed within the user'"'"'s natural capability. A detector detects emanations from at least some positions at which the pointer may be aimed; and circuitry identifies the locations of positions at which the user is aiming the pointer. The interface enables the user to point to a position within his field of view and beyond his reach, move his hand along any arbitrary path and at any arbitrary speed within the user'"'"'s natural capability to point to another position, and so on, and to have the circuitry automatically identify the locations of the series of positions to which he has pointed.

Claims

1. Apparatus which enables a user to communicate with an image appearing on a display surface associated with a computer system, the image appearing on the display surface including an object that represents a control that can be rotated in the image to actuate a software feature, the display surface being characterized by predetermined time-varying light emanations from the surface, the time variations of the emanations being sufficient to enable a determination of a localized area on the surface which is the source of portions of the light emanations, the interface comprising

a hand-held pointer having

an axis that can be aimed at the object that represents a control that can be rotated in the image while the pointer is held at a location remote from the surface, the pointer being rotatable about the axis while being aimed, and

sensors configured to receive portions of the light emanations from the object that represents a control that can be rotated in the image, and

circuitry configured to determine that the hand-held pointer is being rotated based on signals from the sensors and to cause rotation of the object to actuate the software feature based on determination of the rotation.

3. The apparatus of claim 1 or 2 wherein

the emanations differ at different positions, and

the circuitry identifies the location of each localized area in a series of localized areas based on differences in the emanations.

4. The apparatus of claim 1 or 2 wherein the differences comprise differences in time of occurrence.

5. The apparatus of claim 1 or 2 wherein the differences in time of occurrence are associated with raster scanning of the positions.

6. The apparatus of claim 1 or 2 wherein the emanations comprise electromagnetic radiation.

7. The apparatus of claim 1 or 2 wherein the emanations comprise visible light or non-visible radiation.

8. The apparatus of claim 1 or 2 wherein the emanations originate at the localized area.

9. The apparatus of claim 1 or 2 wherein the emanations are reflected from the localized area.

10. The apparatus of claim 1 or 2 wherein the reflections are of scanned emanations.

11. The apparatus of claim 1 wherein the scanned emanations are directed to at least one passive everyday object.

12. The apparatus of claim 1 or 2 wherein the circuitry generates a monopulse position signal as a result of a monopulse processing technique and the monopulse position signal carries information about the timing of the emanations.

13. The apparatus of claim 1 or 2 wherein the circuitry derives a timing signal from the monopulse position signal.

14. The apparatus of claim 1 or 2 wherein the circuitry is part of or attached to the pointer.

16. The apparatus of claims 1 or 2 wherein there are two or more of said pointers.

17. The apparatus of claims 1 or 2 further comprising

an event mechanism which enables the user to generate events associated with a localized area at which he is aiming the pointer.

18. The apparatus of claim 1 or 2 wherein

the event mechanism is part of or attached to the pointer.

19. The apparatus of claim 1 or 2 wherein

the events generated by the event mechanism are represented as event signals delivered to the circuitry.

20. The apparatus of claim 1 or 2 wherein

the event signals are carried by wire.

21. The apparatus of claim 1 or 2 wherein

the event signals are carried by electromagnetic radiation.

22. The apparatus of claim 1 or 2 further comprising a wireless communication channel for carrying the monopulse signal.

23. The apparatus of claim 1 or 2 wherein the event signals are carried on a coded channel.

24. The apparatus of claim 1 or 2 wherein the event signals are carried on a spread spectrum channel.

25. The apparatus of claim 1 or 2 wherein

the circuitry generates a monopulse position signal as a result of a monopulse processing technique.

26. The apparatus of claim 1 or 2 further comprising a wireless communication channel for carrying the monopulse position signal.

27. The apparatus of claim 1 or 2 wherein the communication channel comprises a coded channel.

28. The apparatus of claim 1 or 2 wherein the channel comprises a spread-spectrum channel.

2. Apparatus which enables a user of a computer system to operate interactively with an object that has a surface characterized by visible variations, comprising

a scanner controllable to impose a modulating pattern of radiation on the object to cause a modulated pattern of reflected emanations that is different from the visible variations, the pattern being sufficient to enable a determination, from the modulated pattern of reflected emanations, of a localized area of the visible variations on the object which is a source of reflected portions of the radiation, the modulated pattern of reflected emanations comprising a vertical scan of a horizontal line preceded or followed by a horizontal scan of a vertical line, and

a hand-held pointer having

an axis that can be aimed at a localized area of the visible variations of the object, and

a sensor that receives portions of the modulated pattern of reflected emanations from the localized area of the visible variations, and

circuitry configured to determine the localized area of the visible variations of the object to which the pointer is aimed based on signals from the sensor.

15. The apparatus of claims 2 further comprising a communication channel from the circuitry which identifies the localized areas at which the user is aiming the pointer, to the pointer, and wherein the pointer includes an indicator for conveying information to a user based on data received via the communication link.

29. The apparatus of claim 2 wherein the modulating pattern of radiation comprises a time varying pattern.

30. Apparatus which enables a user to interact with a display surface, the display surface displaying an object that represents a control that can be rotated in the image to actuate a software feature, comprising

a hand-held pointer that includes a sensor arranged so that when the pointer is aimed at the object displayed on the display surface of the surface, from a location remote from the surface, light emanating from the object is received at the sensor, and

circuitry connected to the hand-held pointer and configured to determine changes in the angular orientation of the pointer about an axis generally extending between the pointer and the surface and to cause rotation of the object to actuate the software feature based on determination of the rotation.

31. The apparatus of claim 30, in which the sensor includes three sensing elements and the circuitry determines the changes based on analysis of output signals from the three sensing elements.

32. Apparatus which enables a user to interact with a passive object by pointing at selected visibly-distinct parts of a surface of the object comprising:

a source that projects radiation onto the surface, the radiation being modulated in a pattern that permits different locations of the surface to be distinguished on the basis of radiation reflected from them, the modulated pattern comprising a vertical scan of a horizontal line preceded or followed by a horizontal scan of a vertical line,

a hand-held pointer that includes a sensor arranged so that when the pointer is aimed at one of the visibly distinct parts of the surface, the sensor receives radiation reflected from the part, and

circuitry connected to the sensor and configured to determine a visibly-distinct part of the surface to which the pointer is being aimed based on the modulated pattern of radiation received at the sensor.

33. The apparatus of claim 32 in which the radiation is modulated as a pseudo raster scan.

34. The apparatus of claim 32 in which the circuitry also uses information about the state of modulation of the radiation in determining the part to which the pointer is aimed.

35. The apparatus of claim 32 in which the modulation includes projecting radiation on different parts of the surface at different times.

36. The apparatus of claim 32 in which the surface of the object is two-dimensional.