Method and apparatus for isolating and manipulating graphic objects on computer video monitor

• US 4,746,770 A
• Filed: 02/17/1987
• Issued: 05/24/1988
• Est. Priority Date: 02/17/1987
• Status: Expired due to Fees

• Current Assignee: Sensor Frame Corporation
• Original Assignee: Sensor Frame Corporation
• Inventors: McAvinney, Paul
• Primary Examiner(s): SCHREYER, STAFFORD
• Application Number:
  US07/014,943
• Time in Patent Office: 462 Days
• Field of Search: 178/18, 178/19, 340/365 R, 340/365 P, 340/712, 250/578, 250/221, 250/222.1, 250/224
• US Class Current: 178/18.09
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G10H 1/34 Switch arrangements, e.g. keyboards or mechanical switches specially adapted for electrophonic musical instruments
• CPC Class Codes: G10H 2220/411 Light beams key actuation detection using light G10H2220/305

Abstract

A method of selection and/or manipulating graphic objects such as may appear on a computer video monitor is provided as is associated apparatus. A frame member cooperates with a plurality of optical sensors and a plurality of light occluding objects which tend to obstruct a portion of the light normally received by the optical sensors. The sensors are preferably dynamic RAM sensors. Determination of the position and orientation of the light occluding objects and, if desired, the velocity of the same, permits identification of or alteration of graphic objects. Fingers may be employed as the light occluding objects. This approach eliminates the need for burdensome identification of and changes to graphic objects through a computer keyboard or by means of a "mouse".

Claims

1. A method of manipulating graphic objects comprising

providing a frame, and a plurality of optical sensors generally facing the frame interior,

illuminating at least a portion of the space defined by said frame,

employing a plurality of light occluding objects to obstruct a portion of the light from reaching at least certain elements of said sensors, and

emitting signals from said sensors corresponding to the positions of said light occluding objects with respect to said frame.

2. The method of claim 1 including

providing a video monitor, and

positioning said frame adjacent to said video monitor.

3. The method of claim 1 including

periodically combining said sensor signals to provide information regarding changes in position of said light occluding objects.

4. The method of claim 3 including

storing a first set of said sensor signals corresponding to a first position of said light occluding objects and obtained at a first time, and

subsequently comparing said first set of sensor signals with a second set of said sensor signals obtained at a second time to determine changes in position of said light occluding objects.

5. The method of claim 4 including

employing said comparison to determine the velocity of said light occluding objects.

6. The method of claim 4 including

employing said comparison to determine the orientation of said light occluding objects.

7. The method of claim 4 including

comparing a plurality of sets of said sensor signals with each being obtained at a different time, and

employing said sets to determine changes in position of said light occluding objects.

8. The method of claim 4 including

employing said sensor signals to move said graphic objects.

9. The method of claim 8 including

scanning said sensors at a rate of at least about 30 times per second.

10. The method of claim 8 including

employing at least four said sensors.

11. The method of claim 10 including

providing a generally rectangular frame, and

positioning said sensors generally at the corners of said rectangle.

12. The method of claim 4 including

employing computer means to store and compare said sensor signals.

13. The method of claim 2 including

employing portions of the human body as said light occluding objects.

14. The method of claim 13 including

employing a plurality of fingers as said light occluding objects.

15. The method of claim 13 including

determining the position and velocity and orientation of said light occluding objects.

16. The method of claim 15 including

providing said frame as a closed frame, and

illuminating the interior of said frame substantially uniformly.

17. The method of claim 16 including

employing dynamic RAM sensors as said sensors.

18. The method of claim 17 including

positioning said frame adjacent to said video monitor, and

employing a rectangular frame with four said sensors.

19. The method of claim 18 including

employing said light occluding objects to select and manipulate at least one graphic object.

20. The method of claim 19 including

employing said light occluding objects information to modify said graphic object.

21. The method of claim 19 including

employing at least two fingers to define a said graphic object.

22. The method of claim 21 including

employing said sensors to monitor zones disposed remote from said sensors with respect to the presence of light occluding objects therein.

23. The method of claim 22 including

providing said illumination from sources extending about the periphery of the frame.

24. The method of claim 1 including

determining the shape of said occluding objects.

25. The method of claim 7 including

positioning said frame in front of a video monitor.

26. The method of claim 7 including

maintaining said sensor cells in a "one" state before introduction of said light occluding objects into light obstructing positions, and

maintaining said "one" state on cells having light occluded by said light occluding objects.

27. The method of claim 6 including

determining the attributes of raypaths bound to partially or fully occluded objected by monitoring ray edge-binding and object-binding attributes of said objects.

28. Apparatus for manipulating graphic objects comprising

frame means for at least partially receiving a plurality of light occluding objects,

light source means at least partially illuminating the region adjacent said frame means,

sensor means operatively associated with said frame means for receiving at least a portion of said illumination and emitting responsive signals, and

signal processing means for employing said sensor signals to determine the position of said plurality of light occluding objects.

29. The apparatus of claim 28 including

a video monitor disposed adjacent to said frame means.

30. The apparatus of claim 29 including

said frame means being disposed in front of said video monitor with at least a portion of said monitor being visible through said frame.

31. The apparatus of claim 28 including

said signal processing means having means for comparing a first set of said sensor signals taken at a first time with a second set of signals taken subsequently in order to determine changes in position and orientation of said light occluding objects.

32. The apparatus of claim 31 including

said signal processing means having means for storing and comparing a plurality of sensor signal sets to determine changes in position in said light occluding objects.

36. The apparatus of claim 28 including

said sensor means having a plurality of dynamic RAM sensors.

38. The apparatus of claim 36 including

said sensor means employing at least four said dynamic RAM sensors.

39. The apparatus of claim 38 including

lens means disposed adjacent said dynamic RAM sensors for focusing light thereon.

40. The apparatus of claim 39 including

said frame means being generally rectangular, and

said dynamic RAM sensors disposed generally in the corners thereof.

41. The apparatus of claim 36 wherein

said dynamic RAM sensors having rectilinear arrays of sensor cells.

42. The apparatus of claim 40 including

said sensors being secured to said frame.

33. Apparatus for manipulatign graphic objects comprising

frame means for at least partially receiving light occluding objects,

light source means at least partially illuminating the region adjacent said frame means,

sensor means operatively associated with said frame means for receiving at least a portion of said illumination and emitting responsive signals,

signal processing means for employing said sensor signals to determine the position of said light occluding objects,

said signal processing means having means for comparing a first set of said sensor signals taken at a first time with a second set of signals taken subsequently in order to determine changes in position of said light occluding objects,

said signal processing means having means for storing and comparing a plurality of sensor signal sets to determine changes in position in said light occluding objects, and

said signal processing means having means for determining velocity and orientation of said light occluding objects.

34. The apparatus of claim 33 including

said signal processing means having computer means.

35. The apparatus of claim 33 including

said signal processing means having means for manipulating said graphic objects responsive to the information received from said sensors.

37. The apparatus of claim 33 including

said signal processing means having means for determining angle and z axis position of said light occluding objects.