Quasi-three-dimensional method and apparatus to detect and localize interaction of user-object and virtual transfer device

• US 20050024324A1
• Filed: 12/30/2003
• Published: 02/03/2005
• Est. Priority Date: 02/11/2000
• Status: Abandoned Application

• Current Assignee: Abbas Rafii, Carlo Tomasi
• Original Assignee: Abbas Rafii, Carlo Tomasi
• Inventors: Tomasi, Carlo, Rafii, Abbas
• Application Number:
  US10/750,452
• Publication Number:
  US 20050024324A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0221 Arrangements for reducing keyboard size for transport or storage, e.g. foldable keyboards, keyboards with collapsible keys G06F3/0216 takes precedence; for keyboards integrated in a laptop computer G06F1/1666
• CPC Class Codes: G06F 3/0423 using sweeping light beams, e.g. using rotating or vibrating mirror
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/228 of three-dimensional handwriting, e.g. writing in the air

Abstract

A system used with a virtual device inputs or transfers information to a companion device, and includes two optical systems OS1, OS2. In a structured-light embodiment, OS1 emits a fan beam plane of optical energy parallel to and above the virtual device. When a user-object penetrates the beam plane of interest, OS2 registers the event. Triangulation methods can locate the virtual contact, and transfer user-intended information to the companion system. In a non-structured active light embodiment, OS1 is preferably a digital camera whose field of view defines the plane of interest, which is illuminated by an active source of optical energy. Preferably the active source, OS1, and OS2 operate synchronously to reduce effects of ambient light. A non-structured passive light embodiment is similar except the source of optical energy is ambient light. A subtraction technique preferably enhances the signal/noise ratio. The companion device may in fact house the present invention.

Claims

1-13. -13. (Cancelled)

14. A system enabling a user-manipulated user-object used with a virtual transfer device to transfer information to a companion device, the system comprising:

a central processor unit including memory storing at least one software routine;

a first optical system defining a plane substantially parallel-to and spaced-above a presumed location of said virtual transfer device;

a second optical system having a relevant field of view encompassing at least portions of said plane and responsive to user-object penetration of said plane to interact with said virtual transfer device;

means for determining relative position of a portion of said user-object on said plane;

wherein said system transfers information to said companion device enabling user-object with said virtual transfer device to affect operation of said companion device

15. The system of claim 14, wherein said means for determining includes determining said relative position using triangulation analysis.

16. The system of claim 14, wherein said means for determining includes said processor unit executing said routine to determine said relative position.

17. The system of claim 14, wherein:

said first optical system includes means for generating a plane of optical energy; and

said second optical system includes a camera sensor that detects a reflected portion of said optical energy when said user-object penetrates said plane.

18. The system of claim 14, wherein:

said first optical system includes at least one of (i) a laser to generate said plane, and (ii) an LED to generate said plane; and

said second optical system includes a camera sensor that detects a reflected portion of said optical energy when said user-object penetrates said plane.

19. The system of claim 14, further including means for enhancing responsiveness of said second optical system to said user-object penetration while decreasing said responsiveness to ambient light.

20. The system of claim 19, wherein said means for enhancing includes at least one of (a) providing a signature associated with generation of said plane, (b) selecting a common wavelength for energy within said plane defined by said first optical system and for responsiveness of said second optical system, and (c) synchronizing operation of said first optical system and operation of said second optical system.

21. The system of claim 14, wherein said first optical system includes a first camera sensor that defines said plane.

22. The system of claim 14, wherein:

said first optical system includes a first camera sensor that defines said plane;

said second optical system includes a second camera to sense said penetration;

and further including;

a source of optical energy directed generally toward said virtual transfer device; and

means for synchronizing operation of at least two of same first optical system, said second optical system, and said source of optical energy;

wherein effects of ambient light upon accuracy of information obtained with said system are reduced.

23. The system of claim 14, wherein:

said first optical system includes a generator of optical energy of a desired wavelength; and

said second optical system is sensitive substantially only to optical energy of said desired wavelength.

24. The system of claim 14, wherein said companion device includes at least one of (i) a PDA, (ii) a portable communication device, (iii) an electronic device, (iv) an electronic game device, and (v) a musical instrument, and said virtual transfer device is at least one of (I) a virtual keyboard, (II) a virtual mouse, (III) a virtual trackball, (IV) a virtual pen, (V) a virtual trackpad, and (VI) a user-interface selector.

25. The system of claim 14, wherein said virtual transfer device is mapped to a work surface selected from at least one of (i) a table top, (ii) a desk top, (iii) a wall, (iv) a point-of-sale appliance, (v) a point-of-service appliance, (vi) a kiosk, (vii) a surface in a vehicle, (viii) a projected display, (ix) a physical display, (x) a CRT, and (xi) an LCD.

26. The system of claim 14, wherein at least one of said first operating system and said second operating system is a camera sensor having a lens and an image plane;

wherein at least one of said lens ad said image plane is tilted to enhance at least one of resolution and depth of field.

27. The system of claim 14, further including means for enhancing distinguishment of said user-object from a background object.