INTERACTIVITY IN A LARGE FLAT PANEL DISPLAY

• US 20080309641A1
• Filed: 06/14/2008
• Published: 12/18/2008
• Est. Priority Date: 06/15/2007
• Status: Abandoned Application

• Current Assignee: Luidia Incorporated (Luidia Global Co., Ltd.)
• Original Assignee: Luidia Incorporated (Luidia Global Co., Ltd.)
• Inventors: Harel, Jacob, Holtzman, Raphael, Ding, Yao, Hill, Fredrick N., Muir, Arthur H. III
• Application Number:
  US12/139,452
• Publication Number:
  US 20080309641A1
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0433 in which the acoustic waves are either generated by a movable member and propagated within a surface layer or propagated within a surface layer and captured by a movable member

Abstract

An apparatus is provided that comprises a hand held device comprising a first transmitter and a second transmitter; a flat panel display having a surface and a plurality of receivers placed near or proximate to the surface for receiving signals transmitted from the hand held device; and one or more processors coupled to the receivers such that when the hand held device is placed in close proximity to or on the surface of the flat panel display, a working area within the surface is defined, the working area interacting with the hand held device for determining a location of the hand held device in relation to the surface.

Claims

1. An apparatus comprising:

a hand held device comprising a first transmitter that includes a mechanical wave transmitted and a second transmitter that includes an electromagnetic wave transmitter;

a flat panel display having a surface and a plurality of receivers placed near or proximate to the surface for receiving signals transmitted from the hand held device; and

one or more processors coupled to the receivers such that when the hand held device is placed in close proximity to or on the surface of the flat panel display, a working area within the surface is defined, the working area interacting with the hand held device for determining a location of the hand held device in relation to the surface;

wherein at least some of the receivers include electromagnetic wave based sensors, and a plurality of mechanical wave based sensors,

wherein the one or more processors are configured to correct for non-uniformities to the mechanical wave propagation time.

2. The apparatus as recited in claim 1, wherein the hand held device comprises a stylus.

3. The apparatus as recited in claim 1, wherein the receivers include at least one environmental sensor for sensing one or more environmental parameters that affect the mechanical wave propagation time.

4. The apparatus as recited in claim 1, wherein the mechanical wave transmitter comprises an ultrasound transmitter.

5. The apparatus as recited in claim 1, wherein the electromagnetic wave transmitter comprises an infrared transmitter.

6. The apparatus as recited in claim 1, wherein the stylus comprises one or more buttons such that the receivers are able to detect that some of the one or more buttons are pressed and a location of the stylus at the time of pressing.

7. The apparatus as recited in claim 1, wherein the hand held device comprises a primary inductor and a secondary inductor both storing magnetic flux that are closely coupled, winding turn ratio of the secondary to the primary inductor being greater than 1.

8. The apparatus as recited in claim 1, wherein, in the case there are a plurality of sensors, the plurality of sensors are coupled in parallel, and are placed with the same x and y coordinate on the working area having a z direction offset.

9. The apparatus as recited in claim 1, wherein at least two of the mechanical wave transducers are oriented with a known geometric relationship to one another, and wherein the known geometric relation is used together with measurements that are affected by the propagation time to correct for the non-uniformities.

10. The apparatus as recited in claim 9, wherein a first receiver is placed in substantially orthogonal orientation to a second receiver.

11. The apparatus as recited in claim 1, wherein the receivers have fixed or adjustable Z directional offset, the z-direction being orthogonal to the surface of the flat panel display, and wherein the offset is compensated for using a first calibration method.

12. The apparatus as recited in claim 1, wherein the hand held device has fixed or adjustable Z directional offset, the Z-direction being the direction orthogonal to the surface, and wherein the offset is compensated for using a second calibration method.

13. A method comprising the steps of:

placing a handheld device in a proximity of a flat surface;

receiving or sensing a least one ultrasound signal at a plurality of ultrasound sensors, and at least one infrared signal at one or more infrared sensors, the sensors located on or close to the flat surface; and

processing the sensed at least one ultrasound signal and at least one infrared signals to determine the location of the handheld device, including correcting for non-uniformities to the mechanical wave propagation time.

14. The method as recited in claim 13, further comprising one or more of:

Stacking up multiple mechanical wave sensors to reduce temperature dependency of the propagation time;

increasing the sensitivity of the mechanical wave sensors, e.g., ultrasound sensors, by using, for example, use a discrete low noise amplifier;

connecting multiple receivers to a digital signal procession unit to create actively switched zoning based on where the pen or stylus; and

/or

using temperature sensor to compensate the variation in the time of flight of the mechanical waves, e.g., ultrasound. When more than one set of receivers 208 are used, a compensation method associated therewith can calculate the temperature based on redundant sets of coordinates; and

/or

adding z direction offset in the receiver 208 and stylus 202, where the z-direction is perpendicular to the plane of the surface 14.

15. The method as recited in claim 13 further comprising the step of sensing one or more environmental parameters which affect the ultrasound propagation time, and using the sensed one or more environmental parameters in the correcting for non-uniformities.

16. The method as recited in claim 15, wherein the environmental parameters comprise a temperature.

17. The method as recited in claim 13, wherein more than one sensor is places at the same x and y location, including stacking different sensors with a z-coordinate offset.

18. The method as recited in claim 13, wherein two or more of the ultrasound sensors are places at known geometric orientation to one another, and wherein the correcting for non-uniformities uses the known orientation.

19. The method as recited in claim 13, wherein the two or more of the ultrasound sensors are placed substantially orthogonal to one another such that the known orientation is substantial orthogonality.

20. The method as recited in claim 13, wherein the processing step comprises using at least one processor coupled to the sensors.

21. The method as recited in claim 20, wherein the coupling between the sensors and the at least one processor is via wireless means.

22. The method as recited in claim 20, wherein the coupling between the sensors and the at least one processor is via wire line means.

23. An apparatus comprising:

a stylus including an electromagnetic wave transmitter and a mechanical wave transmitter,

a plurality of receivers placed near the surface of a flat panel display, the receivers including at least one electromagnetic wave based sensor, and a plurality of mechanical wave based sensors; and

one or more processors coupled to the receivers, the processors configured to determine the position of the stylus on a working area defined on the surface when the stylus is transmitting, including correct for variation in propagation time of the mechanical waves.

24. The apparatus as recited in claim 23, further comprising:

at least one environmental sensor for sensing one or more environmental parameters which affect the mechanical wave propagation time;

wherein the determining of the position takes into account the environmental parameter or parameters sensed by the at least one environmental sensor to correct for propagation time variation.

25. The apparatus as recited in claim 23, wherein at least two of the mechanical wave sensors have known geometric orientation to one another, such that the correcting for variation in propagation time of the mechanical waves takes into account the known geometric orientation.

26. An apparatus as recited in claim 23, wherein the one or more processors are coupled to a host processing system, and wherein the host processing system is operative to receive series of states and locations of the stylus, including whether or not any buttons are pressed in the case the stylus includes buttons.

27. An apparatus as recited in claim 23, wherein the flat panel display includes one or more of an LCD display, a plasma display, and/or a rear projection display, or combination of such displays.

28. An apparatus as recited in claim 23, wherein the mechanical wave sensors are ultrasound sensors, and wherein each mechanical wave receiver includes one or a plurality of sensors on each location along the surface, wherein, in the case there are a plurality of sensors, the plurality are coupled in parallel, and are placed with the same x and y coordinate on the working area with z direction offset.

29. An apparatus recited in claim 23, comprising a plurality of sets of receivers arranged to extend the area of the working surface compared to only using a pair of mechanical wave sensors and one electromagnetic wave sensor, wherein the plurality of receivers are coupled to an equal or smaller number of processors.

30. An apparatus as recited in claim 29, wherein a memory is included containing software when executed by one or more processors implementing a method to actively manage the plurality of said receivers and only couple the signals of interest to one or more of the relevant processing units.

31. An apparatus as recited in claim 29, wherein at least two of the mechanical wave sensors are places with orthogonal or near orthogonal orientation to one another, such that the correcting for variation in propagation time of the mechanical waves takes into account the known geometric orientation

32. An apparatus as recited in claim 31, wherein near orthogonal orientation is between 85 and 95 degrees to each other.

33. An apparatus as recited in claim 32, wherein near orthogonal orientation is between 80 and 100 degrees to each other.

34. An apparatus as recited in claim 29, wherein a memory is included containing software when executed by one or more processors implementing a method to calculate the styli position based on the coordinates from a plurality of receivers, with greater weighting associated with the coordinate which has less sensitivity to the ultrasound time-of-flight variation, and lesser weighting associated with the coordinate which has greater sensitivity to the ultrasound time-of-flight variation.

35. An apparatus as recited in claim 29, wherein the receivers include a plurality channels of ambient IR sensors, the ambient IR sensors'"'"' most sensitive directions being positioned away from the working area.

36. An apparatus as recited in claim 35, wherein a memory is included containing software when executed by one or more processors implementing a method to adjust the gain and includes an amplifier of said ambient IR sensors, and to actively subtract the ambient IR noise from the main said IR sensors.

37. An apparatus as recited in claim 35, wherein a memory is included containing software when executed by one or more processors implementing a method to implement an infrared phase locked loop method and a method of running in a free run IR state, wherein the output of the method is the result of either the infrared phase locked loop method or the free run IR state method, with switch-over between the two states actively managed based on one or more pre-defined conditions.

38. An apparatus as recited in claim 37, further comprising a memory to store timing data for received signals when the phase locked loop state is active, wherein the free run IR method recreates IR data based on the data generated by the phase locked loop method stored in the memory, with greater weighting associated with the most recent data, and wherein the free run IR method is operative to generate the same IR data when the phase lock loop method stops operating, and wherein the free run IR method is operative to track the phase locked loop method output immediately when the phase locked loop method resumes.

39. An apparatus as recited in claim 26, wherein the host processing system includes a memory containing software that when executed is operative to comprise an algorithm to correct any mismatch between coordinates generated from a free run IR method and subsequent resumption of a phase locked loop method of generating an IR signal.

40. An apparatus as recited in claim 29, further comprising at least one environmental sensor for sensing environmental parameters which affect the ultrasound propagation time, wherein a memory is included containing software when executed by one or more processors implementing a method to calculate the current speed of ultrasound based on the parameters coupled from the one or more environmental sensor.

41. An apparatus as recited in claim 29, wherein a memory is included containing software when executed by one or more processors implementing a method to calculate the current speed of ultrasound based on redundant coordinates generated from a plurality of said receivers.

42. An apparatus as recited in claim 23, wherein the receivers have fixed or adjustable Z directional offset, the z-direction being orthogonal to the surface of the flat panel display, and wherein the offset is compensated for using a calibration method.

43. An apparatus as recited in claim 23, wherein the stylus has fixed or adjustable Z directional offset, the Z-direction being the direction orthogonal to the surface, and wherein the offset is compensated for using a calibration method.

44. An apparatus as recited in claim 23, wherein the stylus includes a tip and has a temperature sensor located at the tip, and sends the temperature around the tip to said receivers encoded in one or more signals transmitted by the stylus.

45. An apparatus as recited in claim 23, wherein the mechanical wave sensors are ultrasound transducers that can serve as both sensors and transmitters, such that a calibration method can be used using the one or more ultrasound transducers transmitting and one or more transducers receiving.

46. An apparatus as recited in claim 27, further comprising one or more secondary working areas next to a main working area, such as a table where the flat panel displays are placed on, and wherein a plurality of receivers are placed in the secondary working area for capturing the position and possibly state of the stylus when the stylus moves into the secondary working area.