Pointing input system and method using one or more array sensors

• US 7,956,842 B2
• Filed: 06/10/2005
• Issued: 06/07/2011
• Est. Priority Date: 06/11/2004
• Status: Expired due to Fees

• Current Assignee: Micro-Nits Co., Ltd.
• Original Assignee: Micro-Nits Co., Ltd.
• Inventors: Ting, Albert
• Primary Examiner(s): Patel; Nitin
• Application Number:
  US11/149,426
• Publication Number:
  US 20060007134A1
• Time in Patent Office: 2,188 Days
• Field of Search: 345156-158, 345180-183
• US Class Current: 345/156
• CPC Class Codes: G06F 3/0386 for light pen
• CPC Class Codes: G06F 3/04182 Filtering of noise external to the device and not generated by digitiser components
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

In a pointing input system and method, a pointer applies a light spot on a screen, an array sensor shoots on the screen to generate a first data for an identification system to retrieve a second data therefrom, the second data includes the position information of the light spot for an information system to apply a correlated output on the screen. The second data is generated based on an optical distortion information and a spatial rotation and displacement information determined by an alignment procedure that comprises applying an input for alignment on the screen for the array sensor to shoot to generate an alignment data, and comparing the alignment data with a reference data.

Claims

1. A pointing input method for an input system, the input system including a display system, a screen, an array sensor fixedly positioned in the vicinity of the screen, an optical lens in the array sensor, and a pointer isolated from the array sensor, said pointing input method comprising the steps of:

using the array sensor to shoot a test pattern on the screen in order to obtain alignment data including image information of the test pattern;

performing an alignment procedure for obtaining/using optical distortion information about the optical lens in the array sensor, spatial rotation, and displacement information from the alignment data;

applying a light spot on the screen by the pointer;

observing the screen with the array sensor to generate first data including image information of the light spot;

generating second data including position information of the light spot from the first data based on the optical distortion information and spatial rotation and displacement information generated from the alignment procedure; and

applying a correlated output on the screen based on the second data by the display system,

wherein the step of generating second data comprises checking the first data to find a group of nearby pixels that constitute a first data spot based on at least one optical feature.

2. The method of claim 1, wherein alignment data is generated using the test pattern that is shot on the screen, and wherein the step of performing an alignment procedure comprises the step of:

comparing the alignment data with reference data for generating the optical distortion information and spatial rotation and displacement information.

3. The method of claim 2, wherein the test pattern comprises optical spots displayed at a plurality of positions on the screen.

4. The method of claim 2, wherein the test pattern comprises a chessboard pattern, a square pattern, or a cross pattern.

5. The method of claim 2, wherein the reference data comprises a plurality of spatial coordinates of the test pattern.

6. The method of claim 2, wherein the step of performing an alignment procedure further comprises a bilinear interpolation for coordinates computations.

7. The method of claim 1, further comprising identifying a shooting range of the array sensor on the screen.

8. The method of claim 1, wherein the screen comprises a planar surface, a regularly curved surface, or an irregularly curved surface.

9. The method of claim 1, wherein the screen comprises a scattering screen, a CRT screen, an LCD screen, a plasma screen or a rear projection screen.

10. The method of claim 1, wherein the pointer comprises a plurality of light sources having different properties, a plurality of buttons and means for generating various trigger signals representative of different actions.

11. The method of claim 10, wherein the pointer further comprises a switch for determining when to apply the light spot to the screen for conducting different inputs.

12. The method of claim 1, wherein the array sensor comprises an optical lens and an optical filter for mapping onto the array sensor and filtering out optical noise to thereby enhance an identification effect.

13. The method of claim 1, wherein the display system comprises a front projector or a rear projector.

14. The method of claim 1, wherein the step of generating second data further comprises the step of:

obtaining information including a size, an average brightness, a hue, a length, a width and an area of the first data spot.

15. The method of claim 1, wherein the at least one optical feature comprises at least one of the brightness, the hue, the size and the shape.

16. The method of claim 1, wherein the at least one optical feature is used for determining a trigger signal of an action.

17. The method of claim 1, further comprising monitoring a continuous movement of the light spot and tracking a relative position of the continuous movement.

18. The method of claim 17, wherein the relative position is used for determining a trigger signal of an action or a pen stroke.

19. A pointing input system comprising:

an array sensor for shooting a test pattern on a screen;

a first buffer for storing reference data and alignment data generated following the shooting of the test pattern on the screen by the array sensor;

a processor for obtaining/using optical distortion information and spatial rotation and displacement information by analyzing the alignment data in comparison with the reference data;

a second buffer for storing the optical distortion information and spatial rotation and displacement information;

a pointer, isolated from the array sensor, for applying a light spot on the screen for the array sensor to observe in order to generate first data including image information of the light spot; and

an identification system for retrieving second data including position information of the light spot from the first data based on the optical distortion information and the spatial rotation and displacement information for a display system to apply a correlated output on the screen,

wherein the identification system comprises;

a register to provide data about at least one optical feature,

a third buffer for storing the first data, and

a second processor for checking the first data to find a group of nearby pixels that constitute a first data spot based on the at least one optical feature.

20. The pointing input system of claim 19, wherein the array sensor comprises a viewing window or a light beam for identifying a shooting range of the array sensor on the screen.

21. The pointing input system of claim 19, wherein the array sensor comprises an optical lens and an optical filter for mapping onto the array sensor and filtering out an optical noise to thereby enhance identification effect.

22. The pointing input system of claim 19, wherein the pointer comprises a plurality of light sources having different properties, a plurality of buttons and means for generating various trigger signals representative of different actions.

23. The pointing input system of claim 22, wherein the pointer further comprises a switch for determining when to apply the light spot to the screen for conducting different inputs.

24. The pointing input system of claim 19, wherein:

the second processor additionally checks the first data to obtain information including size, average brightness, hue, length, width and area of the first data spot.

25. The pointing input system of claim 19, wherein the at least one optical feature comprises at least one of brightness, hue, size and shape.

26. The pointing input system of claim 19, wherein the at least one optical feature is used for determining a trigger signal of an action.

27. The pointing input system of claim 19, wherein the identification system further monitors a continuous movement of the light spot and tracks a relative position of the continuous movement.

28. The pointing input system of claim 27, wherein the relative position is used for determining a trigger signal of an action or a pen stroke.

29. The pointing input system of claim 19, wherein the display system comprises a front projector or a rear projector.

30. The pointing input system of claim 19, wherein the screen comprises a planar surface, a regularly curved surface, or an irregularly curved surface.

31. The pointing input system of claim 19, wherein the screen comprises a scattering screen, a CRT screen, an LCD screen, a plasma screen or a rear projection screen.

32. A pointing input method for an input system, the input system including a display system, a screen, an array sensor fixedly positioned in the vicinity of the screen, an optical lens in the array sensor, and a pointer isolated from the array sensor, said pointing input method comprising the steps of:

using the array sensor to shoot a test pattern on the screen in order to obtain alignment data including the image information of the test pattern;

performing an alignment procedure for obtaining/using optical distortion information about the optical lens and the array sensor, spatial rotation and displacement information from the alignment data;

applying a light spot on the screen by the pointer;

observing the screen with the array sensor to generate a first data including image information of the light spot;

generating second data including position information of the light spot from the first data based on the optical distortion information and spatial rotation and displacement information generated from the alignment procedure; and

applying a correlated output on the screen based on the second data by the display system,

wherein the step of generating second data comprises;

checking the first data to find a group of nearby pixels that constitute a first data spot based on at least one optical feature; and

obtaining information including a size, an average brightness, a hue, a length, a width and an area of the first data spot.

33. A pointing input system comprising:

an array sensor for shooting a test pattern on a screen;

a first buffer for storing reference data and alignment data generated following the shooting of the test pattern on the screen by the array sensor;

a processor for obtaining/using optical distortion information and spatial rotation and displacement information by analyzing the alignment data in comparison with the reference data;

a second buffer for storing the optical distortion information and spatial rotation and displacement information;

a pointer, isolated from the array sensor, for applying a light spot on the screen for the array sensor to observe in order to generate first data including image information of the light spot; and

an identification system for retrieving second data including a position information of the light spot from the first data based on the optical distortion information and the spatial rotation and displacement information for a display system to apply a correlated output on the screen,

wherein the identification system comprises;

a register to provide data about at least one optical feature;

a third buffer for storing the first data; and

a second processor for checking the first data to find a group of nearby pixels that constitute a first data spot based on the at least one optical feature, and for checking the first data to obtain an information including size, average brightness, hue, length, width and area of the first data spot.