Interactive input system and method

• US 9,207,812 B2
• Filed: 01/11/2013
• Issued: 12/08/2015
• Est. Priority Date: 01/11/2012
• Status: Active Grant

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Rattray, Robbie, McGibney, Grant, Barton, Rob, Taylor, Cameron, Holmgren, David, Shivkumar, Peter, Martin, Nicole, Elman, Dmitry, Guillemin, Alexis, Marcil, Patrick, Zhang, Yun
• Primary Examiner(s): Dowling, William C
• Assistant Examiner(s): HOWARD, RYAN D
• Application Number:
  US13/823,483
• Publication Number:
  US 20140313166A1
• Time in Patent Office: 1,061 Days
• Field of Search: 353/122, 353/121, 353/29, 178/18.11, 178/18.09
• US Class Current: 1/1
• CPC Class Codes: F21V 21/14 Adjustable mountings
• CPC Class Codes: F21V 5/04 of lens shape
• CPC Class Codes: G03B 21/134 Projectors combined with typing apparatus or with printing apparatus
• CPC Class Codes: G06F 3/0354 with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
• CPC Class Codes: G06F 3/03542 Light pens for emitting or receiving light
• CPC Class Codes: G06F 3/0386 for light pen
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: H04N 25/50 Control of the SSIS exposure
• CPC Class Codes: H04N 9/3179 Video signal processing therefor
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A method for adjusting the position of a light curtain emitted by an illumination assembly over a surface, comprises determining a position of the light curtain; calculating a difference between the determined position of the light curtain and a desired position of the light curtain; and adjusting the position of the illumination assembly based on the calculated difference.

Claims

1. A method for adjusting the position of a light curtain emitted by an illumination assembly over a surface, comprising:

determining a position of the light curtain;

calculating a difference between the determined position of the light curtain and a desired position of the light curtain; and

adjusting the position of the illumination assembly based on the calculated difference, wherein determining the position of the light curtain comprises;

adjusting the position of the illumination assembly such that the light curtain intersects with the surface at a first location and capturing a first image frame of the surface;

tilting the illumination assembly by a first known angle such that the light curtain intersects with the surface at a second location and capturing a second image frame of the surface;

tilting the illumination assembly by a second known angle such that the light curtain intersects with the surface at a third location and capturing a third image frame of the surface; and

processing the captured image frames to determine the position of the light curtain.

2. The method of claim 1 wherein the adjusting comprises adjusting the position of the illumination assembly until the light curtain is substantially at the desired position.

3. The method of claim 1 further comprising:

processing the captured image frames to determine the line of intersection between the light curtain and the surface at each of said first, second and third locations, and using the lines of intersection to determine the position of the light curtain.

4. The method of claim 1 wherein the first known angle is with respect to one of an X′

and Y′

axis of the plane of the light curtain and the second known angle is with respect to the other of the X′ and

Y′

axis of the plane of the light curtain.

5. The method of claim 1 further comprising prompting a user to adjust the position of the illumination assembly.

6. The method of claim 5 wherein the prompting comprises displaying a set of instructions to the user via a projection module or a display unit.

7. The method of claim 5 wherein the prompting comprises broadcasting a set of instructions to the user via a speaker system.

8. The method of claim 2 further comprising:

processing the captured image frames to determine the line of intersection between the light curtain and the surface at each of said first, second and third locations, and using the lines of intersection to determine the position of the light curtain.

28. The method of claim 7 wherein the prompting comprises displaying a set of instructions to the user via a projection module or a display unit.

9. A method for adjusting the position of a light curtain emitted by an illumination assembly over a surface, comprising:

determining a position of the light curtain by measuring the distance between the light curtain and the surface at least at two locations;

calculating a difference between the determined position of the light curtain and a desired position of the light curtain by comparing the distance between the light curtain and the surface at a first location with the distance between the light curtain and the surface at a second location; and

adjusting the position of the illumination assembly based on the calculated difference.

10. The method of claim 9 wherein the measuring comprises placing a gauge tool adjacent to the surface.

11. The method of claim 10 wherein said measuring comprises processing captured image frames of said gauge tool to determine the distance at least at the two locations.

12. The method of claim 11 wherein an imaged surface of said gauge tool comprises at least one measurable parameter having a value determined by the distance between the surface and the light curtain.

13. The method of claim 9 further comprising prompting a user to adjust the position of the illumination assembly.

14. The method of claim 13 wherein the prompting comprises displaying a set of instructions to the user via a projection module or a display unit.

15. The method of claim 13 wherein the prompting comprises broadcasting a set of instructions to the user via a speaker system.

16. The method of claim 9 wherein the adjusting comprises adjusting the position of the illumination assembly until the light curtain is substantially at the desired position.

17. A method for adjusting the position of a light curtain emitted by an illumination assembly over a surface, comprising:

determining a position of the light curtain;

calculating a difference between the determined position of the light curtain and a desired position of the light curtain; and

adjusting the position of the illumination assembly based on the calculated difference, wherein said determining comprises;

capturing image frames of a gauge tool moving across said surface and processing the image frames to obtain a set of data points of light curtain reflections;

fitting the data points to a plane represented by a mathematical model to determine the coefficients of the mathematical model; and

using the coefficients to determine the required light curtain adjustment.

18. The method of claim 17 wherein the mathematical model of the plane is selected assuming the surface is planar.

19. The method of claim 18 wherein the determined light curtain adjustment is the adjustment required to orient the light curtain generally parallel to the surface.

20. The method of claim 17 wherein the data points are fitted to the plane via a least-squares method and said coefficients are solved via singular value decomposition.

21. The method of claim 19 wherein the data points of said set are processed to reduce outliers.

22. The method of claim 21 wherein random consensus fitting is employed to determine a consensus set of data points to use during fitting.

23. The method of claim 17 wherein said gauge tool is moved across different portions of said surface in different patterns and wherein a set of data points of light curtain reflections for each pattern are obtained, the sets of data points being used to determine the coefficients of the mathematical model.

24. The method of claim 17 wherein the mathematical model of the plane is selected assuming the surface is warped or curved.

25. The method of claim 24 wherein the determined light curtain adjustment is the adjustment required to orient the light curtain so that the minimum separation between the light curtain and any location on the surface is beyond a threshold.

26. The method of claim 24 wherein the data points are fit to said plane using an M-estimate fitting approach.

27. The method of claim 17 further comprising prompting a user to adjust the position of the illumination assembly.

29. The method of claim 27 wherein the prompting comprises broadcasting a set of instructions to the user via a speaker system.

30. The method of claim 17 wherein the adjusting comprises adjusting the position of the illumination assembly until the light curtain is substantially at the desired position.

31. An interactive input system comprising:

an illumination assembly configured to illuminate a light curtain over a surface;

an imaging module configured to capture image frames of the surface; and

processing structure configured to process captured image frames to determine the location of one or more pointers brought into proximity with said surface and breaking the plane of said light curtain, said processing structure, during calibration, further being configured to perform the method of;

determining a position of the light curtain;

calculating a difference between the determined position of the light curtain and a desired position of the light curtain; and

adjusting the position of the illumination assembly based on the calculated difference, wherein determining the position of the light curtain comprises;

adjusting the position of the illumination assembly such that the light curtain intersects with the surface at a first location and capturing a first image frame of the surface;

tilting the illumination assembly by a first known angle such that the light curtain intersects with the surface at a second location and capturing a second image frame of the surface;

tilting the illumination assembly by a second known angle such that the light curtain intersects with the surface at a third location and capturing a third image frame of the surface; and

processing the captured image frames to determine the position of the light curtain.

32. A method for adjusting the position of a light curtain emitted by an illumination assembly over a surface, comprising:

determining a position of the light curtain by measuring the distance between the light curtain and the surface at least at two locations;

calculating a difference between the determined position of the light curtain and a desired position of the light curtain; and

adjusting the position of the illumination assembly based on the calculated difference, wherein the measuring comprises placing a gauge tool adjacent to the surface and processing captured image frames of said gauge tool to determine the distance at least at the two locations.

33. The method of claim 32 wherein an imaged surface of said gauge tool comprises at least one measurable parameter having a value determined by the distance between the surface and the light curtain.

34. The method of claim 32 further comprising prompting a user to adjust the position of the illumination assembly.

35. The method of claim 34 wherein the prompting comprises displaying a set of instructions to the user via a projection module or a display unit.

36. The method of claim 34 wherein the prompting comprises broadcasting a set of instructions to the user via a speaker system.

37. The method of claim 32 wherein the adjusting comprises adjusting the position of the illumination assembly until the light curtain is substantially at the desired position.

38. An interactive input system comprising:

an illumination assembly configured to illuminate a light curtain over a surface;

an imaging module configured to capture image frames of the surface; and

processing structure configured to process captured image frames to determine the location of one or more pointers brought into proximity with said surface and breaking the plane of said light curtain, said processing structure, during calibration, further being configured to perform the method of;

determining a position of the light curtain by measuring the distance between the light curtain and the surface at least at two locations;

calculating a difference between the determined position of the light curtain and a desired position of the light curtain by comparing the distance between the light curtain and the surface at a first location with the distance between the light curtain and the surface at a second location; and

adjusting the position of the illumination assembly based on the calculated difference.

39. An interactive input system comprising:

an illumination assembly configured to illuminate a light curtain over a surface;

an imaging module configured to capture image frames of the surface; and

processing structure configured to process captured image frames to determine the location of one or more pointers brought into proximity with said surface and breaking the plane of said light curtain, said processing structure, during calibration, further being configured to perform the method of;

determining a position of the light curtain;

calculating a difference between the determined position of the light curtain and a desired position of the light curtain; and

adjusting the position of the illumination assembly based on the calculated difference, wherein said determining comprises;

capturing image frames of a gauge tool moving across said surface and processing the image frames to obtain a set of data points of light curtain reflections;

fitting the data points to a plane represented by a mathematical model to determine the coefficients of the mathematical model; and

using the coefficients to determine the required light curtain adjustment.