APPARATUS AND METHOD FOR DETECTING A POINTER RELATIVE TO A TOUCH SURFACE

• US 20090146972A1
• Filed: 02/12/2009
• Published: 06/11/2009
• Est. Priority Date: 05/05/2004
• Status: Abandoned Application

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Holmgren, David E., MORRISON, GERALD D.
• Application Number:
  US12/369,904
• Publication Number:
  US 20090146972A1
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/85 Stereo camera calibration

Abstract

An apparatus for detecting a pointer relative to a touch surface includes at least two spaced imaging assemblies having overlapping fields of view encompassing the touch surface. The imaging assemblies see the touch surface in three-dimensions as a perspective view. The imaging assemblies acquire overlapping images from different locations. A processor receives and processes image data generated by at least one of the imaging assemblies to determine the location of the pointer relative to the touch surface.

Claims

1. An interactive input system comprising:

at least two imaging devices having overlapping fields of view of an input surface, each of said imaging devices comprising a lens having at least one form of distortion; and

processing structure determining at least one distortion parameter of each said lens during calibration.

2. An interactive input system according to claim 1 wherein said at least one form of distortion is radial distortion.

3. An interactive input system according to claim 1 wherein said at least one form of distortion is decentering distortion.

4. An interactive input system according to claim 1 wherein said at least one form of distortion is radial and decentering distortion.

5. An interactive input system according to claim 1 wherein the distortion of each said lens is expressed by:

Δ

_x=(x−

x₀)K₁((x−

x₀)²+(y−

y₀)²)

Δ

_y=(y−

y₀)K₁((x₁−

x₀)²+(y−

y₀)²)

where;

(Δ

_x, Δ

_y) represent radial and decentering distortion of the lens;

(x, y) are the co-ordinates of a point in an image captured by the imaging device corresponding to a point (X, Y, Z) in a three-dimensional scene;

(x₀, y₀) are the co-ordinates of the principal point of the imaging device, the location at which the optical axis of the imaging device meets the focal plane of the imaging device with the optical axis being approximately normal to the focal plane; and

K₁ is a lens parameter.

6. An interactive input system according to claim 1 wherein said processing structure determines said at least one distortion parameter of each said lens during a self-calibration routine.

7. An interactive input system according to claim 6 wherein said at least one form of distortion is radial distortion.

8. An interactive input system according to claim 6 wherein said at least one form of distortion is decentering distortion.

9. An interactive input system according to claim 6 wherein said at least one form of distortion is radial and decentering distortion.

10. An interactive input system according to claim 6 wherein the distortion of each said lens is expressed by:

Δ

_x=(x−

x₀)K₁((x−

x₀)²+(y−

y₀)²)

Δ

_y=(y−

y₀)K₁((x₁−

x₀)²+(y−

y₀)²)

where;

(Δ

_x, Δ

_y) represent radial and decentering distortion of the lens;

(x, y) are the co-ordinates of a point in an image captured by the imaging device corresponding to a point (X, Y, Z) in a three-dimensional scene;

(x₀, y₀) are the co-ordinates of the principal point of the imaging device, the location at which the optical axis of the imaging device meets the focal plane of the imaging device with the optical axis being approximately normal to the focal plane; and

K₁ is a lens parameter.

11. An interactive input system according to claim 1 wherein said input surface is one of planar, curved and non-planar.

12. An interactive input system according to claim 11 wherein said processing structure determines said at least one distortion parameter of each said lens during a self-calibration routine.

13. An interactive input system according to claim 12 wherein said at least one form of distortion is radial distortion.

14. An interactive input system according to claim 12 wherein said at least one form of distortion is decentering distortion.

15. An interactive input system according to claim 12 wherein said at least one form of distortion is radial and decentering distortion.

16. An interactive input system according to claim 12 wherein the distortion of each said lens is expressed by:

Δ

_x=(x−

x₀)K₁((x−

x₀)²+(y−

y₀)²)

Δ

_y=(y−

y₀)K₁((x₁−

x₀)²+(y−

y₀)²)

where;

(Δ

_x, Δ

_y) represent radial and decentering distortion of the lens;

(x, y) are the co-ordinates of a point in an image captured by the imaging device corresponding to a point (X, Y, Z) in a three-dimensional scene;

(x₀, y₀) are the co-ordinates of the principal point of the imaging device, the location at which the optical axis of the imaging device meets the focal plane of the imaging device with the optical axis being approximately normal to the focal plane; and

K₁ is a lens parameter.

17. An interactive input system according to claim 1 wherein said input surface is any surface within the overlapping fields of view of said imaging devices.

18. An interactive input system according to claim 1 wherein each of said imaging devices sees said input surface in three-dimensions as a perspective view and wherein said processing structure further processes image data generated by at least one of said imaging devices to determine the location of a pointer relative to said input surface.

19. An interactive input system according to claim 18 wherein each imaging device is calibrated to establish the relationship between points (X, Y, Z) in its perspective view and points (x, y) in acquired images, each imaging device generating pointer co-ordinate data when a pointer is captured in an acquired image.

20. An interactive input system according to claim 19 wherein said processing structure triangulates the pointer co-ordinate data to determine the location of the pointer relative to said input surface.

21. An interactive input system according to claim 20 wherein each imaging device is positioned relative to said input surface so that at a minimum the entire periphery of the input surface is within its field of view.

22. An apparatus according to claim 21 wherein said input surface is bordered by a bezel.

23. An interactive input system according to claim 20 wherein said processing structure determines said at least one distortion parameter of each said lens during a self-calibration routine.

24. An interactive input system according to claim 23 wherein said at least one form of distortion is radial distortion.

25. An interactive input system according to claim 23 wherein said at least one form of distortion is decentering distortion.

26. An interactive input system according to claim 23 wherein said at least one form of distortion is radial and decentering distortion.

27. An interactive input system according to claim 23 wherein the distortion of each said lens is expressed by:

Δ

_x=(x−

x₀)K₁((x−

x₀)²+(y−

y₀)²)

Δ

_y=(y−

y₀)K₁((x₁−

x₀)²+(y−

y₀)²)

where;

(Δ

_x, Δ

_y) represent radial and decentering distortion of the lens;

(x, y) are the co-ordinates of a point in an image captured by the imaging device corresponding to a point (X, Y, Z) in the three-dimensional scene;

(x₀, y₀) are the co-ordinates of the principal point of the imaging device, the location at which the optical axis of the imaging device meets the focal plane of the imaging device with the optical axis being approximately normal to the focal plane; and

K₁ is a lens parameter.

28. An interactive input system according to claim 23 wherein said input surface is one of planar, curved and non-planar.

29. An interactive input system according to claim 20 wherein each imaging device generates a certainty value representing the degree of certainty that the imaging device has positively identified the pointer in the acquired image.

30. An interactive input system according to claim 29 wherein said certainty value is used by said processing structure to determine pointer co-ordinate data to be used to determine the position of said pointer relative to said input surface.

31. An interactive input system according to claim 30 wherein said processing structure ignores pointer co-ordinate data generated by said imaging device when the certainty value associated therewith is below a threshold level.

32. An interactive input system according to claim 20 wherein the imaging device that detects a pointer in its acquired image first communicates data to the other imaging device to assist that imaging device to detect the pointer in its acquired image.

33. An interactive input system according to claim 32 wherein each imaging device also generates a certainty value representing the degree of certainty that the imaging device has positively identified the pointer in the acquired image.

34. An interactive input system according to claim 33 wherein said certainty value is used by said processing structure to determine pointer co-ordinate data to be used to determine the position of said pointer relative to said input surface.

35. An interactive input system according to claim 34 wherein said processing structure ignores pointer co-ordinate data generated by said imaging device when the certainty value associated therewith is below a threshold level.

36. An interactive input system according to claim 20 wherein each imaging device processes a subset of pixels in each acquired image.

37. An interactive input system according to claim 17 wherein said imaging devices are portable.

38. A camera-based interactive input system comprising:

a touch surface on which contacts are made using a pointer;

camera devices looking at said touch surface from different vantages and having overlapping fields, said camera devices acquiring images of said touch surface, each of said camera devices having an imperfect lens; and

processing structure receiving and processing said image data to determine the location of said pointer relative to said touch surface via triangulation, said processing structure compensating for image distortion as a result of the imperfect lens of each said camera device.

39. A camera-based interactive input system according to claim 38 wherein said processing structure determines at least one distortion parameter of each said lens during a self-calibration routine.

40. A camera-based interactive input system according to claim 39 wherein said at least one form of distortion is radial distortion.

41. A camera-based interactive input system according to claim 40 wherein said at least one form of distortion is decentering distortion.

42. A camera-based interactive input system according to claim 40 wherein said at least one form of distortion is radial and decentering distortion.

43. A camera-based interactive input system according to claim 40 wherein the distortion of each said lens is expressed by:

Δ

_x=(x−

x₀)K₁((x−

x₀)²+(y−

y₀)²)

Δ

_y=(y−

y₀)K₁((x₁−

x₀)²+(y−

y₀)²)

where;

(Δ

_x, Δ

_y) represent radial and decentering distortion of the lens;

(x, y) are the co-ordinates of a point in an image captured by the camera device corresponding to a point (X, Y, Z) in a three-dimensional scene;

(x₀, y₀) are the co-ordinates of the principal point of the camera device, the location at which the optical axis of the camera device meets the focal plane of the camera device with the optical axis being approximately normal to the focal plane; and

K₁ is a lens parameter.

44. A camera-based interactive input system according to claim 39 wherein said touch surface is one of planar, curved and non-planar.

45. A camera-based interactive input system according to claim 39 wherein each camera device generates a certainty value representing the degree of certainty that the camera device has positively identified the pointer in the acquired image.

46. A camera-based interactive input system according to claim 45 wherein said certainty value is used by said processing structure to determine pointer co-ordinate data to be used to determine the position of said pointer relative to said touch surface.

47. A camera-based interactive input system according to claim 46 wherein said processing structure ignores pointer co-ordinate data generated by said camera device when the certainty value associated therewith is below a threshold level.

48. A camera-based interactive input system according to claim 39 wherein the camera device that detects a pointer in its acquired image first communicates data to the other camera device to assist that camera device to detect the pointer in its acquired image.

49. A camera-based interactive input system according to claim 48 wherein each camera device also generates a certainty value representing the degree of certainty that the camera device has positively identified the pointer in the acquired image.

50. A camera-based interactive input system according to claim 49 wherein said certainty value is used by said processing structure to determine pointer co-ordinate data to be used to determine the position of said pointer relative to said touch surface.

51. A camera-based interactive input system according to claim 50 wherein said processing structure ignores pointer co-ordinate data generated by said camera device when the certainty value associated therewith is below a threshold level.

52. A camera-based interactive input system according to claim 39 wherein each camera device processes a subset of pixels in each acquired image.

53. A camera-based interactive input system according to claim 44 wherein said camera devices are portable.