Size/scale and orientation determination of a pointer in a camera-based touch system

• US 6,954,197 B2
• Filed: 11/15/2002
• Issued: 10/11/2005
• Est. Priority Date: 11/15/2002
• Status: Expired due to Term

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Su, Scott, Morrison, Gerald, Akitt, Trevor
• Primary Examiner(s): Shankar, Vijay
• Assistant Examiner(s): DHARIA, PRABODH M
• Application Number:
  US10/294,917
• Publication Number:
  US 20040095318A1
• Time in Patent Office: 1,061 Days
• Field of Search: 345156-158, 345168-175, 345180-183, 345/39, 178 1801- 1804
• US Class Current: 345/158
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

A touch system includes a touch surface and at least two cameras associated with the touch surface. The at least two cameras acquire images of the touch surface from different locations and having overlapping fields of view. A digital signal processor is associated with each camera. The digital signal processors process pixel data acquired by selected pixel subsets of the at least two digital cameras to generate pointer characteristic data when a pointer exists in the acquired images. The pointer characteristic data identifies edges of the pointer. A master digital signal processor triangulates the edge information in the pointer characteristic data to determine a bounding area that represents a pointer perimeter.

Claims

1. A touch system comprising:

at least two optical recording devices associated with a touch surface, said at least two optical recording devices acquiring images of said touch surface from different locations and having overlapping fields of view; and

processing circuitry processing image data acquired by said at least two optical recording devices to detect edges of a pointer in said images and to triangulate the detected edges to determine a bounding area that defines a pointer perimeter.

2. A touch system according to claim 1 wherein said processing circuitry further determines the center of the bounding area thereby to determine the center of said pointer.

3. A touch system according to claim 2 wherein said processing circuitry examines the shape of the bounding area to determine the orientation of said pointer relative to said touch surface.

4. A touch system according to claim 3 wherein said touch surface is generally rectangular and wherein an optical recording device is positioned adjacent each corner of said touch surface, said processing circuitry triangulating detected edges in images captured by multiple pairs of optical recording devices to yield multiple bounding areas, the smallest of said bounding areas being selected as the bounding area defining said pointer perimeter.

5. A touch system according to claim 4 wherein said processing circuitry averages the centers of said multiple bounding areas to determine the center of said pointer.

6. A touch system according to claim 5 wherein said optical recording devices are digital cameras.

7. A touch system according to claim 6 wherein said digital cameras are CMOS digital cameras having selectable pixel arrays, pixel data from subsets of said pixel arrays being processed by said processing circuitry.

8. A touch system according to claim 7 wherein said processing circuitry includes a camera processor associated with each digital camera, each camera processor processing pixel data to detect edges of said pointer.

9. A touch system according to claim 8 wherein said processing circuitry further includes a master processor triangulating the detected edges to determine said multiple bounding areas and to select said smallest bounding area.

10. A touch system according to claim 1 wherein said processing circuitry superimposes a model over said bounding area, said model having a size proportional to said bounding area and defining said pointer perimeter.

11. A touch system according to claim 10 wherein said processing circuitry determines the center of said model thereby to determine the center of said pointer.

12. A touch system according to claim 11 wherein said model is generally rectangular and is centred over said bounding area.

13. A touch system according to claim 1 wherein said optical recording devices are digital cameras having selectable pixel arrays, pixel data at least from pixel rows capturing the region of said pointer in contact with said touch surface being processed to determine said bounding area.

14. A touch system according to claim 13 wherein said processing circuitry examines the shape of said bounding area to determine the orientation of said pointer relative to said touch surface.

15. A touch system according to claim 14 wherein pixel data from pixel rows capturing regions of said pointer along its length are processed to determine a series of bounding areas, said series of bounding areas being stacked to generate a volumetric representation of said pointer.

16. A touch system comprising:

at least two digital cameras associated with a touch surface, said at least two digital cameras acquiring images of said touch surface from different locations and having overlapping fields of view; and

processing circuitry processing pixel data acquired by said at least two digital cameras to detect edges of a pointer in said images and to triangulate the detected edges to determine at least one bounding area that represents a pointer perimeter.

17. A touch system according to claim 16 wherein said processing circuitry further determines the center of the bounding area thereby to determine the center of said pointer.

18. A touch system according to claim 17 wherein said processing circuitry examines the shape of the bounding area to determine the orientation of said pointer relative to said touch surface.

19. A touch system according to claim 18 wherein said at least two digital cameras are CMOS digital cameras having selectable pixel arrays, said processing circuitry processing pixel data from subsets of said pixel arrays.

20. A touch system according to claim 19 wherein said processing circuitry includes a camera processor associated with each digital camera, each camera processor processing pixel data to detect edges of said pointer.

21. A touch system according to claim 20 wherein said processing circuitry further includes a master processor triangulating the detected edges to determine said bounding area.

22. A touch system according to claim 16 wherein said touch surface is generally rectangular and wherein a digital camera is positioned adjacent each corner of the touch surface, said processing circuitry triangulating detected edges in images captured by multiple pairs of digital cameras to yield multiple bounding areas, the smallest bounding area being selected to represent said pointer perimeter.

23. A touch system according to claim 22 wherein said processing circuitry averages the centers of said multiple bounding areas to determine the center of said pointer.

24. A touch system according to claim 22 wherein said processing circuitry superimposes a model over each of said bounding areas, the model superimposed on the smallest bounding area defining said pointer perimeter.

25. A touch system according to claim 24 wherein said processing circuitry averages the centers of said models thereby to determine the center of said pointer.

26. A touch system according to claim 16 wherein said digital cameras have selectable pixel arrays, pixel data at least from pixel rows capturing the region of said pointer in contact with said touch surface being processed to determine said bounding area.

27. A touch system according to claim 26 wherein said processing circuitry examines the shape of said bounding area to determine the orientation of said pointer relative to said touch surface.

28. A touch system according to claim 27 wherein pixel data from pixel rows capturing regions of said pointer along its length are processed to determine a series of bounding areas, said series of bounding areas being stacked to generate a volumetric representation of said pointer.

29. A touch system comprising:

a touch surface;

at least two cameras associated with said touch surface, said at least two cameras acquiring images of said touch surface from different locations and having overlapping fields of view;

a digital signal processor associated with each camera, the digital signal processors associated with said at least two cameras processing pixel data acquired by selected pixel subsets of the at least two digital cameras to generate pointer characteristic data when a pointer exists in said acquired images, said pointer characteristic data identifying edges of said pointer; and

a master digital signal processor triangulating the edge information in said pointer characteristic data to determine a bounding area defining a pointer perimeter.

30. A touch system according to claim 29 wherein said processing circuitry further determines the center of the bounding area thereby to determine the center of said pointer.

31. A touch system according to claim 30 wherein said processing circuitry examines the shape of the bounding area to determine the orientation of said pointer relative to said touch surface.

32. A touch system according to claim 31 wherein said at least two cameras are CMOS digital cameras having selectable pixel arrays, said processing circuitry processing pixel data from subsets of said pixel arrays.

33. A touch system according to claim 32 wherein said processing circuitry includes a camera processor associated with each digital camera, each camera processor processing pixel data to detect the edges of said pointer.

34. A touch system according to claim 33 wherein said processing circuitry further includes a master processor triangulating the detected edges to determine said bounding area.

35. A touch system according to claim 34 wherein said touch surface is generally rectangular and wherein a digital camera is positioned adjacent each corner of the touch surface, said processing circuitry triangulating detected edges in images captured by multiple pairs of digital cameras to yield multiple bounding areas, the smallest bounding area being selected to represent said pointer perimeter.

36. A touch system according to claim 35 wherein said processing circuitry averages the centers of said multiple bounding areas to determine the center of said pointer.

37. A touch system according to claim 35 wherein said processing circuitry superimposes a model over each of said bounding areas, the model superimposed on the smallest bounding areas defining said pointer perimeter.

38. A touch system according to claim 37 wherein said processing circuitry averages the centers of said models thereby to determine the center of said pointer.

39. A method of detecting the perimeter of a pointer relative to a touch surface comprising the steps of:

acquiring multiple images of a pointer relative to said touch surface;

processing pixel data resulting from said images to detect edges of said pointer; and

triangulating the detected edges to determine a bounding area representing said perimeter.

40. The method of claim 39 further comprising the step of determining the center of the bounding area thereby to determine the center of the pointer.

41. The method of claim 40 further comprising examining the shape of the bounding area to determine the orientation of the pointer relative to the touch surface.

42. The method claim 39 wherein during said processing pixel data associated with the region of said pointer in contact with said touch surface is processed and triangulated.

43. The method of claim 42 further comprising determining the center of the bounding area thereby to determine the center of the pointer.

44. The method of claim 43 further comprising examining the shape of the bounding area to determine the orientation of the pointer relative to the touch surface.

45. The method of claim 44 wherein pixel data associated with regions of said pointer along its length are processed and triangulated to yield as series of bounding areas, said series of bounding areas being stacked to generate a volumetric representation of said pointer.

46. A method of erasing electronic ink presented on a touch surface in response to movement of a pointer over said touch surface comprising the steps of:

tracking movement of said pointer over said touch surface;

determining the perimeter of the pointer that is in contact with said touch surface; and

erasing electronic ink contacted by said determined perimeter.

47. The method of claim 46 wherein said perimeter is represented by a model having a size proportional to said perimeter.

48. The method of claim 46 further comprising the steps of examining the shape of said perimeter to determine the orientation of said pointer relative to said touch surface.

49. A touch system comprising:

at least two imaging devices associated with a touch surface, said at least two imaging devices acquiring images of said touch surface from different locations and having overlapping fields of view; and

processing structure processing image data acquired by said at least two imaging devices to detect a pointer in said images and to determine a bounding area encompassing a tip of said pointer using triangulation.

50. A touch system according to claim 49 wherein said processing structure further determines the center of the bounding area thereby to determine the center of said pointer.

51. A touch system according to claim 50 wherein said processing structure examines the shape of the bounding area to determine the orientation of said pointer relative to said touch surface.

52. A touch system according to claim 51 wherein said touch surface is generally rectangular and wherein an imaging device is positioned adjacent each corner of said touch surface, said processing structure triangulating pointer data in images captured by multiple pairs of imaging devices to yield multiple bounding areas, the smallest of said bounding areas being selected as the bounding area encompassing said pointer tip.

53. A touch system according to claim 52 wherein said processing structure averages the centers of said multiple bounding areas to determine the center of said pointer.

54. A touch system according to claim 49 wherein said processing structure superimposes a model over said bounding area, said model having a size proportional to said bounding area and defining said pointer tip.

55. A touch system according to claim 54 wherein said processing structure determines the center of said model thereby to determine the center of said pointer.

56. A touch system according to claim 55 wherein said model is generally rectangular and is centred over said bounding area.

57. A touch system comprising:

at least two digital cameras associated with a touch surface, said at least two digital cameras acquiring images of said touch surface from different locations and having overlapping fields of view; and

processing structure processing pixel data acquired by said at least two digital cameras to detect a pointer in said images and to determine at least one bounding area that encompasses a tip of said pointer using triangulation.

58. A touch system according to claim 57 wherein said processing structure further determines the center of the bounding area thereby to determine the center of said pointer.

59. A touch system according to claim 58 wherein said processing structure examines the shape of the bounding area to determine the orientation of said pointer relative to said touch surface.

60. A touch system according to claim 59 wherein said at least two digital cameras comprise CMOS digital cameras having selectable pixel arrays, said processing circuitry processing pixel data from subsets of said pixel arrays.

61. A touch system according to claim 57 wherein said touch surface is generally rectangular and wherein a digital camera is positioned adjacent each corner of the touch surface, said processing structure triangulating pointer data in images captured by multiple pairs of digital cameras to yield multiple bounding areas, the smallest bounding area being selected as the bounding area encompassing said pointer tip.

62. A touch system according to claim 61 wherein said processing structure averages the centers of said multiple bounding areas to determine the center of said pointer.

63. A touch system according to claim 57 wherein said processing structure superimposes a model over said bounding area, said model having a size proportional to said bounding area and defining said pointer tip.

64. A touch system according to claim 63 wherein said processing structure determines the center of said model thereby to determine the center of said pointer.

65. A touch system according to claim 64 wherein said model is generally rectangular and is centred over said bounding area.

66. A method of detecting a pointer relative to a touch surface comprising the steps of:

acquiring multiple images of a pointer relative to said touch surface;

processing pixel data resulting from said images to detect said pointer; and

determining a bounding area encompassing a tip of said pointer using triangulation.

67. The method of claim 66 further comprising determining the center of the bounding area thereby to determine the center of the pointer.

68. The method of claim 66 further comprising examining the shape of the bounding area to determine the orientation of the pointer relative to the touch surface.