Camera-based touch system

• US 9,176,627 B2
• Filed: 04/01/2010
• Issued: 11/03/2015
• Est. Priority Date: 07/05/2000
• Status: Expired due to Term

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Morrison, Gerald, Holmgren, David
• Primary Examiner(s): Boddie, William
• Assistant Examiner(s): Shapiro, Leonid
• Application Number:
  US12/753,017
• Publication Number:
  US 20100188370A1
• Time in Patent Office: 2,042 Days
• Field of Search: 345156-158, 345180-183
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

A camera-based touch system (50) includes a passive touch surface (60) and at least two cameras (63) associated with the touch surface. The at least two cameras (63) have overlapping fields of view (FOV) encompassing the touch surface. The at least two cameras (63) acquire images of the touch surface from different locations and generate image data. A processor (54) receives and processes image data generated by the at least two cameras to determine the location of the pointer relative to the touch surface when the pointer is captured in images acquired by the at least two cameras. Actual pointer contact with the touch surface and pointer hover above the touch surface can be determined.

Claims

1. An imaging assembly comprising:

a substantially rectangular frame to surround a region of interest in front of a display surface; and

at least two digital cameras, each digital camera having a selectable pixel array and being mounted adjacent a different corner of said frame, each digital camera having a field of view that (i) images at least a portion of said display surface, (ii) images a least a portion of said frame, and (iii) extends at least beyond an adjacent peripheral edge of said frame, said digital cameras being oriented to capture overlapping images of said region of interest, each digital camera further comprising a processor configured to process a subset of pixel data of images captured by the digital camera to detect the existence of a pointer therein, wherein during processing of each captured image, said processor forms a difference image representing differences between pixel data in the captured image and corresponding pixel data in a background image, generates a histogram comprising a plurality of elements by selectively summing pixel intensities of said difference image, compares the elements of the histogram to a noise threshold, determines the existence of a pointer when a plurality of elements of the histogram exceed the noise threshold and determines the center of the pointer by determining the location of the element of the plurality having the highest value.

2. An imaging assembly according to claim 1, wherein each digital camera comprises a CMOS digital camera.

3. An imaging assembly according to claim 1, wherein imaging components and the processor of each digital camera are mounted on a common board.

4. An imaging assembly according to claim 3, wherein a subset of pixels in the selectable pixel array of each digital camera provides the subset of pixel data to said associated processor.

5. An imaging assembly according to claim 4, wherein each subset of pixels includes contiguous rows of pixels.

6. An imaging assembly according to claim 5, wherein the subset of pixels of each digital camera pixel array is aligned so that the subset of pixels looks generally along a plane substantially coplanar to said display surface.

7. An imaging assembly according to claim 6, wherein the row of pixels in the subset of pixels of each digital camera pixel array, that acquires pixel data corresponding to a tip of said pointer within said region of interest, is determined to enable the associated processor to detect pointer contact and pointer hover over said display surface.

8. An imaging assembly according to claim 1 wherein said processor sums the intensities of the pixels of each column of the difference image to form the histogram.

9. An imaging assembly comprising:

a substantially rectangular frame to surround a region of interest in front of a display surface;

at least two imaging devices, each imaging device having a selectable pixel array and being mounted adjacent a different corner of said frame, said imaging devices being oriented so that the field of view of each imaging device (i) encompasses at least a portion of said display surface, (ii) encompasses a least a portion of said frame, and (iii) extends at least beyond an adjacent peripheral edge of said frame, said imaging devices configured to capture overlapping images of said region of interest; and

a processor associated with each imaging device, each processor configured to process a subset of pixel data of images captured by the associated imaging device to detect the existence of a pointer therein, wherein during processing of each captured image, said processor forms a difference image representing differences between pixel data in the captured image and corresponding pixel data in a background image, generates a histogram comprising a plurality of elements by selectively summing pixel intensities of said difference image, compares the elements of the histogram to a noise threshold, determines the existence of a pointer when a plurality of elements of the histogram exceed the noise threshold and determines the center of the pointer by determining the location of the element of the plurality having the highest value.

10. An imaging assembly according to claim 9, wherein each imaging device comprises a CMOS image sensor.

11. An imaging assembly according to claim 10, wherein each imaging device and associated processor is mounted on a common board.

12. An imaging assembly according to claim 9, wherein each imaging device and associated processor is mounted on a common board.

13. An imaging assembly according to claim 12, wherein a subset of pixels in the selectable pixel array of each imaging device provides the subset of pixel data to said associated processor.

14. An imaging assembly according to claim 13, wherein each subset of pixels includes contiguous rows of pixels.

15. An imaging assembly according to claim 14, wherein the subset of pixels of each imaging device pixel array is aligned so that the subset of pixels looks generally along a plane substantially coplanar to said display surface.

16. An imaging assembly according to claim 15, wherein the row of pixels in the subset of pixels of each imaging device pixel array, that acquires pixel data corresponding to a tip of said pointer within said region of interest, is determined to enable the associated processor to detect pointer contact and pointer hover over said display surface.

17. An imaging assembly according to claim 9 wherein said processor sums the intensities of the pixels of each column of the difference image to form the histogram.

18. A touch system comprising:

a substantially rectangular touch surface;

a region of interest that is in front of, and at least partially includes, said touch surface;

at least two digital cameras, each digital camera having a selectable pixel array and being mounted adjacent a different corner of said touch surface, each digital camera having a field of view that (i) images at least a portion of said touch surface, (ii) images a least a portion of said frame, and (iii) extends at least beyond an adjacent peripheral edge of said touch surface, said digital cameras being oriented to capture overlapping images of said region of interest, each digital camera further comprising a processor configured to process a subset of pixel data of images captured by the digital camera to detect the existence of a pointer therein, wherein during processing of each captured image, said processor forms a difference image representing differences between pixel data in the captured image and corresponding pixel data in a background image, generates a histogram comprising a plurality of elements by selectively summing pixel intensities of said difference image, compares the elements of the histogram to a noise threshold, determines the existence of a pointer when a plurality of elements of the histogram exceed the noise threshold and determines the center of the pointer by determining the location of the element of the plurality having the highest value.

19. A touch system according to claim 18, wherein each digital camera comprises a CMOS digital camera.

20. A touch system according to claim 18, wherein imaging components and the processor of each digital camera are mounted on a common board.

21. A touch system according to claim 20, wherein a subset of pixels in the selectable pixel array of each digital camera provides the subset of pixel data to said associated processor.

22. A touch system according to claim 21, wherein each subset of pixels includes contiguous rows of pixels.

23. A touch system according to claim 21, wherein the subset of pixels of each digital camera pixel array is aligned so that the subset of pixels of each digital camera looks substantially along the plane of said touch surface.

24. A touch system according to claim 23, wherein the row of pixels in the subset of pixels of each digital camera pixel array, that acquires pixel data corresponding to a tip of said pointer on said touch surface, is determined to enable the associated processor to detect pointer contact with said touch surface and pointer hover within said region of interest.

25. A touch system according to claim 18 wherein said processor sums the intensities of the pixels of each column of the difference image to form the histogram.

26. A touch system comprising:

a substantially rectangular touch surface;

a region of interest that is in front of, and at least partially includes, said touch surface;

at least two imaging devices, each imaging device having a selectable pixel array and being mounted adjacent a different corner of said touch surface, said imaging devices being oriented so that the field of view of each imaging device (i) encompasses at least a portion of said touch surface, (ii) encompasses a least a portion of said frame, and (iii) extends at least beyond an adjacent peripheral edge of said touch surface, said imaging devices configured to capture overlapping images of said region of interest; and

a processor associated with each digital camera, each processor configured to process a subset of pixel data of images captured by the associated imaging device to detect the existence of a pointer therein, wherein during processing of each captured image, said processor forms a difference image representing differences between pixel data in the captured image and corresponding pixel data in a background image, generates a histogram comprising a plurality of elements by selectively summing pixel intensities of said difference image, compares the elements of the histogram to a noise threshold, determines the existence of a pointer when a plurality of elements of the histogram exceed the noise threshold and determines the center of the pointer by determining the location of the element of the plurality having the highest value.

27. A touch system according to claim 26, wherein each imaging device comprises a CMOS image sensor.

28. A touch system according to claim 27, wherein each imaging device and associated processor is mounted on a common board.

29. A touch system according to claim 26, wherein each image sensor and associated processor is mounted on a common board.

30. A touch system according to claim 29, wherein a subset of pixels in the selectable pixel array of each digital camera provides the subset of pixel data to said associated processor.

31. A touch system according to claim 30, wherein each subset of pixels includes contiguous rows of pixels.

32. A touch system according to claim 31, wherein the subset of pixels of each digital camera pixel array is aligned so that the subset of pixels of each digital camera looks substantially along the plane of said touch surface.

33. A touch system according to claim 32, wherein the row of pixels in the subset of pixels of each digital camera pixel array, that acquires pixel data corresponding to a tip of said pointer on said touch surface, is determined to enable the associated processor to detect pointer contact with said touch surface and pointer hover within said region of interest.

34. A touch system according to claim 26 wherein said processor sums the intensities of the pixels of each column of the difference image to form the histogram.