Synchronization of cameras in camera-based touch system to enhance position determination of fast moving objects

• US 7,184,030 B2
• Filed: 12/02/2003
• Issued: 02/27/2007
• Est. Priority Date: 06/27/2002
• Status: Active Grant

• Current Assignee: PixArt Imaging Incorporated
• Original Assignee: Smart Technologies Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Akitt, Trevor, Morrison, Gerald, McCharles, Randy, Worthington, Steve
• Primary Examiner(s): Hjerpe; Richard
• Assistant Examiner(s): Said; Mansour M.
• Application Number:
  US10/724,633
• Publication Number:
  US 20040108996A1
• Time in Patent Office: 1,183 Days
• Field of Search: 345156-160, 345173-175, 345177-183, 178/18.01, 178/18.1, 178/19.01
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 5/0733 for distributing synchronisation pulses to different TV cameras

Abstract

A camera-based touch system includes at least one pair of cameras having overlapping fields of view and a touch surface encompassed within the overlapping fields of view across which a pointer is moved. The cameras of the at least one pair acquire images at intervals asynchronously. In order to estimate the position of the pointer relative to the touch surface from image data acquired by the at least one pair of cameras, the images are synthetically synchronized. During this process, for each camera in the pair, each acquired image is processed to determine the position of the pointer therein and the position of the pointer is recorded together with a timestamp representing the time elapsed between a reference point common to the cameras and the time the image was acquired. Successive pairs of recorded positions are interpolated to generate interpolated positions and the interpolated positions are recorded together with synchronization times representing times the images would have been acquired had the cameras been synchronized. Interpolated positions generated by the cameras having equivalent associated synchronization times are determined and these interpolated positions are triangulated to estimate the position of the pointer relative to the touch surface.

Claims

1. In a camera-based touch system including at least one pair of cameras having overlapping fields of view and a touch surface encompassed within said overlapping fields of view across which a pointer is moved, wherein the cameras of said at least one pair acquire images at intervals asynchronously, a method of synchronizing image data acquired by said at least one pair of cameras comprising the step of:

for each camera in said pair;

processing each acquired image to determine the position of said pointer therein and recording the position together with a timestamp representing the time elapsed between a reference point common to said cameras and the time the image was acquired; and

interpolating between pairs of recorded positions to generate interpolated positions and recording each interpolated position together with a synchronization time representing a time each image would have been acquired had said cameras been synchronized.

2. The method of claim 1 wherein said interpolating is performed between each successive pair of recorded positions.

3. The method of claim 2 wherein said reference point is a signal sent to each of said cameras simultaneously.

4. The method of claim 3 further comprising the step of initiating a timer associated with each camera in response to said signal and reading the value of said timer when each image is acquired, the value of said timer constituting said timestamp.

5. The method of claim 1 wherein for each camera, said processing step and interpolating step are performed by a processor of that camera.

6. The method of claim 1 wherein for each camera, said processing step is performed by a processor of that camera and wherein said interpolating step is performed by an external processor receiving the recorded interpolated positions and synchronization times from each camera.

7. The method of claim 1 wherein each interpolated position is calculated using the equation:

X_S=((X₁−

X₀)/(T₁−

T₀))*(T_S−

T₀)+X₀

where;

X₀ and X₁ are successive x-positions of said pointer;

T₁ and T₀ are successive timestamps corresponding to the x-positions X₁ and X₀; and

T_S is a given synchronization time, where T₀≦

T_s≦

T₁.

8. The method of claim 7 wherein each camera records positions and timestamps in a history table, the history table maintained by each camera holding the Nth most recent recorded positions and associated timestamps, the value of N being selected to provide a sufficient number of generated interpolated positions to ensure that at least one interpolated position generated for each camera between resets of said camera has an equivalent synchronization time.

9. In a camera-based touch system including at least one pair of cameras having overlapping fields of view and a touch surface encompassed within said overlapping fields of view across which a pointer is moved, wherein the cameras of said at least one pair acquire images at intervals asynchronously, a method of estimating the position of said pointer relative to said touch surface from image data acquired by said at least one pair of cameras, said method comprising the step of:

for each camera in said pair;

processing each acquired image to determine the position of said pointer therein and recording the position together with a timestamp representing the time elapsed between a reference point common to said cameras and the time the image was acquired; and

interpolating between successive pairs of recorded positions to generate interpolated positions and recording said interpolated positions together with synchronization times representing times the images would have been acquired had said cameras been synchronized; and

determining interpolated positions generated by said cameras having equivalent associated synchronization times and triangulating the interpolated positions to estimate the position of the said pointer relative to said touch surface.

10. The method of claim 9 wherein each camera records positions and timestamps in a history table.

11. The method of claim 10 wherein the history table maintained by each camera holds the Nth most recent recorded positions and associated timestamps, the value of N being selected to provide a sufficient number of generated interpolated positions to ensure that at least one interpolated position generated for each camera between resets of said camera has an equivalent synchronization time.

12. The method of claim 11 wherein the interpolated positions and the associated synchronous times for each camera are stored in a synchronization table.

13. The method of claim 12 wherein N is equal to 6 and wherein each synchronization table holds four interpolated positions and associated synchronous times.

14. The method of claim 11 wherein said reference point is a signal sent to each of said cameras simultaneously.

15. The method of claim 14 further comprising the step of initiating a timer associated with each camera in response to said signal and reading the value of said timer when each image is acquired, the value of said timer constituting said timestamp.

16. The method of claim 15 wherein the intervals between acquisition of successive images by each camera are equal and wherein intervals between succesive synchronization times are equal.

17. The method of claim 16 wherein the intervals between successive synchronization times are greater than or equal to the intervals between acquisition of successive images by each camera.

18. The method of claim 11 wherein for each camera said processing step and interpolating step are performed by a processor of that camera.

19. The method of claim 11 wherein for each camera said processing step is performed by a processor of that camera and wherein said interpolating step is performed by an external processor receiving the recorded interpolated positions and synchronization times from each camera.

20. The method of claim 9 wherein each interpolated position is calculated using the equation:

X_S=((X₁−

X₀)/(T₁−

T₀))*(T_S−

T₀)+X₀

where;

X₀ and X₁ are successive x-positions of said pointer;

T₁ and T₀ are successive timestamps corresponding to the x-positions X₁ and X₀; and

T_S is a given synchronization time, where T₀≦

T_s≦

T₁.

21. A camera-based touch system comprising:

at least one pair of cameras associated with a touch surface and having overlapping fields of view encompassing said touch surface, said at least one pair of cameras acquiring images of said touch surface from different locations and generating image data;

a processor receiving and processing the image data generated by said at least one pair of cameras to determine the location of an object relative to the touch surface by triangulation when the object is captured in images acquired by the at least one pair of cameras; and

a synchronization mechanism to synchronize image data generated by said at least one pair of cameras;

wherein each camera processes each image acquired thereby to determine the position of the object therein and records the position together with a timestamp representing the time elapsed between a reference point common to said cameras and the time the image was acquired, and wherein said synchronization mechanism interpolates between successive pairs of recorded positions to generate interpolated positions, the interpolated positions being recorded together with synchronization times representing times the images would have been acquired had said cameras been synchronized, said processor using interpolated positions generated by the cameras having equivalent associated synchronization times to determine the location of the object using triangulation.

22. A touch system according to claim 21 wherein said synchronization mechanism calculates the interpolated positions using the equation:

X_S=((X₁−

X₀)/(T₁−

T₀))*(T_S−

T₀)+X₀

where;

X₀ and X₁ are successive x-positions of said pointer;

T₁ and T₀ are successive timestamps corresponding to the x-positions X₁ and X₀; and

T_S is a given synchronization time, where T₀≦

T_s≦

T₁.

23. A touch system according to claim 22 wherein each camera records the positions and timestamps in a history table, said history table holding the Nth most recent recorded positions and associated timestamps, the value of N being selected to provide a sufficient number of generated interpolated positions to ensure that at least one interpolated position generated for each camera between resets of said cameras has an equivalent synchronization time.

24. A touch system according to claim 23 wherein said reference point is a signal sent to each of said cameras simultaneously by said processor.