System and method for differentiating between pointers used to contact touch surface

• US 7,532,206 B2
• Filed: 03/11/2003
• Issued: 05/12/2009
• Est. Priority Date: 03/11/2003
• Status: Expired due to Term

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Holmgren, David E., Morrison, Gerald D.
• Primary Examiner(s): Lefkowitz; Sumati
• Assistant Examiner(s): Beck; Alexander S.
• Application Number:
  US10/384,783
• Publication Number:
  US 20040179001A1
• Time in Patent Office: 2,254 Days
• Field of Search: 345157-158, 345/173, 345/175, 345179-183
• US Class Current: 345/179
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A touch system comprises a touch surface and at least one camera acquiring images of the touch surface. A pointer contact data generator generates pointer position data in response to pointer contact with the touch surface, the pointer position data representing where on the touch surface pointer contact is made. A processor communicates with the at least one camera and the pointer contact data generator. The processor analyzes acquired images to determine the type of pointer used to contact the touch surface, and processes the pointer position data in accordance with the determined type of pointer. In the preferred embodiment the processor distinguishes between pointer tip touch surface contacts, pointer backend touch surface contacts and finger touch surface contacts. A writing function is invoked in response to pointer tip touch surface contacts. An erase function is invoked in response to pointer backend touch surface contacts. Mouse events are generated in response to finger touch surface contacts.

Claims

1. A touch system comprising:

a touch surface to be contacted by a pointer;

at least one imaging device having a field of view looking generally across said touch surface; and

at least one processor communicating with said at least one imaging device and analyzing images acquired by said at least one imaging device to;

(i) determine the type of pointer used to contact said touch surface; and

(ii) determine the location on said touch surface where pointer contact is made, wherein during image analysis to determine pointer type, said at least one processor processes the acquired images to determine if at least one three-dimensional formation identifying the pointer type integral with and adjacent the end of the pointer exists and if so, further processes the acquired images to recognize said at least one three-dimensional formation thereby to detect said pointer type.

2. A touch system according to claim l wherein said at least one processor determines whether pointer contacts on said touch surface are made using a pen tool or a finger.

3. A touch system according to claim 2 wherein when pointer contacts on said touch surface are made using a pen tool, said at least one processor further determines whether a tip of said pen tool or a backend of said pen tool is used to contact said touch surface.

4. A touch system according to claim 3 wherein said at least one processor invokes a different function depending on whether said tip or backend is used to contact said touch surface.

5. A touch system according to claim 4 wherein when said at least one processor determines that said backend is used to contact said touch surface, said at least one processor invokes an erase function.

6. A touch system according to claim 4 wherein when said at least one processor determines that said tip is used to contact said touch surface, said at least one processor invokes a write function.

7. A touch system according to claim 6 wherein the at least one three-dimensional formation identifies the color of said pen tool.

8. A touch system according to claim 2 wherein when said at least one processor determines that a finger is used to contact said touch surface, said at least one processor invokes a mouse event.

9. A method of updating image data used to present a displayed image on a touch surface in response to pointer contact on said touch surface, said method comprising:

displaying an image on said touch surface;

generating pointer position data in response to pointer contact with said touch surface and acquiring images of pointers used to contact said touch surface;

calculating a sum along each row of pixels in the images to generate a horizontal intensity profile of the pointer within the images and processing the horizontal intensity profile to determine the type of pointer used to contact the touch surface, the pointer type being selected from one of a pen tool and a finger; and

updating image data used to present the displayed image on said touch surface based on the generated pointer position data and the determined type of pointer, wherein when the determined type of pointer is a pen tool, the images are further analyzed to determine whether a tip or a backend of said pointer is used to contact said touch surface and wherein when a tip is used to contact said touch surface, the images are further analyzed to determine a color assigned to said pen tool.

10. The method of claim 9 wherein when the backend is used to contact said touch surface, an editing function is performed during said updating.

11. The method of claim 9 wherein when a tip is used to contact said touch surface, a write function is performed during said updating.

12. The method of claim 9 wherein during said further analysis, the shape of said tip is examined to determine the color assigned to the pen tool.

13. The method of claim 9 wherein during said further analysis, a glyph on said tip is recognized to determine the color assigned to the pen tool.

14. A touch system comprising:

a touch surface on which an image is presented;

a processor responsive to pointer contacts on the touch surface and generating image data that is used to present said image; and

at least one camera to acquire images of said pointer, wherein said processor examines said images to generate a horizontal intensity profile representing said pointer and process the horizontal intensity profile to determine at least the shape of said pointer thereby to determine whether said pointer is a pen tool or a finger and updates the image data based on the type of pointer and the pointer activity on the touch surface, wherein one end of said pen tool has a generally conical tip and constitutes a writing tool and wherein said tip includes a unique identifier, when a writing tool is used to contact said touch surface, said processor examining said images to recognize said identifier and thereby determine another attribute assigned to the writing tool.

15. A touch system according to claim 14 wherein said processor edits said image data when said pointer is a pen tool and invokes a mouse event when said pointer is a finger.

16. A touch system according to claim 15 wherein said processor examines the orientation of the sides of said pointer to determine if the pointer is a pen tool or finger.

17. A touch system according to claim 14 wherein when said pointer is a finger, said processor updates the image data in accordance with an invoked mouse event.

18. A camera-based touch system comprising;

a pair of cameras having overlapping fields of view encompassing an area on which pointer contacts are to be made, said cameras acquiring images and generating corresponding image data;

a processor receiving and processing the image data generated by said cameras to determine the location of a pointer relative to the area when the pointer is captured in images acquired by the cameras and analyzing the image data to generate a horizontal intensity profile of said pointer and processing the horizontal intensity profile to determine the type of pointer used to contact the area, said pointer type being selected from one of a pen tool and a finger; and

a plurality of different pen tools, each pen tool having a unique three-dimensional tip configuration specifying an attribute assigned to said pen tool, wherein when said processor determines that the tip of one of said pen tools is used to contact said area, said processor further analyzes said acquired images to recognize the unique three-dimensional configuration of said tip.

19. A touch system according to claim 18 wherein an image is presented on said area, said processor updating image data used to generate the image presented on said area in accordance with the locations where contacts are made on the area and the determined type of pointer.

20. A touch system according to claim 18 wherein when said processor determines that the tip of one of said pen tools is used to contact said area, said processor uses said pointer position data to invoke a write function.

21. A touch system according to claim 18 wherein the unique three-dimensional configuration is a unique longitudinal tip shape and wherein said assigned attribute is a pen tool color, when said processor determines that the tip of one of said pen tools is used to contact said area, said processor uses said pointer position and the recognized tip shape to invoke a write function in the color assigned to said pen tool.

22. A touch system according to claim 18 wherein the unique three-dimensional configuration is at least one integral formation on said tip and wherein said assigned attribute is a pen tool color, when said processor determines that the tip of one of said pen tools is used to contact said area, said processor uses said pointer position data and the recognized at least one formation to invoke a write function in the color assigned to said pen tool.

23. A touch system according to claim 18 wherein when said processor determines that a finger is used to contact said area, said processor uses said pointer position data to invoke a mouse event.

24. A touch system comprising:

a touch surface to be contacted by a pointer;

at least one imaging device having a field of view looking generally across said touch surface;

at least one processor communicating with said at least one imaging device and analyzing images acquired by said at least one imaging device to;

(i) determine the type of pointer used to contact said touch surface; and

(ii) determine the location on said touch surface where pointer contact is made; and

a plurality of different pen tools, each pen tool having a unique three-dimensional tip configuration, wherein when said at least one processor determines that a tip of one of said pen tools is used to contact said touch surface, said at least one processor further analyzes said images to recognize the unique three-dimensional tip configuration of said one pen tool.

25. A touch system according to claim 24 wherein the shape of each pen tool tip has an invariant property irrespective of the orientation of the pen tool and wherein said at least one processor examines the invariant property to determine the pen tool used to contact the touch surface.

26. A touch surface according to claim 25 wherein each pen tool has a generally conical tip shape defining an opening angle, the opening angle of the tip being different for each pen tool and constituting said unique three-dimensional tip configuration.

27. A touch surface according to claim 26 wherein said at least one processor examines said images to determine the curve of growth of the sides of the tip and uses the determined curve of growth to determine the pen tool.

28. A touch system according to claim 24 wherein the unique three-dimensional tip configuration is at least one integral formation on said tip that is unique to each pen tool.

29. A touch system according to claim 28 wherein each pen tool is assigned a different color, when said at least one processor determines that said tip is used to contact said touch surface, said at least one processor uses said pointer position data and the recognized formation to invoke a write function in the color assigned to said pen tool.

30. A method of updating image data used to present a displayed image on a touch surface in response to pointer contact on said touch surface, said method comprising the steps of:

displaying an image on said touch surface;

generating pointer position data in response to pointer contact with said touch surface and acquiring images of pointers used to contact said touch surface;

analyzing the acquired images to determine the type of pointer used to contact the touch surface; and

updating image data used to present the displayed image on said touch surface based on the generated pointer position data and the determined type of pointer, wherein when a tip of a pointer is used to contact said touch surface, the images are further analyzed to determine a color assigned to said pen tool, said determined color being based on a recognized three-dimensional pen tool tip configuration.

31. The method of claim 30 wherein when said tip is used to contact said touch surface, a write function is performed during said updating.

32. The method of claim 30 wherein during said further analysis, a formation on said tip is recognized to determine the color assigned to the pen tool.

33. A touch system comprising:

a touch surface to be contacted by at least one of a finger and a pen tool, said pen tool having a unique three-dimensional tip configuration;

at least one imaging device having a field of view looking generally across said touch surface; and

at least one processor communicating with said at least one imaging device and analyzing images acquired by said at least one imaging device to;

(i) determine the type of pointer used to contact said touch surface; and

(ii) determine the location on said touch surface where pointer contact is made, wherein when said at least one processor determines that a tip of a pen tool is used to contact said touch surface, said at least one processor further analyzes said images to recognize the unique three-dimensional configuration of said pen tool tip.

34. A touch system according to claim 33 wherein the shape of said pen tool tip has an invariant property irrespective of the orientation of the pen tool and wherein said at least one processor examines the invariant property to determine the pen tool used to contact the touch surface.

35. A touch surface according to claim 34 wherein the tip shape is conical defining an opening angle, the opening angle of the tip being different for different pen tools and constituting said unique three-dimensional configuration.

36. A touch surface according to claim 35 wherein said at least one processor examines said images to determine the curve of growth of the sides of the tip and uses the determined curve of growth to determine the pen tool.

37. A touch system according to claim 33 wherein the unique three-dimensional configuration is at least one integral formation on said tip that is unique to said pen tool.

38. A touch system according to claim 37 wherein the at least one formation identifies the color of said pen tool.

39. An interactive input system comprising:

at least one imaging device having a field of view looking into a region of interest into which at least one pointer is positioned; and

processing structure communicating with said at least one imaging device and analyzing images acquired by said at least one imaging device to;

(i) determine the type of the at least one pointer; and

(ii) determine the location of the at least one pointer in said region of interest, wherein during image analysis to determine pointer type, said processing structure processes the acquired images to recognize the three-dimensional configuration of the pointer tip.

40. An interactive input system according to claim 39 wherein the tip configuration of said at least one pointer has an invariant property irrespective of the orientation of the pen tool and wherein said processing structure examines the invariant property to determine the type of said at least one pointer.

41. An interactive input system according to claim 40 wherein the tip is conical defining an opening angle, the opening angle of the tip being different for different pointers.

42. An interactive input system according to claim 39 wherein the three-dimensional configuration comprises at least one three-dimensional integral formation, the at least one formation being different for different pointers.

43. An interactive input system according to claim 42 wherein the at least one formation identifies the color of said pointer.