Large scale touch system and methods for interacting with same

• US 8,487,910 B2
• Filed: 05/02/2005
• Issued: 07/16/2013
• Est. Priority Date: 05/02/2005
• Status: Active Grant

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Hill, Douglas, Beckie, Niel, van Ieperen, Taco, Vdovine, Serguei, Sirotich, Roberto, Fletcher, Mark, Tallman, Scott, Williams, Marilyn, Bill, Shane Edward, Goodman, Shannon Patricia
• Primary Examiner(s): Nguyen, Chanh
• Assistant Examiner(s): STONE, ROBERT M
• Application Number:
  US11/118,626
• Publication Number:
  US 20060244734A1
• Time in Patent Office: 2,997 Days
• Field of Search: 345/173, 345/175, 345179-183, 178/18.01, 178/18.09, 178/18.11, 178/19.04, 178/19.05
• US Class Current: 345/175
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A touch system comprises a large scale touch surface on which an image is presented and at least two imaging devices looking across the touch surface and having overlapping fields of view. Processing structure communicates with the imaging devices and processes image data acquired by the imaging devices to determine pointer contacts on the touch surface using triangulation. The processing structure further executes at least one application to facilitate user interaction with the touch surface. Methods of interacting with the touch surface are also disclosed.

Claims

1. A method of facilitating user interaction with an image presented on a touch surface of a large scale touch system, said touch surface comprising a plurality of generally rectangular sub-regions arranged in a row extending the longitudinal length of the touch surface with adjacent sub-regions partially overlapping, said method comprising:

capturing images of the sub-regions of the touch surface with sets of camera devices, each set of camera devices comprising at least one pair of laterally spaced camera devices having overlapping fields of view oriented to look generally across the respective sub-region of said touch surface;

processing the captured images to detect pointer movement within each sub-region and generating pointer coordinate data for each sub-region in which pointer movement is detected;

processing pointer coordinate data generated for adjacent sub-regions that are a result of pointer movement within overlapping portions thereof in accordance with weighted averaging logic to yield a single set of pointer coordinate data representing the pointer movement within the overlapping portions;

processing the pointer coordinate data to detect a predetermined input gesture made at an arbitrary location on said touch surface;

displaying on the touch surface a scaled version of at least a portion of said displayed image at a user accessible location in response to detection of said predetermined input gesture, said user accessible location positioned adjacent to said arbitrary location on said touch surface;

processing the pointer coordinate data to detect user interaction with said touch surface within said displayed scaled version of at least a portion of said displayed image;

calculating a mapped position on the touch surface based on the position of said user interaction and the position of said displayed scaled version; and

mapping said user interaction to said mapped position on the touch surface so that user interactions with the displayed scaled version are translated to interactions with said displayed image.

2. The method of claim 1 wherein during displaying, a scaled version of the entire displayed image is presented within a window.

3. The method of claim 2 wherein said window is positioned adjacent a lower portion of said touch surface.

4. A method of facilitating user interaction with an image presented on a touch surface of a large scale touch system, said touch surface comprising a plurality of generally rectangular sub-regions arranged in a row extending the longitudinal length of the touch surface with adjacent sub-regions partially overlapping, said method comprising:

capturing images of the sub-regions of the touch surface with sets of camera devices, each set of camera devices comprising at least one pair of laterally spaced camera devices having overlapping fields of view oriented to look generally across the respective sub-region of said touch surface;

processing the captured images to detect pointer movement within each sub-region and generating pointer coordinate data for each sub-region in which pointer movement is detected;

processing pointer coordinate data generated for adjacent sub-regions that are a result of pointer movement within overlapping portions thereof in accordance with weighted averaging logic to yield a single set of pointer coordinate data representing the pointer movement within the overlapping portions;

receiving a user command designating a text display region for displaying text on the touch surface;

displaying a textbox window on said touch surface adjacent a user accessible location in response to an associated input command;

processing the pointer coordinate data to detect writing input within said textbox window, the position of said writing input being irrelevant to the position of said text display region;

injecting the detected writing input as text in said text display region; and

processing the pointer coordinate data to detect a predefined pointer interaction with said touch surface and in response changing the position of at least one of the textbox window and the text display region on the touch surface.

5. The method of claim 4 wherein said text display region is initially positioned adjacent an upper portion of said touch surface.

6. The method of claim 5 wherein said textbox window is initially positioned adjacent a lower portion of said touch surface.

7. A method of facilitating user interaction with an image presented on a touch surface of a large scale touch system, said touch surface comprising a plurality of generally rectangular sub-regions arranged in a row extending the longitudinal length of the touch surface with adjacent sub-regions partially overlapping, said method comprising:

capturing images of the sub-regions of the touch surface with sets of camera devices, each set of camera devices comprising at least one pair of laterally spaced camera devices having overlapping fields of view oriented to look generally across the respective sub-region of said touch surface;

processing the captured images to detect pointer movement within each sub-region and generating pointer coordinate data for each sub-region in which pointer movement is detected;

processing pointer coordinate data generated for adjacent sub-regions that are a result of pointer movement within overlapping portions thereof in accordance with weighted averaging logic to yield a single set of pointer coordinate data representing the pointer movement within the overlapping portions;

processing the pointer coordinate data to detect a predetermined input gesture made at an arbitrary location on said touch surface and in response duplicating a portion of said displayed image on said touch surface at a user accessible location; and

mapping pointer coordinate data generated in response to user interaction with said touch surface within said duplicated portion of said displayed image to a corresponding position on said touch surface so that user interactions with the duplicated portion are translated to interactions with the corresponding portion of said displayed image.

8. The method of claim 7 wherein said duplicated portion is the upper region of said displayed image.

9. The method of claim 8 wherein said upper region of said displayed image is duplicated adjacent the bottom portion of said displayed image.

10. The method of claim 9 wherein each side portion of said displayed image is duplicated adjacent the opposite side of said touch surface.

11. A method of facilitating user interaction with an image presented on a touch surface of a large scale touch system, said touch surface comprising a plurality of generally rectangular sub-regions arranged in a row extending the longitudinal length of the touch surface with adjacent sub-regions partially overlapping, said method comprising:

capturing images of the sub-regions of the touch surface with sets of camera devices, each set of camera devices comprising at least one pair of laterally spaced camera devices having overlapping fields of view oriented to look generally across the respective sub-region of said touch surface;

processing the captured images to detect pointer movement within each sub-region and generating pointer coordinate data for each sub-region in which pointer movement is detected;

processing pointer coordinate data generated for adjacent sub-regions that are a result of pointer movement within overlapping portions thereof in accordance with weighted averaging logic to yield a single set of pointer coordinate data representing the pointer movement within the overlapping portions;

displaying a mouse object on said touch surface at a user accessible location and remote from a cursor displayed on said touch surface, wherein said mouse object comprises a first area, and wherein pointer interaction within said first area determines the movement of said cursor;

processing the pointer coordinate data to detect pointer interactions on said first area of said mouse object;

positioning said mouse object on said touch surface based on detected pointer interactions on said first area;

positioning the cursor on said touch surface based on detected pointer interactions on said first area; and

processing events corresponding to said detected pointer interactions in a manner to avoid input focus being shifted to said mouse object.

12. The method of claim 11 wherein said processing comprises injecting mouse events into a mouse event queue of the touch system operating system.

13. The method of claim 12 wherein said mouse object includes at least one second area, wherein user interaction within said second area causes a mouse-down event.

14. The method of claim 13 wherein said mouse object is moveable across said touch surface in response to pointer interactions with said touch surface.

15. The method of claim 11 further comprising:

automatically positioning said mouse object to a predefined location when said mouse object is at a predefined border.

16. The method of claim 11 further comprising:

saving said pointer coordinate data when said pointer interaction is a predefined type of pointer interaction;

automatically positioning said mouse object to a predefined location when said mouse object is at a predefined border; and

resuming said saved pointer interaction.

17. A method of facilitating user interaction with an image presented on a display surface comprising a plurality of generally rectangular sub-regions arranged in a row extending the longitudinal length of said display surface with adjacent sub- regions partially overlapping, said method comprising:

capturing images of the sub-regions of the display surface with sets of camera devices, each set of camera devices being associated with a respective one of said sub-regions and comprising at least one pair of laterally spaced camera devices having overlapping fields of view oriented to look generally across the respective sub-region of said display surface;

processing the captured images to detect pointer movement within each sub-region and generating pointer coordinate data for each sub-region in which pointer movement is detected;

processing pointer coordinate data generated for adjacent sub-regions that are a result of pointer movement within overlapping portions thereof in accordance with weighted averaging logic to yield a single set of pointer coordinate data representing the pointer movement within the overlapping portions;

detecting at least one graphical object in said presented image associated with a running computer program, wherein user interaction with said image is required to dismiss said graphic object in order for the running computer program to advance beyond its current state;

applying a dark overlay to said display surface, said overlay having an opening therein determined at least by the position, size and shape of said at least one graphical object;

processing pointer coordinate data to detect user interaction with said image signifying the dismissal of said graphical object; and

removing said overlay.

18. The method of claim 17 wherein said at least one displayed graphical object is a modal box.

19. The method of claim 17 further comprising:

displaying concentric rings about said modal box.