Interactive input system and method therefor

• US 9,292,129 B2
• Filed: 10/30/2013
• Issued: 03/22/2016
• Est. Priority Date: 10/30/2012
• Status: Active Grant

• Current Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Smart Technologies ULC (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Tsang, Kenny, Advani, Dinesh, Zhang, Daniel
• Primary Examiner(s): Bibbins, Latanya
• Application Number:
  US14/067,113
• Publication Number:
  US 20140118284A1
• Time in Patent Office: 874 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 40/171 by use of digital ink

Abstract

A method for determining the location of a touch input in an application window on an interactive surface of a display device is described. The application window comprises a canvas configured to display at least a portion of a user interface. The method comprises the following steps. A first location of the touch input associated with a first coordinate space is determined. The first coordinate space is associated with the interactive surface. The touch input is used to emulate a mouse event. A second location of the touch input associated with a second coordinate space is determined in response to the emulated mouse event. The second coordinate space is associated with the user interface. At least one offset parameter is calculated, correlating the first coordinate space with the second coordinate space. While the first coordinate space does not change with respect the second coordinate space, the at least one offset parameter is used to determine the location of subsequent touch inputs based on corresponding locations associated with the first coordinate space. A computer readable medium and an interactive input system for implementing the method are also described.

Claims

1. A method for determining the location of a touch input in an application displayed in an application window on an interactive surface of a display device, the application window comprising a canvas configured to display at least a portion of a user interface;

the method comprising;

determining a first location of the touch input associated with a first coordinate space, the first coordinate space being associated with the interactive surface;

using the touch input to emulate a mouse event;

injecting the mouse event into the application window;

receiving a second location of the touch input associated with a second coordinate space from the application in response to the injected emulated mouse event;

the second coordinate space being associated with the user interface;

calculating at least one offset parameter correlating the first coordinate space with the second coordinate space; and

while the first coordinate space does not change with respect the second coordinate space, using the at least one offset parameter to determine the location of subsequent touch inputs based on corresponding locations associated with the first coordinate space.

2. The method of claim 1, wherein the at least one offset parameter is recalculated when the user interface is loaded into a corresponding application.

3. The method of claim 1, wherein the at least one offset parameter is recalculated when the application window is moved.

4. The method of claim 1, wherein the at least one offset parameter is recalculated when the canvas moves or changes size.

5. The method of claim 1, wherein components of the application window are changed.

6. The method of claim 1, wherein the at least one offset parameter is recalculated when the application window is resized.

7. The method of claim 1, wherein the at least one offset parameter is recalculated when the user interface is panned to display a different portion on the canvas.

8. The method of claim 1, wherein the touch input includes a plurality of other parameters in addition to location.

9. The method of claim 8, wherein the plurality of other parameters includes a pointer colour.

10. The method of claim 8, wherein the plurality of other parameters includes a pointer type.

11. The method of claim 10, wherein the touch input is used for injecting user commands or digital ink, depending on the pointer type.

12. The method of claim 11, comprising the further steps of:

processing the digital ink to determine if it overlaps a user input area;

converting the digital ink to text; and

populating the overlapped user input area with the text.

13. The method of claim 12, comprising the further step of removing the digital ink.

14. The method of claim 1, wherein the user interface is a webpage.

15. The method of claim 14, wherein the at least one offset parameter is recalculated when the user interface is loaded into a web browser.

16. A non-transitory computer readable medium having stored thereon instructions for determining the location of a touch input in an application displayed in an application window on an interactive surface of a display device, the application window comprising a canvas configured to display at least a portion of a user interface, the instructions, when executed by a processor, cause the processor to implement the steps of:

determining a first location of the touch input associated with a first coordinate space, the first coordinate space being associated with the interactive surface;

using the touch input to emulate a mouse event;

injecting the mouse event into the application window;

receiving a second location of the touch input associated with a second coordinate space from the application in response to the injected emulated mouse event;

the second coordinate space being associated with the user interface;

calculating at least one offset parameter correlating the first coordinate space with the second coordinate space; and

while the first coordinate space does not change with respect the second coordinate space, using the at least one offset parameter to determine the location of subsequent touch inputs based on corresponding locations associated with the first coordinate space.

17. The non-transitory computer readable medium of claim 16, wherein the at least one offset parameter is recalculated when the user interface is loaded into a corresponding application.

18. The non-transitory computer readable medium of claim 16, wherein the at least one offset parameter is recalculated when the application window is moved.

19. The non-transitory computer readable medium of claim 16, wherein the at least one offset parameter is recalculated when the canvas moves or changes size.

20. The non-transitory computer readable medium of claim 16, wherein the at least one offset parameter is recalculated when the application window is resized.

21. The non-transitory computer readable medium of claim 16, wherein the at least one offset parameter is recalculated when the user interface is panned to display a different portion on the canvas.

22. The non-transitory computer readable medium of claim 16, wherein the touch input includes a plurality of other parameters in addition to location.

23. The non-transitory computer readable medium of claim 22, wherein the plurality of other parameters includes a pointer type.

24. The non-transitory computer readable medium of claim 23, wherein the touch input is used for injecting user commands or digital ink, depending on the pointer type.

25. The non-transitory computer readable medium of claim 24, comprising further instruction for:

processing the digital ink to determine if it overlaps a user input area;

converting the digital ink to text; and

populating the overlapped user input area with the text.

26. The non-transitory computer readable medium of claim 25, comprising further instruction for removing the digital ink.

27. The non-transitory computer readable medium of claim 16, wherein the user interface is a webpage.

28. The non-transitory computer readable medium of claim 27, wherein the at least one offset parameter is recalculated when the user interface is loaded into a web browser.

29. The computer readable medium of claim 28, wherein the instructions are injected into the webpage using a bookmarklet.

30. The computer readable medium of claim 16, wherein the instructions are automatically injected into a markup language document used to generate the user interface.

31. An interactive processing system comprising a processor and a non-transitory computer readable medium having stored thereon instructions, which when executed cause the processor to implement the steps of:

determining a first location of the touch input associated with a first coordinate space, the first coordinate space being associated with the interactive surface;

using the touch input to emulate a mouse event;

injecting the mouse event into an application window;

receiving a second location of the touch input associated with a second coordinate space from an application associated with the application window in response to the injected emulated mouse event;

the second coordinate space being associated with the user interface;

calculating at least one offset parameter correlating the first coordinate space with the second coordinate space; and

while the first coordinate space does not change with respect the second coordinate space, using the at least one offset parameter to determine the location of subsequent touch inputs based on corresponding locations associated with the first coordinate space.

32. The interactive processing system of claim 31, wherein the instructions are automatically injected into a markup language document used to generate the user interface.

33. The interactive processing system of claim 31, wherein the user interface is a webpage, the at least one offset parameter is recalculated when the user interface is loaded into a web browser, and the instructions are injected into the webpage using a bookmarklet.