METHOD AND APPARATUS FOR EXECUTING APPLICATION USING MULTIPLE INPUT TOOLS ON TOUCHSCREEN DEVICE

• US 20150054784A1
• Filed: 08/26/2014
• Published: 02/26/2015
• Est. Priority Date: 08/26/2013
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: CHUNG, Hyun-kwon, JANG, Yong-seok, KIM, Min-sung, KIM, Hye-soo, LIM, Janine June C., CRUZ, Jonathan Martin S., DALANGIN, Marwell D., SINGH, Nicolai Andrew F., SANTOS, Timothy Israel D.
• Application Number:
  US14/468,537
• Publication Number:
  US 20150054784A1
• Time in Patent Office: Days
• US Class Current: 345/174
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0393 Accessories for touch pads or touch screens, e.g. mechanical guides added to touch screens for drawing straight lines, hard keys overlaying touch screens or touch pads
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/046 by electromagnetic means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Provided are a method of performing an event action by using a touch screen device based on operation gestures that are simultaneously input by using multiple operation tools and the touch screen device. The method includes: identifying a first operation tool based on contact by the first operation tool, the contact being sensed on the touch screen device; setting an operation area on the touch screen device based on an area designated by the contact by the first operation tool; identifying a second operation tool based on access by the second operation tool, the access being sensed on the touch screen device; sensing an operation gesture of the second operation tool within the operation area by using the second operation tool.

Claims

1. A method of operating a touch screen device, the method comprising:

identifying a first operation tool based on contact by the first operation tool, the contact being sensed on the touch screen device;

setting an operation area on the touch screen device based on an area designated by the contact by the first operation tool;

identifying a second operation tool based on access by the second operation tool, the access being sensed on the touch screen device;

sensing an operation gesture of the second operation tool within the operation area by using the second operation tool, wherein the second operation tool moves on the first operation tool and the first operation tool is in contact with the touch screen device; and

performing an action corresponding to the sensed operation gesture of the second operation tool from among actions that are previously registered in an interaction database (DB) of the touch screen device.

2. The method of claim 1, wherein the identifying a first operation tool comprises determining a position where the first operation tool contacts the touch screen device, by using an electrostatic sensor of the touch screen device, wherein the identifying a second operation tool comprises determining an input position of the second operation tool by using an electromagnetic induction sensor of the touch screen device.

3. The method of claim 1, wherein the identifying a first operation tool comprises identifying the first operation tool based on a sensed contacting state of the first operation tool, wherein the sensed contacting state of the first operation tool is from among identification information of operation tools registered with an operation tool register DB of the interaction DB,

wherein the setting an operation area on the touch screen device comprises determining an operation area of the identified first operation tool based on form information of operation tools that are previously registered with the operation tool register DB in the operation tool operation area,

wherein the identification information comprises at least one of a number of contact points of the first operation tool, a form of each of the contact points, a distance between the contact points, and a surface area of each of the contact points.

4. The method of claim 1, wherein the identifying a second operation tool comprises identifying the second operation tool based on a sensed access state of the second operation tool from among identification information of operation tools previously registered with an operation tool register DB of the interaction DB,

wherein the identification information comprises at least one of a sensitivity of pressing an auxiliary button of the second operation tool and a release sensitivity of the auxiliary button of the second operation tool.

5. The method of claim 1, wherein the identifying a first operation tool comprises:

storing, in an operation tool register DB, identification information of the first operation tool including at least one of a number of contact points of the first operation tool, a form of each of the contact points, a distance between the contact points, and a surface area of each of the contact points, which are stored in the interaction DB; and

storing information of an operation area determined based on a form of the first operation tool in the operation tool register DB.

6. The method of claim 1, wherein the identifying a second operation tool comprises:

storing identification information of the second operation tool including at least one of a sensitivity of pressing an auxiliary button of the second operation tool and a release sensitivity of the auxiliary button of the second operation tool, in an operation tool register DB; and

storing operation information of the second operation tool including at least one of a contact sensitivity or a release sensitivity of a contacting portion of the second operation tool and a distance between the contacting portion and the touch screen device, in the operation tool register DB.

7. The method of claim 1, wherein the interaction DB includes information about an action corresponding to an operation gesture of at least one of the first and second operation tools,

wherein the operation gesture of the at least one of the first and second operation tools is a single, previously set input or a set of a series of previously set inputs.

8. The method of claim 1, wherein the performing an operation corresponding to the sensed operation gesture of the second operation tool comprises determining an event action corresponding to a series of operation gestures which are input by using at least one of the first and second operation tools, from among the event actions that are previously registered in the interaction DB.

9. The method of claim 1, further comprising executing an application for performing an event determined based on an operation gesture of at least one of the first and second operation tools,

wherein the performing an operation corresponding to a sensed operation gesture of the second operation tool comprises:

mapping information about a virtual operation area defined in an application installed in the touch screen device to an event corresponding to an operation gesture of the at least one of the first and second operation tools, to the event actions previously registered with the interaction DB; and

performing, when a current operation gesture of the second operation tool is sensed within the virtual operation area as the application is executed, an action of an event corresponding to the current operation gesture.

10. The method of claim 1, wherein the performing an action corresponding to the sensed operation gesture of the second operation tool comprises displaying a result screen generated by the performing of the action, on the touch screen device.

11. The method of claim 1, wherein the performing an action corresponding to the sensed operation gesture of the second operation tool comprises:

receiving an output request which has been submitted to an external device;

transmitting image data about a current display screen of the touch screen device to the external device, based on the output request;

displaying a virtual operation area of the first operation tool on the touch screen device; and

transmitting information about a position and a form of the virtual operation area of the first operation tool, to the external device,

wherein when the current display screen and the virtual operation area are displayed on the external device, an operation gesture is sensed within the virtual operation area by using an operation tool of the external device.

12. The method of claim 1, further comprising:

receiving activity information including user identification information, lesson identification information, activity identification information, and activity page identification information, from each of a plurality of touch screen devices in which a same application is installed;

displaying an activity list including icons indicating activities and an activity page corresponding to the activity list, on the touch screen device and displaying, on each of the icons indicating the activities, a number indicating how many touch screen devices are displaying on each icon among the icons an activity from among the plurality of touch screen devices; and

displaying, when an input about the number is received, activity information of a user of a touch screen device that is displaying the activity page corresponding to the activity information of the user from among the touch screen devices.

13. The method of claim 1, further comprising transmitting activity information including user identification information, lesson identification information, activity identification information, and activity page identification information, to a management device from among a plurality of touch screen devices in which a same application is installed.

14. A touch screen device, comprising:

a touch screen unit that includes a display unit and a touch panel for outputting a display screen by converting image data to an electrical image signal;

a first operation tool sensing unit that senses contact by a first operation tool on the touch screen device and determines a position at which the first operation tool contacts the touch screen device;

a second operation tool sensing unit that senses access by a second operation tool on the touch screen and determines an input position of the second operation tool;

an operation action management unit that determines an action corresponding to an operation gesture of the second operation tool sensed in an operation area by the second operation tool, which moves on the first operation tool, from among actions that are previously registered in an interaction database (DB) of the touch screen device, and that outputs a control signal so that the action is performed; and

a network unit that transmits or receives data to or from an external device.

15. A method of operating a touch screen device, the method comprising:

identifying a first operation tool based on contact by the first operation tool, the contact being sensed on a touch screen; and

setting an operation area on the touch screen based on an area designated by the contact by the first operation tool,

wherein the identifying of a first operation tool comprises identifying based on a pattern formed of positions of a plurality of contact points arranged on the sensed first operation tool.

16. The method of claim 15, wherein the identifying a first operation tool comprises determining a position where the first operation tool contacts the touch screen device, by using an electrostatic sensor of the touch screen device.

17. The method of claim 15, wherein the identifying a first operation tool comprises identifying the first operation tool based on a sensed contacting state of the first operation tool, wherein the sensed contacting state of the first operation tool is from among identification information of operation tools registered with an operation tool register DB of the interaction DB,

wherein the setting an operation area on the touch screen device comprises determining an operation area of the identified first operation tool based on form information of operation tools that are previously registered with the operation tool register DB in the operation tool operation area,

wherein the identification information comprises at least one of a number of contact points of the first operation tool, a form of each of the contact points, a distance between the contact points, and a surface area of each of the contact points.

18. The method of claim 15, wherein the plurality of contact points arranged on the first operation tool are located around a contact point having a previously set form from among the contact points of the first operation tool, and are expressed as a combination of two-dimensional coordinate values.

19. The method of claim 15, wherein the identifying a first operation tool comprises:

storing, in an operation tool register DB, identification information of the first operation tool including at least one of a number of contact points of the first operation tool, a form of each of the contact points, a distance between the contact points, and a surface area of each of the contact points, which are stored in the interaction DB; and

storing information of an operation area determined based on a form of the first operation tool in the operation tool register DB.

20. The method of claim 15, wherein the setting an operation area on the touch screen comprises setting the operation area based on a rotational state of a contact point having the previously set form from among the contact points of the first operation tool.