Mobile terminal and touch recognizing method therein

• US 9,128,527 B2
• Filed: 11/01/2011
• Issued: 09/08/2015
• Est. Priority Date: 12/29/2010
• Status: Expired due to Fees

• Current Assignee: Industry Foundation Of Chonnam National University (Chonnam National University), LG Electronics, Inc. (LG Corporation)
• Original Assignee: Industry Foundation Of Chonnam National University (Chonnam National University), LG Electronics, Inc. (LG Corporation)
• Inventors: Kim, Minjoo, Lee, Chilwoo, Seo, Yungho, Oh, Chimin, Kwak, Jaedo, Kim, Jonggu
• Primary Examiner(s): Chang, Kent
• Assistant Examiner(s): Lu, William
• Application Number:
  US13/287,062
• Publication Number:
  US 20120169670A1
• Time in Patent Office: 1,407 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A mobile terminal and touch recognizing method therein are disclosed, by which a multi-touch can be recognized. The present invention includes receiving a plurality of touches simultaneously using a touchscreen, consecutively receiving a plurality of images including a plurality of the received touches for a predetermined time using a camera provided under the touchscreen, determining patterns and pattern changes of a plurality of the touches included in a plurality of the images, respectively, and recognizing a gesture by a plurality of the touches in accordance with a result of the determining step. Moreover, the touch pattern includes at least one selected from the group consisting of a touch number, a touch point, a touch size and a touch figure.

Claims

1. A mobile terminal, comprising:

a touchscreen configured to simultaneously receive a plurality of touches and to display information;

a camera provided under the touchscreen, the camera configured to capture a plurality of images of the plurality of received touches; and

a controller configured to;

detect the plurality of touches from the plurality of captured images;

detect a first touch scheme, a second touch scheme, a third touch scheme, and a fourth touch scheme from the plurality of touches, the first touch scheme and the second touch scheme each comprising at least two touches;

determine patterns and pattern changes of a first intermediate point of the at least two touches of the first touch scheme and a second intermediate point of the at least two touches of the second touch scheme;

recognize a gesture pattern of the plurality of touches based on the patterns and pattern changes of the first intermediate point and the second intermediate point;

recognize the third touch scheme as an enlarging gesture, wherein the third touch scheme comprises a first touch and a second touch of the plurality of touches, the second touch located to a right side of the first touch, and wherein the first touch is rotated clockwise while the second touch is rotated counter-clockwise at a same time, each of the first and second touches rotated while maintaining a same respective touch location;

cause enlargement of the displayed information based on the recognized third touch scheme;

recognize the fourth touch scheme as a reducing gesture, wherein the fourth touch scheme comprises a third touch and a fourth touch of the plurality of touches, the third and fourth touches simultaneously received via the touchscreen, and the fourth touch located to a right side of the third touch, and wherein the third touch is rotated counter-clockwise while the fourth touch is rotated clockwise at a same time, each of the third and fourth touches rotated while maintaining a same respective touch location; and

cause reduction of the displayed information based on the recognized fourth touch scheme.

2. The mobile terminal of claim 1, wherein the controller is further configured to recognize the gesture pattern by detecting at least a number of touches, a position of at least one touch, a size of at least one touch, or a shape of at least one touch.

3. The mobile terminal of claim 1, wherein the controller is further configured to recognize the gesture pattern by detecting at least a moved distance, a moved direction, or a change in distance from a particular point of each of the plurality of touches.

4. The mobile terminal of claim 3, wherein the controller is further configured to detect:

a reducing gesture pattern when the moved direction of each of two or more of the plurality of touches is a direction converging toward a particular point; and

an enlarging gesture pattern when the moved direction of each of two or more of the plurality of touches is a direction expanding away from a particular point.

5. The mobile terminal of claim 3, wherein the controller is further configured to:

group at least two touches of the plurality of touches, wherein the moved direction of each of the at least two touches comprises a same direction; and

generate a representative touch of the grouped at least two touches.

6. The mobile terminal of claim 5, wherein the controller is further configured to detect at least a shift gesture pattern or a scroll gesture pattern in response to a moved distance or a moved direction of the representative touch.

7. The mobile terminal of claim 2, wherein the controller is further configured to:

recognize an enlarging gesture pattern when the size of each of the plurality of touches increases; and

recognize a reducing gesture pattern when the size of each of the plurality of touches decreases.

8. The mobile terminal of claim 2, wherein the controller is further configured to recognize a rotating gesture pattern by detecting a rotating movement of each of the plurality of touches in a first direction.

9. The mobile terminal of claim 1, further comprising a memory configured to store information related to a plurality of gesture patterns corresponding to the plurality of touches.

10. The mobile terminal of claim 1, wherein the controller is further configured to cause the touchscreen to perform a display operation corresponding to the recognized gesture pattern.

11. A method for recognizing touch inputs in a mobile terminal, the method comprising:

displaying, on a touchscreen of the mobile terminal, information;

capturing, via a camera of the mobile terminal, a plurality of images of a plurality of touches to the touchscreen;

detecting the plurality of touches from the plurality of captured images;

detecting a first touch scheme, a second touch scheme, a third touch scheme, and a fourth touch scheme from the plurality of touches, the first touch scheme and the second touch scheme each comprising at least two touches;

determining patterns and pattern changes of a first intermediate point of the at least two touches of the first touch scheme and a second intermediate point of the at least two touches of the second touch scheme;

recognizing a gesture pattern of the plurality of touches based on the patterns and pattern changes of the first intermediate point and the second intermediate point;

recognizing the third touch scheme as an enlarging gesture, wherein the third touch scheme comprises a first touch and a second touch of the plurality of touches, the first and second touches simultaneously received via the touchscreen, and the second touch located to a right side of the first touch, and wherein the first touch is rotated clockwise while the second touch is rotated counter-clockwise at a same time, each of the first and second touches rotated while maintaining a same respective touch location;

enlarging the displayed information based on the recognized third touch scheme;

recognizing the fourth touch scheme as a reducing gesture, wherein the fourth touch scheme comprises a third touch and a fourth touch of the plurality of touches, the fourth touch located to a right side of the third touch, and wherein the third touch is rotated counter-clockwise while the fourth touch is rotated clockwise at a same time, each of the third and fourth touches rotated while maintaining a same respective touch location; and

reducing the displayed information based on the recognized fourth touch scheme.

12. The method of claim 11, wherein the gesture pattern is further recognized by detecting at least a moved distance, a moved direction, or a change in distance from a particular point of each of the plurality of touches.

13. The method of claim 11, wherein recognizing the gesture pattern further comprises:

recognizing a reducing gesture pattern when the moved direction of each of the plurality of touches is a direction converging toward a particular point;

recognizing an enlarging gesture pattern when the moved direction of each of the plurality of touches is a direction expanding away from a particular point; and

recognizing a rotating gesture pattern when the moved direction of each of the plurality of touches is circular about a particular point in a first direction.

14. The method of claim 12, further comprising:

grouping at least two touches of the plurality of touches, wherein a detected moved direction of each of the at least two touches comprise a same direction; and

generating a representative touch of the grouped at least two touches.

15. The method of claim 14, further comprising detecting at least a shift gesture pattern or a scroll gesture pattern in response to a moved distance or a moved direction of the representative touch.