Apparatus and method for processing user input using motion of object

• US 9,696,812 B2
• Filed: 05/22/2014
• Issued: 07/04/2017
• Est. Priority Date: 05/29/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Jun Haeng, Kim, Jun Seok, Park, Keun Joo, Ryu, Hyun Surk, Woo, Jooyeon, Shin, Chang-Woo, Lee, Kyoobin
• Primary Examiner(s): Abdin, Shaheda
• Application Number:
  US14/285,156
• Publication Number:
  US 20140354537A1
• Time in Patent Office: 1,139 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A user input processing apparatus using a motion of an object to determine whether to track the motion of the object, and track the motion of the object using an input image including information associated with the motion of the object.

Claims

1. An apparatus for processing a user input, the apparatus comprising:

a sensor configured to output event signals in response to sensing a movement of an object, wherein the event signals comprise information indicating pixels sensing an event corresponding to a part, which the movement is sensed, of the object, among sensing pixels of the sensor, wherein each of the pixels senses the event based on a change of input light due to the movement; and

a processor configured to determine whether the event signals correspond to a first mode input for a tracking operation, by comparing a scattering of the plurality of pixels to a threshold value, and to track the movement in response to the determination that the event signals correspond to the first mode input,

wherein the threshold value is greater than a first scattering of pixels corresponding to at least a part of a finger, and the threshold value is smaller than a second scattering of pixels corresponding to a hand.

2. The apparatus of claim 1, wherein the processor is further configured to determine that the event signals correspond to the first mode input, when the scattering of the plurality of pixels is greater than the threshold value.

3. The apparatus of claim 2, wherein the processor is further configured to determine that the event signals correspond to a second mode input for a selecting operation, when the scattering of the plurality of pixels is smaller than the threshold value.

4. The apparatus of claim 1, wherein the processor is further configured to calculate the scattering of the plurality of pixels in a predetermined direction.

5. The apparatus of claim 1, wherein the processor is further configured to:

calculate a centroid of the plurality of pixels, detect a dominant line associated with the plurality of pixels, and detect a portion of the object based on the centroid and the dominant line.

6. The apparatus of claim 5, wherein the processor is further configured to:

obtain a plurality of lines by performing a Hough transformation on the plurality of pixels, and obtain the dominant line based on the plurality of lines.

7. The apparatus of claim 5, wherein the processor is further configured to:

project the centroid onto the dominant line, and explore the dominant line from the projected location toward the portion of the object positioned in a predetermined direction.

8. The apparatus of claim 1, wherein the processor is further configured to:

calculate an orientation vector of the plurality of pixels, divide the plurality of pixels into a predetermined number of sections, calculate feature vectors of the sections based on the orientation vector, and recognize a portion of the object based on the feature vectors.

9. The apparatus of claim 8, wherein a direction of the orientation vector is quantized to be one of a predetermined number of quantization directions.

10. The apparatus of claim 8, wherein the processor is further configured to:

obtain a section corresponding to the portion of the object from among the predetermined number of sections by inputting a previously learned parameter and the feature vectors to a classifier.

11. The apparatus of claim 1, wherein the processor is further configured to determine a pattern of the plurality of pixels corresponds to a predetermined pose instructing suspension of the tracking.

12. The apparatus of claim 11, wherein the processor is further configured to:

calculate a centroid of the plurality of pixels;

detect, among the plurality of pixels, an area pattern of a point included in an area, separate from the centroid by at least a predetermined distance, in a predetermined direction; and

determine whether the area pattern corresponds to the predetermined pose.

13. The apparatus of claim 11, wherein the processor is further configured to:

extract a component of a predetermined direction by applying a line filter method to the plurality of pixels,

detect a component pattern of the plurality of pixels based on the component of the predetermined direction, and determine whether the component pattern corresponds to the predetermined pose.

14. The apparatus of claim 1, wherein the at least the part of the finger corresponds to an index finger.

15. A method of processing a user input, the method comprising:

obtaining event signals output by a sensor in response to sensing a movement of an object, wherein the event signals comprise information indicating pixels sensing an event corresponding to a part, which the movement is sensed, of the object, among sensing pixels of the sensor, wherein each of the pixels senses the event based on a change of input light due to the movement;

determining whether the event signals correspond to a selection input processing mode for a selecting operation, by comparing a scattering of the plurality of pixels to a threshold value, wherein the threshold value is greater than a first scattering of pixels corresponding to at least a part of a finger, and the threshold value is smaller than a second scattering of pixels corresponding to a hand;

calculating a centroid of the plurality of pixels;

extracting a dominant line included of the plurality of pixels; and

recognizing a portion of the object based on the centroid and the dominant line.

16. The method of claim 15, wherein the obtaining comprises:

classifying the plurality of pixels into a plurality of blocks;

selecting blocks including at least a predetermined number of pixels from among the plurality of blocks; and

obtaining pixels included in the selected blocks.

17. The method of claim 15, further comprising:

calculating the scattering of the plurality of pixels;

comparing the scattering to the threshold value; and

switching an operation mode to the selection input processing mode based on a result of the comparing.

18. The method of claim 15, further comprising:

determining whether a pattern of the plurality of pixels corresponds to a predetermined pose indicating suspension of a tracking mode; and

switching an operation mode to a suspension mode based on a result of the determining.

19. A non-transitory computer-readable medium comprising a program which, when executed, causes a computer to perform the method of claim 15.

20. A user input processing method comprising:

obtaining event signals output by a sensor in response to sensing a movement of an object, wherein the event signals comprise information indicating pixels sensing an event corresponding to a part, which the movement is sensed, of the object, among sensing pixels of the sensor, wherein each of the pixels senses the event based on a change of input light due to the movement;

determining whether the event signals correspond to a selection input processing mode for a selecting operation, by comparing a scattering of the plurality of pixels to a threshold value, wherein the threshold value is greater than a first scattering of pixels corresponding to at least a part of a finger, and the threshold value is smaller than a second scattering of pixels corresponding to a hand;

dividing the plurality of pixels into a predetermined number of sections;

calculating orientation vectors of the plurality of pixels;

calculating feature vectors of the sections based on the orientation vectors; and

recognizing a portion of the object based on the feature vectors.

21. The method of claim 20, wherein the calculating of the orientation vectors comprises quantizing directions of the orientation vectors to be one of a predetermined number of quantization directions.

22. The method of claim 20, wherein the calculating of the feature vectors comprises calculating a vector sum of the orientation vectors of the pixels included in each of the sections, and generating a feature vector of each of the sections based on the vector sum.

23. The method of claim 20, wherein when the directions of the orientation vectors are quantized to be one of the predetermined number of quantization directions, the calculating of the feature vectors comprises:

counting a number of pixels corresponding to the quantization directions among the pixels included in each of the sections; and

generating a feature vector of each of the sections based on the number of pixels counted.

24. The method of claim 20, wherein the feature vector for each of the sections comprises elements corresponding to the quantization directions, and each of the elements is generated based on the number of pixels counted correspondingly to each of the quantization directions.

25. The method of claim 20, wherein the feature vector of each of the sections is generated based on an orientation vector corresponding to a quantization direction of which a number of pixels is the greatest among the quantization directions.

26. The method of claim 20, wherein the calculating of the feature vectors comprises:

extracting the most dominant vector from among the orientation vectors of the pixels in each of the sections; and

generating a feature vector for each of the sections based on the most dominant vector.

27. The method of claim 20, wherein the recognizing comprises acquiring a section corresponding to the portion of the object among the predetermined number of sections by inputting a learned parameter and the feature vectors to a classifier.

28. The method of claim 20, wherein the learned parameter is previously learned based on feature vectors of a plurality of training images and sections corresponding to the portion of the object in the plurality of training images.

29. A non-transitory computer-readable medium comprising a program which, when executed, causes a computer to perform the method of claim 20.