APPARATUS AND METHOD OF USING EVENTS FOR USER INTERFACE

• US 20150095818A1
• Filed: 06/12/2014
• Published: 04/02/2015
• Est. Priority Date: 10/01/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: LEE, Kyoobin, RYU, Hyun Surk
• Granted Patent: US 10,310,615 B2
• Time in Patent Office: Days
• US Class Current: 715/765
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 25/48 specially adapted for particular use

Abstract

An apparatus for use interface and a method of user interface are provided. The apparatus may include a classifier configured to classify an event as corresponding to a class among at least two classes, an updater configured to update class information related to the class corresponding to the event, and a processor configured to determine a user input corresponding to the event based on the updated class information.

Claims

1. An apparatus for user interface, the apparatus comprising:

a classifier configured to classify an event as corresponding to a class among at least two classes;

an updater configured to update class information related to the class corresponding to the event; and

a processor configured to determine a user input corresponding to the event based on the updated class information.

2. The apparatus of claim 1, wherein the classifier is configured to classify the event as corresponding to the class among the at least two classes based on representative values corresponding to the at least two classes.

3. The apparatus of claim 2, wherein the representative values correspond to coordinates or position-related information regarding each of the at least two classes, or to frequency, amplitude or intensity related information regarding each of the at least two classes.

4. The apparatus of claim 2, wherein the classifier comprises:

a comparator configured to compare an event value corresponding to the event to the representative values; and

a detector configured to detect a class having a representative value most similar to the event value.

5. The apparatus of claim 1, wherein the updated class information comprises a representative value representing the class of the classified event.

6. The apparatus of claim 1, wherein the at least two classes correspond to at least two objects associated with the user input.

7. The apparatus of claim 6, wherein the processor is configured to track the at least two objects to determine the user input.

8. The apparatus of claim 1, further comprising:

at least one event-based sensor configured to detect the event.

9. The apparatus of claim 8, wherein the at least one event-based sensor comprises at least one of an event-based vision sensor, an event-based sound sensor, and an event-based touch sensor.

10. The apparatus of claim 1, wherein the processor comprises:

an optical flow calculator configured to calculate optical flows corresponding to the at least two classes; and

a velocity component calculator configured to calculate velocity components corresponding to the at least two classes based on the calculated optical flows.

11. The apparatus of claim 10, wherein the velocity components comprise at least one of:

a first velocity component at which an object generating the event moves in an x-axial direction on a virtual plane used to detect an event;

a second velocity component at which the object moves in a y-axial direction on the virtual plane;

a third velocity component at which the object moves in a direction orthogonal to the virtual plane; and

a fourth velocity component at which the object rotates on the virtual plane.

12. The apparatus of claim 10, wherein the processor further comprises:

a recognizer configured to recognize the user input by analyzing the velocity components based on a user input context.

13. The apparatus of claim 10, wherein the processor further comprises:

a recognizer configured to recognize the user input by analyzing a relative velocity between the class of the classified event and at least one remaining class based on the velocity components.

14. The apparatus of claim 1, wherein the processor comprises:

a recognizer configured to recognize the user input by analyzing a representative value representing the class of the classified event and a representative value representing at least one remaining class based on a user input context.

16. The method of claim 14, wherein the identifying information of the event comprises an index identifying the event, a coordinate or position-related information of the event, or frequency, amplitude or intensity related information regarding the event;

and the updating comprises updating the value of the element stored in a memory.

17. The method of claim 16, wherein the selecting comprises:

comparing the index to the representative values of the plurality of classes; and

detecting a class having a representative value most similar to the index.

18. The method of claim 16, wherein the determining comprises:

calculating a velocity vector corresponding to the updated element based on the updated element and ambient elements of the updated element;

calculating a plurality of velocity components corresponding to the selected class based on the calculated velocity vector corresponding to the updated element; and

recognizing the user input by analyzing the calculated plurality of velocity components.

19. The method of claim 18, wherein the calculating of the velocity vector comprises:

calculating a time difference between the updated element and the ambient elements;

calculating a distance difference between the updated element and the ambient elements; and

calculating the velocity vector corresponding to the updated element based on the calculated time difference and the calculated distance difference.

20. The method of claim 18, wherein the plurality of velocity components comprises:

a first velocity component at which an object generating the event moves in an x-axial direction on a virtual plane used to detect an event;

a second velocity component at which the object moves in a y-axial direction on the virtual plane;

a third velocity component at which the object moves in a direction orthogonal to the virtual plane; and

a fourth velocity component at which the object rotates on the virtual plane.

21. The method of claim 20, wherein the calculating of the plurality of velocity components comprises:

extracting elements corresponding to the selected class from the plurality of elements;

calculating the first velocity component and the second velocity component by averaging velocity vectors of the extracted elements;

calculating a central point of the selected class based on magnitudes of the velocity vectors of the extracted elements;

calculating the third velocity component by calculating inner products of the velocity vectors of the extracted elements and vectors toward the extracted elements from the central point; and

calculating the fourth velocity component by calculating outer products of the velocity vectors of the extracted elements and the vectors toward the extracted elements from the central point.

15. A method of user interface, the method comprising:

receiving a signal comprising a timestamp of an event and an identifying information of the event;

selecting a class from a plurality of predetermined classes based on the identifying information;

updating a representative value of the selected class based on the identifying information;

updating a value of an element corresponding to the event in an event map, based on the timestamp; and

determining a user input corresponding to the event based on at least one of the event map and representative values of the plurality of classes.

22. The method of claim 15, wherein the determining comprises analyzing the updated representative value of the selected class and representative values of remaining classes.

23. The method of claim 15, wherein the signal comprises an output signal of an event-based sensor configured to detect the event.

28. A non-transitory computer-readable storage medium comprising a program comprising instructions to cause a computer to perform the method of claim 15.

24. A method of user interface, the method comprising:

receiving events corresponding to a plurality of objects;

classifying each of the events as corresponding to a class among a plurality of classes that correspond to the plurality of objects;

updating class information relating to one or more of the classes based on the classified events; and

determining a user input based on the updated class information.

25. The method of claim 24, wherein the classifying comprises detecting, for each of the events, a class having a representative value most similar to a value of the corresponding event, among the plurality of classes.

26. The method of claim 24, wherein the class information comprises at least one of pixels included in the classes, events included in the classes, and representative values of the classes stored in a memory.

27. The method of claim 24, wherein the determining comprises:

calculating optical flows for the classes;

calculating velocity components of the classes based on the calculated optical flows; and

recognizing a command to process the user input based on the velocity components of the classes.

29. An apparatus for user interface, the apparatus comprising:

a classifier configured to classify an event as corresponding to an object among a plurality of objects;

an updater configured to update representative values corresponding to at least one of the plurality of objects based on the classified event; and

a processor configured to determine a user input corresponding to the event based on the updated representative values.

30. The apparatus of claim 29, wherein the event comprises at least one of a brightness-changing event, a sound-changing event or a touch event detected by a sensor.

31. The apparatus of claim 29, wherein the representative values correspond to coordinates or position-related information regarding each of the at least two classes, or to frequency, amplitude or intensity related information regarding each of the at least two classes.

32. The apparatus of claim 29, wherein the processor is configured to determine the user input by determining a direction of change regarding one or more of the plurality of objects based on the updated representative values.

33. The apparatus of claim 29, wherein the processor is configured to determine the user input based on the updated representative values by determining either a relative movement of one object to another object among the plurality of objects or a change in frequency, amplitude, or intensity of sound produced by one or more of the plurality of objects.