SYSTEM AND METHOD FOR CONTEXT-BASED INTERACTION FOR ELECTRONIC DEVICES

• US 20180336009A1
• Filed: 12/28/2017
• Published: 11/22/2018
• Est. Priority Date: 05/22/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yoganandan, Arun Rakesh, Akiyoshi, Kumi, Mauk, Tais C., Zhu Jin, Chang Long, Hong, Jung Won
• Granted Patent: US 11,221,823 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method includes receiving a voice input at an electronic device. An ambiguity of the voice input is determined. The ambiguity is resolved based on contextual data. The contextual data includes at least one of: an image, a non-voice input comprising a gesture, a pointer of a pointing device, a touch, or a combination thereof.

Claims

1. A method, comprising:

receiving a voice input at an electronic device;

determining an ambiguity of the voice input; and

resolving the ambiguity based on contextual data,

wherein the contextual data includes at least one of;

an image, a non-voice input comprising a gesture, a pointer of a pointing device, a touch, or a combination thereof.

2. The method of claim 1, wherein the ambiguity relates to identification of an object or a position that the voice input applies to.

3. The method of claim 2, wherein the image is one of captured by a camera coupled to the electronic device, displayed on a display device coupled to the electronic device, or a combination thereof.

4. The method of claim 2, wherein the non-voice input is sensed by at least one sensor coupled to the electronic device.

5. The method of claim 2, further comprising:

resolving the ambiguity by determining identification of the object based on the image containing the object or the non-voice input indicating the object.

6. The method of claim 5, wherein the ambiguity includes a demonstrative determiner, and the contextual data further includes information affecting an action applicable to the object.

7. The method of claim 3, further comprising:

resolving the ambiguity by enabling the camera upon a determination that no non-voice input occurred.

8. The method of claim 2, wherein the object is the electronic device, and the electronic device adjusts an interface corresponding to the voice input with the ambiguity resolved.

9. The method of claim 6, wherein the action applicable to the object comprises one of:

receiving information, assisting with a purchase, calendaring an event, applying features to content, selecting at least one content associated with the object, moving the object on a display, or a combination thereof.

10. An electronic device comprising:

a memory storing instructions; and

at least one processor executing the instructions including a process configured to;

receive a voice input;

determine an ambiguity of the voice input; and

resolve the ambiguity based on contextual data,

wherein the contextual data includes at least one of;

an image, a non-voice input comprising a gesture, a pointer of a pointing device, a touch, or a combination thereof.

11. The electronic device of claim 10, wherein the ambiguity relates to identification of an object or a position that the voice input applies to;

and the image is one of captured by a camera coupled to the electronic device, displayed on a display associated with the electronic device, or a combination thereof.

12. The electronic device of claim 11, wherein the non-voice input is sensed by at least one sensor coupled to the electronic device.

13. The electronic device of claim 11, wherein:

the process is further configured to;

resolve the ambiguity by determining identification of the object based on the image containing the object or the non-voice input indicating the object;

the ambiguity includes a demonstrative determiner; and

the contextual data further includes information affecting an action applicable to the object.

14. The electronic device of claim 11, wherein the process is further configured to:

resolve the ambiguity by enabling the camera upon a determination that no non-voice input occurred.

15. The electronic device of claim 11, wherein the object is the electronic device, and the electronic device adjusts an interface corresponding to the voice input with the ambiguity resolved.

16. The electronic device of claim 13, wherein the action applicable to the object comprises one of:

receiving information, assisting with a purchase, calendaring an event, applying features to content, selecting at least one content associated with the object, moving the object on a display, or a combination thereof.

17. A non-transitory processor-readable medium that includes a program that when executed by a processor performing a method comprising:

receiving a voice input at an electronic device;

determining an ambiguity of the voice input; and

resolving the ambiguity based on contextual data,

wherein the contextual data includes at least one of;

an image, a non-voice input comprising a gesture, a pointer of a pointing device, a touch, or a combination thereof.

18. The non-transitory processor-readable medium of claim 17, wherein:

the ambiguity relates to identification of an object or a position that the voice input applies to;

the image is one of captured by a camera coupled to the electronic device, displayed on a display associated with the electronic device, or a combination thereof; and

the non-voice input is sensed by at least one sensor coupled to the electronic device.

19. The non-transitory processor-readable medium of claim 18, wherein the method further comprises:

resolving the ambiguity by;

determining identification of the object based on the image containing the object or the non-voice input indicating the object, or enabling the camera upon a determination that no non-voice input occurred;

wherein;

the ambiguity includes a demonstrative determiner;

the contextual data further includes information affecting an action applicable to the object; and

the action applicable to the object comprises one of;

receiving information, assisting with a purchase, calendaring an event, applying features to content, selecting at least one content associated with the object, moving the object on a display, or a combination thereof.

20. The non-transitory processor-readable medium of claim 17, wherein the object is the electronic device, and the electronic device adjusts an interface corresponding to the voice input with the ambiguity resolved.