METHOD AND APPARATUS FOR PROCESSING INPUT IN MOBILE TERMINAL

• US 20120293406A1
• Filed: 03/16/2012
• Published: 11/22/2012
• Est. Priority Date: 05/16/2011
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: PARK, Hyung-Kil, SEO, Min-Hwan
• Granted Patent: US 9,170,645 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning

Abstract

A method for processing an input in a mobile terminal that recognizes inputs such as a visual input and/or voice input is provided. In the method, a position on a screen corresponding to a viewing direction of a user'"'"'s eyes is determined. An indicator corresponding to the determined position is displayed on the screen. When a relevant (i.e. actuating) signal is detected, an indicator is arranged on the desired object for selection.

Claims

1. A method for processing an input in a mobile terminal, the method comprising:

determining by a controller a viewing position on a display screen corresponding to a tracking of a user'"'"'s eyes and displaying an indicator on the display screen corresponding to the determined position on the screen being viewed; and

when a sensor of the mobile terminal detects an actuating signal, selecting by the controller an object arranged on the indicator of the display screen.

2. The method of claim 1, further comprising, when the actuating signal is detected but the indicator is not arranged on the object displayed by the display screen, outputting a message of a character or voice form that informs the object has not been selected.

3. The method of claim 1, wherein the actuating signal occurs when it is detected that the user closes and opens both his eyes for a threshold time or longer.

4. The method of claim 1, wherein the actuating signal occurs when it is detected that a when “

select'"'"' voice is detected.

5. The method of claim 1, wherein object comprises an icon displayed by the display screen.

6. A method for processing an input in a mobile terminal, the method comprising:

determining by a controller a viewing position on a display screen corresponding to a tracking of a user'"'"'s eyes and displaying an indicator on the display screen corresponding to the determined position on the screen being viewed;

when a first signal is detected by at least one sensor of a plurality of sensors, the controller selecting an object arranged on the display by placing the indicator on an object designated to be rearranged in the display;

when a second signal is detected by at least one sensor of the plurality of sensors, moving a display of the selected object to correspond with movement of a user'"'"'s eyes; and

when a third signal is detected by at least one sensor of the plurality of sensors, arranging the selected object on a relevant position of the display screen.

7. The method of claim 6, further comprising, when the first signal is detected but the indicator has not been arranged on the object, providing a message in character or voice form indicating the object has not been selected.

8. The method of claim 6, wherein the first signal is detected by the mobile terminal when the user closes a first eye for a threshold time or longer, the second signal is detected when an eyeball of a second eye which is open moves, and the third signal occurs is detected when the user opens the closed first eye.

9. The method of claim 8, wherein the first signal is generated when a “

select”

voice is detected, the second signal is generated when a “

drag”

voice is detected, and the third signal is generated when a “

drop”

voice is detected.

10. A method for processing an input in a mobile terminal, the method comprising:

dividing a screen into one or more display regions to which a relevant function has been designated;

determining a viewing position on the display screen corresponding to a user'"'"'s eyes and displaying an indicator corresponding to the determined position on the screen; and

determining a region of the regions to which the indicator belongs, and performing the relevant function.

11. The method of claim 10, wherein the dividing of the screen into one or more displayed regions to which a relevant function has been designated comprises:

dividing the screen into a plurality of display regions to which a function for expanding content of the screen has been designated and a region to which a function for reducing content of the screen has been designated, or

dividing the screen into one or more regions to which a function for scrolling content of the screen to different directions has been designated, or

configuring regions replacing one or more key buttons configured outside the screen.

12. The method of claim 10, wherein determining a region of the one or more regions to which the indicator belongs, and performing the relevant function comprises:

when eye flickering is detected by a sensor of the mobile terminal, performing a relevant function designated to the display region to which the indicator belongs, or when a “

select”

voice is detected, performing the relevant function designated to the region to which the indicator belongs.

13. An apparatus for processing an input in a mobile terminal, the apparatus comprising:

an output unit for outputting voice data and/or display data;

a non-transitory storage that stores a predetermined program for controlling an overall operation and storing various data input/output while a control operation is performed; and

a controller receiving information regarding a user'"'"'s eye movement from a camera, detecting a voice command provided from a microphone, and performing a control of an overall operation,

wherein the controller determines a position on a screen corresponding to a viewing position of the user'"'"'s eyes, and displays an indicator corresponding to the determined position on the screen, and when an actuating signal is detected, selects an object put on the indicator.

14. The apparatus of claim 13, wherein when the actuating signal is detected the indicator is not positioned on an object for selection , the controller outputs a message of a character or voice form indicating the no object selection has been made.

15. The apparatus of claim 13, wherein the actuating signal occurs when the controller has determined the user closes and opens both eyes for a threshold time or longer.

16. The apparatus of claim 13, wherein the actuating signal occurs when the controller has determined, a “

select'"'"' voice is detected.

17. An apparatus for processing an input in a mobile terminal, the apparatus comprising:

an output unit for outputting voice data and display data;

a storage for storing a predetermined program for controlling an overall operation and storing various data input/output while a control operation is performed; and

a controller receiving information regarding a user'"'"'s eye movement from a camera, detecting a voice command provided from a microphone, and performing a control of an overall operation,

wherein the controller determines a viewing position on a display screen corresponding to the user'"'"'s eyes, displays an indicator corresponding to the determined viewed position on the screen, and when a first signal is detected, selects an object that has the indicator placed thereon as an object to be rearranged, when a second signal is detected, moves and displays the selected object along with corresponding movement of the user'"'"'s eyes, and when a third signal is detected, arranges and displays the selected object on a relevant position.

18. The apparatus of claim 17, wherein when the first signal is detected but the indicator is not placed on an object for selection, the controller outputs a character or voice message indicating that no object has been selected.

19. The apparatus of claim 17, wherein the first signal occurs when it is detected that the user closes a first eye for a threshold time or longer, the second signal occurs when it is detected that an eyeball of a second eye which is open and moves, and the third signal occurs when it is detected that the user opens the closed first eye.

20. The apparatus of claim 17, wherein the first signal occurs when a “

select”

voice is detected, the second signal occurs when a “

drag”

voice is detected, and the third signal occurs when a “

drop”

voice is detected.

21. An apparatus for processing an input in a mobile terminal, the apparatus comprising:

an output unit that outputs voice data and display data;

a non-transitory storage that stores a predetermined program for controlling an overall operation and storing various data input/output while a control operation is performed; and

a controller receiving information regarding a user'"'"'s eye movement from a camera, detecting a voice command provided from a microphone, and performing a control of an overall operation,

wherein the controller divides display of the screen into one or more regions to which a relevant function has been designated, determines a position on the screen corresponding to the user'"'"'s eyes, displays an indicator corresponding to the determined position on the screen, determines a region of the regions to which the indicator belongs, and performs the relevant function.

22. The apparatus of claim 21, wherein the controller divides the display of the screen into a region to which a function for expanding content of the screen has been designated and a region to which a function for reducing content of the screen has been designated, or divides the display of screen into one or more regions to which a function for scrolling content of the screen to respective directions has been designated, or configures regions replacing one or more key buttons configured outside the screen.

23. The apparatus of claim 21, wherein when eye flickering occurs, the controller determines a region of the region to which the indicator belongs, and performs an associated relevant function designated to the determined region, or when a “

select”

voice is detected, the controller determines a region of the region to which the indicator belongs, and performs an associated relevant function designated to the determined region.