Display apparatus and controlling method thereof

• US 10,191,554 B2
• Filed: 03/13/2015
• Issued: 01/29/2019
• Est. Priority Date: 03/14/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Bong-seok, Lee, Dong-Hun
• Primary Examiner(s): Michaud, Robert
• Application Number:
  US14/657,462
• Publication Number:
  US 20150261305A1
• Time in Patent Office: 1,418 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A display apparatus and a controlling method thereof are provided. The controlling method of a display apparatus includes displaying a mode conversion user interface (UI) that provides a guideline to enter a motion task mode; recognizing a first motion that corresponds to the guideline; entering the motion task mode that enables a second motion to control a function of the display apparatus in response to the first motion being recognized while the guideline is displayed on the display apparatus; and displaying a motion task UI to perform the motion task mode.

Claims

1. A controlling method of a display apparatus, comprising:

based on a captured image of an object corresponding to a predetermined shape while the display apparatus is operated in a general driving mode, activating an intermediate mode and displaying a mode conversion user interface (UI) that provides a guideline of a first motion for switching from the intermediate mode to a motion task mode that allows a user to control at least one function of the display apparatus via a second motion different from the first motion;

based on recognizing the first motion that corresponds to the guideline within a predetermined time since the intermediate mode is activated, activating the motion task mode and displaying a motion task UI to control the display apparatus in accordance with the second motion;

based on recognizing the second motion to control the at least one function of the display apparatus while the display apparatus is operated in the motion task mode, controlling the at least one function corresponding to the second motion,

wherein the general driving mode is switched to the motion task mode through the intermediate mode that occurs between the general driving mode and the motion task mode, based on the first motion being recognized within the predetermined time since the intermediate mode is activated.

2. The method as claimed in claim 1, further comprising recognizing, in the general driving mode, a shape of the object in the captured image as a triggering input that activates the intermediate mode.

3. The method as claimed in claim 2, wherein further comprising recognizing, in the general driving mode, a voice command as the triggering input that activates the intermediate mode.

4. The method as claimed in claim 1, wherein the motion conversion UI comprises a mode conversion icon configured to activate the motion task mode and a pointer configured to move towards the mode conversion icon according to the first motion.

5. The method as claimed in claim 4, wherein the entering the motion task mode comprises entering the motion task mode in response to the pointer moving to and being placed on the mode conversion icon according to the first motion.

6. The method as claimed in claim 1, wherein the guideline indicates a predetermined motion pattern set to perform the first motion.

7. The method as claimed in claim 6, wherein the entering the motion task mode comprises entering the motion task mode in response to the first motion corresponding to the predetermined motion pattern.

8. The method as claimed in claim 1, further comprising:

operating the display apparatus in the general driving mode before the mode conversion UI is displayed, and

in response to the mode conversion UI being displayed, converting the general driving mode to a power driving mode in which resolution and power consumption of the display apparatus increases in comparison with the general driving mode to perform motion recognition, the power driving mode comprising the intermediate mode and the motion task mode.

9. The method as claimed in claim 8, further comprising converting the power driving mode into the general driving mode in response to the second motion not being recognized with the predetermined time after the mode conversion UI is displayed.

10. The method as claimed in claim 1, wherein the motion task mode is controlled by tracking a motion of the object.

11. The method as claimed in claim 10, wherein the object is one of a hand, a finger, an arm, and a face of the user, or an inanimate object.

12. A display apparatus, comprising:

a motion recognition unit configured to recognize a first motion and a second motion of a user;

a display configured to display a user interface (UI); and

a controller configured to;

based on a captured image of an object corresponding to a predetermined shape through the motion recognition unit while the display apparatus is operated in a general driving mode, activate an intermediate mode and control the display to display a mode conversion user interface (UI) that provides a guideline of a first motion for switching from an intermediate mode to a motion task mode that allows the user to control at least one function of the display apparatus via a second motion different from the first motion,

based on recognizing the first motion corresponding to the guideline within a predetermined time since the intermediate mode is activated, activating the motion task mode and control the display to a motion task UI to control the display apparatus in accordance with the second motion,

based on recognizing the second motion to control the at least one function of the display apparatus while the display apparatus is operated in the motion task mode, control the at least one function corresponding to the second motion,

wherein the general driving mode is switched to the motion task mode through the intermediate mode that occurs between the general driving mode and the motion task mode, based on the first motion being recognized within the predetermined time since the intermediate mode is activated.

13. The apparatus as claimed in claim 12, wherein the controller is further configured to control the display to display a mode conversion icon configured to activate the motion task mode and a pointer configured to move toward the mode conversion icon according to the first motion.

14. The apparatus as claimed in claim 13, wherein the controller is further configured to enter the motion task mode in response to the pointer moving to and being placed on the mode conversion icon according to the first motion.

15. The apparatus as claimed in claim 12, wherein the controller is further configured to control the display to display the guideline through a motion pattern guide UI which guides the user to take a predetermined motion to enter the motion task mode.

16. The apparatus as claimed in claim 15, wherein the controller is further configured to enter the motion task mode in response to a pattern of the first motion that corresponds to the motion pattern guide UI being recognized.

17. The apparatus as claimed in claim 12, wherein the controller is further configured to drive the display in the general driving mode before the motion conversion UI being displayed, and in response to the motion conversion UI being displayed, convert the general driving mode to a power driving mode in which power consumption of the display apparatus increases in comparison with the general driving mode to perform motion recognition, the power driving mode comprising the intermediate mode and the motion task mode.

18. The apparatus as claimed in claim 17, wherein the controller is further configured to convert the power driving mode to the general driving mode in response to a predetermined motion not being recognized within a predetermined time after the mode conversion UI is displayed.

19. The apparatus as claimed in claim 12, wherein the controller is further configured to recognize an object and control the display to display the mode conversion UI in response to the object being recognized, and wherein the motion task mode is controlled by tracking a motion of the recognized object.

20. The apparatus as claimed in claim 12, wherein the object is one of a hand, a finger, an arm, and a face of the user, or an inanimate object.

21. A controlling method of a display apparatus, comprising:

recognizing a triggering input while the display apparatus is operated in a general driving mode;

controlling the display apparatus to operate in an intermediate mode to display a guideline of an action input that activates a motion task mode before activating the motion task mode that allows a user to control at least one function of the display apparatus via a motion input;

switching from the intermediate mode to the motion task mode based on a recognizing a first motion that corresponds to the action input within a predetermined time since the intermediate mode is activated; and

recognizing a second motion to control the at least one function of the display apparatus while the display apparatus is operated in the motion task mode,

controlling the at least one function corresponding to the second motion

wherein the general driving mode is switched to the motion task mode through the intermediate mode that occurs between the general driving mode and the motion task mode, based on the first motion being recognized within the predetermined time since the intermediate mode is activated, and

wherein the triggering input, the first motion, and the second motion are different from each other.

22. The method as claimed in claim 21, further comprising identifying whether the first motion is performed within a predetermined size of a region that covers an area of the displayed guideline.

23. The method as claimed in claim 21, further comprising switching the intermediate mode to the general driving mode in response to the display apparatus failing to recognize the first motion within the predetermined time.

24. The method as claimed in claim 21, further comprising changing a visual indicator of the guideline while the first motion is being recognized and processed.

25. The method as claimed in claim 24, wherein the changing the visual indicator of the guideline comprises changing a typeface or a color of the guideline while the first motion is being recognized and processed.

26. The method as claimed in claim 24, wherein the changing the visual indicator of the guideline comprises keeping the visual indicator flashing while the first motion is being recognized and processed.