Multi display device and control method thereof

• US 9,317,198 B2
• Filed: 10/10/2013
• Issued: 04/19/2016
• Est. Priority Date: 10/10/2012
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kwak, Ji-yeon, Kim, Dae-hyun, Kim, Sang-ho
• Primary Examiner(s): Nguyen, Kevin
• Application Number:
  US14/050,871
• Publication Number:
  US 20140101578A1
• Time in Patent Office: 922 Days
• Field of Search: 715761-765, 715/778, 715851-853, 715840-841
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1618 the display being foldable up to the back of the other housing with a single degree of freedom, e.g. by 360° rotation over the axis defined by the rear edge of the base enclosure
• CPC Class Codes: G06F 1/1647 including at least an additional display G06F1/1692 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A multi display apparatus includes a first body comprising a first display, a second body comprising a second display, a hinge to connect the first and second bodies to each other, a first imaging unit provided on the first body, a second imaging unit provided on the second body, and a controller to recognize a user'"'"'s gesture using a plurality of images photographed at the first and second imaging units, and perform a corresponding control operation in response to the recognized user gesture. The controller recognizes the user'"'"'s gesture using a movement of a user object within recognition ranges of the respective imaging units. As a result, the operation of the multi display apparatus can be controlled more easily and conveniently.

Claims

1. A multi display apparatus, comprising:

a first body comprising a first display;

a second body comprising a second display;

a hinge to connect the first and second bodies to each other;

a first imaging unit provided on the first body and having a first recognition range;

a second imaging unit provided on the second body and having a second recognition range; and

a controller to recognize a user'"'"'s gesture using a plurality of images photographed at the first and second imaging units, and perform a corresponding control operation in response to the recognized user gesture, the controller recognizing the user'"'"'s gesture within an overlapping range between the first recognition range and the second recognition range, using a direction of movement of a user object moving in the first recognition range and a direction of movement of the user object moving in the second recognition range.

2. The multi display apparatus of claim 1, wherein the controller detects an angle between the first and second bodies with reference to the hinge, and activates the first and second imaging units when the detected angle is within a preset angle range.

3. The multi display apparatus of claim 1, wherein the controller recognizes the user'"'"'s gesture three-dimensionally.

4. The multi display apparatus of claim 1, wherein the controller recognizes the user'"'"'s gesture as a stereo image.

5. The multi display apparatus of claim 1, wherein, when the user object moves, thus forming one successive pattern, the controller successively performs a plurality of control operations corresponding to respective user'"'"'s gestures forming the successive pattern.

6. The multi display apparatus of claim 1, wherein the user'"'"'s gesture comprises one of a single finger gesture, a two finger gesture, a multi finger gesture, a palm gesture, a multi palm gesture, a first gesture, and a multi first gesture.

7. The multi display apparatus of claim 1, wherein the controller performs an operation of selecting an icon, when recognizing a grip gesture with respect to the icon displayed on the first or second display.

8. A control method of a multi display apparatus comprising a first body comprising a first display, a second body comprising a second display, and a hinge which connects the first and second bodies to each other, the control method comprising:

photographing a user object, using a first imaging unit provided on the first body and a second imaging unit provided on the second body;

recognizing a user'"'"'s gesture within an overlapping range between a first recognition range of the first imaging unit and a second recognition range of the second imaging unit, using a plurality of images of the user object photographed by the first and second imaging units, respectively, to determine a direction of movement of the user object moving in the first recognition range and a direction of movement of the user object moving in the second recognition range; and

performing a corresponding control operation in response to the recognized user'"'"'s gesture.

9. The control method of claim 8, wherein the recognizing the user'"'"'s gesture comprises:

detecting an angle between the first and second bodies with reference to the hinge; and

activating the first and second imaging units when the detected angle is within a preset angle range.

10. The control method of claim 8, wherein the user'"'"'s gesture is recognized three-dimensionally.

11. The control method of claim 8, wherein the user'"'"'s gesture is recognized as a stereo image.

12. The control method of claim 8, wherein, when the user object moves, thus forming one successive pattern, the performing the control operation comprises:

successively performing a plurality of control operations corresponding to respective user'"'"'s gestures forming the successive pattern.

13. The control method of claim 8, wherein the user'"'"'s gesture comprises:

one of a single finger gesture, a two finger gesture, a multi finger gesture, a palm gesture, a multi palm gesture, a first gesture, and a multi first gesture.

14. The control method of claim 8, wherein the performing the control operation comprises:

performing an operation of selecting an icon, when a grip gesture with respect to the icon displayed on the first or second display is recognized.

15. A non-transitory computer-readable medium to contain computer-readable codes as a program to execute the control method of claim 8.

16. A multi display apparatus, comprising:

a plurality of displays;

a first imaging unit having a first recognition range;

a second imaging unit having a second recognition range; and

a controller to determine a user gesture based on a plurality of images of a first user object captured by the first and second imaging units in a gesture recognition area where the first and second recognition ranges overlap, and to perform a command operation corresponding to the determined user gesture.

17. The multi display apparatus of claim 16, wherein the first user object comprises:

at least one of a user'"'"'s hand, a user'"'"'s face, a user'"'"'s eye, or a stylus pen.

18. The multi display apparatus of claim 16, wherein the user gesture corresponds to a movement of the first user object determined by the controller based on the plurality of images of the first user object captured by the first and second imaging units.

19. The multi display apparatus of claim 16, wherein the plurality of displays are connected to one another by a flexible connector.

20. The multi display apparatus of claim 16, further comprising:

a sensor to determine an orientation of the plurality of displays relative to each other;

wherein the controller performs a command operation corresponding to the determined user gesture and the determined orientation of the plurality of displays.

21. The multi display apparatus of claim 16, wherein:

the first and second imaging units capture a plurality of images of a plurality of user objects; and

the controller determines the user gesture based on the plurality of images of the plurality of user objects captured by the first and second imaging units.

22. The multi display apparatus of claim 16, wherein the command operation comprises transferring content from a first one of the plurality of displays to a second one of the plurality of displays.

23. The multi display apparatus of claim 22, wherein the content comprises:

an application executable by the controller, the application being executed after it is transferred to the second one of the plurality of displays.

24. The multi display apparatus of claim 16, wherein at least one of the first and second recognition ranges may be adjusted independently of an orientation of the displays.

25. The multi display apparatus of claim 16, wherein at least one of the plurality of displays is a touch screen.

26. The multi display apparatus of claim 25, wherein the touch screen is configured to accept input from the first user object when the first user object approaches within a predetermined distance from the touch screen.

27. The multi display apparatus of claim 16, wherein the plurality of displays are connected wirelessly to each other and the controller.

28. A control method of a multi display apparatus having a plurality of displays, the method comprising:

capturing a plurality of images of a first user object within a first recognition range;

capturing a plurality of images of the first user object within a second recognition range;

determining a user gesture within a gesture recognition area where the first and second recognition ranges overlap, based on the captured images of the user object; and

performing a command operation corresponding to the determined user gesture.