Movable display apparatus, robot having movable display apparatus and display method thereof

• US 20100185990A1
• Filed: 12/22/2009
• Published: 07/22/2010
• Est. Priority Date: 01/20/2009
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Han, Woo Sup, Choi, Byung Kwon, Ha, Tae Sin
• Granted Patent: US 9,298,254 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

A movable display apparatus, a robot having the movable display apparatus, and a display method thereof, and, more particularly, a display method of a robot having an apparatus to display an image according to a visual point of a user are provided. It is possible to provide a convenient extended image service, by movably mounting the apparatus to display the image according to the visual point of the user and mounting the movable display apparatus in the robot so as to accurately display the image according to the visual point of the user using the mobility and motion of the robot. In addition, it is possible to provide an image, which is viewed like a three-dimensional image, via a two-dimensional display apparatus, by changing the displayed image according to a variation in the sightline of the user.

Claims

1. A movable display apparatus comprising:

a display unit to display an image;

a camera provided on the display unit to acquire an image of a user;

a head mounted device (HMD) to measure a head rotation direction and a pupil direction of the user;

a control unit to recognize a sightline of the user using the image of the user acquired by the camera and the head rotation direction and the pupil direction measured by the HMD and to control a posture of the display unit according to the recognized sightline of the user; and

a driving unit to move the display unit under the control of the control unit.

2. The movable display apparatus according to claim 1, further comprising a camera image processing unit to extract a face direction of the user from the user image acquired by the camera, wherein the control unit estimates the head rotation direction for recognizing the sightline of the user using the extracted face direction and the measured head rotation direction of the user.

3. The movable display apparatus according to claim 1, further comprising a camera image processing unit to extract the pupil direction of the user from the user image acquired by the camera, wherein the control unit estimates the pupil direction for recognizing the sightline of the user using the extracted pupil direction and the measured pupil direction of the user.

4. The movable display apparatus according to claim 2, wherein the control unit recognizes the sightline direction of the user using the estimated head rotation direction and pupil direction, controls the posture of the display unit according to the recognized sightline direction of the user, and provides the displayed image according to the sightline of the user.

5. The movable display apparatus according to claim 1, wherein the control unit decides the posture of the display unit such that the center of mass (COM), according to the motion of the display unit, is positioned on the center of the display unit.

6. A robot comprising:

a display unit movably provided on a manipulator to display an image;

a camera provided on the display unit to acquire an image of a user;

a head mounted device (HMD) to measure a head rotation direction and a pupil direction of the user;

a control unit to recognize a sightline of the user using the image of the user acquired by the camera and the head rotation direction and the pupil direction measured by the HMD and to control a posture of the display unit according to the recognized sightline of the user; and

a driving unit to move the manipulator and change the posture of the display unit under the control of the control unit.

7. The robot according to claim 6, further comprising a camera image processing unit to extract a face direction of the user from the image of the user acquired by the camera, wherein the control unit estimates the head rotation direction for recognizing the sightline of the user using the extracted face direction and the measured head rotation direction of the user.

8. The robot according to claim 6, further comprising a camera image processing unit to extract the pupil direction of the user from the image of the user acquired by the camera, wherein the control unit estimates the pupil direction for recognizing the sightline of the user using the extracted pupil direction and the measured pupil direction of the user.

9. The robot according to claim 7, wherein the control unit recognizes the sightline direction of the user using the estimated head rotation direction and pupil direction, computes a difference between the sightline position of the user and the current position of the display unit, and generates a motion path of the manipulator.

10. The robot according to claim 9, wherein the control unit checks a variation in the center of mass (COM) when the display unit is positioned at the sightline position of the user, and moves a main body of the robot.

11. The robot according to claim 6, wherein the control unit changes the image displayed on the display unit according to a variation in the sightline of the user.

12. A display method of a robot, the method comprising:

providing an image service using a display unit movably provided on a manipulator;

acquiring an image of a user who receives the image service;

measuring a head rotation direction and a pupil direction of the user; and

recognizing a sightline of the user using the acquired image of the user and the measured head rotation direction and pupil direction of the user and controlling a posture of the display unit according to the recognized sightline of the user to track the sightline of the user.

13. The method according to claim 12, further comprising extracting a face direction of the user from the acquired image of the user, wherein the recognizing of the sightline of the user includes estimating the head rotation direction of the user using the extracted face direction and the measured head rotation direction of the user.

14. The method according to claim 12, further comprising extracting the pupil direction of the user from the acquired image of the user, wherein the recognizing of the sightline of the user includes estimating the pupil direction of the user using the extracted pupil direction and the measured pupil direction of the user.

15. The method according to claim 13, wherein the recognizing of the sightline of the user includes recognizing a sightline position of the user using the head rotation direction and the pupil direction.

16. The method according to claim 15, wherein the tracking of the sightline of the user includes computing a difference between the sightline position of the user and the current position of the display unit and generating a motion path of the manipulator.

17. The method according to claim 15, wherein the tracking of the sightline of the user includes checking a variation in the center of mass (COM) of the robot when the display unit is positioned at the sightline position of the user, and moving a main body of the robot.

18. The method according to claim 12, wherein the tracking of the sightline of the user includes changing the image displayed on the display unit according to a variation in the sightline of the user.