Display system and method using hybrid user tracking sensor

• US 9,171,371 B2
• Filed: 06/13/2011
• Issued: 10/27/2015
• Est. Priority Date: 08/12/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Sung, Gee Young, Nam, Dong Kyung, Kim, Yun Tae, Park, Ju Yong
• Primary Examiner(s): CHEN, CHIA WEI A
• Application Number:
  US13/067,600
• Publication Number:
  US 20120038627A1
• Time in Patent Office: 1,597 Days
• Field of Search: 345/633, 345/659, 345/619, 348/222.1, 348/158, 348/148, 348/77, 34833301-33313, 382/103, 455/556.1
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Example embodiments of a display system and method using a hybrid user tracking sensor are described. The display system may determine a final location of a user based on information of a face or eyes of a user generated based on information received from a plurality of cameras and posture information generated based on information received from a plurality of sensors.

Claims

1. A display system comprising:

a processor to control one or more processor-executable units;

a location information provider unit to provide location information of a face or eyes of a user generated based on information received from a plurality of cameras;

a posture information provider unit to provide posture information of the display system generated based on information received from a plurality of sensors; and

a final location determining unit to determine a final location of the user with respect to the display system based on the location information of the face or the eyes of the user and the posture information of the display system,

wherein the final location determining unit searches for a location of the face or the eyes of the user within a predetermined range within an image from the plurality of cameras,

wherein the predetermined range is determined based on X and Y coordinates of the user on the image with respect to the display system,

wherein each sensor type differs among the plurality of sensors, and

wherein the final location determining unit comprises;

a relative location calculator to calculate the X and Y coordinates based on the posture information in response to a posture of the display system being changed.

2. The display system of claim 1, wherein:

the plurality of sensors comprises;

an acceleration sensor and a terrestrial magnetism sensor, and

the posture information provider unit comprises;

a posture angle calculator to calculate a posture angle with respect to the direction of gravity based on information received from the acceleration sensor;

an azimuth angle calculator to calculate an azimuth angle based on information received from the terrestrial magnetism sensor; and

a provider to provide posture information comprising at least the posture angle and the azimuth angle.

3. The display system of claim 2, wherein:

the posture angle comprises a pitch and a roll, and

the azimuth angle comprises a yaw.

4. The display system of claim 1, wherein:

the plurality of cameras comprises;

a stereo camera comprising two cameras, and

the location information provider unit comprises;

a location information generator to generate the location information corresponding to three dimensional (3D) location information of the user, using a stereo image received from the stereo camera.

5. The display system of claim 1, wherein the final location determining unit comprises:

a posture change determining unit to determine whether the posture of the display system has changed based on the posture information; and

a determining unit to determine the final location based on whether the posture has changed.

6. The display system of claim 5, wherein the determining unit comprises:

a first determining unit to determine a location based on the location information as the final location when the posture remains unchanged; and

a second determining unit to determine the final location based on the X and Y coordinates when the posture has changed.

7. The display system of claim 6, wherein the second determining unit searches for the location of the face or the eyes of the user in an image within the predetermined range determined according to the X and Y coordinates, among images received from the plurality of cameras, and determines the found location as the final location.

8. The display system of claim 1, further comprising:

a display unit to generate a multiview image based on the final location.

9. The display system of claim 8, wherein the display unit changes and displays the multiview image based on the final location through a subpixel rendering using a lenticular lens.

10. The display system of claim 8, wherein the display unit changes and displays the multiview image based on the final location through a subpixel rendering using a cycle and a width of a variable parallax barrier, and the variable parallax barrier.

11. A display method comprising:

providing location information of a face or eyes of a user generated based on information received from a plurality of cameras;

providing posture information of a display apparatus generated based on information received from a plurality of sensors; and

determining, by way of a processor, a final location of the user with respect to the display system based on the location information of the face or the eyes of the user and the posture information of the display apparatus,

wherein the determining the final location comprises searching for a location of the face or the eyes of the user within a predetermined range within an image from the plurality of cameras,

wherein the predetermined range is determined based on X and Y coordinates of the user on the image with respect to the display apparatus,

wherein each sensor type differs among the plurality of sensors, and

wherein the determining comprises;

calculating the X and Y coordinates based on the posture information in response to a posture of the display system being changed.

12. The display method of claim 11, wherein:

the plurality of sensors comprises;

an acceleration sensor and a terrestrial magnetism sensor, and

the providing posture information comprises;

calculating a posture angle with respect to the direction of gravity based on information received from the acceleration sensor;

calculating an azimuth angle based on information received from the terrestrial magnetism sensor; and

providing posture information comprising at least the posture angle and the azimuth angle.

13. The display method of claim 11, wherein:

the plurality of cameras comprises;

a stereo camera comprising two cameras, and

the providing location information comprises;

generating the location information corresponding to three dimensional (3D) location information of the user, using a stereo image received from the stereo camera.

14. The display method of claim 11, wherein the determining comprises:

determining whether the posture of the display system has changed based on the posture information; and

determining the final location based on whether the posture changed.

15. The display method of claim 14, wherein the determining comprises:

determining a location based on the location information as the final location when the posture remains unchanged; and

determining the final location based on the X and Y coordinates when the posture has changed.

16. The display method of claim 15, wherein the determining the final location based on the X and Y coordinates comprises searching for the location of the face or the eyes of the user in an image within the predetermined range determined according to the X and Y coordinates, among images received from the plurality of cameras, and determining the found location as the final location.

17. The display method of claim 11, further comprising:

generating a multiview image based on the final location.

18. A non-transitory computer-readable medium comprising a program for instructing a computer to perform the method of claim 11.

19. A display apparatus comprising:

a processor to control one or more processor-executable units;

a location information provider unit to obtain information about a location of at least one eye of a user;

a posture information provider unit to sense a posture of the display apparatus based on one or more sensors; and

a final location determining unit to determine a final location of the user with respect to the display apparatus based on the location information provided by the location information provider and the posture information of the display apparatus provided by the posture information provider,

wherein the final location determining unit searches for a location of the face or the eyes of the user within a predetermined range within an image from the plurality of cameras,

wherein the predetermined range is determined based on a relative location of the user on the image with respect to the display apparatus,

wherein each sensor type differs among the one or more sensors, and

wherein the final location determining unit comprises;

a relative location calculator to calculate X and Y coordinates based on the posture information in response to a posture of the display system being changed.

20. The display apparatus of claim 19, further comprising:

a display adjustment unit to adjust a display of the display apparatus according to the final location information determined by the final location determining unit.

21. The display apparatus of claim 20, wherein the adjusting of the display by the display adjustment unit comprises adjusting a multiview image based on the final location information using a subpixel rendering.

22. The display apparatus of claim 19, wherein the posture information provider unit comprises one or more of an acceleration sensor and a terrestrial magnetism sensor.

23. The display apparatus of claim 19, wherein the location information provider unit comprises a stereo camera to obtain the information about the location of the at least one eye of the user.

24. A display method of a display apparatus, the method comprising:

sensing information about a location of at least one eye of a user;

sensing a posture of the display apparatus based on one or more sensors; and

determining, by way of a processor, a final location of the at least one eye of the user based on the location information of the at least one eye of the user and the posture information of the display apparatus,

wherein the determining the final location comprises searching for a location of the face or the eyes of the user within a predetermined range within an image from the plurality of cameras,

wherein the predetermined range is determined based on a relative location of the user on the image with respect to the display apparatus,

wherein each sensor type differs among the one or more sensors, and

wherein the determining comprises;

calculating X and Y coordinates based on the posture information in response to a posture of the display system being changed.

25. The display method of claim 24, further comprising adjusting a display of the display apparatus according to the final location information.

26. A non-transitory computer-readable medium comprising a program for instructing a computer to perform the method of claim 24.