Image display apparatus and pointing method for same

• US 9,910,507 B2
• Filed: 10/01/2014
• Issued: 03/06/2018
• Est. Priority Date: 10/02/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Okuno, Yoshiaki, Hirasawa, Kousuke, Yamamoto Murakami, Edwardo Arata, Choi, Sang-on
• Primary Examiner(s): Marinelli, Patrick F
• Application Number:
  US15/027,108
• Publication Number:
  US 20160239096A1
• Time in Patent Office: 1,252 Days
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0308 comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device
• CPC Class Codes: G08C 17/00 Arrangements for transmitting signals characterised by the use of a wireless electrical link
• CPC Class Codes: G08C 2201/32 Remote control based on movements, attitude of remote control device
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

An image display apparatus that automatically adjust a pointing direction of a remote controller to almost match with a position of a pointer on a screen, and a pointing method performed by the image display apparatus are provided. The image display apparatus includes a reference angle detector configured to detect a reference angle formed by the pointing direction of the remote controller with respect to a reference direction set for a display; a remote control direction detector configured to detect a remote control direction which is a direction in which the remote controller is viewed, from a reference point set on the display; and a standard direction adjuster configured to adjust a standard direction set in the remote controller so that the standard direction reaches the reference point on the display as viewed from the remote controller, based on the reference angle and the remote control direction.

Claims

1. An image display apparatus comprising:

a reference angle detector configured to detect a reference angle formed by a pointing direction of a remote controller with respect to a reference direction set for a display;

a remote control direction detector configured to detect a remote control direction which is a direction in which the remote controller is viewed, from a reference point set on the display;

a standard direction adjuster configured to adjust a standard direction set in the remote controller so that the standard direction reaches the reference point set on the display as viewed from the remote controller, based on the reference angle and the remote control direction; and

a controller configured to control a pointer to be displayed at a corresponding position on the display, based on a pointing angle that is formed by the pointing direction of the remote controller with respect to the standard direction.

2. The image display apparatus of claim 1, wherein the reference angle detector comprises a reference angle calculator configured to calculate the reference angle based on a reference direction set as a direction value output by a direction sensor provided in the remote controller when the pointing direction of the remote controller is perpendicular to the display and based on a remote control direction set as a direction value output by the direction sensor in correspondence with the pointing direction of the remote controller.

3. The image display apparatus of claim 1, further comprising a pointing angle calculator configured to calculate the pointing angle based on a reference pointing angle and an angular velocity output by an angular velocity sensor provided in the remote controller,

wherein the standard direction adjuster sets an angle formed by the pointing direction with respect to the standard direction, as the reference pointing angle, in the pointing angle calculator.

4. The image display apparatus of claim 3, wherein the pointing angle calculator stores a previous remote control direction corresponding to when the standard direction was adjusted in the past, and calculates the pointing angle when a difference between the previous remote control direction and a current remote control direction is equal to or greater than a threshold value.

5. The image display apparatus of claim 1, wherein the remote control direction detector comprises:

a camera disposed on the display to be able to photograph the remote controller; and

a remote control direction angle calculator configured to calculate a remote control direction angle formed by the remote control direction with respect to a direction perpendicular to the display, based on an image captured by the camera.

6. The image display apparatus of claim 5, wherein

the camera is disposed on a horizontal edge of the display to have a photographing direction parallel to a vertical plane perpendicular to the display,

the remote control direction angle calculator comprises:

a first remote control detector configured to calculate a first remote control direction angle made by the direction perpendicular to the display and a direction obtained by projecting the remote control direction to a horizontal plane, based on the image captured by the camera; and

a second remote control detector configured to calculate a second remote control direction angle made by the direction perpendicular to the display and a direction obtained by projecting the remote control direction to the vertical plane, based on the image captured by the camera, a photographing angle made by the display and the photographing direction, a distance from the camera to the remote controller, and a length of a vertical edge of the display, and

the standard direction adjuster adjusts the standard direction so that the standard direction reaches the reference point on the display as viewed from the remote controller, based on the reference angle, the first remote control direction angle, and the second remote control direction angle.

7. The image display apparatus of claim 1, wherein

the controller comprises:

a position calculator configured to calculate a corresponding position on the display, based on the pointing angle; and

a display controller configured to control the pointer to be displayed on the corresponding position, and

the position calculator calculates the corresponding position so that a distance from the reference point to the corresponding position for each unit angle of the pointing angle increases with an increase in a distance from the display to the remote controller.

15. The image display apparatus of claim 1, wherein the remote controller comprising:

a light-emission device having at least one light source;

a direction sensor configured to sense a remote control direction value representing a direction in which the remote controller points;

a manipulator configured to manipulate pointing to the image display apparatus; and

a controller configured to turn on the at least one light source of the light-emission device according to a pointing manipulation of the manipulator, modulate the remote control direction value sensed by the direction sensor into a remote control code, and output the remote control code to the image display apparatus via the light-emission device,

wherein the remote control direction value transmitted to the image display apparatus is used by the image display apparatus to determine a pointing position on a display of the image display apparatus.

16. The image display apparatus of claim 1, wherein the remote controller is configured to:

turning on at least one light source of a light-emission device according to a pointing manipulation input and sensing a remote control direction value representing a direction in which the remote controller points;

transforming the sensed remote control direction value into a remote control code; and

outputting the remote control code to the image display apparatus via the light-emission device,

wherein the remote control direction value transmitted to the image display apparatus is used by the image display apparatus to determine a pointing position on a display of the image display apparatus.

8. A pointing method comprising:

detecting a reference angle formed by a pointing direction of a remote controller with respect to a reference direction set for a display;

detecting a remote control direction which is a direction in which the remote controller is viewed, from a reference point set on the display;

adjusting a standard direction set in the remote controller so that the standard direction reaches the reference point set on the display as viewed from the remote controller, based on the reference angle and the remote control direction; and

controlling a pointer to be displayed at a corresponding position on the display, based on a pointing angle that is formed by the pointing direction of the remote controller with respect to the standard direction.

9. The pointing method of claim 8, wherein the detecting of the reference angle comprises calculating the reference angle based on a reference direction set as a direction output by a direction sensor provided in the remote controller when the pointing direction of the remote controller is perpendicular to the display and based on a remote control direction set as a direction output by the direction sensor in correspondence with the pointing direction of the remote controller.

10. The pointing method of claim 8, further comprising calculating the pointing angle based on a reference pointing angle and an angular velocity output by an angular velocity sensor provided in the remote controller,

wherein an angle formed by the pointing direction with respect to the standard direction is set as the reference pointing angle.

11. The pointing method of claim 8, wherein the detecting of the remote control direction comprises calculating a remote control direction angle formed by the remote control direction with respect to a direction perpendicular to the display, based on an image captured by a camera mounted on the display to be able to photograph the remote controller.

12. The pointing method of claim 11, wherein

the camera is mounted on an edge of the display parallel to one of a horizontal plane and a vertical plane perpendicular to the display, to have a photographing direction parallel to a surface other than one of the horizontal plane and the vertical plane,

the calculating of the remote control direction angle comprises:

calculating a first remote control direction angle made by the direction perpendicular to the display and a direction obtained by projecting the remote control direction to the one plane, based on the image captured by the camera; and

calculating a second remote control direction angle made by the direction perpendicular to the display and a direction obtained by projecting the remote control direction to the other plane, based on the image captured by the camera, a photographing angle made by the display and the photographing direction, a distance from the camera to the remote controller, and a length of an edge of the display that is parallel to the other plane, and

the adjusting of the standard direction comprises adjusting the standard direction so that the standard direction reaches the reference point on the display as viewed from the remote controller, based on the reference angle, the first remote control direction angle, and the second remote control direction angle.

13. The pointing method of claim 8, wherein the adjusting of the standard direction comprises adjusting the standard direction at regular intervals.

14. The pointing method of claim 8, wherein

the controlling comprises:

calculating a corresponding position on the display, based on the pointing angle; and

controlling the pointer to be displayed on the corresponding position, and

the calculating of the corresponding position comprises calculating the corresponding position so that a distance from the reference point to the corresponding position for each unit angle of the pointing angle increases with an increase in a distance from the display to the remote controller.