Method for guiding controller to move to within recognizable range of multimedia apparatus, the multimedia apparatus, and target tracking apparatus thereof

• US 9,838,573 B2
• Filed: 09/18/2013
• Issued: 12/05/2017
• Est. Priority Date: 09/18/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Choi, Jin-Wuk, Kwon, Dong-Wook, Kim, Jung-Seop, Kim, Kyung-Il, Kim, Min-Ho, Lee, Gi-Sang, Lee, Sang-Bo, Lee, Jin-Kyung, Jung, Sun-Tae, Jin, Young-Gu
• Primary Examiner(s): Patel, Jayanti K
• Assistant Examiner(s): Matt, Marnie
• Application Number:
  US14/030,340
• Publication Number:
  US 20140078311A1
• Time in Patent Office: 1,539 Days
• Field of Search: 348169
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44231 Monitoring of peripheral device or external card, e.g. to detect processing problems in a handheld device or the failure of an external recording device configuring of peripheral devices in general G06F9/4411; reporting information sensed by appliance or service execution status of appliance services in a home automation network H04L12/2823; monitoring the status of connected device in data switched networks H04L43/0817
• CPC Class Codes: H04N 5/2228 Video assist systems used in motion picture production, e.g. video cameras connected to viewfinders of motion picture cameras or related video signal processing

Abstract

Methods, systems, and devices for guiding a subject back within the recognizable visual range of a multimedia system are described. According to one of the described methods, when it is determined that the target has left the recognizable range of the multimedia system, sensor information is acquired from a portable electronic device (or controller) the user has been using to control the multimedia system, and the acquired sensor information is used to determine where the user is, relative to the recognizable range. In one example, the user is asked to make a gesture with the portable electronic device, and the sensor information concerning that gesture is used to determine the user'"'"'s relative location. In another example, the sensor information recorded at the time the user left the recognizable range is used to determine the user'"'"'s relative location.

Claims

1. A method for guiding a user of a controller of a multimedia apparatus to move within an angle-of-view range of a camera connected to the multimedia apparatus, comprising:

determining, using the camera, whether a target is within the angle-of-view range of the camera; and

if it is determined that the target is not within the angle-of-view range;

acquiring sensor information generated by a sensor included in the controller, the sensing information indicating movements of the controller corresponding to the target;

estimating the location of the controller corresponding to the target based on the sensor information; and

outputting, by the multimedia apparatus, guidance information based on the estimated location of the controller to guide the target to move within the angle-of-view range of the camera,

wherein the sensor information is generated outside the angle-of-view range of the camera,

wherein the multimedia apparatus is configured to determine a movement of the target within the angle-of-view range of the camera as an input of the multimedia apparatus using the camera.

2. The method of claim 1, wherein, if it is determined that the target is not within the angle-of-view range, further comprising:

outputting instruction information instructing the target to perform a gesture with the controller.

3. The method of claim 1, wherein determining whether the target is in the angle-of-view range of the camera comprises:

receiving an image through the camera;

recognizing the target from the received image; and

tracking the target.

4. The method of claim 1, further comprising, if no target is detected in the angle-of-view range of the camera, outputting guidance information for guiding the target to move toward the multimedia apparatus.

5. The method of claim 1, wherein the sensor information acquired from the controller comprises sensor information of changes in the controller'"'"'s position, orientation, and/or direction when the target makes a predetermined gesture in response to instruction information.

6. The method of claim 1, further comprising:

calculating an angle that a reference axis of the controller tilts with respect to a predetermined reference axis of the multimedia apparatus, based on the sensor information of the controller;

generating direction information to be displayed on a screen of the controller, based on the calculated tilting angle; and

transmitting the direction information to the controller.

7. The method of claim 1, wherein estimating the location of the controller comprises determining whether the controller is located to the right or left of the angle-of-view range of the camera, based on the sensor information of the controller.

8. The method of claim 7, wherein outputting guidance information corresponding to the estimated location of the controller comprises outputting information for guiding the target to move in a direction opposite to the direction in which the controller is outside of the angle-of-view range.

9. The method of claim 1, wherein outputting guidance information corresponding to the estimated location of the controller comprises outputting guidance information through at least one of a display unit, a speaker, and a vibration device of the multimedia apparatus.

10. The method of claim 1, wherein the sensor information is acquired from at least one of an acceleration sensor and a gyro sensor in the controller.

11. A multimedia apparatus for guiding a user of a controller of the multimedia apparatus to move within an angle-of-view range of a camera connected to the multimedia apparatus, comprising:

a display; and

a processor configured to;

determine, using the camera, whether a target is within the angle-of-view range of the camera,

acquire sensor information generated by a sensor included in the controller, the sensing information indicating movements of the controller corresponding to the target when no target is detected within the angle-of-view range of the camera,

estimate the location of the controller corresponding to the target using the sensor information of the controller, and

output, using the display, guidance information based on the estimated location of the controller to guide the target to move within the angle-of-view range of the camera,

wherein the sensor information is generated outside the angle-of-view range of the camera,

wherein the processor is further configured to determine a movement of the target within the angle-of-view range of the camera as an input of the multimedia apparatus using the camera.

12. The multimedia apparatus of claim 11, further comprising:

one or more output units configured to output instruction information when no target is detected within the angle-of-view range, the instruction information comprising instructions for the target to perform a predetermined gesture.

13. The multimedia apparatus of claim 11, further comprising:

an image input unit configured to receive one or more images acquired by the camera; and

a short-range communication unit configured to communicate with the controller, to receive sensor information from the controller, and to relay the sensor information from the controller to the control unit.

14. The multimedia apparatus of claim 13, wherein the image input unit and the short-range communication unit are installed in a set-top box connected to the multimedia apparatus in at least one of a wired and wireless manner.

15. The multimedia apparatus of claim 13, wherein if no target is detected within the angle-of-view range, the control unit outputs guidance information for guiding the user to move toward the multimedia apparatus, the guidance information being output through at least one of the display unit, a speaker, and a vibration device.

16. The multimedia apparatus of claim 11, wherein the sensor information from the controller comprises information indicating a change in at least one of position, orientation, and direction of the controller when the target makes a predetermined gesture in response to the guidance information.

17. The multimedia apparatus of claim 16, wherein the sensor information includes coordinates of the controller before the user makes the predetermined gesture, and the coordinates of the controller after the target makes the predetermined gesture.

18. The multimedia apparatus of claim 11, wherein the control unit calculates an angle at which the controller is tilting with respect to a predetermined reference axis of the multimedia apparatus, based on the sensor information, and estimates the relative location of the controller using the tilting angle of the controller.

19. The multimedia apparatus of claim 18, wherein the control unit determines whether the controller is located to the left or right side outside the angle-of-view range of the camera, based on the tilting angle of the controller.

20. The multimedia apparatus of claim 19, wherein the control unit outputs information for guiding the target to move in a direction opposite to the left or right side outside the angle-of-view range of the camera the controller is located in, and wherein the information is output through at least one of the display unit, a speaker, and a vibration device.

21. The multimedia apparatus of claim 11, wherein the controller comprises:

a sensor configured to acquire sensor information indicating a change in position, orientation, and/or direction of the controller;

a short-range communication unit configured to communicate with the multimedia apparatus; and

a control unit configured to calculate an angle at which a reference axis of the controller tilts with respect to a predetermined reference axis of the multimedia apparatus, based on the sensor information, to generate information about a direction in which the controller is to rotate in consideration of the calculated tilting angle, and to output the information about the direction for rotation.

22. A portable electronic device, comprising:

a short-range communication unit configured to transmit short-range communication signals;

at least one sensor configured to acquire information concerning at least one of orientation, movement, and location of the portable electronic device; and

at least one non-transitory computer-readable medium having program instructions recorded thereon, the program instructions configured to have the portable electronic device perform the steps of;

receiving, via the short-range communication unit, a request from a multimedia system for sensor information corresponding to a user of the portable electronic device when the user of the portable electronic device is outside an angle-of-view range of the multimedia system; and

transmitting, via the short-range communication unit, the requested sensor information to the multimedia system, the multimedia system subsequently using the received sensor information to determine which direction outside its angle-of-view visual range the user is in,

wherein the sensor information is generated outside the angle-of-view range of the multimedia system,

wherein a movement of the user is determined as an input of the multimedia system within the angle-of-view range of the multimedia system.

23. The portable electronic device of claim 22, wherein the program instructions recorded on the at least one non-transitory computer-readable medium are further configured to have the portable electronic device perform the step of:

acquiring, via the at least one sensor, information concerning change in orientation of the portable electronic device after the multimedia system instructs the user of the portable electronic device to perform an action with the portable electronic device.