Apparatus and method for executing a menu in a wireless terminal

• US 10,048,782 B2
• Filed: 10/28/2009
• Issued: 08/14/2018
• Est. Priority Date: 10/28/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Se-Hwan
• Primary Examiner(s): Rayan, Mihir K
• Application Number:
  US12/607,651
• Publication Number:
  US 20100103105A1
• Time in Patent Office: 3,212 Days
• CPC Class Codes: G06F 3/0386 for light pen
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42206 characterized by hardware details
• CPC Class Codes: H04N 21/42209 for displaying non-command information, e.g. electronic program guide [EPG], e-mail, messages or a second television channel
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42221 Transmission circuitry, e.g. infrared [IR] or radio frequency [RF]
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43637 involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11] arrangements for wireless networking or broadcasting of information in indoor or near-field type systems H04B10/114
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

A menu execution apparatus and method for conveniently providing a menu in a wireless terminal are provided. The apparatus includes a pointer unit for indicating a specific object in a menu execution recognition mode. A camera photographs the object indicated by the pointer unit in the menu execution recognition mode. A controller controls an operation for recognizing the object indicated by the pointer unit and photographed by the camera and displaying a menu for controlling the recognized object.

Claims

1. A wireless terminal for executing a menu, comprising:

a touch screen;

a light emitting pointer unit for emitting light for pointing at an external object;

a camera configured to photograph an external object;

a sensor configured to sense a motion of the wireless terminal; and

a controller configured to at least;

control an operation for simultaneously operating the light emitting pointer unit and the camera upon sensing a predetermined input of one of an input unit and the touch screen,

control the camera to photograph an external object pointed at by the light emitted from the light emitting pointer unit, in response to determining a motion of the wireless terminal is absent during a given time,

control to recognize the photographed external object pointed at by the light emitted from the light emitting pointer unit, and

control to display a menu on the touch screen for controlling the recognized external object.

2. The wireless terminal of claim 1, wherein the light emitting pointer unit and the camera are mounted at a same position.

3. The wireless terminal of claim 1, wherein the controller is further configured to:

control an operation for comparing a first image, captured by photographing the external object pointed at by the light emitted from the light emitting pointer unit using the camera, to a second image captured by photographing the external object after the light emitting pointer unit is terminated, and

extract values of pixels hidden by the light emitted from the light emitting pointer unit displayed on the first image from the second image.

4. The wireless terminal of claim 3, wherein the controller is further configured to:

control an operation for setting a minimal unit area around the light emitted from the light emitting pointer unit displayed on the first image, and

determine a recognition type through an image included in the minimal unit area.

5. The wireless terminal of claim 4, wherein the recognition type is determined by applying a priority for recognition.

6. The wireless terminal of claim 4, wherein the controller is further configured to:

control an operation for recognizing letters by extending the minimal unit area in a given area unit when the recognition type is letters and displaying a menu for controlling the letters, and

control an operation for recognizing an external device corresponding to a shape by extending the minimal unit area in the given area unit when the recognition type is the shape and displaying a menu for controlling the external device.

7. The wireless terminal of claim 1, wherein the light emitting pointer unit points at the external object in a menu execution recognition mode, and

wherein the menu execution recognition mode starts when the predetermined input of the input unit or the touch screen is sensed.

8. The wireless terminal of claim 7, wherein the controller controls to notify a user of a start of the menu execution recognition mode by generating at least one of a start sound and a vibration.

9. The wireless terminal of claim 1, wherein the controller is further configured to:

control an external device, if the recognized external object is the external device.

10. A method for executing a menu in a wireless terminal including a light emitting pointer unit and a camera, the method comprising:

simultaneously operating the light emitting pointer unit and the camera upon sensing a predetermined input of one of an input unit and a touch screen;

photographing, by the camera, an external object pointed at by the light emitted from the light emitting pointer unit, in response to determining a motion of the wireless terminal is absent during a given time;

recognizing the photographed external object pointed at by the light emitted from the light emitting pointer unit; and

displaying a menu for controlling the recognized external object.

11. The method of claim 10, wherein displaying the menu comprises:

storing a first image captured by photographing the external object pointed at by the light emitted from the light emitting pointer unit;

terminating an operation of the light emitting pointer unit, after the first image is captured;

storing a second image captured by photographing the external object previously pointed at by the light emitted from the light emitting pointer unit, after the light emitting pointer unit is terminated;

extracting pixel values of a portion of the external object hidden by the light emitted from the light emitting pointer unit displayed on the first image from the second image by comparing the first image to the second image;

extracting an image included in an area set around the light emitted from the light emitting pointer unit displayed on the first image; and

displaying a menu for controlling the extracted image.

12. The method of claim 11, wherein displaying the menu further comprises:

setting a minimal unit area around the light emitted from the light emitting pointer unit displayed on the first image;

extracting an image included in a minimal unit area and determining a recognition type through the extracted image;

extracting letters while extending the minimal unit area in a given area unit when the recognition type is the letters, and displaying a menu for controlling the extracted letters; and

determining an external device corresponding to a shape while extending the minimal unit area in the given area unit when the recognition type is the shape, and displaying a menu for controlling the external device.

13. The method of claim 12, wherein the recognition type is determined by applying a priority for recognition.

14. The method of claim 10, wherein the light emitting pointer unit points at the external object in a menu execution recognition mode, and

wherein the menu execution recognition mode starts when the predetermined input of the input unit or the touch screen is sensed.

15. The method of claim 14, further comprising:

notifying a user of a start of the menu execution recognition mode by generating at least one of a start sound and a vibration.

16. The method of claim 10, further comprising:

if the recognized object is an external device, controlling the external device.

17. A method for executing a menu in a wireless terminal, comprising:

simultaneously operating a light emitting pointer unit and a camera, when the wireless terminal switches to a menu execution recognition mode;

capturing, by the camera, a first image including an external object pointed at by light emitted from the light emitting pointer unit;

determining whether a plurality of points are designated by the light emitted from the light emitting pointer unit on captured image;

extracting, from the captured image, pixel values corresponding to the plurality of points, when the plurality of points are designated;

extracting an area having the extracted pixel values as boundary values from the captured image; and

displaying a menu for controlling a second image included in the area.