Method and apparatus for providing service using a sensor and image recognition in a portable terminal

• US 9,915,544 B2
• Filed: 08/08/2016
• Issued: 03/13/2018
• Est. Priority Date: 09/24/2009
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Jae-Myeon, Hong, Hyun-Su, Jeong, Mi-Jin
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US15/231,632
• Publication Number:
  US 20160349071A1
• Time in Patent Office: 582 Days
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/3492 employing speed data or traffic data, e.g. real-time or historical traffic control systems for road vehicles involving transmission of navigation instructions to the vehicle G08G1/0968
• CPC Class Codes: G01C 21/3623 using a camera or code reader, e.g. for optical or magnetic codes
• CPC Class Codes: G01C 21/3644 Landmark guidance, e.g. using POIs or conspicuous other objects
• CPC Class Codes: G01C 21/3679 Retrieval, searching and output of POI information, e.g. hotels, restaurants, shops, filling stations, parking facilities G01C21/3611 takes precedence
• CPC Class Codes: G03B 13/10 adjusting viewfinders field
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3245 of image modifying data, e.g. handwritten addenda, highlights or augmented reality information
• CPC Class Codes: H04N 2201/3252 Image capture parameters, e.g. resolution, illumination conditions, orientation of the image capture device H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3273 Display
• CPC Class Codes: H04N 2201/328 Processing of the additional information
• CPC Class Codes: H04N 23/61 : based on recognised objects
• CPC Class Codes: H04N 23/632 : for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/635 : Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

A method and an apparatus provides a service using a sensor and image recognition in a portable terminal that supports Augmented Reality (AR). The method includes determining whether the portable terminal is parallel to ground using an acceleration sensor. When the portable terminal is parallel to the ground, either a map including nearby Point Of Interest (POI) information or constellation information is displayed.

Claims

1. An electronic device comprising:

a camera;

a display;

a storage storing instructions; and

a processor configured to execute the instructions to at least;

control to obtain real-time images from the camera,

control to identify an object within the obtained real-time images,

control to display a graphical object related to the identified object on the display while obtaining the real-time images,

in response to receiving an input corresponding to a capture event while displaying the graphical object on the display, control to generate an image by combining the graphical object and one of the obtained real-time images,

control to store the generated image in the storage, and

control to display the generated image stored in the storage in response to executing an application related to the generated image including the graphical object.

2. The electronic device of claim 1, wherein the graphical object comprises at least one of name information, location information, category information, phone number information, or related image information of the identified object.

3. The electronic device of claim 1, wherein the processor is configured to execute the instructions further to control to obtain information regarding the identified object from an external electronic device.

4. A method performed by an electronic device, the method comprising:

obtaining real-time images from a camera of the electronic device;

identifying an object within the obtained real-time images;

displaying a graphical object related to the identified object on a display of the electronic device while obtaining the real-time images;

in response to receiving an input corresponding to a capture event while displaying the graphical object on the display, generating an image by combining the graphical object and one of the obtained real-time images;

storing the generated image in a storage of the electronic device; and

in response to executing an application related to the generated image including the graphical object, displaying the generated image stored in the storage.

5. The method of claim 4, wherein the graphical object comprises at least one of name information, location information, category information, phone number information, or related image information of the identified object.

6. The method of claim 4, further comprising:

obtaining information regarding the identified object from an external electronic device.

7. A method performed by an electronic device, the method comprising:

controlling to obtain real-time images from a camera of the electronic device;

controlling to identify an object within the obtained real-time images;

controlling to display a graphical object related to the identified object on a display of the electronic device while obtaining the real-time images;

in response to receiving an input corresponding to a capture event while displaying the graphical object on the display, controlling to generate an image by combining the graphical object and one of the obtained real-time images;

controlling to store the generated image in a storage of the electronic device; and

controlling to display the generated image stored in the storage in response to executing an application related to the generated image including the graphical object.

8. The method of claim 7, wherein the graphical object comprises at least one of name information, location information, category information, phone number information, or related image information of the identified object.

9. The method of claim 7, further comprising:

controlling to obtain information regarding the identified object from an external electronic device.

10. A non-transitory computer readable recording medium storing instructions executable by a processor of an electronic device to cause the processor to at least:

control to obtain real-time images from a camera of the electronic device;

control to identify an object within the obtained real-time images;

control to display a graphical object related to the identified object on a display of the electronic device while obtaining the real-time images;

in response to receiving an input corresponding to a capture event while displaying the graphical object on the display, control to generate an image by combining the graphical object and one of the obtained real-time images;

control to store the generated image in the storage of the electronic device; and

control to display the generated image included in the storage in response to executing an application related to the generated image including the graphical object.

11. The non-transitory computer readable recording medium of claim 10, wherein the graphical object comprises at least one of name information, location information, category information, phone number information, or related image information of the identified object.

12. The non-transitory computer readable recording medium of claim 10, wherein the instructions cause the processor further to:

control to obtain information regarding the identified object from an external electronic device.