Method and apparatus for providing service using a sensor and image recognition in a portable terminal

• US 20110071757A1
• Filed: 09/23/2010
• Published: 03/24/2011
• Est. Priority Date: 09/24/2009
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Jae-Myeon, Hong, Hyun-Su, Jeong, Mi-Jin
• Granted Patent: US 9,410,810 B2
• Time in Patent Office: Days
• US Class Current: 701/208
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/3492 employing speed data or traffic data, e.g. real-time or historical traffic control systems for road vehicles involving transmission of navigation instructions to the vehicle G08G1/0968
• CPC Class Codes: G01C 21/3623 using a camera or code reader, e.g. for optical or magnetic codes
• CPC Class Codes: G01C 21/3644 Landmark guidance, e.g. using POIs or conspicuous other objects
• CPC Class Codes: G01C 21/3679 Retrieval, searching and output of POI information, e.g. hotels, restaurants, shops, filling stations, parking facilities G01C21/3611 takes precedence
• CPC Class Codes: G03B 13/10 adjusting viewfinders field
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3245 of image modifying data, e.g. handwritten addenda, highlights or augmented reality information
• CPC Class Codes: H04N 2201/3252 Image capture parameters, e.g. resolution, illumination conditions, orientation of the image capture device H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3273 Display
• CPC Class Codes: H04N 2201/328 Processing of the additional information
• CPC Class Codes: H04N 23/61 : based on recognised objects
• CPC Class Codes: H04N 23/632 : for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/635 : Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

A method and an apparatus provides a service using a sensor and image recognition in a portable terminal that supports Augmented Reality (AR). The method includes determining whether the portable terminal is parallel to ground using an acceleration sensor. When the portable terminal is parallel to the ground, either a map including nearby Point Of Interest (POI) information or constellation information is displayed.

Claims

1. A method for providing a service using a sensor and image recognition in a portable terminal, comprising:

determining whether the portable terminal is parallel to the ground using an acceleration sensor; and

when the portable terminal is parallel to the ground, displaying at least one of a map comprising nearby Point Of Interest (POI) information and constellation information.

2. The method of claim 1, further comprising:

when the portable terminal is parallel to the ground, determining a screen direction of the portable terminal; and

when the screen of the portable terminal faces the sky, displaying the map comprising the nearby POI information; and

when the screen of the portable terminal faces the ground, displaying the constellation information,

wherein the map comprising the nearby POI information indicates an orientation of the portable terminal.

3. The method of claim 1, further comprising:

when the portable terminal is parallel to the ground, collecting current location information; and

obtaining one of the nearby POI information and constellation information corresponding to the current location information from one of a pre-stored database and a network.

4. A method for providing a service using a sensor and image recognition in a portable terminal, comprising:

conducting a preview by driving a camera;

determining, using an acceleration sensor, whether the portable terminal is moved backward or forward over a threshold distance; and

when the portable terminal is moved backward or forward over the threshold distance, adjusting a zoom of the camera by a first preset level according to the distance of movement.

5. The method of claim 4, further comprising:

determining, using the acceleration sensor, whether the portable terminal is bounced by a user; and

when the portable terminal is bounced, adjusting the zoom by a second preset level according to at least one of an amount of the bounce and a number of the bounces.

6. The method of claim 4, further comprising:

determining, using the acceleration sensor, whether the portable terminal is snapped; and

when the portable terminal is snapped, adjusting the zoom by a third preset level according to at least one of an intensity of the snap and a number of the snaps,

wherein the portable terminal is snapped with a wrist of a user.

7. A method for providing a service using a sensor and image recognition in a portable terminal, comprising:

conducting a preview by driving a camera;

obtaining current location information and displaying Point Of Interest (POI) information in a preview image;

when an image is captured, synthesizing a graphical object indicating the POI information over the captured image; and

tagging the synthesized image as an image of a POI corresponding to the POI information.

8. The method of claim 7, wherein synthesizing the graphical object comprises:

determining whether the captured image comprises a user'"'"'s face; and

when the user'"'"'s face is detected, synthesizing the graphical object indicating the POI information around the user'"'"'s face.

9. The method of claim 7, further comprising:

when an Internet search function is executed while the POI information is displayed, extracting at least one search keyword from the POI information; and

conducting the Internet search with at least one extracted search keyword.

10. The method of claim 7, wherein obtaining the current location information and displaying the POI information in the preview image comprises:

determining and displaying at least one of a travel distance and a travel time by considering an actual path leading to the POI corresponding to the POI information displayed in the preview image,

wherein the travel time is determined based on at least one of a user travel speed of the portable terminal and an average speed of a plurality of users over a certain time.

11. An apparatus for providing a service using a sensor and image recognition in a portable terminal, comprising:

an acceleration sensor configured to obtain an attitude angle of the portable terminal;

a display unit configured to display a user interface under control of the a controller; and

a controller configured to determine whether the portable terminal is parallel to the ground using the attitude angle, and when the portable terminal is parallel to the ground, control the display unit to display at least one of a map comprising nearby Point Of Interest (POI) information and constellation information.

12. The apparatus of claim 11, wherein the controller determines a screen direction of the portable terminal using the attitude angle when the portable terminal is parallel to the ground, controls the display unit to display the map comprising the nearby POI information when the screen of the portable terminal faces the sky, and controls the display unit to display the constellation information when the screen of the portable terminal faces the ground, and

the map comprising the nearby POI information indicates an orientation of the portable terminal.

13. The apparatus of claim 11, further comprising:

a Global Positioning System (GPS) receiver configured to collect current location information,

wherein the controller functions to obtains one of the nearby POI information and constellation information corresponding to the current location information from one of a pre-store database and a network.

14. An apparatus for providing a service using a sensor and image recognition in a portable terminal, comprising:

a camera configured to capture an image;

a display unit configured to provide a preview by displaying the image captured from the camera;

an acceleration sensor configured to measure movement of the portable terminal; and

a controller configured to determine, using the measurement of the acceleration sensor, whether the portable terminal is moved backward or forward over a threshold distance, and when the portable terminal is moved backward or forward over the threshold distance, adjusting a zoom of the camera by a first preset level according to the distance of movement.

15. The apparatus of claim 14, wherein the controller determines, using the acceleration sensor, whether the portable terminal is bounced by a user, and adjusts the zoom by a preset level according to at least one of an amount of the bounce and a number of the bounces.

16. The apparatus of claim 14, wherein the controller determines, using the acceleration sensor, whether the portable terminal is snapped, and adjusts the zoom by a preset level according to at least one of an intensity of the snap and a number of the snaps,

wherein the portable terminal is snapped with a wrist of a user.

17. An apparatus for providing a service using a sensor and image recognition in a portable terminal, comprising:

a camera configured to capture an image;

a display unit configured to provide a preview by displaying the image from the camera;

a Global Positioning System (GPS) receiver configured to collect current location information; and

a controller configured to obtain Point Of Interest (POI) information using the current location information, synthesize a graphical object indicating the POI information over the captured image when an image is captured, and tag the synthesized image as an image of a POI corresponding to the POI information.

18. The apparatus of claim 17, wherein the controller determines whether the captured image comprises a user'"'"'s face, and synthesizes the graphical object indicating the POI information around the user'"'"'s face when the user'"'"'s face is detected.

19. The apparatus of claim 17, further comprising:

a communication module configured to transmit and receive data,

wherein when an Internet search function is executed while the POI information is displayed, the controller extracts a at least one search keyword from the POI information and conducts the Internet search through the communication module using at least one extracted search keyword.

20. The apparatus of claim 17, wherein the controller controls to determine and display at least one of a travel distance and a travel time by considering an actual path leading to the POI corresponding to the POI information displayed in the preview image, and determines the travel time based on at least one of a user movement speed of the portable terminal and an average speed of a plurality of users over a certain time.