Providing navigation information to a point of interest on real-time street views using a mobile device

• US 9,699,373 B2
• Filed: 04/28/2013
• Issued: 07/04/2017
• Est. Priority Date: 04/28/2013
• Status: Active Grant

• Current Assignee: Tencent Technology Shenzhen Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Inventors: Kuang, Junbin
• Primary Examiner(s): Daniels, Anthony J
• Application Number:
  US13/989,735
• Publication Number:
  US 20140320674A1
• Time in Patent Office: 1,528 Days
• Field of Search: 348113, 348119
• CPC Class Codes: H04N 1/32144 embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
• CPC Class Codes: H04N 2201/3245 of image modifying data, e.g. handwritten addenda, highlights or augmented reality information
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

Method of and systems for providing point of interest (POI) information on a mobile device is described. The method of providing point of interest (POI) information on a mobile device including a non-transitory computer readable medium including a program, a processor, and a computer executable program code includes: initiating capture of a real-time image by a camera module of a mobile device; determining location information of the mobile device; sending the location information to a server; receiving POI information from the server, wherein the POI information relates to at least one object near the mobile device and currently appearing in the real-time image; and displaying the POI information, superimposed on the real-time image, on the mobile device.

Claims

1. A method of providing point of interest (POI) information on a mobile device comprising a non-transitory computer readable medium including a program, a processor, and a computer executable program code, the method comprising:

initiating capture of a real-time image of at least one object by a camera module of a mobile device;

receiving user-provided destination information and sending the user-provided destination information and current location of the mobile device to a server, wherein the server is configured to request the mobile device to provide additional navigation information after finding the user-provided destination information within a predefined distance from the current location;

in response to the request from the server, determining updated current location, speed, orientation, rotation, and inclination information of the mobile device;

sending the updated current location, speed, orientation, rotation, and inclination information of the mobile device to the server without sending the real-time image, wherein the server is configured to search for POI information associated with the user-provided destination information in accordance with the updated current location, speed, orientation, rotation, and inclination information of the mobile device and return the POI information to the mobile device;

receiving the POI information from the server, wherein the POI information includes label information associated with the at least one object within the predefined distance from the mobile device and navigation information from the updated current location of the mobile device to the destination, and wherein the POI information is determined in accordance with the updated current location, speed, orientation, rotation, inclination information of the mobile device, and the destination information without reliance upon the real-time image captured by the camera; and

displaying the POI information, superimposed on the real-time image of the at least one object, on the mobile device, further including displaying the label information on the real-time image next to the at least one object and displaying a directional indicator on the real-time image indicating a navigation path from the updated current location of the mobile device to the destination based on the navigation information.

2. The method of claim 1, further comprising:

determining the updated current location, speed, orientation, rotation, and inclination information of the mobile device;

sending the updated current location, speed, orientation, rotation, and inclination information to the server;

receiving updated POI information from the server, wherein the updated POI information is at least in part based on the updated current location, speed, orientation, rotation, inclination information of the mobile device, and the destination information; and

updating the previously-superimposed POI information with the updated POI information on the mobile device such that the label information appears attached to the at least one object and the navigation information moves in a direction towards the destination in the real-time image.

3. The method of claim 1, further includes:

determining the updated current location, speed, orientation, rotation, and inclination information of the mobile device;

updating the POI information according to the updated current location, speed, orientation, rotation, inclination information, and the destination information; and

updating the previously-superimposed POI information with the updated POI information on the mobile device such that the label information appears attached to the at least one object and the navigation information moves in a direction towards the destination in the real-time image.

4. The method of claim 1, further comprising:

detecting a user selection of the label information;

in response to the user selection of the label information, sending a query to the server, the query including an identity of the at least one object;

receiving additional information related to the at least one object from the server; and

in response to the additional information, activating an interactive application on the mobile device.

5. A method for providing navigation information to a mobile device, the method comprising:

at a server having a processor and memory for storing programs to be executed by the processor;

receiving current location and user-provided destination information from the mobile device after the mobile device initiates capture of a real-time image of at least one object by a camera module of the mobile device;

requesting the mobile device to provide additional navigation information after finding the user-provided destination information within a predefined distance from the current location;

receiving updated current location, speed, orientation, rotation, and inclination information of the mobile device from the mobile device without receiving the real-time image;

identifying POI information associated with the user-provided destination information in accordance with the updated current location, speed, orientation, rotation, and inclination information of the mobile device without reliance upon the real-time image captured by the camera, wherein the POI information includes label information associated with at least one object near the mobile device and navigation information from the updated current location of the mobile device to the destination; and

returning the POI information to be superimposed on the real-time image of the at least one object captured by the camera of the mobile device, wherein on the mobile device, the label information is displayed on the real-time image next to the at least one object and a directional indicator is displayed on the real-time image indicating a navigation path from the updated current location of the mobile device to the destination based on the navigation information.

6. The method of claim 5, further comprising:

receiving the updated current location, speed, orientation, rotation, and inclination information from the mobile device, wherein the updated current location, speed, orientation, rotation, and inclination information is generated by respective sensors in the mobile device;

determining updated POI information based on the updated current location, speed, orientation, rotation, inclination information of the mobile device, and the destination information; and

returning the updated POI information to the mobile device such that the label information appears attached to the at least one object and the navigation information moves in a direction towards the destination in the real-time image.

7. The method of claim 5, further comprising:

receiving a query from the mobile device, wherein the query was generated by a user selection of the label information;

in response to the query, generating a response including additional information related to the at least one object; and

returning the response to the mobile device, wherein the response includes instructions for activating an interactive application on the mobile device.

8. A server, comprising:

a processor;

memory; and

a plurality of programs stored in the memory, the programs including instructions for;

receiving current location and user-provided destination information from the mobile device after the mobile device initiates capture of a real-time image of at least one object by a camera module of the mobile device;

requesting the mobile device to provide additional navigation information after finding the user-provided destination information within a predefined distance from the current location;

receiving updated current location, speed, orientation, rotation, and inclination information of the mobile device from the mobile device without receiving the real-time image;

identifying POI information associated with the user-provided destination information in accordance with the updated current location, speed, orientation, rotation, and inclination information of the mobile device, without reliance upon the real-time image captured by the camera, wherein the POI information includes label information associated with at least one object near the mobile device and navigation information from the updated current location of the mobile device to the destination; and

returning the POI information to be superimposed on the real-time image of the at least one object captured by the camera of the mobile device, wherein on the mobile device, the label information is displayed on the real-time image next to the at least one object and a directional indicator is displayed on the real-time image indicating a navigation path from the updated current location of the mobile device to the destination based on the navigation information.