Displaying image data and geographic element data

• US 9,329,052 B2
• Filed: 08/05/2009
• Issued: 05/03/2016
• Est. Priority Date: 08/07/2007
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Varanasi, Anantakrishna, Wong, Yoon Kean, Ho, Junius
• Primary Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US12/535,841
• Publication Number:
  US 20100035637A1
• Time in Patent Office: 2,463 Days
• Field of Search: 348/333.02, 455/556.1
• US Class Current: 1/1
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams
• CPC Class Codes: G01C 21/3682 output of POI information on a road map G01C21/3614, G01C21/3685 take precedence
• CPC Class Codes: G06F 16/29 Geographical information databases

Abstract

A mobile computing device includes a housing, a camera, a display, a memory, and a processing circuit. The housing is configured to be carried by a user while in use. The camera is configured to output image data. The memory is configured to store geographic element data representing one or more geographic elements. The processing circuit is configured to receive the geographic element data for the plurality of geographic elements. The processing circuit is configured to determine a camera orientation. The processing circuit is configured to concurrently display the image data and geographic element data for the plurality of geographic elements on the display.

Claims

1. A mobile computing device comprising:

a housing;

a camera;

a memory that stores instructions;

a display; and

a processing circuit that executes the instructions to cause the mobile computing device to;

present, on the display, image data captured by the camera;

identify a geographic element that is in the camera'"'"'s view and depicted in the image data when the mobile computing device is oriented in a first direction;

receive geographic element data corresponding to the geographic element from a remote server;

select the geographic element data that corresponds to the geographic element based on a membership association between an entity related to the geographic element and a user of the mobile computing device;

concurrently display the geographic element data with the image data, the geographic element data being displayed in alignment with the geographic element; and

in response to the mobile computing device changing orientation, continue to display the geographic element data in alignment with the geographic element.

2. The mobile computing device of claim 1, wherein the mobile computing device accesses a database of geographic element data for the geographic element data that corresponds to the geographic element.

3. The mobile computing device of claim 1, wherein the memory further stores personal data of the user indicating the membership association.

4. The mobile computing device of claim 3, wherein the processing circuit receives the personal data from the memory.

5. The mobile computing device of claim 1, wherein the processing circuit is configured to acquire a position fix of the mobile computing device.

6. The mobile computing device of claim 1, wherein the geographic element data comprises textual, image, video and/or audio data.

7. The mobile computing device of claim 1, wherein the processing circuit is configured to determine the orientation of the mobile computing device based at least in part on an image processing algorithm.

8. The mobile computing device of claim 1, wherein the processing circuit executes the instructions to cause the mobile computing device to display a distinguishing indicia which distinguishes the geographic element from another geographic element.

9. The mobile computing device of claim 1, wherein the processing circuit executes the instructions to cause the mobile computing device to detect a boundary and/or an edge of the displayed geographic element by detecting sharp changes in image brightness.

15. The mobile computing device of claim 2, wherein the processing circuit executes instructions to cause the mobile computing device to:

extract words from the image data captured by the camera; and

store, in the database, the words as part of the geographic element data.

10. A computer-implemented method for displaying geographic element data on a mobile computing device comprising a camera and one or more processors, the method being implemented by the one or more processors of the mobile computing device and comprising:

presenting, on the display, image data captured by the camera;

identifying a geographic element that is in the camera'"'"'s view and depicted in the image data when the mobile computing device is oriented in a first direction;

receiving geographic element data corresponding to the geographic element from a remote server;

selecting the geographic element data that corresponds to the geographic element based on a membership association between an entity related to the geographic element and a user of the mobile computing device;

concurrently displaying the geographic element data with the image data, the geographic element data being displayed in alignment with the geographic element; and

in response to the mobile computing device changing orientation, continuing to display the geographic element data in alignment with the geographic element.

11. The method of claim 10, further comprising accessing a database of geographic element data for the geographic element data that corresponds to the geographic element.

12. The method of claim 10, wherein the geographic element data comprises textual, image, video and/or audio data.

13. The method of claim 10, further comprising:

acquiring a position fix of the mobile computing device.

14. The method of claim 13, further comprising:

determining the orientation of the mobile computing device based at least in part on an image processing algorithm, and wherein identifying the geographic element is based on the acquired position fix and the determined orientation of the camera.

16. The method of claim 11, further comprising:

extracting words from the image data captured by the camera; and

storing, in the database, the words as part of the geographic element data.

17. A mobile computing device comprising:

a housing;

a camera;

a memory that stores instructions;

a display; and

a processing circuit that executes the instructions to cause the mobile computing device to;

determine a current location of the mobile computing device;

present, on the display, image data captured by the camera;

identify a geographic element that is in the camera'"'"'s field of view and depicted in the image data when the mobile computing device is oriented in a first direction;

receive geographic element data corresponding to the geographic element from a remote server;

select the geographic element data that corresponds to the geographic element based on personal, non-geographic data of a user of the mobile computing device;

concurrently display the geographic element data with the image data, the geographic element data being displayed in alignment with the geographic element; and

in response to the mobile computing device changing orientation, continue to display the geographic element data in alignment with the geographic element.

18. The mobile computing device of claim 17, wherein the processing circuit executes the instructions to cause the mobile computing device to select the geographic element data on the remote server by receiving a search query from a search bar shown on the display, and searching the remote server for the geographic element data corresponding to the search query.

19. The mobile computing device of claim 17 wherein the selected geographic element data comprises review data or ratings data associated with the geographic element.

20. The mobile computing device of claim 17, wherein the processing circuit executes the instructions to cause the mobile computing device to, in response to the mobile computing device changing orientation, continue to display the geographic element data in alignment with the geographic element by performing operations comprising repositioning the geographic element data by applying a set of motion vectors to a current location of the geographic element data.

21. The mobile computing device of claim 20, wherein the repositioning the geographic element data comprises:

partitioning each frame of the image data into a plurality of blocks;

calculating a correlation between a first block in a first frame of the image data with each of a number of neighboring blocks that surround the first block in a subsequent frame of the image data;

identifying a neighboring block having the highest correlation with the first block in the first frame of the image data;

determining the set of motion vectors corresponding to the identified neighboring block to identify, for the subsequent frame of the image data, a change in the field of view of the camera;

creating a new frame of geographic indicators based on the first frame and the set of motion vectors; and

superimposing the new frame on the subsequent frame of the image data.

27. The mobile computing device of claim 17, wherein the personal, non-geographic data of the user includes a membership or non-geographic affiliation between the user and an entity associated with the geographic element.

22. A computer-implemented method for displaying geographic element data on a mobile computing device comprising a camera and one or more processors, the method being implemented by the one or more processors of the mobile computing device and comprising:

determining a current location of the mobile computing device;

presenting, on a display, image data captured by the camera;

identifying a geographic element that is in the camera'"'"'s field of view and depicted in the image data when the mobile computing device is oriented in a first direction;

receiving the geographic element data corresponding to the geographic element from a remote server;

selecting the geographic element data that corresponds to the geographic element based on personal, non-geographic data of a user of the mobile computing device; and

concurrently displaying the geographic element data with the image data, the geographic element data being displayed in alignment with the geographic element; and

in response to the mobile computing device changing orientation, continuing to display the geographic element data in alignment with the geographic element.

23. The computer-implemented method of claim 22, wherein selecting the geographic element data on the remote server comprises receiving a search query from a search bar presented on the display and searching the remote server for the geographic element data corresponding to the search query.

24. The computer-implemented method of claim 22, wherein the selected geographic element data comprises reviews or ratings associated with the geographic element.

25. The computer-implemented method of claim 22, wherein continuing to display the geographic element data in alignment with the geographic element comprises repositioning the geographic element data by applying a set of motion vectors to a current location of the geographic element data.

26. The computer-implemented method of claim 25, wherein the repositioning the geographic element data comprises:

partitioning each frame of the image data into a plurality of blocks;

calculating a correlation between a first block in a first frame of the image data with each of a number of neighboring blocks that surround the first block in a subsequent frame of the image data;

identifying a neighboring block having the highest correlation with the first block in the first frame of the image data;

determining the set of motion vectors corresponding to the identified neighboring block to identify, for the subsequent frame of the image data, a change in the field of view of the camera;

creating a new frame of geographic indicators based on the first frame and the set of motion vectors; and

superimposing the new frame on the subsequent frame of the image data.

28. The method of claim 22, wherein the personal, non-geographic data of the user includes a membership or non-geographic affiliation between the user and an entity associated with the geographic element.