SYSTEM AND METHOD FOR DISPLAYING OBJECT LOCATION IN AUGMENTED REALITY

• US 20120088526A1
• Filed: 10/08/2010
• Published: 04/12/2012
• Est. Priority Date: 10/08/2010
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: Research In Motion Limited (Blackberry Limited)
• Inventors: Lindner, Jeffery
• Granted Patent: US 8,315,674 B2
• Time in Patent Office: Days
• US Class Current: 455/457
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04M 2250/52 including functional features of a camera
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/025 using location based information parameters
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

A system and a method are provided for displaying location information on a mobile device. The location information can include direction, distance, positional coordinates, etc. The mobile device'"'"'s display displays an image captured using the mobile device'"'"'s camera. A selection input is received to identify an object in the image. A facing direction of the mobile device is detected using the mobile device'"'"'s magnetometer. The mobile device determines a bearing to the object relative to the mobile device'"'"'s facing direction. The mobile device then determines a distance between the mobile device and the object. The obtained or computed location data is overlaid on the image, thereby augmenting the image. The location data can include at least a direction indicator of the object and the distance between the mobile device and the object, whereby the direction indicator can be determined using the bearing.

Claims

1. A method of displaying location information on a mobile device, the method comprising:

displaying on the mobile device'"'"'s display an image captured using the mobile device'"'"'s camera;

receiving a first selection input to select a first object image of a first object shown in the image;

detecting a facing direction of the mobile device using the mobile device'"'"'s facing direction finder;

determining a bearing to the first object relative to the facing direction;

determining a distance between the mobile device and the first object;

displaying, overlaid on the image, at least a direction indicator of the first object and the distance between the mobile device and the first object, determined from at least the bearing.

2. The method of claim 1 wherein the bearing is determined by obtaining a relationship between an angle of view of the camera and a pixel resolution of the image, and correlating a pixel location of the first object image with the relationship.

3. The method of claim 1 wherein the direction indicator is at least one of:

a true bearing relative to true North;

the bearing relative the facing direction; and

the magnetic bearing.

4. The method of claim 1 wherein the first object image is highlighted on the display upon receiving the first selection input.

5. The method of claim 1 further comprising:

detecting a coordinate location of the mobile device; and

determining a coordinate location of the first object based on the coordinate location of the mobile device, the distance between the mobile device and the first object, and the direction of the bearing.

6. The method of claim 1 further comprising determining an elevation angle of the object relative to the mobile device by obtaining a relationship between a vertical angle of view of the camera and a pixel resolution of the image, and correlating a pixel location of the first object image with the relationship.

7. The method of claim 1 further comprising:

receiving a second selection input to select a second object in the image;

determining a second bearing to the second object relative to the facing direction;

determining a second distance between the mobile device and the second object;

displaying, overlaid on the image, at least a second direction indicator of the second object and the second distance between the mobile device and the second object, determined from at least the second bearing.

8. The method of claim 7 further comprising:

determining an angle and a distance between the first object and the second object; and

displaying the angle and the distance between the first object and the second object overlaid the image.

9. The method of claim 8 wherein, upon detecting at least one of the first object, second object and the mobile device has changed orientation or position, updating the display of at least one of the direction indicator, the second direction indicator, the distance, and the second distance.

10. The method of claim 9 further comprising:

upon detecting that one of the first or the second object is no longer within the image, displaying an off-screen indicator notifying that the one of the first or the second object can be viewed by moving or rotating the camera in a certain direction.

11. The method of claim 1 wherein the facing direction finder is in communication with a magnetometer of the mobile device.

12. A computer readable medium comprising computer executable instructions for displaying location information on a mobile device, said computer readable medium comprising instructions for:

displaying on the mobile device'"'"'s display an image captured using the mobile device'"'"'s camera;

receiving a first selection input to select a first object image of a first object shown in the image;

detecting a facing direction of the mobile device using the mobile device'"'"'s facing direction finder;

determining a bearing to the first object relative to the facing direction;

determining a distance between the mobile device and the first object;

displaying, overlaid on the image, at least a direction indicator of the first object and the distance between the mobile device and the first object, determined from at least the bearing.

13. A mobile device, comprising:

a camera;

a magnetometer;

a display;

an input device; and

a processor coupled to the camera, a magnetometer, the display and the input device, the processor configured to;

display on the display an image captured by the camera, upon receiving at the input device a first selection input selecting a first object image of a first object shown in the image;

detect a facing direction of the mobile device using the magnetometer;

determine a bearing to the first object relative to the facing direction;

determine a distance between the mobile device and the first object, and display overlaid on the image, at least a direction indicator of the first object and the distance between the mobile device and the first object, determined from at least the bearing.

14. The mobile device of claim 13 wherein the bearing is determined by obtaining a relationship between an angle of view of the camera and a pixel resolution of the image, and correlating a pixel location of the first object image with the relationship.

15. The mobile device of claim 13 wherein the direction indicator is at least one of:

a true bearing relative to true North;

the bearing relative the facing direction; and

the magnetic bearing.

16. The mobile device of claim 13 wherein the first object image is highlighted on the display upon receiving the first selection input.

17. The mobile device of claim 13 wherein the processor is further configured to:

detect a coordinate location of the mobile device; and

determine a coordinate location of the first object based on the coordinate location of the mobile device, the distance between the mobile device and the first object, and the direction of the bearing.

18. The mobile device of claim 13 wherein the processor is further configured to determine an elevation angle of the object relative to the mobile device by obtaining a relationship between a vertical angle of view of the camera and a pixel resolution of the image, and correlating a pixel location of the first object image with the relationship.

19. The mobile device of claim 13 wherein the processor is further configured to:

receive a second selection input to select a second object in the image;

determine a second bearing to the second object relative to the facing direction;

determine a second distance between the mobile device and the second object;

display, overlaid on the image, at least a second direction indicator of the second object and the second distance between the mobile device and the second object, determined from at least the second bearing.

20. The mobile device of claim 19 wherein the processor is further configured to:

determine an angle and a distance between the first object and the second object; and

display the angle and the distance between the first object and the second object overlaid the image.

21. The mobile device of claim 20 wherein, upon detecting at least one of the first object, second object and the mobile device has changed orientation or position, the process is configured to update the display of at least one of the direction indicator, the second direction indicator, the distance, and the second distance.

22. The mobile device of claim 21 wherein the processor is further configured to:

upon detecting that one of the first or the second object is no longer within the image, display an off-screen indicator notifying that the one of the first or the second object can be viewed by moving or rotating the camera in a certain direction.