Directional and x-ray view techniques for navigation using a mobile device

• US 10,354,452 B2
• Filed: 03/13/2018
• Issued: 07/16/2019
• Est. Priority Date: 02/26/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Jiajian, Chao, Hui
• Primary Examiner(s): McDowell, Jr., Maurice L.
• Assistant Examiner(s): Ricks, Donna J.
• Application Number:
  US15/919,721
• Publication Number:
  US 20180204386A1
• Time in Patent Office: 490 Days
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels

Abstract

Techniques for displaying navigation information on a mobile device are provided that include a method that includes obtaining an indication of a position and an indication of a direction associated with the mobile device, using the indication of the position, the indication of the direction, information regarding identities of POIs within a geographic region of interest, and information regarding areas associated with the POIs to determine at least one relevant POI, of the POIs, that is associated with the position and direction, and displaying at least one visual indication associated with each of the at least one relevant POI on the mobile device. The appearance of the at least one visual indication is dependent on at least one of a distance from the mobile device of the relevant POI associated with the visual indication or presence of a known physical barrier between the mobile device and that relevant POI.

Claims

1. A method of generating one or more augmented views, the method comprising:

determining a plurality of points of interest within a geographic area of interest based on a field of view of a camera;

obtaining an image of a portion of the geographic area of interest captured using the camera;

obtaining a first image associated with a first point of interest from at least one database, the first point of interest being disposed within the portion of the geographic area of interest;

obtaining a second image associated with a second point of interest from the at least one database, the second point of interest being disposed within the portion of the geographic area of interest;

generating an augmented view including a composite image of at least the portion of the geographic area, wherein generating the augmented view includes;

overlaying, at a first transparency level, the first image over the image of the portion of the geographic area of interest; and

overlaying, at a second transparency level, the second image over the image of the portion of the geographic area of interest, wherein the first image is displayed as being in front of at least a portion of the second image based on the first point of interest being disposed at least partially behind the second point of interest in the geographic area of interest relative to the camera, the second transparency level of the second image being different than the first transparency level of the first image; and

displaying, on a mobile device, the augmented view.

2. The method of claim 1, wherein at least a portion of the first image does not overlap at least a portion of the second image.

3. The method of claim 1, wherein at least one of the first image or the second image has a size that is dependent on a distance from a mobile device of at least one of the first point of interest or the second point of interest.

4. The method of claim 1, wherein the second transparency level has a lower transparency than the first transparency level.

5. The method of claim 1, wherein one or more visual indications associated with one or more points of interest that are not visible in the field of view of the camera are displayed as at least partially transparent based on the one or more points of interest being at least partially obstructed from view by one or more objects in the geographic area of interest.

6. The method of claim 1, wherein a transparency level of at least one of the first image or the second image is dependent on a presence of one or more physical barriers between a mobile device and at least one of the first point of interest or the second point of interest.

7. The method of claim 1, wherein at least one of the first image or the second image is displayed as opaque based on at least one of the first point of interest or the second point of interest being visible in the field of view.

8. The method of claim 1, further comprising generating an updated augmented view, wherein at least one of the first transparency level of the first image or the second transparency level of the second image is adjusted in the updated augmented view based on movement of the camera relative to at least one of the first point of interest or the second point of interest.

9. The method of claim 1, wherein the geographic area of interest comprises an indoor environment.

10. The method of claim 9, wherein the mobile device is in the indoor environment, wherein the second point of interest is in the indoor environment, and wherein the first point of interest is outside of the indoor environment.

11. The method of claim 1, wherein the mobile device is in a first indoor environment, wherein the second point of interest is in the first indoor environment, and wherein the first point of interest is in a second indoor environment.

12. The method of claim 1, wherein the first transparency level of the first image and the second transparency level of the second image are based on the first point of interest being disposed at least partially behind the second point of interest in the geographic area of interest.

13. The method of claim 1, wherein the first point of interest is disposed within the portion of the geographic area of interest at a first distance from the mobile device and the second point of interest is disposed within the portion of the geographic area of interest at a second distance from the mobile device, and wherein the first transparency level of the first image is based on the first distance and the second transparency level of the second image is based on the second distance.

14. The method of claim 1, wherein the first point of interest is disposed within the portion of the geographic area of interest at a distance from the second point of interest, and wherein the first transparency level of the first image and the second transparency level of the second image are based on the distance.

15. The method of claim 1, wherein the first point of interest is disposed within the portion of the geographic area of interest at a first distance from the mobile device and the second point of interest is disposed within the portion of the geographic area of interest at a second distance from the mobile device, and wherein the first image is displayed as being in front of at least the portion of the second image based on the first distance being greater than the second distance.

16. The method of claim 1, wherein the first image and the second image are different images captured using one or more image capture devices.

17. The method of claim 1, wherein the first image and the second image are captured using a same image capture device.

18. An apparatus for generating one or more augmented views, comprising:

a memory configured to store one or more images; and

a processor configured to;

determine a plurality of points of interest within a geographic area of interest based on a field of view of a camera;

obtain an image of a portion of the geographic area of interest captured using the camera;

obtain a first image associated with a first point of interest from at least one database, the first point of interest being disposed within the portion of the geographic area of interest;

obtain a second image associated with a second point of interest from the at least one database, the second point of interest being disposed within the portion of the geographic area of interest;

generate an augmented view including a composite image of at least the portion of the geographic area, wherein generating the augmented view includes;

overlaying, at a first transparency level, the first image over the image of the portion of the geographic area of interest; and

overlaying, at a second transparency level, the second image over the image of the portion of the geographic area of interest, wherein the first image is displayed as being in front of at least a portion of the second image based on the first point of interest being disposed at least partially behind the second point of interest in the geographic area of interest relative to the camera, the second transparency level of the second image being different than the first transparency level of the first image; and

a display for displaying the augmented view.

19. The apparatus of claim 18, wherein at least a portion of the first image does not overlap at least a portion of the second image.

20. The apparatus of claim 18, wherein the second transparency level has a lower transparency than the first transparency level.

21. The apparatus of claim 18, wherein one or more visual indications associated with one or more points of interest that are not visible in the field of view of the camera are displayed as at least partially transparent based on the one or more points of interest being at least partially obstructed from view by one or more objects in the geographic area of interest.

22. The apparatus of claim 18, wherein a transparency level of at least one of the first image or the second image is dependent on a presence of one or more physical barriers between the apparatus and at least one of the first point of interest or the second point of interest.

23. The apparatus of claim 18, wherein at least one of the first image or the second image is displayed as opaque based on at least one of the first point of interest or the second point of interest being visible in the field of view.

24. The apparatus of claim 18, wherein the processor is further configured to generate an updated augmented view, wherein at least one of the first transparency level of the first image or the second transparency level of the second image is adjusted in the updated augmented view based on movement of the camera relative to at least one of the first point of interest or the second point of interest.

25. The apparatus of claim 18, wherein the geographic area of interest comprises an indoor environment, wherein the apparatus is in the indoor environment, wherein the second point of interest is in the indoor environment, and wherein the first point of interest is outside of the indoor environment.

26. The apparatus of claim 18, wherein the apparatus is in a first indoor environment, wherein the second point of interest is in the first indoor environment, and wherein the first point of interest is in a second indoor environment.

27. The apparatus of claim 18, wherein the first transparency level of the first image and the second transparency level of the second image are based on the first point of interest being disposed at least partially behind the second point of interest in the geographic area of interest.

28. The apparatus of claim 18, wherein the first point of interest is disposed within the portion of the geographic area of interest at a first distance from the apparatus and the second point of interest is disposed within the portion of the geographic area of interest at a second distance from the apparatus, and wherein the first transparency level of the first image is based on the first distance and the second transparency level of the second image is based on the second distance.

29. The apparatus of claim 18, wherein the first point of interest is disposed within the portion of the geographic area of interest at a first distance from the apparatus and the second point of interest is disposed within the portion of the geographic area of interest at a second distance from the apparatus, and wherein the first image is displayed as being in front of at least the portion of the second image based on the first distance being greater than the second distance.

30. The apparatus of claim 18, further comprising:

a camera for capturing the one or more images.

31. The apparatus of claim 18, wherein the apparatus comprises a mobile device with a camera for capturing pictures.

32. The apparatus of claim 18, wherein the first image and the second image are images captured using one or more image capture devices.

33. The apparatus of claim 18, wherein the first image and the second image are captured using a same image capture device.