Augmented reality navigation

• US 10,663,302 B1
• Filed: 03/18/2019
• Issued: 05/26/2020
• Est. Priority Date: 03/18/2019
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Shen, Chen
• Primary Examiner(s): Brier, Jeffery A
• Application Number:
  US16/356,526
• Time in Patent Office: 435 Days
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A processor of an AR device may receive a plurality of points within a coordinate system corresponding to points in space. The plurality of points may define a path between a starting location within the coordinate system and a navigation destination within the coordinate system. The processor may determine that the AR device is at the starting location. The processor may receive camera data from a camera of the AR device. The camera data may represent at least a portion of the points in space. The processor may generate an AR display. The generating may include overlaying a graphic illustrating the position of at least a portion of the points within the coordinate system over equivalent points in space illustrated by the camera data.

Claims

1. A method of providing an augmented reality (AR) display comprising:

receiving, at a processor of an AR device, an image captured by a camera of the AR device;

determining, by the processor, that the AR device is at a starting location by analyzing the image and determining that a plurality of navigation destinations are available from the starting location;

receiving, by the processor, a user selection of one of the plurality of navigation destinations as a selected navigation destination;

receiving, at the processor, a plurality of points within a coordinate system corresponding to points in space, the plurality of points defining a path between the starting location within the coordinate system and the selected navigation destination within the coordinate system;

receiving, at the processor, camera data from the camera of the AR device, the camera data representing at least a portion of the points in space;

generating, by the processor, an AR display, the generating comprising overlaying a graphic illustrating the position of the portion of the points within the coordinate system that are represented in the camera data over all equivalent points in space illustrated by the camera data to form a path over the camera data;

receiving, at the processor, updated camera data from the camera of the AR device, the updated camera data representing at least a partially different portion of the points in space as a result of the camera having moved through space; and

generating, by the processor, an updated AR display, the generating comprising overlaying an updated graphic illustrating the position of the partially different portion of the points within the coordinate system that are represented in the updated camera data over all equivalent points in space illustrated by the updated camera data to update the path over the updated camera data.

2. The method of claim 1, wherein the determining that the AR device is at the starting location further comprises:

gathering location data defining a position of the AR device with at least one sensor of the AR device; and

comparing the location data with location data defining the starting location.

3. The method of claim 1, wherein the generating comprises changing the graphic to illustrate the position of a different portion of the points within the coordinate system as the AR device moves along equivalent different points in space illustrated by the camera data.

4. The method of claim 1, further comprising generating, by the processor, a map display illustrating the points within the coordinate system overlaid on a map.

5. A method of generating augmented reality (AR) display data comprising:

setting, by a processor of an AR device, a location of the AR device as a starting location within a coordinate system corresponding to points in space;

as the AR device moves through space from the starting location, repeatedly recording, by the processor, locations of the AR device as points within the coordinate system;

determining, by the processor, that the AR device has reached a destination location within the coordinate system;

in response to the determining, generating, by the processor, a route definition comprising at least the starting location within the coordinate system, the points within the coordinate system, and the destination location within the coordinate system;

associating, by the processor, with the starting location as a user-selectable route of a plurality of user-selectable routes originating at the starting location and having different destination locations; and

in response to a user selection of the user-selectable route made in response to a determination that the AR device is at the starting location, generating, by the processor, an AR display, the generating comprising;

receiving, at the processor, camera data from a camera of the AR device, the camera data representing at least a portion of the points in space;

overlaying a graphic illustrating the position of the portion of the points within the coordinate system that are represented in the camera data over all equivalent points in space illustrated by the camera data to form a path over the camera data;

receiving, at the processor, updated camera data from the camera of the AR device, the updated camera data representing at least a partially different portion of the points in space as a result of the camera having moved through space; and

overlaying an updated graphic illustrating the position of the partially different portion of the points within the coordinate system that are represented in the updated camera data over all equivalent points in space illustrated by the updated camera data to update the path over the updated camera data.

6. The method of claim 5, wherein the setting of the location as the starting location comprises receiving a scan of an object at the starting location.

7. The method of claim 5, further comprising transmitting, by the processor, the route definition to a server configured to serve AR display data.

8. The method of claim 5, wherein the determining that the AR device has reached a destination location is performed in response to a user command.

9. The method of claim 5, wherein the determining that the AR device has reached a destination location is performed in response to receiving a scan of an object at the destination location.

10. An augmented reality (AR) device comprising:

a processor;

a camera in communication with the processor;

a display in communication with the processor; and

a non-transitory memory in communication with the processor, the non-transitory memory storing instructions that, when executed by the processor, cause the processor to perform processing comprising;

receiving an image captured by a camera of the AR device;

determining that the AR device is at a starting location by analyzing the image and determining that a plurality of navigation destinations are available from the starting location;

receiving a user selection of one of the plurality of navigation destinations as a selected navigation destination;

receiving a plurality of points within a coordinate system corresponding to points in space, the plurality of points defining a path between the starting location within the coordinate system and the selected navigation destination within the coordinate system;

receiving, by the camera, camera data representing at least a portion of the points in space;

displaying, by the display, an overlay of a graphic illustrating the position of the portion of the points within the coordinate system that are represented in the camera data over all equivalent points in space illustrated by the camera data to form a path over the camera data;

receiving, by the camera, updated camera data representing at least a partially different portion of the points in space as a result of the camera having moved through space; and

displaying, by the display, an updated overlay of an updated graphic illustrating the position of the partially different portion of the points within the coordinate system that are represented in the updated camera data over all equivalent points in space illustrated by the updated camera data to update the path over the updated camera data.

11. The AR device of claim 10, wherein the processing further comprises:

setting a location of the AR device as the starting location within the coordinate system;

as the AR device moves through space from the starting location, repeatedly recording locations of the AR device as points within the coordinate system;

determining that the AR device has reached the destination location within the coordinate system; and

in response to the determining, generating a route definition comprising at least the starting location within the coordinate system, the points within the coordinate system, and the destination location within the coordinate system.

12. The AR device of claim 11, wherein the received plurality of points comprises the route definition.

13. The AR device of claim 11, further comprising a transceiver configured to transmit the route definition to a server configured to serve AR display data.

14. The AR device of claim 10, further comprising at least one location sensor configured to determine the AR device is at the starting location.

15. The AR device of claim 10, further comprising a transceiver configured to receive the plurality of points from a server configured to serve AR display data.

16. The AR device of claim 10, wherein the displaying comprises changing the graphic to illustrate the position of a different portion of the points within the coordinate system as the AR device moves along equivalent different points in space illustrated by the camera data.

17. The AR device of claim 10, wherein the processing further comprises displaying, by the display, a map display illustrating the points within the coordinate system overlaid on a map.