Augmented reality assisted deliveries

• US 10,373,377 B1
• Filed: 10/26/2016
• Issued: 08/06/2019
• Est. Priority Date: 10/26/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Niewiadomski, Robert
• Primary Examiner(s): Wills-Burns, Chineyere D
• Application Number:
  US15/334,458
• Time in Patent Office: 1,014 Days
• Field of Search: 345 8
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/3632 Guidance using simplified or iconic instructions, e.g. using arrows G01C21/365 takes precedence
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01C 21/3697 Output of additional, non-guidance related information, e.g. low fuel level G01C21/3679 takes precedence
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06Q 10/0631 Resource planning, allocation, distributing or scheduling for enterprises or organisations
• CPC Class Codes: G06Q 10/083 Shipping
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

Disclosed are various embodiments for providing augmented reality assisted deliveries. Some embodiments include determining a current location of a client device and a next node in a delivery workflow based at least in part on the current location of the client device. Then, the client device renders the next node in the delivery workflow on an augmented reality display linked to the client device. Various embodiments can also include identifying a plurality of delivery workflows that include a destination node that corresponds to a delivery and selecting select a delivery workflow from the plurality of delivery workflows based at least in part on a delivery constraint. In these embodiments, a delivery route to a routing node is generated, wherein the routing node comprises an entry to the selected delivery workflow. The delivery route and the delivery workflow are then sent to an augmented reality client.

Claims

1. A system, comprising:

a computing device comprising a processor and a memory; and

an application comprising machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to at least;

receive an annotation from a first augmented reality client device for a node in a delivery workflow;

add the annotation to the node in the delivery workflow;

identify a plurality of delivery workflows that include the node;

select the delivery workflow from the plurality of delivery workflows based at least in part the delivery workflow having a routing node that is compatible with a vehicle constraint associated with a delivery, the routing node comprising a navigational waypoint on a delivery route to a destination node;

add the delivery workflow to the delivery route; and

send the delivery route to a second augmented reality client device, wherein the second augmented reality client device is configured to;

render a three-dimensional model of an area associated with the node in the delivery workflow on an augmented reality display associated with the second augmented reality client device, wherein the three-dimensional model comprises a marker indicating a physical location corresponding to the node in the delivery workflow;

render the annotation in association with the node on the augmented reality display associated with the second augmented reality client device; and

wherein the augmented reality display is configured to be worn by an operator of the second augmented reality client device.

2. The system of claim 1, wherein the node represents a parking location.

3. The system of claim 1, wherein the node represents a delivery location.

4. The system of claim 1, wherein the annotation comprises a video captured by the first augmented reality client device as a delivery agent moved from a previous node to the node, and the video is stored in association with the node.

5. A system, comprising:

a computing device comprising a processor and a memory; and

an application comprising machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to at least;

identify a plurality of delivery workflows that include a destination node that corresponds to a delivery;

select a delivery workflow from the plurality of delivery workflows based at least in part on a delivery constraint and the delivery workflow comprising at least one node that is compatible with a vehicle restriction associated with a vehicle being used for the delivery, the at least one node comprising a navigational routing waypoint on a delivery route to the destination node;

generate at least a portion of the delivery route to a routing node, wherein the routing node comprises an entry to the selected delivery workflow; and

send the delivery route and the selected delivery workflow to an augmented reality client, wherein at least a portion of the selected delivery workflow is rendered using a three-dimensional model that comprises a mark of a physical location of the routing node.

6. The system of claim 5, wherein the machine readable instructions that cause the computing device to at least identify the plurality of delivery workflows further comprise machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to at least:

match a first address specified in the destination node to a second address corresponding to the delivery; and

match an addressee specified in the destination node to a recipient corresponding to the delivery.

7. The system of claim 5, wherein:

the selected delivery workflow comprises a first delivery workflow;

the destination node of the first delivery workflow comprises a first destination node; and

the application further comprises machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to further;

identify a second delivery workflow that comprises the routing node of the first delivery workflow, the delivery constraint of the first delivery workflow, and a second destination node; and

add the second delivery workflow to the delivery route to the routing node.

8. The system of claim 5, wherein the routing node represents a navigational waypoint.

9. The system of claim 5, wherein the delivery constraint comprises a method of delivery.

10. The system of claim 5, wherein:

the routing node is limited to a method of delivery; and

the machine readable instructions that cause the computing device to select a delivery workflow from the plurality of delivery workflows based at least in part on the delivery constraint further comprise machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to at least determine that each routing node within the delivery workflow specifies the method of delivery as a valid method of delivery for the routing node.

11. The system of claim 5, wherein the delivery constraint comprises a time of delivery.

12. The system of claim 5, wherein the physical location is a first physical location, and the selected delivery workflow further comprises an access node that includes access instructions for a delivery agent to access a second physical location represented by the destination node.

13. The system of claim 5, wherein the physical location is a first physical location, and wherein the selected delivery workflow further comprises a parking node that specifies a second physical location for a delivery agent to park when attempting delivery to a third physical location represented by the destination node.

14. A method, comprising:

determining, via a client device, a current location of the client device using a receiver associated with the client device;

determining, via the client device, a next node in a delivery workflow based at least in part on the current location of the client device and a vehicle restriction associated with a vehicle being used in association with the client device, wherein the next node comprises a navigational routing waypoint on a delivery route to a destination node and is compatible with the vehicle restriction associated with the vehicle;

rendering, via the client device, the next node in the delivery workflow on an augmented reality display linked to the client device; and

rendering, via the client device, a marker on the augmented reality display, wherein the marker comprises a visual indication of a physical location corresponding to the next node in the delivery workflow.

15. The method of claim 14, further comprising:

rendering, via the client device, a data balloon on the augmented reality display in proximity to the marker, wherein the data balloon comprises information stored within the next node in the delivery workflow.

16. The method of claim 14, wherein the augmented reality display comprises a headset.

17. The method of claim 14, wherein the marker is rendered on a three-dimensional model comprising a holographic projection of the physical location of the next node.

18. The method of claim 14, wherein the next node in the delivery workflow comprises a way point on a road network.

19. The method of claim 14, wherein the next node in the delivery workflow comprises a delivery destination.

20. The method of claim 14, wherein the next node in the delivery workflow comprises a parking point.