Displaying items of interest in an augmented reality environment

• US 10,535,169 B2
• Filed: 10/31/2017
• Issued: 01/14/2020
• Est. Priority Date: 11/02/2016
• Status: Active Grant

• Current Assignee: United Parcel Service Of America Incorporated (United Parcel Service Incorporated)
• Original Assignee: United Parcel Service Of America Incorporated (United Parcel Service Incorporated)
• Inventors: Dotterweich, Andrew, Schenken, Christopher T., Cooper, Jeffrey
• Primary Examiner(s): Sams, Michelle L
• Application Number:
  US15/798,551
• Publication Number:
  US 20180122118A1
• Time in Patent Office: 805 Days
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06K 7/10297 arrangements for handling protocols designed for non-contact record carriers such as RFIDs NFCs, e.g. ISO/IEC 14443 and 18092 cryptographic protocols H04L9/00; network security protocols H04L63/00; real-time communication protocols in data switching networks H04L65/00; network protocols for data switching network services H04L67/00
• CPC Class Codes: G06K 7/1413 1D bar codes
• CPC Class Codes: G06K 7/1417 2D bar codes
• CPC Class Codes: G06Q 10/0838 Historical data
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels

Abstract

Computer program products, methods, systems, apparatus, and computing entities are provided for overcoming the technical problem of providing an augmented reality that displays an actual image of the item or a proportionally dimensioned representation of the item to a user. To overcome this challenge, two separate approaches can be used: a beacon/tag/sensor-based approach and a marker-based approach.

Claims

1. A method for providing an augmented reality display comprising an image of an item, the method comprising:

identifying, via a user computing entity comprising one or more processors and a display, a marker in a field of view of the user computing entity, the marker associated with a powered asset;

requesting, via the user computing entity, at least a portion of a dispatch plan associated with the marker, the dispatch plan including one or more serviceable points along a delivery route of the powered asset and including an item associated with the delivery route;

causing, via the user computing entity, display of an item indicator associated with the item of the dispatch plan, the item indicator including an address associated with the delivery route;

receiving, via the user computing entity, a selection of the item indicator associated with the item;

generating, via the user computing entity, a request for item information for the item, the request for the item information comprising a tracking identifier for the item;

receiving, via the user computing entity, a response with the item information for the item, the response comprising a uniform resource identifier for an image of the item, the image of the item being previously captured at a point during transportation of the item by a carrier and stored in association with the uniform resource identifier;

retrieving, via the user computing entity, the image of the item using the uniform resource identifier; and

generating, via the user computing entity, an augmented reality display layer comprising the image of the item superimposed over a real world background layer, the image of the item being proportionally dimensioned relative to the real world background layer based on a set of dimensions determined for the item from the image of the item captured at the point during transportation of the item by the carrier.

2. The method of claim 1, wherein the marker is selected from a group consisting of text, a barcode, a character string, an Aztec Code, a MaxiCode, a matrix, and a QR Code.

3. The method of claim 1, wherein the image is a stock image based on the set of dimensions of the item.

4. The method of claim 1, wherein the marker is identified in response to an occurrence of a trigger.

5. The method of claim 1, wherein the image further comprises the item information.

6. The method of claim 1, wherein the image is captured by a camera positioned along a conveyor transporting the item.

7. An apparatus comprising a display, at least one processor, and at least one memory including program code, the at least one memory and the program code configured to, with the at least one processor, cause the apparatus to at least:

identify a marker in a field of view of the apparatus, the marker associated with a powered asset;

request at least a portion of a dispatch plan associated with the marker, the dispatch plan including one or more serviceable points along a delivery route of the powered asset and including an item associated with the delivery route;

cause display of an item indicator associated with an item of the dispatch plan, the item indicator including an address associated with the delivery route;

receive a selection of the item indicator associated with the item;

generate a request for item information for the item, the request for the item information comprising a tracking identifier for the item;

receive a response with the item information for the item, the response comprising a uniform resource identifier for an image of the item, the image of the item being previously captured at a point during transportation of the item by a carrier and stored in association with the uniform resource identifier;

retrieve the image of the item using the uniform resource identifier; and

generate an augmented reality display layer comprising the image of the item superimposed over a real world background layer, the image of the item being proportionally dimensioned relative to the real world background layer based on a set of dimensions determined for the item from the image of the item captured at the point during transportation of the item by the carrier.

8. The apparatus of claim 7, wherein the marker is selected from a group consisting of text, a barcode, a character string, an Aztec Code, a MaxiCode, a matrix, and a QR Code.

9. The apparatus of claim 7, wherein the image is a stock image based on the set of dimensions of the item.

10. The apparatus of claim 7, wherein the marker is identified in response to an occurrence of a trigger.

11. The apparatus of claim 7, wherein the image further comprises the item information.

12. A computer program product for providing an augmented reality display comprising an image of an item, the computer program product comprising at least one nontransitory computer-readable storage medium having computer-readable program code stored therein that, when executed by a processor, cause the processor to:

identify a marker in a field of view of a user computing entity, the marker associated with a powered asset;

request at least a portion of a dispatch plan associated with the marker, the dispatch plan including one or more serviceable points along a delivery route of the powered asset and including an item associated with the delivery route;

cause display of an item indicator associated with the item of the dispatch plan, the item indicator including an address associated with the delivery route;

receive a selection of the item indicator associated with the item;

generate a request for item information for the item, the request for the item information comprising a tracking identifier for the item;

receive a response with the item information for the item, the response comprising a uniform resource identifier for an image of the item, the image of the item being previously captured at a point during transportation of the item by a carrier and stored in association with the uniform resource identifier;

retrieve the image of the item using the uniform resource identifier; and

generate an augmented reality display layer for display on the user computing entity comprising the image of the item superimposed over a real world background layer, the image of the item being proportionally dimensioned relative to the real world background layer based on a set of dimensions determined for the item from the image of the item captured at the point during transportation of the item by the carrier.

13. The computer program product of claim 12, wherein the marker is selected from a group consisting of text, a barcode, a character string, an Aztec Code, a MaxiCode, a matrix, and a QR Code.

14. The computer program product of claim 12, wherein the image is a stock image based on the set of dimensions of the item.

15. The computer program product of claim 12, wherein the marker is identified in response to an occurrence of a trigger.