HANDS-FREE AUGMENTED REALITY SYSTEM FOR PICKING AND/OR SORTING ASSETS

• US 20190107411A1
• Filed: 08/14/2018
• Published: 04/11/2019
• Est. Priority Date: 08/15/2017
• Status: Active Grant

• Current Assignee: United Parcel Service Of America Incorporated (United Parcel Service Incorporated)
• Original Assignee: United Parcel Service Of America Incorporated (United Parcel Service Incorporated)
• Inventors: Gil, Julio
• Granted Patent: US 11,156,471 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/3632 Guidance using simplified or iconic instructions, e.g. using arrows G01C21/365 takes precedence
• CPC Class Codes: G01C 21/3641 Personalized guidance, e.g. limited guidance on previously travelled routes
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06Q 10/047 Optimisation of routes or paths, e.g. travelling salesman problem
• CPC Class Codes: G06Q 10/0833 Tracking
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/001 using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP

Abstract

A system is provided for hands-free handling of at least one asset by a user. The system includes a user device configured to be worn by a user and a control system remotely located relative to the user device and configured to exchange asset-related data with the user device, the asset-related data including at least one or more notifications related to the handling of the at least one asset by the user at the asset location. The user device contains a processor configured to determine location data associated with the at least one asset and including a location for the at least one asset, the determination being based, in part, upon the obtained asset identifier data; dynamically generate and display, at the user device, one or more navigational projections configured to guide the user to the asset location; and detect handling of the at least one asset by the user.

Claims

1. A system for providing hands-free handling of at least one asset by a user, the system comprising:

a user device configured to be worn by a user, the user device comprising;

one or more memories; and

one or more processors configured to;

obtain asset identifier data for the at least one asset;

determine location data associated with a location for the at least one asset based, at least in part, upon the obtained asset identifier data;

dynamically generate and display one or more navigational projections configured to guide the user to the location;

detect handling of the at least one asset by the user; and

receive one or more notifications associated with the handling of the at least one asset by the user at the location.

2. The system of claim 1, wherein the user device further comprises a mounting mechanism configured for mounting the user device to either a chest or a top portion of a head of the user.

3. The system of claim 1, wherein the user device further comprises:

a plurality of depth sensors; and

a pivoting laser projector,

wherein;

the pivoting laser projector is configured to perform the dynamically generate and display the one or more navigational projections; and

the one or more navigational projections are made based on the location data of the user device collected via the plurality of depth sensors.

4. The system of claim 1, wherein the one or more navigational projections are two-dimensional projections generated via a pivoting laser projector of the user device.

5. The system of claim 1, wherein the one or more navigational projections are three-dimensional projections generated via a pivoting laser projector of the user device.

6. The system of claim 1, wherein the one or more navigational projections are holographic images generated via a pivoting laser projector of the user device.

7. The system of claim 1, wherein the user device further comprises a plurality of depth sensors configured to map an environment that the user is in to calibrate, prior to utilization of the user device, the user device in an operational mode for asset picking or sorting.

8. The system of claim 1, wherein the user device further comprises at least one speaker configured to transmit audible instructions to the user concurrent with the displaying of the one or more navigational projections.

9. The system of claim 8, wherein the audible instructions are configured to provide feedback in response to detection, by the user device, of a deviation of the user from the one or more navigational projections.

10. A computer implemented method for providing hands-free handling of at least one asset by a user, the method comprising:

receiving, from a remote location relative to a user device that is worn by the user, asset identifier data for at least one asset;

determining, at the user device, first location data associated with the user device and determining second location data associated with the at least one asset, the second location data being determined, based at least in part, on analyzing the received asset identifier data, the first location data being determined, based at least in part, on analyzing a present position of the user device; and

dynamically generating and displaying one or more navigational projections configured to guide the user to a location associated with the second location data, the one or more navigational projections dynamically updating based at least in part on one or more detected changes of a present location of the user device.

11. The method of claim 10, further comprising detecting, at the user device, handling of the at least one asset by the user in response to the present location of the user device matching the location associated with the at least one asset.

12. The method of claim 11, wherein the asset identifier data is obtained by a controller system upon receipt of an order for the at least one asset by a customer.

13. The method of claim 10, wherein the asset identifier data is received, at the user device, via the user device scanning asset identifying data located on the at least one asset.

14. The method of claim 10, wherein:

the user device further comprises a plurality of depth sensors and a pivoting laser projector;

the dynamic generation and display of the one or more navigational projections occurs via the pivoting laser projector; and

the plurality of depth sensors collect environmental data to determine at least one type of the one or more navigational projections to generate and display.

15. The method of claim 14, wherein one of the at least one type of navigational projections is a two-dimensional directional arrow configured to instruct the user wearing the user device to make a change relative to the user present movement pattern.

16. The method of claim 14, wherein one of the at least one type of navigational projections is a two-dimensional frame identifying one or more assets for the user to pick up upon arriving at the location associated with the at least one asset.

17. The method of claim 16, wherein the two-dimensional frame encompasses multiple assets for picking up by the user.

18. The method of claim 16, wherein the two-dimensional frame encompasses a single asset.

19. The method of claim 16, wherein the two-dimensional frame contains text indicia configured to further guide and instruct the user for handling of the one or more assets.

20. A computer program product for providing hands-free handling of at least one asset by a user, the computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising one or more executable portions configured for:

mapping an environment that a user is located in based at least in part on generating a multidimensional graphical representation of the environment;

receiving, at a user device, asset identifier data to identify at least one asset;

associating one or more assets locations within the mapped environment, the one or more asset locations being associated with the at least one asset; and

generating and displaying, based at least on the associating, one or more navigational projections within the environment, the one or more navigational projections configured to guide the user to an asset location within the environment.