System and methods for volume dimensioning for supply chains and shelf sets

• US 10,268,892 B1
• Filed: 05/16/2016
• Issued: 04/23/2019
• Est. Priority Date: 05/15/2015
• Status: Active Grant

• Current Assignee: 4DMobile LLC
• Original Assignee: 4DMobile LLC
• Inventors: Miller, Matthew, Kunert, Steven R., McDermott, Craig, Berry, Josh, Rasmussen, Jon
• Primary Examiner(s): Perez-Fuentes, Luis
• Application Number:
  US15/156,149
• Time in Patent Office: 1,072 Days
• CPC Class Codes: G03H 1/0005 Adaptation of holography to specific applications holographic optical element G02B5/32; holographic scanner G02B26/106; recognition using holographic mask G06V10/88; holographic memories G11B7/0065, G11C13/042
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1656 Details related to functional adaptations of the enclosure, e.g. to provide protection against EMI, shock, water, or to host detachable peripherals like a mouse or removable expansions units like PCMCIA cards, or to provide access to internal components for maintenance or to removable storage supports like CDs or DVDs, or to mechanically mount accessories mounting of accessories to a computer display G06F1/1607; display hoods G06F1/1603; cooling arrangements for portable computers G06F1/203
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/1696 the I/O peripheral being a printing or scanning device
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 10/083 Shipping
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H02J 50/00 Circuit arrangements or systems for wireless supply or distribution of electric power
• CPC Class Codes: H02J 7/0042 : characterised by the mechanical construction
• CPC Class Codes: H04B 1/3877 : Arrangements for enabling portable transceivers to be used in a fixed position, e.g. cradles or boosters
• CPC Class Codes: H04B 1/3888 : Arrangements for carrying or protecting transceivers
• CPC Class Codes: H04B 2001/3894 : Waterproofing of transmission device
• CPC Class Codes: H04M 1/04 : Supports for telephone transmitters or receivers
• CPC Class Codes: H04M 1/185 : Improving the rigidity of the casing or resistance to shocks
• CPC Class Codes: H04M 1/72409 : by interfacing with external accessories hands-free H04M1/60
• CPC Class Codes: H04M 1/724094 : Interfacing with a device worn on the user's body to provide access to telephonic functionalities, e.g. accepting a call, reading or composing a message
• CPC Class Codes: H04M 1/72412 : using two-way short-range wireless interfaces
• CPC Class Codes: H04N 13/204 : using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/254 : in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/296 : Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/51 : Housings
• CPC Class Codes: H04N 23/62 : Control of parameters via user interfaces

Abstract

A system and related methods for volume dimensioning may be implemented by a smartphone, pad, tablet, or similar mobile computing device configured with a three-dimensional imaging system. The system may recognize the edges and corners of an object and infer its dimensions, from which a volume and dimensional weight can be calculated. The system may utilize the determined dimensions to allocate shipping resources to most efficiently and safely transport a shipment through a supply chain, assess damage to an object, assess the compliance of carry-on baggage to size requirements, and quickly assess the stocking of a retailer'"'"'s and a competitor'"'"'s products.

Claims

1. A system for volume dimensioning of products in a supply chain, the system comprising:

a mobile computing device configured with a touchscreen, at least one processor, a memory, and a wireless transceiver configured to establish a wireless link;

at least one three-dimensional (3D) imaging system of the mobile computing device, the at least one 3D imaging system configured to;

a) capture at least one of (1) an image featuring one or more objects including a target object and (2) distance information associated with the image, the distance information including at least one distance between the mobile computing device and one or more of the target object and a marker object;

and

b) transmit the at least one image and the distance information to the at least one processor;

a time sensor of the mobile computing device, the time sensor configured to determine a time of the at least one image;

at least one position sensor of the mobile computing device, the position sensor configured to determine a location of the at least one image;

the at least one processor configured to

a) activate the at least one 3D imaging system;

b) determine at least one dimension of the target object, the dimension including at least one of a width, a length, a height, a volume based on one or more of the image, the distance information, and a known dimension corresponding to the marker object;

c) detect at least one object identifier corresponding to the target object based on the at least one image, the object identifier including at least one of a barcode, a QR code, an RFID code, a text string, a brand identifier, and a design element;

d) associate object data with the target object based on one or more of the image, the determined dimension, the time, the location, reference data, and the at least one object identifier, the associated object data including at least one of a count, a fragility status, a prior status, a shipping origin, a shipping destination, a damage status, a shipping history, and a brand;

e) display, via the touchscreen, one or more of the image, the determined dimension, and the determined object data;

and

f) generate, based on the associated object data, one or more object instructions corresponding to the target object, the object instructions including one or more of a shipping assignment, a storage assignment, a restocking order, an inventory report, and an update to the object data.

2. The system of claim 1, wherein the at least one first target object comprises at least one subject, the count includes a count of the at least one subobject, and the dimension includes at least one subdimension corresponding to the at least one subobject.

3. The system of claim 1, wherein the at least one processor is configured to:

display, via the touchscreen, the at least one image including at least one icon superimposed on the at least one image and corresponding to the at least one dimension; and

select the target object from the one or more featured objects based on user input entered via the touchscreen.

4. The system of claim 1, wherein the at least one processor is configured to generate, based on one or more of the at least one image and the distance information, at least one of a 3D model of the first object and a holographic image of the first object.

5. The system of claim 4, further comprising:

at least one augmented reality device wirelessly linked to the mobile computing device, the at least one augmented reality device including a head worn device (HWD) worn by the user and configured to

display the at least one holographic image to the user;

detect at least one of an eye movement of the user, a gesture of the user, and an audio command;

adjust the at least one holographic image based on at least one of the eye movement, the gesture, and the audio command;

detect the at least one object identifier based on the at least one of the eye movement, the gesture, and the audio command; and

display the object data associated with the at least one object identifier to the user.

6. The system of claim 1, wherein the reference data is at least one of a) stored in the memory or b) located remotely from, and wirelessly linked to, the mobile computing device.

7. The system of claim 1, wherein the shipping assignment and the storage assignment include at least one of:

assigning the target object to at least one of a shipping vehicle and a storage location;

assigning the target to at least one of a subdivision of the shipping vehicle and a subdivision of the storage location, the subdivision including at least one of a shelf, a room, and a compartment;

determining whether the target object may be stacked atop at least one neighbor object; and

determining whether the at least one neighbor object may be stacked atop the target object.

8. A method for volume dimensioning via three-dimensional (3D) imaging, comprising:

activating a 3D imaging system of a mobile computing device;

capturing, via the 3D imaging system, at least one of an image associated with at least one target object and distance information associated with the image, the distance information including at least one distance between the mobile computing device and one or more of the target object and a marker object;

determining, via at least one sensor of the mobile computing device, one or more of a time of the at least one image and a location of the at least one image;

determining, via at least one processor of the mobile computing device, at least one dimension of the target object based on one or more of the image, the distance information, and a known dimension corresponding to the marker object, the dimension including at least one of a length, a height, a width, a volume, and an estimated weight;

detecting, via the processor, at least one object identifier corresponding to the target object based on the at least one image, the object identifier including at least one of a barcode, a QR code, an RFID code, a text string, a brand identifier, and a design element;

associating, via the processor, object data with the target object based on one or more of the at least one image, the dimension, the time, the location, reference data, and the object identifier, the associated object data including one or more of a count, a fragility status, a prior shipping status, a shipping origin, a shipping destination, a damage status, and a brand;

displaying, via a touchscreen of the mobile computing device, at least one of the captured image, the determined dimension, and the associated object data;

and

generating, based on the associated object data, one or more object instructions corresponding to the target object, the object instructions including one or more of a shipping assignment, a storage assignment, a restocking order, and an inventory report.

9. The method of claim 8, further comprising:

modifying the at least one dimension based on at least one icon displayed via the touchscreen, the at least one icon positionable by the user via the touchscreen.

10. The method of claim 9, wherein modifying the at least one dimension based on at least one icon displayed via the touchscreen, the at least one icon positionable by the user via the touchscreen object includes:

selecting the target object from one or more candidate objects by positioning the at least one icon via the touchscreen.

11. The method of claim 8, further comprising:

generating, via the at least one processor, at least one of a 3D model of the first object and a holographic image of the first object based on one or more of the at least one image and the distance information.

12. The method of claim 11, further comprising:

displaying the at least one holographic image via at least one augmented reality device wirelessly connected to the mobile computing device;

detecting, via the at least one augmented reality device, at least one of an eye movement of the user, a gesture of the user, and an audio command; and

adjusting the at least one holographic image based on at least one of the eye movement, the gesture, and the audio command.

13. The method of claim 8, wherein determining, via the at least one processor, object data associated with the at least one first object based on one or more of the at least one image, the distance information, the dimension, the volume, and the dimension weight determining associating, via the at least one processor, object data with the target object based on one or more of the at least one image, the dimension, the time, the location, reference data, and the object identifier, the associated object data including one or more of a count, a fragility status, a prior status, a shipping origin, a shipping destination, a damage status, a shipping history, and a brand includes:

determining the associated object data by decoding the at least one object identifier.

14. The method of claim 8, wherein determining, via the at least one processor, object data associated with the at least one first object based on one or more of the at least one image, the distance information, the dimension, the volume, and the dimension weight determining associating, via the at least one processor, object data with the target object based on one or more of the at least one image, the dimension, the time, the location, reference data, and the object identifier, the associated object data including one or more of a count, a fragility status, a prior status, a shipping origin, a shipping destination, a damage status, a shipping history, and a brand includes:

determining-the associated object data by comparing one or more of the at least one image, the dimension, the time, the location, reference data, and the object identifier to the reference data, the reference data one of a) stored in a memory of the mobile computing device and b) located remotely from, and wirelessly linked to, the mobile computing device.

15. The method of claim 8, wherein performing at least one of a value determination of the first object and a shipping determination of the first object based on one or more of the at least one image, the distance information, the dimension, the volume, and the dimension weight the shipping assignment and the storage assignment include at least one of:

assigning the target object to one or more of a shipping vehicle and a storage location;

assigning the target object to one or more of a subdivision of the shipping vehicle and a subdivision of the storage location, the subdivision including at least one of a shelf, a room, and a compartment;

determining whether the target object may be stacked atop a neighbor object; and

determining whether a third object may be stacked atop the target object based on one or more of the at least one image, the distance information, the dimension, the volume, and the dimension weight.

16. The method of claim 12, wherein determining, via the at least one processor, object data associated with the at least one first object based on one or more of the at least one image, the distance information, the dimension, the volume, and the dimension weight includes:

detecting, via the at least one augmented reality device, at least one object identifier of the first object from the at least one image based on at least one of the eye movement, the gesture, and the audio command; and

determining, via the at least one processor, object data associated with the at least one first object by decoding the at least one object identifier.