Store shelf imaging system and method using a vertical LIDAR
First Claim
Patent Images
1. A retail environment robotic system comprising:
- a mobile base including an operatively associated navigation component configured to determine a location of the mobile base within a retail environment, the retail environment including a plurality of product display units arranged in a plurality of aisles, each product display unit including a plurality of product display fixtures and a plurality of products arranged on one or more of the product display fixtures;
a vertical scanning LIDAR (Light Detection and Ranging) unit operatively mounted to the mobile base, the LIDAR unit configured to scan in a vertical direction and generate a vertical array of distance and angle values representative of a plurality of object distances and angles from the LIDAR unit to a plurality of objects detected along the vertical scan of the LIDAR unit;
a LIDAR data processing component operatively associated with the LIDAR unit, the LIDAR data processing component configured to process the plurality of vertical arrays of object distances and angles to generate a depth map associated with one or more product display units as the mobile base travels along an aisle, the depth map including a plurality of columns and a plurality of rows where each of the plurality of columns is associated with a single vertical scan of the LIDAR unit at one of a plurality of locations along the aisle, and each of the plurality of rows is associated with one of a plurality of vertical distances; and
a depth map processing component configured to identify regions of the depth map associated with regions of the one or more product display units which include one or more of a missing product, the product, a shelf, a product tag attached to the product, and a shelf tag attached to the shelf.
6 Assignments
0 Petitions
Accused Products
Abstract
This disclosure provides a retail environment robotic system including a vertically oriented/scanning LIDAR (Light Detection and Ranging) unit mounted to a mobile base which travels along the aisles of a retail environment such as a store. According to an exemplary embodiment, the LIDAR unit vertically scans product display units along the aisles to generate a depth map associated with detected objects which is further processed to detect missing products, product/shelving tags, shelves, etc.
-
Citations
21 Claims
-
1. A retail environment robotic system comprising:
-
a mobile base including an operatively associated navigation component configured to determine a location of the mobile base within a retail environment, the retail environment including a plurality of product display units arranged in a plurality of aisles, each product display unit including a plurality of product display fixtures and a plurality of products arranged on one or more of the product display fixtures; a vertical scanning LIDAR (Light Detection and Ranging) unit operatively mounted to the mobile base, the LIDAR unit configured to scan in a vertical direction and generate a vertical array of distance and angle values representative of a plurality of object distances and angles from the LIDAR unit to a plurality of objects detected along the vertical scan of the LIDAR unit; a LIDAR data processing component operatively associated with the LIDAR unit, the LIDAR data processing component configured to process the plurality of vertical arrays of object distances and angles to generate a depth map associated with one or more product display units as the mobile base travels along an aisle, the depth map including a plurality of columns and a plurality of rows where each of the plurality of columns is associated with a single vertical scan of the LIDAR unit at one of a plurality of locations along the aisle, and each of the plurality of rows is associated with one of a plurality of vertical distances; and a depth map processing component configured to identify regions of the depth map associated with regions of the one or more product display units which include one or more of a missing product, the product, a shelf, a product tag attached to the product, and a shelf tag attached to the shelf. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A retail environment robotic system comprising:
-
a mobile base including an operatively associated navigating component configured to determine a location of the mobile base within a retail environment including a plurality of product display units arranged in a plurality of aisles, each product display unit including a plurality of product display fixtures, a plurality of products arranged on one or more shelves of the product display fixtures, and a plurality of tags attached to the shelves of the product display fixtures, each tag including product-related data including a barcode associated with a respective product proximately located on the product display fixture shelf near the tag; an image capture assembly operatively mounted on the mobile base, the assembly including a plurality of image capture devices for acquiring images of the product display units; a vertical scanning LIDAR (Light Detection and Ranging) unit operatively mounted to one of the mobile base and the image capture assembly, the LIDAR unit configured to scan in a vertical direction and generate LIDAR data including a vertical array of distance and angle values representative of a plurality of object distances and angles from the LIDAR unit to a plurality of objects detected along the vertical scan of the LIDAR unit; and a master control unit including an operatively associated memory which stores instruction and a processor for executing the instructions, the control unit instructions configured to process the images acquired by the plurality of image capture devices at a sequence of locations during a movement of the mobile base in the retail environment, thereby extracting the product-related data from the images of the product display units captured by the plurality of image capture devices during the movement of the mobile base, and generating a profile of the retail environment indicating locations of one or more of the products, missing products and the tags throughout the retail environment based on the extracted product-related data, the LIDAR data, and the locations of the mobile base at the sequence of locations provided by the navigation component. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A retail environment robotic system comprising:
-
a mobile base including an operatively associated navigating component configured to determine a location of the mobile base within a retail environment including a plurality of product display units arranged in a plurality of aisles, each product display unit including a plurality of product display fixtures, a plurality of products arranged on one or more shelves of the product display fixtures, and a plurality of tags attached to the shelves of the product display fixtures, each tag including product-related data including a barcode associated with a respective product proximately located on the product display fixture shelf near the tag; an image capture assembly operatively mounted on the mobile base, the assembly including a plurality of image capture devices for acquiring images of the product display units; a vertical scanning LIDAR (Light Detection and Ranging) unit operatively mounted to one of the mobile base and the image capture assembly, the LIDAR unit configured to scan in a vertical direction and generate LIDAR data including a vertical array of distance and angle values representative of a plurality of object distances and angles from the LIDAR unit to a plurality of objects detected along the vertical scan of the LIDAR unit; a master control unit including an operatively associated memory which stores instruction and a processor for executing the instructions, the control unit instructions configured to process the images acquired by the plurality of image capture devices at a sequence of locations during a movement of the mobile base in the retail environment, thereby extracting the product-related data from the images of the product display units captured by the plurality of image capture devices during the movement of the mobile base, and generating a profile of the retail environment indicating locations of one or more of the products, missing products and the tags throughout the retail environment based on the extracted product-related data, the LIDAR data, and the locations of the mobile base at the sequence of locations provided by the navigation component; a LIDAR data processing component operatively associated with the LIDAR unit and master control unit, the LIDAR data processing component configured to process the plurality of vertical arrays of object distances and angles to generate a depth map associated with the one or more product display units as the mobile base travels along an aisle, the depth map including a plurality of columns and a plurality of rows where each of the plurality of columns is associated with a single scan of the LIDAR unit at one of a plurality of locations along the aisle, and each of the plurality of rows is associated with one of a plurality of vertical distances; and a depth map processing component configured to identify regions of the depth map associated with regions of the one or more product display units which include one or more of the missing products, the products, the shelves, and the tags. - View Dependent Claims (20, 21)
-
Specification