Automated generation of a three-dimensional space representation and planogram verification

• US 9,547,838 B2
• Filed: 11/06/2013
• Issued: 01/17/2017
• Est. Priority Date: 11/06/2013
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Larsen, Brett J.
• Primary Examiner(s): ZALALEE, SULTANA MARCIA
• Application Number:
  US14/073,231
• Publication Number:
  US 20150123973A1
• Time in Patent Office: 1,168 Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation

Abstract

The present disclosure provides an automated scheme for generating and verifying a three-dimensional (3D) representation of a target space. In one embodiment, the automatic generation of a 3D representation of a target space includes receiving target space data from one or more mobile data capture devices and generating a local point cloud from the target space data. In one embodiment, the local point cloud is incorporated into a master point cloud. In one embodiment, a polygon mesh is generated using the master point cloud and the polygon mesh is rendered, using a plurality of visual images captured from the target space, which generates the 3D representation. In one embodiment, the automatic verification includes comparing a portion of the 3D representation with a portion of an approved layout, and identifying one or more discrepancies between the portion of the 3D representation and the portion of the approved layout.

Claims

1. A computer-implemented method performed by a computing device, where the computing device includes at least a processor for executing instructions from a memory, the method comprising:

controlling, by at least the processor, at least movement of one or more mobile data capture devices within a target space by wirelessly transmitting control instructions, over a wireless computer network, to the one or more mobile data capture devices;

wirelessly receiving target space data from the one or more mobile data capture devices over the wireless computer network, wherein the target space data is captured from the target space by the one or more mobile data capture devices in response to the control instructions;

generating a three-dimensional representation of the target space at least in part by generating a local point cloud using the target space data, and incorporating the local point cloud into a master point cloud representing the target space;

in response to incorporating the local point cloud into the master point cloud, determining, by at least the processor, that a data gap exists in the master point cloud by evaluating data of the master point cloud, wherein the evaluating is not based on providing a visual rendering to a user;

in response to determining that the data gap exists, controlling automatically, by at least the processor, at least movement of the one or more mobile data capture devices within the target space, to capture additional target space data to fill in the data gap in the master point cloud, by wirelessly transmitting second control instructions over the wireless computer network to the one or more mobile data capture devices; and

automatically verifying the three-dimensional representation, wherein the automatically verifying comprises comparing a portion of the three-dimensional representation with a portion of an approved layout of the target space, and

identifying one or more discrepancies between the portion of the three-dimensional representation and the portion of the approved layout.

2. The method of claim 1, wherein the local point cloud comprises a set of three-dimensional (3D) points, the master point cloud comprises a plurality of 3D points, and

the incorporating comprises merging the set of 3D points into the plurality of 3D points.

3. The method of claim 2, wherein

generating the three-dimensional representation of the target space further comprises generating a polygon mesh using the master point cloud, and

rendering the polygon mesh, using a plurality of visual images captured from the target space, to produce the three-dimensional representation.

4. The method of claim 3, wherein the rendering the polygon mesh comprises determining a respective best image for a polygon of the polygon mesh, wherein the respective best image is determined from a subset of the plurality of visual images that are associated with the target space data used to generate the polygon, and

painting a portion of the respective best image onto the polygon.

5. The method of claim 1, wherein the generating the local point cloud comprises translating depth map data into the local point cloud, wherein the target space data comprises the depth map data.

6. The method of claim 1, further comprising:

generating a report that comprises the one or more discrepancies, wherein each discrepancy indicates that a visual design present in the approved layout is not found in an equivalent location in the three-dimensional representation.

7. An apparatus comprising:

a processor;

a memory coupled to the processor and configured to store instructions executable by the processor, wherein the instructions are configured to implement;

a control module configured to control, by at least the processor, at least movement of one or more mobile data capture devices within a target space by wirelessly transmitting control instructions, over a wireless computer network, to the one or more mobile data capture devices;

a receipt module configured to wirelessly receive target space data from the one or more mobile data capture devices over the wireless computer network, wherein the target space data is captured from the target space by the one or more mobile data capture devices in response to the control instructions;

a point cloud module configured to generate a local point cloud from the target space data;

a three-dimensional representation module configured to;

incorporate the local point cloud into a master point cloud representing the target space and automatically generate a three-dimensional representation of the target space based on the master point cloud, and

in response to incorporating the local point cloud into the master point cloud, determine, by at least the processor, that a data gap exists in the master point cloud by evaluating data of the master point cloud, wherein the evaluating is not based on providing a visual rendering to a user;

wherein the control module is further configured to, in response to determining that the data gap exists, automatically control, by at least the processor, at least movement of the one or more mobile data capture devices within the target space, to capture additional target space data to fill in the data gap in the master point cloud, by wirelessly transmitting second control instructions over the wireless computer network to the one or more mobile data capture devices; and

a comparison module configured to automatically verify the three-dimensional representation, at least in part by;

comparing a portion of the three-dimensional representation with a portion of an approved layout of the target space, and

identifying one or more discrepancies between the portion of the three-dimensional representation and the portion of the approved layout.

8. The apparatus of claim 7, further comprising a registration module configured to incorporate the local point cloud into the master point cloud, wherein the local point cloud comprises a set of three-dimensional (3D) points, the master point cloud comprises a plurality of 3D points, and

the set of 3D points are merged into the plurality of 3D points.

9. The apparatus of claim 8, further comprising a polygon mesh engine configured to generate a polygon mesh using the master point cloud, and

an image rendering engine configured to render the polygon mesh, using a plurality of visual images captured from the target space, to produce the three-dimensional representation, wherein the target space data comprises the plurality of visual images.

10. The apparatus of claim 9, wherein the image rendering engine comprises a best image detection module configured to determine a respective best image for a polygon of the polygon mesh, wherein the respective best image is determined from a subset of the plurality of visual images that are associated with the target space data used to generate the polygon, and

a polygon paint module configured to paint a portion of the respective best image onto the polygon.

11. The apparatus of claim 7, wherein the point cloud engine is further configured to translate depth map data into the local point cloud, wherein the target space data comprises the depth map data.

12. The apparatus of claim 7, further comprising a reporting module configured to generate a report that comprises the one or more discrepancies, wherein each discrepancy indicates that a visual design present in the approved layout is not found in an equivalent location in the three-dimensional representation.

13. A non-transitory computer readable storage medium that stores program instructions that, when executed on a processor of a computing device, are configured to cause the computing device to:

control, by at least the processor, at least movement of one or more mobile data capture devices within a target space by wirelessly transmitting control instructions, over a wireless computer network, to the one or more mobile data capture devices;

wirelessly receive a plurality of visual images from the one or more mobile data capture devices over the wireless computer network, wherein the plurality of visual images is captured from the target space by the one or more mobile data capture devices in response to the control instructions;

generate a three-dimensional representation of the target space at least in part by generating a local point cloud from the plurality of visual images, and incorporating the local point cloud into a master point cloud representing the target space;

in response to incorporating the local point cloud into the master point cloud, determine, by at least the processor, that a data gap exists in the master point cloud by evaluating data of the master point cloud, wherein the evaluating is not based on providing a visual rendering to a user;

in response to determining that the data gap exits, control automatically, by at least the processor, at least movement of the one or more mobile data capture devices within the target space, to capture additional visual images to fill in the data gap in the master point cloud, by wirelessly transmitting second control instructions over the wireless computer network to the one or more mobile data capture devices; and

automatically verify the three-dimensional representation, wherein the automatically verifying comprises comparing a portion of the three-dimensional representation with a portion of an approved layout of the target space, and identifying one or more discrepancies between the portion of the three-dimensional representation and the portion of the approved layout.

14. The non-transitory computer readable storage medium of claim 13, wherein the local point cloud comprises a set of three-dimensional (3D) points, the master point cloud comprises a plurality of 3D points, and

the incorporating comprises merging the set of 3D points into the plurality of 3D points.

15. The non-transitory computer readable storage medium of claim 14, wherein the automatically generating the three-dimensional representation of the target space further comprises generating a polygon mesh using the master point cloud, and

rendering the polygon mesh, using at least the plurality of visual images captured from the target space, to produce the three-dimensional representation.

16. The non-transitory computer readable storage medium of claim 15, wherein the rendering the polygon mesh comprises determining a respective best image for a polygon of the polygon mesh, wherein the respective best image is determined from a subset of the plurality of visual images that are associated with the target space data used to generate the polygon, and

painting a portion of the respective best image onto the polygon.

17. The non-transitory computer readable storage medium of claim 13, wherein generating the three-dimensional representation of the target space further comprises translating depth map data into the local point cloud, wherein the depth map data is derived from the plurality of visual images.

18. The non-transitory computer readable storage medium of claim 13, that stores program instructions that, when executed on the processor of the computing device, are configured to cause the computing device to:

generate a report that comprises the one or more discrepancies, wherein each discrepancy indicates that a visual design present in the approved layout is not found in an equivalent location in the three-dimensional representation.