Comparing Virtual and Real Images in a Shopping Experience

• US 20130300729A1
• Filed: 05/11/2012
• Published: 11/14/2013
• Est. Priority Date: 05/11/2012
• Status: Active Grant

• Current Assignee: Dassault Systemes SA
• Original Assignee: Dassault Systemes SA
• Inventors: Grimaud, Jean-Jacques
• Granted Patent: US 8,941,645 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 7/001 using an image reference approach

Abstract

In one embodiment, a computer-based method includes receiving image data of a real world layout. The image data reflects the real world layout across three dimensions (e.g., vertical, horizontal, and orthogonal). Each of the three dimensions has an image range with a beginning and an end. The real world layout has inventory of products distributed across the three dimensions. The computer-based method further includes comparing a 3D model of a planned layout of the environment to the received image data. The 3D model represents the planned layout across the three dimensions. The computer-based method further includes determining whether results of the comparison reach a first particular threshold, and if so, then output a pertinent indication, alert, or the like.

Claims

1. A computer-based method comprising:

receiving image data of a real world layout, the image data reflecting the real world layout across three dimensions, each of the three dimensions having an image range with a beginning and an end, and the real world layout having inventory distributed across the three dimensions;

comparing a 3D model of a planned layout of the real world layout to the received image data, the 3D model representing the planned layout across the three dimensions; and

determining whether results of the comparison reach a first particular threshold.

2. The method of claim 1, further comprising:

extracting, from the image data, at least one product display grouping;

associating the extracted at least one product display grouping with at least one representation of the product display grouping in the 3D model of the planned layout;

comparing the at least one product display groupings to the at least one representation of the product display grouping; and

determining whether the results meet a second particular threshold.

3. The method of claim 1, further comprising:

generating an alert if the results of the comparison of the 3D model to the received image data reaches the first particular threshold, wherein the comparison represents similarity or non-similarity and the first particular threshold represents a particular degree of similarity.

4. The method of claim 1, further comprising generating the 3D model of the planned layout.

5. The method of claim 1, wherein comparing the 3D model to the received image data includes calculating an image difference.

6. The method of claim 1, further comprising:

converting the 3D model to a 2D representation; and

wherein comparing the 3D model to the received image data comprises comparing the 2D representation of the 3D model to the received image data.

7. The method of claim 1, wherein comparing the 3D model of the planned layout to the received image data comprises:

converting the image data to a rendered 3D model; and

comparing the 3D model of the planned layout to the rendered 3D model.

8. The method of claim 1, wherein comparing a 3D model of a planned layout to the image data comprises:

analyzing the image data to determine a first quantity in an inventory, the first quantity in the inventory represented in the real world layout;

retrieving a second quantity in the inventory, the second quantity in the inventory represented in the 3D model of the planned layout; and

comparing the first quantity and the second quantity.

9. The method of claim 8, wherein receiving image data comprises capturing image data of products in a customer display location and in a storage location.

10. The method of claim 8, wherein retrieving a second quantity in the inventory comprises retrieving the second quantity from a transaction accounting system.

11. The method of claim 8, wherein comparing the first quantity and the second quantity comprises calculating the difference of the first quantity and the second quantity, wherein the difference indicates a difference between a quantity of inventory calculated to be in a store and a quantity of inventory actually in the store.

12. The method of claim 1, wherein receiving the image data of a real world layout comprises capturing the image data of the real world layout with a plurality of cameras.

13. The method of claim 12, wherein receiving the image data of the real world layout comprises receiving the image data from the plurality of cameras positioned to be able to generate the 3D model of the real world layout.

14. The method of claim 1, further comprising:

adjusting the received image data to match properties of the 3D model, wherein the properties include one or more of lighting characteristics, contrast, hue, shadow placement, and shadow removal.

15. The method of claim 1, further comprising:

generating an instruction to conform the real world layout to the planned layout of the environment.

16. The method of claim 1, further comprising:

receiving audio data of the real world layout, the audio data reflecting at least one of sounds, voice, or music from the real world layout;

comparing a sound model of the planned layout of the environment to the received audio data, the sound model representing the planned layout; and

determining whether results of the comparison reach a particular threshold.

17. A system comprising:

a reception module configured to receive image data of a real world layout, the image data reflecting the real world layout across three dimensions, each of the three dimensions having an image range with a beginning and an end, and the real world layout having inventory distributed across the three dimensions; and

a comparison module configured to compare a 3D model of a planned layout of the real world layout to the received image data, the 3D model representing the planned layout across the three dimensions; and

a threshold determination module configured to determine whether results of the comparison reach a first particular threshold.

18. The system of claim 17, further comprising:

an grouping extraction module configured to extract, from the image data, at least one product display grouping; and

an association module configured to associate the extracted at least one product display grouping with at least one representation of the product display grouping in the 3D model of the planned layout, wherein the comparison module is further configured to compare the at least one product display groupings to at least one representation of the product display grouping, and the threshold determination module is further configured to determine whether the results meet a second particular threshold.

19. The system of claim 17, further comprising an alert generation module configured to generate an alert if the comparison of the 3D model to the received image data reaches the first particular threshold, wherein the comparison represents similarity or non-similarity and the first particular threshold represents a particular degree of similarity.

20. The system of claim 17, further comprising a model generation module configured to generate the 3D model of a planned layout.

21. The system of claim 17, wherein the comparison module is further configured to compare the 3D model to the received image data by calculating an image difference.

22. The system of claim 17, further comprising:

a conversion module configured to convert the 3D model to a 2D representation; and

wherein the comparison module is further configured to compare the 2D representation of the 3D model to the received image data.

23. The system of claim 17, wherein the comparison module further includes a conversion module configured to convert the received image data to a rendered 3D model, wherein the comparison module is further configured to compare the 3D model of the planned layout to the rendered 3D model.

24. The system of claim 17, wherein the comparison module comprises:

an analysis module configured to analyze the image data to determine a first quantity in an inventory; and

an inventory reception module configured to retrieve a second quantity in an inventory, and wherein the comparison module is further configured to compare the first quantity and the second quantity.

25. The system of claim 24, wherein the image capture module is further configured to receive image data of products in a customer display location and in a storage location.

26. The system of claim 24, wherein the inventory reception module is further configured to retrieve the second quantity of the products from a transaction accounting system.

27. The system of claim 24, wherein the comparison module is further configured to calculate the difference of the first and second inventories, wherein the difference indicates a difference between a quantity of inventory calculated to be in a store and a quantity of inventory actually in the store.

28. The system of claim 17, wherein the reception module is further configured to receive the image data of a real world layout from a plurality of cameras.

29. The system of claim 28, wherein the reception module is further configured to receive the image data from the plurality of cameras positioned to be able to generate a 3D model of the real world layout.

30. The system of claim 17, further comprising:

an image adjustment module configured to adjust the image data to match properties of the 3D model, wherein the properties include one or more of lighting characteristics, contrast, hue, shadow placement, and shadow removal.

31. The system of claim 17, further comprising:

an instruction generation module configured to generate an instruction to conform the real world layout to the planned layout of the environment.

32. The system of claim 17, wherein:

the reception module is further configured to receive audio data of the real world layout, the audio data reflecting at least one of sounds, voice, or music from the real world layout;

the comparison module is further configured to compare a sound model of the planned layout of the environment to the received audio data, the sound model representing the planned layout; and

the threshold determination module is further configured to determine whether results of the comparison reach a particular threshold.

33. A computer comprising:

a memory area configured to store image data reflecting a real world layout across three dimensions, each of the three dimensions having an image range with a beginning and an end, and the real world layout having inventory distributed across the three dimensions and a 3D model of a planned layout, the 3D model representing the planned layout across the three dimensions in said memory area; and

a processor coupled to said memory area, said processor configured to;

receive the image data of the real world layout;

compare the 3D model of the planned layout to the received image data; and

determine whether results of the comparison reach a particular threshold.