Methods and systems for monitoring a retail shopping facility

• US 10,565,554 B2
• Filed: 05/03/2017
• Issued: 02/18/2020
• Est. Priority Date: 06/10/2016
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Inventors: Jones, Nicholaus A., Taylor, Robert J., Vasgaard, Aaron J., Jones, Matthew A.
• Primary Examiner(s): Schnurr, John R
• Application Number:
  US15/585,317
• Publication Number:
  US 20170357939A1
• Time in Patent Office: 1,021 Days
• CPC Class Codes: G06Q 10/0875 Itemisation or classification of parts, supplies or services, e.g. bill of materials
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H02K 7/1846 structurally associated with wheels or associated parts dynamos arranged in the wheel hub of cycles B62J6/12
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/65 Control of camera operation in relation to power supply

Abstract

In some embodiments, apparatuses and methods are provided herein useful for a retail shopping facility to maintain stocked shelves, in part, from obtaining images of the store shelves via cameras mounted onto shopping carts. By one approach, the images obtained from the shopping carts are compared with planogram images to determine where retail products are needed to restock the shelves. In some examples, the shopping cart has a control circuit that determines when to capture electronic images based, in part, upon a sensor mounted onto the shopping cart, instructs the camera to capture the electric image, and stores the captured electronic image in the memory.

Claims

1. A shopping cart apparatus comprising:

a cart frame with a plurality of cart wheels coupled to the cart frame; and

an image capturing assembly including;

an electronic image capturing device coupled to the cart frame, the electronic image capturing device configured to capture electronic images of shelves within a retail shopping facility, the electronic images usable to indicate a stock level of retail items within the retail shopping facility;

a memory configured to store at least some of the electronic images taken with the electronic image capturing device until the at least some of the electronic images are transferred to a central computer;

at least one sensor configured to measure cart conditions effecting image quality of the electronic images captured by the electronic image capturing device;

a power storage device configured to store electrical energy sufficient to periodically capture electronic images and store the at least some of the electronic images; and

a control circuit operably coupled to the electronic image capturing device, the memory, the at least one sensor, and the power storage device, the control circuit configured to;

determine when to capture the electronic images based, in part, on information from the at least one sensor;

instruct the electronic image capturing device to capture an electronic image; and

store the captured electronic image in the memory;

wherein the control circuit instructs the electronic image capturing device to capture the electronic image when the at least one sensor indicates that the cart frame is within a predetermined distance from one of the shelves and when the shopping cart apparatus is moving below a predetermined speed or the shopping cart apparatus is not moving.

2. The shopping cart apparatus of claim 1 further including a cart handle coupled to the cart frame and configured to be grasped by a user and the electronic image capturing device coupled to the cart handle.

3. The shopping cart apparatus of claim 2 wherein the at least one sensor comprises at least one of:

a proximity sensor configured to determine a current location of the cart frame relative to one of the shelves within the retail shopping facility;

an accelerometer configured to determine cart movement; and

a wheel power generator assembly coupled to at least one of the plurality of cart wheels, the wheel power generator assembly configured to convert kinetic energy from a rotation of the at least one of the plurality of cart wheels to electrical energy.

4. The shopping cart apparatus of claim 3 wherein the control circuit is further configured to transfer the at least some of the electronic images to the central computer and the shopping cart further includes at least one of:

a wireless transceiver to transfer the at least some of the electronic images to the central computer; and

a wired connection configured to transfer the at least some of the electronic images at a cart corral or a point of sale terminal.

5. The shopping cart apparatus of claim 3 wherein the control circuit is configured to capture the electronic images when readings from the at least one sensor are within a predetermined range and a predetermined amount of time or space has passed since a previous electronic image was captured.

6. The shopping cart apparatus of claim 5 wherein the control circuit is configured to capture the electronic image when the proximity sensor indicates that the cart frame is within a predetermined distance from one of the shelves and when at least one of the following occurs:

the accelerometer indicates that the shopping cart apparatus is moving below a certain speed;

the accelerometer indicates that the shopping cart apparatus is not moving;

the wheel power generator assembly is generating power below a certain amount; and

the wheel power generator assembly is not generating power.

7. The shopping cart apparatus of claim 3 wherein the proximity sensor is disposed at a lower portion of the cart frame or is disposed at the cart handle.

8. The shopping cart apparatus of claim 3 wherein the power storage device is charged via at least one of:

the wheel power generator assembly coupled to at least one of the plurality of cart wheels;

a direct electrical connection between the shopping cart apparatus and a cart corral; and

an electrical connection between the shopping cart apparatus and a charging pad at a point of sale terminal.

9. The shopping cart apparatus of claim 8 wherein the direct electrical connection between the shopping cart apparatus and the cart corral includes at least one of a wire brush connection, a cart extension contact or one of the cart wheels extending within a track, and a plug-in connection.

10. A shopping cart apparatus comprising:

a cart frame with a plurality of cart wheels coupled to a lower portion thereof; and

an image capturing assembly including;

an electronic image capturing device associated with the cart frame, the electronic image capturing device configured to capture electronic images of shelves within a retail shopping facility;

at least one sensor configured to measure cart conditions effecting image quality of the electronic images captured by the electronic image capturing device; and

a portable electronic device configured to be operably coupled to the electronic image capturing device and the at least one sensor, the portable electronic device configured to determine when to capture the electronic images based, in part, on information from the at least one sensor, instruct the electronic image capturing device to capture the electronic images; and

store the electronic image;

wherein the portable electronic device instructs the electronic image capturing device to capture an electronic image when the at least one sensor indicates that the cart frame is within a predetermined distance from one of the shelves and when the shopping cart apparatus is moving below a predetermined speed or the shopping cart apparatus is not moving.

11. The shopping cart apparatus of claim 10 wherein the portable electronic device has battery power and transmission capabilities.

12. The shopping cart apparatus of claim 10 wherein the portable electronic device is mountable to the cart frame.

13. A method of obtaining electronic images comprising:

coupling an image capturing assembly to a shopping cart, the shopping cart having a cart frame, a plurality of cart wheels, and a cart handle, the image capturing device having an electronic image capturing device associated with the cart frame and at least one sensor configured to measure cart conditions;

capturing a first electronic image with the electronic image capturing device and storing the electronic image;

sensing the cart conditions to determine whether cart speed and location are suitable for capturing electronic images;

determining whether sufficient time or distance have lapsed since the first electronic image is captured; and

capturing a second electronic image if sufficient time or distance have lapsed and if cart conditions are suitable for capturing electronic images;

wherein the step of sensing the cart conditions includes determining whether the accelerometer indicates that the shopping cart is moving below a certain speed and whether the wheel power generator is generating below a certain amount of power.

14. The method of claim 13 further comprising transferring the captured electronic images to a central computer and analyzing the captured electronic images to determine at least one of:

areas within the retail shopping facility requiring restocking; and

integrity of shelf displays compared with a planogram.

15. The method of claim 13 further comprising generating power via at least one of the plurality of cart wheels with a wheel power generator assembly configured to convert kinetic energy from a rotation of the at least one of the plurality of cart wheels to electrical energy.

16. The method of claim 13 further comprising powering the electronic image capturing assembly, in part, by at least one of:

a wheel power generator assembly coupled to one of the plurality of cart wheels, a direct electrical connection between the shopping cart and a cart corral, and an electrical connection between the shopping cart and a charging pad at a point of sale terminal.

17. The method of claim 13 further comprising processing captured images via image contrast enhancement or dynamic range reduction.