Computer vision navigation

• US 9,354,066 B1
• Filed: 11/25/2014
• Issued: 05/31/2016
• Est. Priority Date: 11/25/2014
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Wal-Mart Stores Texas LLC (WalMart Inc.)
• Inventors: Gupta, Ankit, Shah, Saurav, Sowdaboina, Venkata Pranay Kumar
• Primary Examiner(s): CHEUNG, CALVIN K
• Application Number:
  US14/553,510
• Publication Number:
  US 20160146614A1
• Time in Patent Office: 553 Days
• US Class Current: 1/1
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/024 Guidance services

Abstract

A system for computer-aided visual recognition of products may help a customer navigate to a selected product in a retail store. A customer can direct a mobile computing device camera at one or more targeted products displayed on store shelves. Image recognition operations can be carried out to compare the targeted product image(s) against images from a prepopulated product image store of known products. Upon a positive match, shelf location information of the identified targeted product(s) may be used to determine the current location of the customer. An in-store destination location is defined by the shelf location for the selected product. A route may be calculated to navigate the customer from the current location to the destination location.

Claims

1. A computer-implemented method for navigating to a selected product, comprising:

receiving, at a processing device of a navigation server, one or more signals from a mobile computing device corresponding to one or more images of a targeted product;

receiving at least one image from a product image store;

determining, at the processing device, that the one or more images of the targeted product matches the at least one image stored at the product image store by;

comparing, at the processing device, the one or more images of the targeted product to the at least one image stored at the product image store, thereby resulting in a comparison score; and

determining, at the processing device, that the comparison score meets a minimum score threshold;

in response to the comparison score meeting the minimum score threshold, recalling, from a store map database, information regarding an in-store location of the targeted product;

mapping a current location based on the in-store location of the targeted product; and

transmitting, from the processing device, directions and actively navigating from the current location to the in-store location of the targeted product.

2. The method of claim 1, wherein the information regarding the in-store location of the targeted product is selected from the group consisting of:

an aisle number, a bay number, a display number, a shelf number, a location coordinate, and combinations thereof.

3. The method of claim 1, further comprising initially defining the current location as an out-of-store location.

4. The method of claim 1, wherein determining, at the processing device, if the one or more images of the targeted product matches at least one image stored at the product image store further comprises applying a feature detection algorithm to the one or more images of the targeted product.

5. The method of claim 4, wherein applying the feature detection algorithm to the one or more images of the targeted product comprises carrying out a Speeded-Up Robust Features (SURF) feature detection algorithm.

6. The method of claim 1, wherein determining, at the processing device, if the one or more images of the targeted product matches at least one image stored at the product image store further comprises:

splitting the one or more images of the targeted product into multiple constituent images and comparing each of the multiple constituent images to the at least one image stored at the product image store.

7. The method of claim 1, further comprising calculating a route from the current location to the in-store location of the targeted product.

8. A computer-implemented system for navigating to a selected product comprising:

a navigation server comprising a memory device and a processing device, the memory device storing computer-readable instructions directing the processing device to;

receive one or more signals from a mobile computing device corresponding to one or more images of a targeted product;

receive at least one image from a product image store;

determine that the one or more images of the targeted product matches the at least one image stored at the product image store by;

comparing the one or more images of the targeted product to the at least one image stored at the product image store, thereby resulting in a comparison score; and

determining that the comparison score meets a minimum score threshold;

in response to the comparison score meeting the minimum score threshold, obtain, from a store map database, information regarding an in-store location of the targeted product;

map a current location based on the in-store location of the targeted product; and

transmit directions and actively navigate from the current location to the in-store location of the targeted product.

9. The system of claim 8, wherein the information regarding the in-store location of the targeted product is selected from the group consisting of:

an aisle number, a bay number, a display number, a shelf number, a location coordinate, and combinations thereof.

10. The system of claim 8, wherein the computer-readable instructions further direct the processing device to initially define the current location as an out-of-store location.

11. The system of claim 8, wherein the computer-readable instructions further direct the processing device to apply a feature detection algorithm to the one or more images of the targeted product.

12. The system of claim 11 wherein the computer-readable instructions further direct the processing device to carry out a Speeded-Up Robust Features (SURF) feature detection algorithm.

13. The system of claim 8, wherein the computer-readable instructions further direct the processing device to:

split the one or more images of the targeted product into multiple constituent images and

compare each of the multiple constituent images to the at least one image stored at the product image store.

14. The system of claim 8, wherein the computer-readable instructions further direct the processing device to calculate a route from the current location to the in-store location of the targeted product.

15. A system, comprising:

a plurality of computer-readable instructions in the form of an application executable by a processor of a mobile computing device to;

generate one or more signals corresponding to one or more images of a targeted product; and

a navigation server comprising a memory device and a processing device, the memory device storing computer-readable instructions directing the processing device to;

receive the one or more signals corresponding to the one or more images of the targeted product from the application of the mobile computing device;

receive at least one image from a product image store;

determine that the one or more images of the targeted product matches the at least one image stored at the product image store by;

comparing the one or more images of the targeted product to the at least one image stored at the product image store, thereby resulting m a comparison score; and

determining that the comparison score meets a minimum score threshold,

in response to the comparison score meeting the minimum score threshold, obtain, from the store map database, information regarding an in-store location of the targeted product;

map a current location based on the in-store location of the targeted product; and

transmit directions to the application of the mobile device and actively navigate from the current location to the in-store location of the targeted product.

16. The system of claim 15, wherein the information regarding the in-store location of the targeted product is selected from the group consisting of:

an aisle number, a bay number, a display number, a shelf number, a location coordinate, and combinations thereof.

17. The system of claim 15, wherein the computer-readable instructions further direct the processing device to initially define the current location as an out-of-store location.

18. The system of claim 15, wherein the computer-readable instructions further direct the processing device to apply a feature detection algorithm to the one or more images of the targeted product.

19. The system of claim 18, wherein the computer-readable instructions further direct the processing device to carry out a Speeded-Up Robust Features (SURF) feature detection algorithm.

20. The system of claim 15, wherein the computer-readable instructions further direct the processing device to:

split the one or more images of the targeted product into multiple constituent images and

compare each of the multiple constituent images to the at least one image stored at the product image store.