Self-service method and device

• US 10,636,024 B2
• Filed: 05/21/2018
• Issued: 04/28/2020
• Est. Priority Date: 11/27/2017
• Status: Active Grant

• Current Assignee: Shanghai Yuepu Investment Center
• Original Assignee: Shenzhen Malong Technologies Co., Ltd.
• Inventors: Scott, Matthew Robert, Huang, Dinglong, Fu, Kai
• Primary Examiner(s): Hayles, Ashford S
• Application Number:
  US15/985,413
• Publication Number:
  US 20190164142A1
• Time in Patent Office: 708 Days
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06K 7/1092 sensing by means of TV-scanning
• CPC Class Codes: G06Q 20/208 Input by product or record sensing, e.g. weighing or scanner processing
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G07G 1/0036 Checkout procedures
• CPC Class Codes: G07G 1/0072 with means for detecting the weight of the article of which the code is read, for the verification of the registration weighing combined with recording means, e.g. in weighing scale, G01G19/415
• CPC Class Codes: G07G 1/0081 the reader being a portable scanner or data reader G07G1/009 takes precedence

Abstract

Provided are a self-service method and device. The method includes: acquiring by a camera video data associated with a shopping cart at a distance from a shelf less than a preset value; determining whether an article is being placed into the shopping cart, based on identification of the video data; if yes, acquiring an ID of the shopping cart, determining the placed article and associating the ID with the article; displaying a total sum of all articles associated with the ID to the user when detecting the shopping cart corresponding to the ID is located in a preset payment area in the shopping mall; and executing a permitting procedure for allowing the shopping cart corresponding to the ID to pass upon reception of a message fed back from the payment area and indicating that the total sum corresponding to the ID has been cleared.

Claims

1. A computer-implemented method, comprising:

in response to a shopping cart being within a preset distance from a shelf, acquiring video data associated with the shopping cart;

determining, based on the video data, an article being placed into the shopping cart;

determining one or more product types associated with the shelf at a present period of time;

retrieving pre-stored images of the one or more product types;

identifying a product type of the article based on a two-stage matching procedure, comprising (1) performing a first matching procedure between an image of the article and the pre-stored images of the one or more product types associated with the shelf at the present period of time; and

(2) in response to a nonmatch at the first matching procedure, performing a second matching procedure between the image of the article and pre-stored images of all product types;

acquiring an ID of the shopping cart;

associating the ID of the shopping cart with the product type of the article;

in response to the shopping cart being in a payment area, displaying to a user the product type of the article based on the ID of the shopping cart; and

executing a permitting procedure for allowing the shopping cart to pass the payment area based on a payment of the article.

2. The method according to claim 1, further comprising:

acquiring the ID of the shopping cart from an electronic tag reader associated with the shelf.

3. The method according to claim 1, further comprising:

capturing the image of the article from the video data.

4. The method according to claim 1, further comprising:

capturing the image of the article directly from the article.

5. The method according to claim 1, further comprising:

in response to another nonmatch in the second matching procedure, generating a prompt message to prompt the user to make payment in an alternative method.

6. A system, comprising:

a camera configured to acquire, in response to a shopping cart being within a preset distance from a shelf, video data associated with the shopping cart;

a determining module configured to determine, based on the video data, an article being placed into the shopping cart;

an associating module configured to identify a product type of the article based on a two-stage matching procedure, comprising (1) performing a first matching procedure between an image of the article and pre-stored images of one or more product types associated with the shelf at a present period of time; and

(2) in response to a nonmatch at the first matching procedure, performing a second matching procedure between the image of the article and pre-stored images of all product types;

a displaying module configured to display, in response to the shopping cart being in a payment area, to a user the product type of the article; and

an executing module configured to execute a permitting procedure for allowing the shopping cart to pass the payment area based on a payment of the article.

7. The system according to claim 6, wherein

the associating module is further configured to

acquire an ID of the shopping cart via an electronic tag reader.

8. The system according to claim 6, wherein the camera is further configured to

shoot the article to capture the image of the article.

9. The system according to claim 6, wherein the camera is further configured to capture the image of the article from the video data.

10. The system according to claim 6, further comprising:

a prompting module configured to generate a prompt message to prompt the user to make payment in an alternative method.

11. A non-transitory computer-readable medium storing a program causing a computer to execute a process, the process comprising:

in response to a shopping cart being within a preset distance from a shelf, acquiring video data associated with the shopping cart;

determining, based on the video data, an article being placed into the shopping cart;

capturing an image of the article from the video data;

determining one or more product types associated with the shelf at a present period of time;

retrieving pre-stored images of the one or more product types;

identifying a product type of the article based on a two-stage matching procedure, comprising (1) performing a first matching procedure between the image of the article and the pre-stored images of the one or more product types associated with the shelf at the present period of time; and

(2) in response to a nonmatch at the first matching procedure, performing a second matching procedure between the image of the article and pre-stored images of all product types;

acquiring an ID of the shopping cart;

associating the ID of the shopping cart with the product type of the article;

in response to the shopping cart being in a payment area, displaying to a user the product type of the article based on the ID of the shopping cart; and

enabling, based on the product type of the article, the user to make a payment of the article.

12. The medium according to claim 11, the process further comprising:

executing a permitting procedure for allowing the shopping cart to leave the payment area based on the payment of the article.