Method of merchandising for checkout lanes

• US 20050189412A1
• Filed: 02/02/2005
• Published: 09/01/2005
• Est. Priority Date: 02/27/2004
• Status: Active Grant

• Current Assignee: Datalogic Adc, Inc. (Datalogic S.p.A.)
• Original Assignee: Evolution Robotics Inc. (iRobot Corporation)
• Inventors: Cremean, Michael, Hudnut, Alec, Simonini, Alex, Morgan, Howard
• Granted Patent: US 7,246,745 B2
• Time in Patent Office: Days
• US Class Current: 235/383
• CPC Class Codes: A47F 9/046 Arrangement of recording means in or on check-out counters record carriers or readers per se G06K
• CPC Class Codes: G07F 7/02 by keys or other credit registering devices
• CPC Class Codes: G07G 1/0081 the reader being a portable scanner or data reader G07G1/009 takes precedence
• CPC Class Codes: G07G 3/003 Anti-theft control theft alarms G08B13/00

Abstract

Methods and computer readable media for recognizing and identifying items located on the belt of a counter and/or in a shopping cart of a store environment for the purpose of reducing/preventing bottom-of-the-basket loss, checking out the items automatically, reducing the checkout time, preventing consumer fraud, increasing revenue and replacing a conventional UPC scanning system to enhance the checking out speed. The images of the items taken by visual sensors may be analyzed to extract features using the scale-invariant feature-transformation (SIFT) method. Then, the extracted features are compared to those of trained images stored in a database to find a set of matches. Based on the set of matches, the items are recognized and associated with one or more instructions, commands or actions without the need for personnel to visually see the items, such as by having to come out from behind a check out counter or peering over a check out counter.

Claims

1. A method of increasing a rate of revenue at a point-of-sale, comprising:

(a) moving in a substantially horizontal direction an object past a visual sensor;

(b) receiving visual image data of the object;

(c) comparing the visual image data with data stored in a database to find a set of matches;

(d) determining if the set of matches is found; and

(e) sending a recognition alert, wherein the set of matches is used to expedite a transaction process at the point-of-sale.

2. The method of claim 1, wherein the transaction process comprises the steps of:

identifying automatically one or more bottom-of-the-basket (BoB) items prior to completing the transaction process at the point-of-sale.

3. The method of claim 2, wherein the transaction process further comprises:

retrieving automatically price information of BoB items to be included in a checkout transaction at the point-of-sale prior to completion of the transaction process.

4. The method of claim 3, wherein the transaction process includes an automatic checkout process at the point of sale.

5. The method of claim 1, further comprising:

analyzing the visual image data to extract one or more visual features.

6. The method of claim 5, wherein the step of comparing comprises:

finding a match for each of the one or more features;

associating a quality measure with the match, the quality measure being a match confidence that ranges from 0 to 100%; and

if the associated quality measure exceeds a predetermined threshold, including the match in the set of matches.

7. The method of claim 5, wherein the step of comparing comprises:

finding a match for each of the one or more features;

associating a quality measure with the match, wherein the quality measure is a match confidence that ranges from 0 to 100%;

selecting a particular match associated with a highest quality measure; and

including the particular match in the set of matches.

8. The method of claim 1, further comprising, prior to the step of sending a recognition alert:

(f) checking if each element of the set of matches is reliable; and

(g) if all elements of the set of matches are unreliable, repeating the steps (a)-(f).

9. The method of claim 8, wherein the step of checking comprises:

a step of recognizing each element of the set of matches for a plurality of process cycles or a step of computing a statistical probability that each of the one or more visual features can be recognized.

10. The method of claim 1, wherein the step of receiving visual image data comprises:

capturing a plurality of images;

comparing two consecutive ones of the plurality of images to detect a motion; and

if the motion is detected, taking later one of the two consecutive images.

11. A computer readable medium embodying program code with instructions for increasing a rate of revenue at a point-of-sale, said computer readable medium comprising:

program code for moving in a substantially horizontal direction an object past a visual sensor;

program code for receiving visual image data of the object;

program code for comparing the visual image data with data stored in a database to find a set of matches;

program code for determining if the set of matches is found; and

program code for sending a recognition alert, wherein the set of matches is used to expedite a transaction process at the point-of-sale.

12. The computer readable medium of claim 11, further comprising:

program code for analyzing the visual image data to extract one or more visual features.

13. The computer readable medium of claim 11, further comprising:

program code for checking if each element of the set of matches is reliable; and

program code for repeating operation of the program code for receiving visual image data to the program code for checking if each element of the set of matches is reliable.

14. A method of preventing merchandise fraud, comprising:

(a) receiving visual image data of merchandise to be checked out, said merchandise located in a shopping cart;

(b) comparing the visual image data with data stored in a database to find a set of matches;

(c) determining if the set of matches is found; and

(d) sending a recognition alert to a point-of-sale, wherein the recognition alert is used to prevent bottom-of-the-basket (BoB) fraud.

15. The method of claim 14, wherein the step of sending a recognition alert comprises:

ringing up one or more BoB items if a customer attempts to leave without paying for the one or more BoB items; and

preventing the customer from closing out a transaction until the one or more BoB items are acknowledged and accepted.

16. The method of claim 14, further comprising:

analyzing the visual image data to extract one or more visual features.

17. The method of claim 16, wherein the step of comparing comprises:

finding a match for each of the one or more features.

18. The method of claim 17, wherein the step of comparing further comprises:

associating a quality measure with the match, the quality measure being a match confidence that ranges from 0 to 100%; and

if the associated quality measure exceeds a predetermined threshold, including the match in the set of matches.

19. The method of claim 17, wherein the step of comparing further comprises:

associating a quality measure with the match, wherein the quality measure is a match confidence that ranges from 0 to 100%;

selecting a particular match associated with a highest quality measure; and

including the particular match in the set of matches.

20. The method of claim 14, further comprising, prior to the step of sending a recognition alert:

(e) checking if each element of the set of matches is reliable; and

(f) if all elements of the set of matches are unreliable, repeating the steps (a)-(e).

21. The method of claim 20, wherein the step of checking comprises:

a step of recognizing each element of the set of matches for a plurality of process cycles.

22. The method of claim 20, wherein the step of checking comprises:

a step of computing a statistical probability that each of the one or more visual features can be recognized.

23. The method of claim 14, further comprising, after the step of sending a recognition alert:

(e) determining if there are more objects to be checked out; and

(f) if the determination in the step (e) is positive, repeating the steps (a)-(e).

24. The method of claim 14, wherein the step of receiving visual image data comprises:

capturing a plurality of images.

25. The method of claim 24, wherein the step of receiving visual image data further comprises:

comparing two consecutive ones of the plurality of images to detect a motion; and

if the motion is detected, taking the later one of the two consecutive images.

26. The method of claim 14, wherein the point-of-sale is located in a self-checkout lane.

27. A computer readable medium embodying program code with instructions for preventing merchandise fraud, said computer readable medium comprising:

program code for receiving visual image data of merchandise to be checked out, said merchandise located in a shopping cart;

program code for comparing the visual image data with data stored in a database to find a set of matches;

program code for determining if the set of matches is found; and

program code for sending a recognition alert to a point-of-sale, wherein the recognition alert is used to prevent bottom-of-the-basket (BoB) fraud.

28. The computer readable medium of claim 27, further comprising:

program code for analyzing the visual image data to extract one or more visual features.

29. The computer readable medium of claim 27, further comprising:

program code for checking if each element of the set of matches is reliable; and

program code for repeating operation of the program code for receiving visual image data to the program code for checking if each element of the set of matches is reliable.

30. The computer readable medium of claim 27, further comprising:

program code for determining if there is any more object to be checked out; and

program code for repeating operation of the program code for receiving visual image data to the program code for determining if there is any more object to be checked out.

31. A method of automatically including merchandise in a checkout sale transaction to reduce checkout waiting in line time for a store customer, comprising:

(a) receiving visual image data of merchandise to be checked out, said merchandise located in a shopping cart;

(b) comparing the visual image data with data stored in a first database to find a set of matches;

(c) determining if the set of matches is found;

(d) retrieving merchandise information from a second database; and

(e) sending the merchandise information to a point-of-sale, wherein the merchandise information is included in a sale transaction automatically.

32. The method of claim 31, further comprising:

analyzing the visual image data to extract one or more visual features.

33. The method of claim 32, wherein the step of comparing comprises:

finding a match for each of the one or more features; and

associating a quality measure with the match, the quality measure being a match confidence that ranges from 0 to 100%.

34. The method of claim 33, wherein the step of comparing further comprises:

if the associated quality measure exceeds a predetermined threshold, including the match in the set of matches.

35. The method of claim 33, wherein the step of comparing further comprises:

selecting a particular match associated with a highest quality measure; and

including the particular match in the set of matches.

36. The method of claim 31, further comprising, prior to the step of sending the merchandise information:

(f) checking if each element of the set of matches is reliable.

37. The method of claim 36, further comprising:

(g) if all elements of the set of matches are unreliable, repeating the steps (a)-(f).

38. The method of claim 36, wherein the step of checking comprises the step of recognizing each element of the set of matches for a plurality of process cycles or the step of computing a statistical probability that each of the one or more visual features can be recognized.

39. The method of claim 31, further comprising, after the step of sending the merchandise information:

(f) determining if there are any more objects to be checked out.

40. The method of claim 39, further comprising

(g) if the determination in the step (f) is positive, repeating the steps (a)-(f).

41. The method of claim 31, wherein the step of receiving visual image data comprises:

capturing a plurality of images;

comparing two consecutive ones of the plurality of images to detect a motion; and

if the motion is detected, taking later one of the two consecutive images.

42. The method of claim 31, wherein the point-of-sale includes a checkout subsystem on a plug-and-play configuration or a non-interfering parallel processing basis and provides a cashier with the merchandise information.

43. A computer readable medium embodying program code with instructions for automatically including merchandise in a checkout sale transaction to reduce checkout waiting in line time for a store customer, said computer readable medium comprising:

program code for receiving visual image data of merchandise to be checked out, said merchandise located in a shopping cart;

program code for comparing the visual image data with data stored in a first database to find a set of matches;

program code for determining if the set of matches is found;

program code for retrieving merchandise information from a second database; and

program code for sending the merchandise information to a point-of-sale, wherein the merchandise information is included in a sale transaction automatically.

44. The computer readable medium of claim 43, further comprising:

program code for analyzing the visual image data to extract one or more visual features.

45. The computer readable medium of claim 43, further comprising:

program code for checking if each element of the set of matches is reliable; and

program code for repeating operation of the program code for receiving visual image data to the program code for checking if each element of the set of matches is reliable.

46. The computer readable medium of claim 43, further comprising:

program code for determining if there is any more object to be checked out; and

program code for repeating operation of the program code for receiving visual image data to the program code for determining if there is any more object to be checked out.

47. A method of monitoring behavior of a cashier, comprising:

comparing a detection log of one or more bottom-of-the basket (BoB) items with a transaction log of the one or more BoB items;

recording an action taken by the cashier to process each of the one or more BoB items; and

correlating the action over a predetermined period to characterize the behavior of the cashier.

48. A computer readable medium embodying program code with instructions for monitoring behavior of a cashier, said computer readable medium comprising:

program code for comparing a detection log of one or more bottom-of-the basket (BoB) items with a transaction log of the one or more BoB items;

program code for recording an action taken by the cashier to process each of the one or more BoB items; and

program code for correlating the action over a predetermined period to characterize the behavior of the cashier.

49. A method for processing at least one bottom-of-the-basket (BoB) item at a point-of-sale, comprising:

(a) receiving match data;

(b) displaying a BoB list using the match data, the BoB list including at least one BoB item;

(c) selecting a particular BoB item from the BoB list;

(d) determining if quantity of the particular BoB item needs to be changed;

(e) determining if the particular BoB item needs to be deleted from the BoB list;

(f) adding the particular BoB item to a transaction log;

(g) sending the particular BoB item to a transaction;

(h) deleting the particular BoB item from the BoB list; and

(i) determining if the transaction is finished.

50. The method of claim 49, further comprising:

modifying the quantity if the determination in the step (d) is affirmative.

51. The method of claim 49, further comprising, prior to the step (b):

determining if a cashier is ready to process the at least one BoB item.

52. The method of claim 49, wherein the quantity is set to 1 by default.

53. The method of claim 49, wherein the step (e) includes the step of querying whether a customer intends to purchase the particular BoB item.

54. A computer readable medium embodying program code with instructions for processing bottom-of-the-basket (BoB) items at a point-of-sale, said computer readable medium comprising:

program code for receiving match data;

program code for displaying a BoB list using the match data, the BoB list including at least one BoB item;

program code for selecting a particular BoB item from the BoB list;

program code for determining if quantity of the particular BoB item needs to be changed;

program code for determining if the particular BoB item needs to be deleted from the BoB list;

program code for adding the particular BoB item to a transaction log;

program code for sending the particular BoB item to a transaction;

program code for deleting the particular BoB item from the BoB list; and

program code for determining if the transaction is finished.

55. The computer readable medium of claim 54, further comprising:

program code for modifying the quantity.

56. The computer readable medium of claim 54, further comprising:

program code for determining if a cashier is ready to process the at least one BoB item.

57. A method of automatically including merchandise in a checkout sale transaction to increase revenue, comprising:

(a) receiving visual image data of merchandise to be checked out;

(b) analyzing the visual image data to extract one or more visual features;

(c) comparing the one or more visual features with feature data stored in a database to find a set of matches;

(d) determining if the set of matches is found;

(e) sending a recognition alert to a point of sale, wherein the recognition alert is used to prevent bottom-of-the-basket fraud; and

(f) sending merchandise information to the point-of-sale, wherein the merchandise information is included in a checkout sale transaction automatically.

58. The method of claim 57, wherein the step of comparing is based on a scale invariant feature transform (SIFT) method.

59. The method of claim 57, wherein the step of comparing comprises:

finding a match for each of the one or more features.

60. The method of claim 59, wherein the step of comparing further comprises:

associating a quality measure with the match.

61. The method of claim 60, wherein the step of comparing further comprises:

if the quality measure exceeds a predetermined threshold, including the match in the set of matches.

62. The method of claim 61, wherein the quality measure is a match confidence that ranges from 0 to 100%.

63. The method of claim 60, wherein the step of comparing further comprises:

selecting a particular match associated with a highest quality measure.

64. The method of claim 63, wherein the step of comparing further comprises:

including the particular match in the set of matches.

65. The method of claim 57, further comprising, prior to the step of sending a recognition alert:

computing a statistical probability that each of the one or more visual features can be recognized.

66. The method of claim 57, further comprising, prior to the step of sending a recognition alert:

(g) checking if each element of the set of matches is reliable.

67. The method of claim 66, further comprising:

(h) if all elements of the set of matches are unreliable, repeating the steps (a)-(g).

68. The method of claim 66, wherein the step of checking comprises:

recognizing each element of the set of matches for a plurality of process cycles.

69. The method of claim 57, wherein the step of receiving virtual image data comprises:

capturing a plurality of images.

70. The method of claim 69, wherein the step of receiving virtual image data further comprises:

comparing two consecutive ones of the plurality of images to detect a motion; and

if the motion is detected, taking the later one of the two consecutive images.

71. A computer readable medium embodying program code with instructions for automatically including merchandise in a checkout sale transaction to increase revenue, said computer readable medium comprising:

program code for receiving visual image data of merchandise to be checked out;

program code for analyzing the visual image data to extract one or more visual features;

program code for comparing the one or more visual features with feature data stored in a database to find a set of matches;

program code for determining if the set of matches is found;

program code for sending a recognition alert to a point of sale, wherein the recognition alert is used to prevent bottom-of-the-basket fraud; and

program code for sending merchandise information to the point-of-sale, wherein the merchandise information is included in a checkout sale transaction automatically.

72. The computer readable medium of claim 71, further comprising:

program code for computing a statistical probability that each of the one or more visual features can be recognized.

73. The computer readable medium of claim 72, further comprising:

program code for checking if each element of the set of matches is reliable; and

program code for repeating operation of the program code for receiving visual image data to the program code for checking if each element of the set of matches is reliable

74. A method of increasing accuracy in including merchandise in a checkout sale transaction to account for a store inventory, comprising:

(a) receiving visual image data of merchandise to be checked out;

(b) comparing the visual image data with data stored in a database to find a set of matches;

(c) determining if the set of matches is found; and

(d) sending a recognition alert to a point of sale, wherein the recognition alert is used to prevent bottom-of-the-basket fraud.

75. The method of claim 74, further comprising:

(e) retrieving automatically price information of the object to be included in the checkout sale transaction; and

(f) updating the store inventory upon completion of the checkout sale transaction.

76. The method of claim 74, further comprising:

(e) checking if each element of the set of matches is reliable; and

(f) if all elements of the set of matches are unreliable, repeating the steps (a)-(e).

77. The method of claim 76, wherein the step of checking comprises:

a step of recognizing each element of the set of matches for a plurality of process cycles.

78. The method of claim 76, wherein the step of checking comprises:

a step of computing a statistical probability that each of the one or more visual features can be recognized.

79. The method of claim 74, further comprising:

analyzing the visual image data to extract one or more visual features based on a scale invariant feature transform (SIFT) method.

80. The method of claim 74, wherein the step of comparing comprises:

finding a match for each of the one or more features; and

associating a quality measure with the match, the quality measure being a match confidence that ranges from 0 to 100%.

81. The method of claim 80, wherein the step of comparing further comprises:

if the associated quality measure exceeds a predetermined threshold, including the match in the set of matches.

82. The method of claim 80, wherein the step of comparing further comprises:

selecting a particular match associated with a highest quality measure; and

including the particular match in the set of matches.

83. The method of claim 74, wherein the step of receiving visual image data comprises:

capturing a plurality of images.

84. The method of claim 83, wherein the step of receiving visual image data further comprises:

comparing two consecutive ones of the plurality of images to detect a motion; and

if the motion is detected, taking later one of the two consecutive images.

85. A computer readable medium embodying program code with instructions for increasing accuracy in including merchandise in a checkout sale transaction to account for a store inventory, said computer readable medium comprising:

program code for receiving visual image data of merchandise to be checked out;

program code for comparing the visual image data with data stored in a database to find a set of matches;

program code for determining if the set of matches is found; and

program code for sending a recognition alert to a point of sale, wherein the recognition alert is used to prevent bottom-of-the-basket fraud.

86. The computer readable medium of claim 85, further comprising:

program code for analyzing the visual image data to extract one or more visual features.

87. The computer readable medium of claim 85, further comprising:

program code for retrieving automatically price information of the object to be included in the checkout sale transaction; and

program code for updating the store inventory upon completion of the checkout sale transaction.

88. The computer readable medium of claim 85, further comprising:

program code for checking if each element of the set of matches is reliable; and

program code for repeating operation of the program code for receiving visual image data to the program code for checking if each element of the set of matches is reliable.

89. A method of linking a visual image of merchandise to a checkout sale transaction, comprising:

(a) receiving visual image data of merchandise to be checked out;

(b) identifying the merchandise using the visual image data; and

(c) sending merchandise information to the point-of-sale, wherein the merchandise information is included in a checkout sale transaction automatically.

90. The method of claim 89, wherein the step of identifying automatically the merchandise comprises:

comparing the visual image data with data stored in a database to find a set of matches;

determining if the set of matches is found.

91. The method of claim 90, further comprising:

analyzing the visual image data to extract one or more visual features.

92. The method of claim 91, wherein the step of comparing comprises:

finding a match for each of the one or more features;

associating a quality measure with the match, the quality measure being a match confidence that ranges from 0 to 100%; and

if the associated quality measure exceeds a predetermined threshold, including the match in the set of matches.

93. The method of claim 91, wherein the step of comparing comprises:

finding a match for each of the one or more features;

associating a quality measure with the match, wherein the quality measure is a match confidence that ranges from 0 to 100%;

selecting a particular match associated with a highest quality measure; and

including the particular match in the set of matches.

94. The method of claim 89, further comprising, prior to the step of sending a recognition alert:

(d) checking if each element of the set of matches is reliable; and

(e) if all elements of the set of matches are unreliable, repeating the steps (a)-(d).

95. The method of claim 94, wherein the step of checking comprises the step of recognizing each element of the set of matches for a plurality of process cycles or the step of computing a statistical probability that each of the one or more visual features can be recognized.

96. The method of claim 89, wherein the step of receiving visual image data comprises:

capturing a plurality of images;

comparing two consecutive ones of the plurality of images to detect a motion; and

if the motion is detected, taking later one of the two consecutive images.

97. A computer readable medium embodying program code with instructions for linking a visual image of merchandise to a checkout sale transaction, said computer readable medium comprising:

program code for receiving visual image data of merchandise to be checked out;

program code for identifying the merchandise using the visual image data; and

program code for sending merchandise information to the point-of-sale, wherein the merchandise information is included in a checkout sale transaction automatically.

98. The computer readable medium 97, further comprising:

program code for analyzing the visual image data to extract one or more visual features based on a scale invariant feature transform (SIFT) method.

99. The computer readable medium 97, further comprising:

program code for checking if each element of the set of matches is reliable; and

program code for repeating operation of the program code for receiving visual image data to the program code for checking if each element of the set of matches is reliable.

100. A method for processing at least one bottom-of-the-basket (BoB) item at a point-of-sale, comprising:

(a) receiving visual image data of at least one BoB item;

(b) receiving match data;

(c) displaying a BoB list using the match data, the BoB list including at least one BoB item; and

(d) freezing a sale transaction until a human intervention is performed to include the BoB item in the transaction.

101. A computer readable medium embodying program code with instructions for processing at least one bottom-of-the-basket (BoB) item at a point-of-sale, said computer readable medium comprising:

program code for receiving visual image data of merchandise to be checked out;

program code for receiving match data

program code for displaying a BoB list using the match data, the BoB list including at least one BoB item; and

program code for freezing a sale transaction until a human intervention is performed to include the BoB item in the transaction.