COMPREHENSIVE NO-TOUCH INVENTORY MANAGEMENT SYSTEM
A system directed to inventory in an enclosure employs sensors and data processing to provide real time inventory management without requiring additonal user action. Adaptable to storage units such as refrigerators, cabinets and pantries, and particularly useful for tracking perishable items, object recognition rechnology generates time-stamped data identifying objects as they are placed in and removed from the enclosure, thereby enabling an associated database to maintain inventory of objects in the enclosure in real time. Embodiments further comprise sensors providing weight data for an object as it is placed in the enclosure, enabling the inventory database to track item weights. The system database enables software means to provide users actionable information such as expiration alerts, recipes, shopping lists, dietary recommendations and other information concerning food items in the enclosure.
- 1. -12 (canceled)
- 13. A system for managing inventory in a storage unit, comprising:
at least one image sensor disposed within the storage unit, the image sensor configured to transmit image data of the contents of the storage unit; an image processing system configured to receive the image data from the at least one image sensor, the image processing system further comprising means for processing the image data to derive item identification data identifying items by item type contained in the storage unit in real-time, the image processing system further configured to transmit the item identification data; a data processing system configured to receive item identification data from the image processing system, the data processing system further configured to provide an inventory database with item identification data, the inventory database containing a listing of each item by item type in the storage unit; and a user interface to the system for managing inventory.
- View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21, 22)
- 23. A method for managing inventory in a storage unit, comprising the steps of
obtaining image data of the contents of the storage unit, processing the image data to derive item identification data, updating an item inventory database with item identification data, and providing at least one of a user interface and a provider interface to the item inventory database.
This application claims the benefit of U.S. provisional patent application No. 62/654,522, filed Apr. 8, 2018.
This invention relates to inventory management systems. More specifically, this invention relates to systems adapted to automatically manage an inventory of items, such as foodstuffs, which may have limited usable lifespans.
It has been estimated that the average household in a developed nation wastes 40% of their yearly food purchases. Such waste is largely the result of inadequate management of inventories of perishable food items.
Earliest efforts to manage perishable inventories entailed tedious manual tabulation of items as they were entered into and removed from storage. Some more recent efforts to automate inventory management have been limited to items packaged with bar codes such as the Universal Product Code, enabling an inventory management system to identify and track such items by scanning them as they move in and out of inventory. Other recent efforts have required users to label each item in inventory with a radio frequency identification marker to enable automated item identification and tracking. More recent efforts to apply artificial intelligence technology to the task have been limited to particular kinds of items such as fresh vegetable produce.
What is needed is a system for tracking perishable items automatically as they are placed in and removed from storage in an enclosure such as a refrigerator, cabinet or pantry. What is needed further is such a system that tracks such items in the form in which they are acquired by the user, without requiring additional labeling or other intervention by the user. Further, what is needed is such a system with universal capability, adaptively tracking perishable items of all kinds. Yet further, what is needed is an automated system that provides users with actionable information and functionality to optimize the management of perishable foods or other items with limited lifespans.
This system uses at least one sensor such as a camera or lidar scanner disposed within a storage unit such as a refrigerator, cabinet or pantry, to acquire data on the contents of the storage unit. The system applies object identification technology to processes the data thus acquired, automatically identifying items as they are placed in and removed from the storage unit. Time-stamped records of the automatically identified items are recorded in a system database. Embodiments further include weight sensors enabling item records further to include item weight data. A user interface enables actionable and supervisory information exchange between the user and the system. A provider interface enables item ordering transactions with item providers.
Objects of the present invention as well as advantages, features and characteristics, in addition to methods of operation, function of related elements of structure, and the combination of parts and economies of manufacture, will become apparent upon consideration of the following description and claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures, and wherein:
In a typical embodiment of the invention, one or more image sensors are disposed within the target storage unit. Image sensors may be cameras, lidar scanners, sonar, radar or other devices suitable for obtaining a representation of the contents of the storage unit in the form of image data. Sensors are so situated within the storage unit that the image obtained by the sensors encompasses objects stored within the storage unit. Image sensors may be statically mounted to receive a constant perspective of the interior of the storage unit. Alternatively, image sensors may employ reciprocating motors to pivot and scan wide areas of the storage unit.
The image sensors are in communication with image processing equipment which receives storage unit content image data from the sensors. It will be understood by those of skill in the art that the image processing equipment associated with this system may be local to the storage unit, or that image processing equipment may be located distantly from the storage unit, receiving image sensor data via network or other transmission means. In yet other embodiments, the system image processing functionality may be widely distributed over equipment and networks employed by the system for this purpose.
The image processing functionality of this system operates on image data received from the sensors to recognize objects in the storage unit. The system latency for object recognition is on the order of seconds rather than minutes, sufficiently short as to be characterized as real-time. Embodiments may practice object recognition by image processing employing software systems that have been trained in the deep learning technologies of artificial intelligence, relying, for example, on convolutional network architecture, in some cases Space Displacement Neural Networks in particular, to ensure some degree of shift, scale and distortion invariance in the process of image recognition training. In some such systems, a regularized linear model of a three dimensional object is created in software and the model is associated with the identity of the object. Thereby an initial dataset is created that correlates idealized representations of image data with object identity. Dataset augmentation is carried out by presenting an object recognition software system in training with various transformations of object representations that do not change the class of object presented. Such transformations include random rotations, translations and in some cases random perturbation of colors and nonlinear distortions of the object representations. After many iterations of dataset augmentation, the image recognition software system has been trained to identify objects represented in the image data, reliably and in real or near-real time.
While it will be recognized by those of skill in the art that many different programming and training approaches employing a range of supporting architectures can be used to implement object recognition in a software system, any image data processing means for providing reliable near-real time object identification from the system'"'"'s image data, without requiring user input or labeling of items, is adaptable to be incorporated in this system. In accordance with the teachings of this invention, then, system image processing receives image data as input and, through object recognition technology, provides item identification data as output.
A system inventory database receives item identification data from the image processing equipment, creating inventory data records as items are placed in and removed from the storage unit. Thereby, the system maintains and updates its database, providing real-time inventory of items in the storage unit. By time-stamping inventory data records, the system can track item aging. Embodiments further include weight sensors, enabling item weight data to be included in the inventory data record for each item.
A user interface to the system inventory database enables a user to determine the contents of the storage unit at any time. The user interface can aggregate data on like or identical items in storage to report an inventory of unique items to the user. When the system includes time-stamping of inventory data records, the user interface can provide the user with information regarding the aging of perishable items in the storage unit. Additional software functionality enables the system to provide the user with alerts regarding deadlines for perishable items and shortfalls of item quantities. Other software may provide the user with item usage suggestions such as recipes and nutritional recommendations. Yet other software in conjunction with a provider interface may enable manual or automatic (no-click) ordering of new items from item providers based upon inventory data and/or user specified parameters such as desired replenishment frequency.
Referring now to the figures, wherein like numerals indicate like or corresponding parts throughout the several views, an exemplary prior art refrigerator is generally shown at
Weight data 20, item identification data 28 and time stamp data 30 are communicated to system inventory database 32 for database processing 34. Database processing 34 determines 36 whether the item is new to the inventory or whether an item of the same type is already existing in inventory. If the item is new, its initial weight is recorded 42, a database record is created for the item and an expiration date is calculated 46, optionally using expiration data from an external source 48, the expiration date then associated with the database record for the item.
If database processing 34 determines 36 that the item is of the same sort as other items existing in inventory, then the total weight of those items is calculated 38 and the database record for that item type in inventory is updated 40.
An exemplary user action made possible with this system is illustrated in
Having calculated and stored the expiration date for an item in the inventory database (
The system may determine that restocking of items is required, for example when the inventory database indicates that quantities of an item have fallen below a certain pre-set threshold, or, as shown at 58 in
The system can further assist the user in optimizing a user meal plan. Turning to
In the depicted embodiment, a user interface 111 is provided to enable server 110 to interact with client 130 via the Internet over the World Wide Web to provide the user with supervisory and actionable inventory management capabilities via common network client functionalities such as web browser 124 and smart phone app 126. It will be appreciated by those of skill in the art, however, that the system may use various communication channels, such as local area networks (LANs), wide area networks or peer-to-peer systems, alone or in combination, for communication between the various system components in keeping with the spirit of the present invention.
The depicted embodiment also comprises a provider interface 123, employing a communication channel to interact with external providers of items 132. Provider interface 123 enables ordering of new inventory items from providers 132, either as directed by the user via user interface 111 or automatically under control of server engine 112 such as when shortfalls are indicated in inventory database 114.
Persons of skill in the art will recognize that user interface 111 need not be implemented in whole within server 110 but rather may at least be partially implemented in client 130. Persons of skill in the art will further recognize that server 110 may communicate with a provider client system (not illustrated) at a provider 132 whereby part or all of provider interface 123 is implemented in such provider client system.
Although the present system has been described in terms of various embodiments, it is not intended that the system be limited to these embodiments. For example, the system can be utilized for inventory management in other environments where no-touch inventory recording would be preferred. The data created by the system may also be used in different ways. For example, new items can be announced via voice command spoken to the user.
While the invention has been described with a certain degree of particularity, it should be recognized that elements thereof may be altered by persons skilled in the art without departing from the spirit and scope of the invention. Accordingly, the present invention is not intended to be limited to the specific forms set forth herein, but on the contrary, it is intended to cover such alternatives, modifications and equivalents as can be reasonably included within the scope of the invention. The invention is limited only by the following claims and their equivalents.