REFRIGERATOR AND METHOD OF FOOD MANAGEMENT THEREOF

• US 20180335252A1
• Filed: 05/17/2018
• Published: 11/22/2018
• Est. Priority Date: 05/18/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Oh, Jeong-il
• Granted Patent: US 10,724,792 B2
• Time in Patent Office: Days
• CPC Class Codes: F25D 23/028 Details
• CPC Class Codes: F25D 2400/361 Interactive visual displays
• CPC Class Codes: F25D 2500/06 Stock management
• CPC Class Codes: F25D 2700/04 Sensors detecting the presence of a person
• CPC Class Codes: F25D 2700/06 Sensors detecting the presence of a product
• CPC Class Codes: F25D 29/00 Arrangement or mounting of control or safety devices
• CPC Class Codes: F25D 29/005 Mounting of control devices
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables

Abstract

A refrigerator and a food management method are provided. The refrigerator includes a main body that includes a storage. The refrigerator also includes a door configured to be hinged to the main body to rotate on a front side of one side of the storage and include a display. The refrigerator also includes a camera positioned on a front surface of the door and configured to photograph food to be recognized. The refrigerator also includes a controller configured to control the display, and the first camera. The controller is configured to display on the display a food name corresponding to an image of food, photographed by the first camera.

Claims

1. A refrigerator, comprising:

a main body including a storage;

a door configured to be hinged to the main body to rotate on a front side of one side of the storage and include a display on a front surface of the door;

a first camera be positioned on a front surface of the door and configured to photograph food to be recognized; and

a controller configured to;

control the display, and

control the first camera,

wherein the controller displays on the display a food name corresponding to an image of food, photographed by the first camera.

2. The refrigerator as claimed in claim 1, further comprising:

a user approach detector configured to detect an approach of a user with respect to the refrigerator,

wherein the user approach detector includes at least one of a microphone, the first camera and a proximity sensor.

3. The refrigerator as claimed in claim 2, wherein the controller is configured further to display a food management application screen on the display in response to detecting the approach of the user.

4. The refrigerator as claimed in claim 3, wherein the controller is further configured to recognize the image of the food photographed by the first camera through a learning model by using the food management application.

5. The refrigerator as claimed in claim 4, further comprising:

a memory, wherein the memory is configured to store training data corresponding to the learning model,

wherein the controller is further configured to recognize the image of the food through the learning model.

6. The refrigerator as claimed in claim 5, further comprising:

a communication interface, wherein the communication interface is configured to be connected with an external apparatus,

wherein to recognize the image of the food, the training data is stored in at least one of the memory, an external server, a server of an external store, and a web search engine.

7. The refrigerator as claimed in claim 6, wherein the controller is further configured to recognize the image of the food through the learning model in a determined order of the memory, the external server, a server of an external store and web search engine based on the stored training data.

8. The refrigerator as claimed in claim 2, wherein the proximity sensor is positioned to be adjacent to the first camera.

9. The refrigerator as claimed in claim 1, wherein the first camera is positioned to be inclined from a front surface of the door at a set angle.

10. The refrigerator as claimed in claim 1, wherein the controller is further configured to a preview the food by using the first camera.

11. The refrigerator as claimed in claim 1, wherein the controller is further configured to:

display on the display, food information corresponding to the food that is recognized, and

wherein the food information includes at least two or more of a name of the food, a location where the food is stored, a date that the food is stored and a period that the food is stored.

12. The refrigerator as claimed in claim 11, wherein the controller is further configured to display through the display, the location where the food is stored in the storage by using a second camera to photograph the storage.

13. The refrigerator as claimed in claim 12, wherein the controller is further configured to display on the display, a storage period of the food, in response to the food stored at the location in the storage.

14. The refrigerator as claimed in claim 11, wherein the controller is further configured to display on the display, an arrival of a storage period limit of the food by using a storage period of the food information.

15. The refrigerator as claimed in claim 13, wherein the controller is further configured to display on the display, an arrival of a storage period limit of the food by using the storage period of the food information.

16. A food management method of a refrigerator, the method comprising:

detecting a user approach through a user approach detector positioned on a front surface of a door of the refrigerator;

displaying a screen of a food management application on a display positioned on the front surface of the door of the refrigerator in response to the user approach being detected;

photographing food to be recognized through a first camera, wherein the first camera is adjacent to the display;

recognizing a food image corresponding to the photographed food; and

displaying food information corresponding to the food on the display according to a result of the recognition.

17. The food management method as claimed in claim 16, further comprising:

distinguishably displaying a location at which the food is stored in a storage within the refrigerator according to the food information.

18. The food management method as claimed in claim 17, wherein the location at that the food is stored is distinguishably displayed on an inside of the storage photographed by a second camera.

19. The food management method as claimed in claim 16, wherein to photograph the food to be recognized comprises displaying a preview of the food on the display prior to the photographing.

20. The food management method as claimed in claim 16, further comprising:

displaying an arrival of a storage period limit of the food based on the food information.

21. A refrigerator, comprising:

a main body including a storage;

a memory configured to store a first comparison food image;

a door configured to be hinged to the main body to rotate on a front side of one side of the storage and include a display on a front surface of the door;

a camera configured to be positioned on a front surface of the door and photograph food to be recognized; and

a controller configured to control the memory, the display and the camera, wherein the controller is configured to;

preferentially recognizes a food image photographed by the camera according to a food recognition search order by using the first comparison food image of the memory, and

display on the display a food name corresponding to the photographed food image.

22. The refrigerator as claimed in claim 21, further comprising:

a communication interface,

wherein the controller is further configured to;

recognize the food image photographed by the camera according to the food recognition search order based on a second comparison food image stored in a memory of an external server connected via the communication interface, and

display on the display a food name corresponding to the photographed food image.

23. The refrigerator as claimed in claim 22, wherein the controller is further configured to:

recognize the food image photographed by the camera according to the food recognition search order by using a third comparison food image stored in a memory of a store server connected via the communication interface; and

display on the display a food name corresponding to the photographed food image.

24. The refrigerator as claimed in claim 23, wherein the controller is further configured to recognize the food image photographed by the camera by using a search engine via the communication interface according to the food recognition search order;

and

display on the display a food name corresponding to the photographed food image.