Virtual representation of activity within an environment

• US 10,431,061 B2
• Filed: 11/29/2017
• Issued: 10/01/2019
• Est. Priority Date: 11/29/2016
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Inventors: Tovey, David, Mattingly, Todd D., High, Donald R., Webb, Tim W., Sunday, Eugene P.
• Primary Examiner(s): Foxx, Chico A
• Application Number:
  US15/826,476
• Publication Number:
  US 20180151040A1
• Time in Patent Office: 671 Days
• Field of Search: None
• CPC Class Codes: G08B 13/1672 using sonic detecting means, e.g. a microphone operating in the audio frequency range
• CPC Class Codes: G08B 13/19682 Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
• CPC Class Codes: G08B 13/19684 Portable terminal, e.g. mobile phone, used for viewing video remotely

Abstract

In some embodiments, apparatuses and methods are provided herein useful to presenting a virtual representation of a user'"'"'s environment based on activity in the user'"'"'s environment. In some embodiments, a system comprises one or more sensors, wherein the one or more sensors are located about the user'"'"'s environment and configured to detect the activity within the user'"'"'s environment and transmit, to a control circuit, indications of the activity, the control circuit configured to receive, from the one or more sensors, the indications of the activity within the user'"'"'s environment, generate the virtual representation of the user'"'"'s environment, and render, based on the indications of the activity, the virtual representation of the user'"'"'s environment to include representations of the activity within the user'"'"'s environment, and a display device, the display device configured to present the virtual representation of the user'"'"'s environment including the representations of the activity within the user'"'"'s environment.

Claims

1. A system for presenting a virtual representation of a user'"'"'s environment based on activity in the user'"'"'s environment, the system comprising:

one or more sensors, wherein the one or more sensors are located about the user'"'"'s environment and configured to;

detect the activity within the user'"'"'s environment;

detect, for multiple items within the user'"'"'s environment, remaining useful lives; and

transmit, to a control circuit, indications of the activity within the user'"'"'s environment and the remaining useful lives;

the control circuit configured to;

receive, from a mobile device associated with the user, a scan of the user'"'"'s environment;

receive, from the one or more sensors, the indications of the activity within the user'"'"'s environment and the remaining useful lives;

generate, based on the scan of the user'"'"'s environment, the virtual representation of the user'"'"'s environment;

render, based on the indications of the activity within the user'"'"'s environment, the virtual representation of the user'"'"'s environment to include representations of the activity within the user'"'"'s environment; and

reorder, automatically for the user based on the remaining useful lives for the multiple items within the user'"'"'s environment, those of the multiple items having a remaining useful life below a threshold; and

a display device, the display device configured to present, based on the rendering of the virtual representation of the user'"'"'s environment, the virtual representation of the user'"'"'s environment including the representations of the activity within the user'"'"'s environment.

2. The system of claim 1, wherein the one or more sensors include one or more of image sensors, motion sensors, light sensors, sound sensors, water usage sensors, energy usage sensors, proximity sensors, and door closure sensors.

3. The system of claim 1, wherein the control circuit is further configured to:

generate a user interface, wherein the user interface allows the user to interact with the system; and

receive, via the user interface, user input.

4. The system of claim 3, wherein the user input is to set an alert based on a trigger condition, and wherein the control circuit is further configured to:

determine, based on the indications of the activity within the user'"'"'s environment, that the trigger condition has occurred;

generate, based on the occurrence of the trigger condition, an alert; and

transmit, to the user, the alert.

5. The system of claim 3, wherein the user input is to set a limit for one or more devices within the user'"'"'s environment, wherein the control circuit is further configured to:

transmit, to the one or more devices within the user'"'"'s environment, an indication of the limit, wherein the indication of the limit causes the one or more devices within the user'"'"'s environment to adhere to the limit.

6. The system of claim 5, wherein the one or more devices are one or more of appliances and utilities.

7. The system of claim 3, wherein the user input is to modify a program for one or more devices in the user'"'"'s environment, and wherein the control circuit is further configured to modify the program for the one or more devices in the user'"'"'s environment based on the user input.

8. The system of claim 1, wherein the control circuit is further configured to:

update, based on images captured by cameras associated with the system, the virtual representation of the user'"'"'s environment.

9. The system of claim 1, wherein one or more devices in the user'"'"'s environment operate based on a program, and wherein the control circuit is further configured to:

analyze the indications of activity within the user'"'"'s environment; and

develop, based on the analysis of the indications of activity within the user'"'"'s environment, suggestions for program modifications.

10. The system of claim 1, wherein the control circuit renders the virtual representation of the user'"'"'s environment to include representations of the activity within the user'"'"'s environment in real time.

11. A method for presenting a virtual representation of a user'"'"'s environment based on activity in the user'"'"'s environment, the method comprising:

monitoring, via one or more sensors located about the user'"'"'s environment, the activity within the user'"'"'s environment;

monitoring, via the one or more sensors located about the user'"'"'s environment, remaining useful lives for multiple items within the user'"'"'s environment;

receiving, by a control circuit from a mobile device associated with the user, a scan of the user'"'"'s environment;

receiving, by the control circuit from the one or more sensors, indications of the activity within the user'"'"'s environment and the remaining useful lives for multiple items in the user'"'"'s environment;

generating, by the control circuit based on the scan of the user'"'"'s environment, the virtual representation of the user'"'"'s environment;

rendering, based on the indications of the activity within the user'"'"'s environment, the virtual representation of the user'"'"'s environment to include representations of the activity within the user'"'"'s environment;

reordering, automatically for the user based on the remaining useful lives for the multiple items within the user'"'"'s environment, those of the multiple items having a remaining useful life below a threshold; and

presenting, via a display device based on the rendering of the virtual representation of the user'"'"'s environment, the virtual representation of the user'"'"'s environment including the representations of the activity within the user'"'"'s environment.

12. The method of claim 11, wherein the one or more sensors include one or more of image sensors, motion sensors, light sensors, sound sensors, water usage sensors, energy usage sensors, proximity sensors, and door closure sensors.

13. The method of claim 11, further comprising:

generating a user interface, wherein the user interface allows the user to interact with the system; and

receiving, via the user interface, user input.

14. The method of claim 13, wherein the user input is to set an alert based on a trigger condition, the method further comprising:

determining, based on the indications of the activity within the user'"'"'s environment, that the trigger condition has occurred;

generating, based on the occurrence of the trigger condition, an alert; and

transmitting, to the user, the alert.

15. The method of claim 13, wherein the user input is to set a limit for one or more devices within the user'"'"'s environment, the method further comprising:

transmitting, to the one or more devices within the user'"'"'s environment, an indication of the limit, wherein the indication of the limit causes the one or more devices within the user'"'"'s environment to adhere to the limit.

16. The method of claim 15, wherein the one or more devices are one or more of appliances and utilities.

17. The method of claim 13, wherein the user input is to modify a program for one or more devices in the user'"'"'s environment, the method further comprising:

modifying the program for the one or more devices in the user'"'"'s environment based on the user input.

18. The method of claim 11, wherein the display device is one or more of a television, a computer, and a mobile device.

19. The method of claim 11, wherein one or more devices in the user'"'"'s environment operate based on a program, the method further comprising:

analyzing the indications of the activity within the user'"'"'s environment; and

developing, based on the analyzing the indications of the activity within the user'"'"'s environment, suggestions for program modifications.

20. The method of claim 11, wherein the rendering the virtual representation of the user'"'"'s environment to include representation of the activity within the user'"'"'s environment occurs in real time.