USER GENERATED VIRTUAL ROOM-BASED USER INTERFACE

• US 20140181704A1
• Filed: 02/26/2014
• Published: 06/26/2014
• Est. Priority Date: 06/03/2009
• Status: Active Grant

• Current Assignee: Savant Systems, LLC
• Original Assignee: Savant Systems, LLC
• Inventors: Madonna, Robert P., Silva, Michael C., Cipollo, Nicholas J., Hamm, Andrew R.
• Granted Patent: US 10,775,960 B2
• Time in Patent Office: Days
• US Class Current: 715/762
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: H05B 47/155 Coordinated control of two or more light sources
• CPC Class Codes: H05B 47/175 by remote control

Abstract

In one embodiment, a technique is utilized to generate a virtual-room of a virtual room-based user interface that controls one or more devices within a physical room of a structure. A user is prompted to select one or more services provided by the one or more devices within the physical room. A plurality of images of the physical room are obtained that each share substantially a same perspective. The plurality of images include an all-off image, in which all of the one or more of the services are in an inactive state, and one or more service-specific images, in which a single service is in an active state and any other services are in the inactive state. The plurality of images are aligned to create an aligned image stack. One or more hit boxes are designated. The virtual room is generated based on the aligned image stack and the designated one or more hit boxes.

Claims

1. A method for generating a virtual-room of a virtual room-based user interface that controls one or more devices within a physical room of a structure, comprising:

prompting a user of a mobile device to select one or more services provided by the one or more devices within the physical room, each device providing a corresponding service;

obtaining, on the mobile device, a plurality of images of the physical room that each share substantially a same perspective, the plurality of images including an all-off image, in which all of the one or more of the services are in an inactive state, and one or more service-specific images, in which a single service is in an active state and any other services are in the inactive state;

aligning the plurality of images to create an aligned image stack;

designating, on the mobile device, one or more hit boxes, each hit box to coincide with one of the one or more devices; and

generating the virtual room based on the aligned image stack and the designated one or more hit boxes, the virtual room usable by a user to control the one or more devices within the physical room.

2. The method of claim 1, wherein the virtual room provides a photo-realistic depiction of the physical room, wherein the photo-realistic depiction of the physical room includes photo-realistic depictions of the one or more devices.

3. The method of claim 2, wherein the photo-realistic depictions of the one or more devices each depict a current state of a respective device.

4. The method of claim 1, wherein the obtaining further comprises:

capturing each of the plurality of images using a camera of the mobile device.

5. The method of claim 4, wherein the capturing is performed while services in the physical room are deactivated and then sequentially activated one at a time.

6. The method of claim 1, wherein the obtaining further comprises:

assigning previously-captured images from an image library.

7. The method of claim 1, wherein the aligning further comprises:

displaying a first image of the plurality of images overlaid upon at least a second image of the plurality of images; and

adjusting the first image and the second image in response to user input so that features of the first image and the second image coincide.

8. The method of claim 1, wherein the designating is based upon input in relation to the aligned image stack.

9. The method of claim 1, further comprising:

testing the generated virtual room; and

in response to the testing, either finalizing the generated the virtual room or repeating one or more of the steps of selecting, obtaining, aligning, designating or generating to refine the virtual room.

10. The method of claim 1, wherein the generating is performed by the mobile device.

11. A mobile device for generating a virtual-room of a virtual room-based user interface that controls one or more devices within a physical room of a structure, comprising:

a touch sensitive display screen;

a camera;

a memory configured to store a configuration application;

a processor configured to execute the configuration application, the configuration application, when executed, operable to;

prompt a user of the mobile device to select one or more services provided by the one or more devices within the physical room,

capture a plurality of images of the physical room with the camera of the mobile device, wherein each image shares substantially a same perspective,

align the plurality of images to create an aligned image stack,

designate, in response to input on the touch sensitive display screen of the mobile device, one or more hit boxes, each hit box to coincide with one of the one or more devices, and

generate the virtual room based on the aligned image stack and the designated one or more hit boxes, the virtual room usable by a user to control the one or more devices within the physical room.

12. The mobile device of claim 11, wherein the virtual room provides a photo-realistic depiction of the physical room, wherein the photo-realistic depiction of the physical room includes photo-realistic depictions of the one or more devices.

13. The mobile device of claim 12, wherein the photo-realistic depictions of the one or more devices each depict a current state of a respective device.

14. The mobile device of claim 11, wherein the configuration application, when executed, is further operable to cause a home automation or device control system to deactivate and then sequentially activate services one at a time, the plurality of images captured while the configuration application is so causing the home automation or device control system to deactivate and then sequentially activate services.

15. The mobile device of claim 11, wherein the plurality of images includes an all-off image, in which all of one or more services provided in the physical room are in an inactive state, and one or more service-specific images, in which a single service of the one or more services is in an active state and any other services are in the inactive state.

16. The mobile device of claim 11, wherein the configuration application, when executed, is further operable to display a first image of the plurality of images overlaid upon at least a second image of the plurality of images on the touch sensitive display screen, and, in response to touch input from the user on the touch sensitive display screen, adjust the first image and the second image so that features of the first image and the second image coincide.

17. The mobile device of claim 11, wherein the configuration application, when executed, is further operable to designate the hit boxes based upon touch input from the user on the touch sensitive display screen.

18. The mobile device of claim 11, wherein the configuration application, when executed, is further operable to test the generated virtual room, and either finalize the generated virtual room or repeat operations to refine the generated virtual room.

19. A non-transitory computer-readable medium having software encoded thereon, the software, when executed on one or more processors, operable to:

prompt a user in a configuration user interface to select one or more services provided by one or more devices within a physical room of a structure;

obtain a plurality of images of the physical room that each share substantially a same perspective, the plurality of images including an all-off image, in which all of one or more services provided in the room are in an inactive state, and one or more service-specific images, in which a single service of the one or more services is in an active state and any other services are in the inactive state;

align, in the configuration user interface, the plurality of images to create an aligned image stack;

designate, in the configuration user interface, one or more hit boxes, each hit box to coincide with one of the one or more devices; and

generate a virtual room of a virtual room-based user interface using the aligned image stack and the designated one or more hit boxes, the virtual room usable by a user to control the one or more devices within the physical room.

20. The non-transitory computer-readable medium of claim 19, wherein the virtual room provides a photo-realistic depiction of the physical room, wherein the photo-realistic depiction of the physical room includes photo-realistic depictions of the one or more devices.