Methods and systems of generating 3D user interface for physical environment

• US 8,085,265 B2
• Filed: 04/23/2008
• Issued: 12/27/2011
• Est. Priority Date: 04/23/2008
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Chen, Henry, Gu, Ji, Li, Cheng Jun, Zhang, Yajun-Edwin, Hajdukiewicz, John R., Plocher, Tom
• Primary Examiner(s): NGUYEN, PHU K
• Application Number:
  US12/108,467
• Publication Number:
  US 20090271365A1
• Time in Patent Office: 1,343 Days
• Field of Search: 345/419, 709/227, 709/228, 709/229
• US Class Current: 345/419
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

One embodiment of the application provides a method of generating a 3D user interface (UI) in a controller for an appliance in a physical environment. The method includes: abstracting the appliance to obtain to a appliance functionality description having variables of the appliance, operations of the state variables, and relationship among the state variables; converting the state variables into corresponding 3D widgets in the controller, wherein the state variable are mapped respectively into the 3D widgets; laying out the 3D widgets to form the 3D UI in the controller; constructing communication objects in the controller from the state variables, the state variables being mapped into the communication objects; and providing interaction between the 3D UI and the appliance by a communication protocol integrated with the appliance.

Claims

1. A system of automatically generating a 3D user interface (UI) for an appliance in a physical environment, comprising:

a functionality specification abstracted from the appliance; and

a controller coupled to the appliance comprising;

a functionality analyzer to analyze the functionality specification to extract state variables of the appliance, operations of the state variables, and relationships among the state variables;

a 3D widget generator to automatically convert the state variables into corresponding 3D widgets;

a 3D UI generation module to generate the 3D UI using the 3D widgets in the controller; and

a communication object manager to construct communication objects from the state variables in the controller, wherein the state variables are mapped into the communication objects.

2. The system of claim 1, wherein a communication protocol is integrated with the appliance to provide interaction between the 3D UI in the controller when displayed on a display device and the appliance in the physical environment via the communication objects.

3. The system of claim 1, wherein the controller further comprising:

a state variable manager applicable to capture any interaction of a user with the 3D widgets, and to update the corresponding state variables by the operations thereof.

4. The system of claim 3, wherein the state variable manager is applicable to capture any update of the communication objects resulted from an action to the appliance, and to update the state variables by the operations thereof.

5. The system of claim 1, wherein the controller further comprises a 3D virtual device delegate module to capture any update of the state variables of the appliance and in turn to update corresponding 3D virtual device.

6. The system of claim 1, wherein the controller further comprising:

a 3D UI layout module to layout the 3D widgets and display the 3D widgets on a display device separate from the appliance.

7. A method of generating a 3D user interface (UI) in a controller for an appliance in a physical environment, comprising:

abstracting the appliance to obtain to an appliance functionality description having variables of the appliance, operations of the state variables, and relationship among the state variables;

converting the state variables into corresponding 3D widgets in the controller, wherein the state variable are mapped respectively into the 3D widgets;

laying out the 3D widgets to form the 3D UI in the controller;

constructing communication objects in the controller from the state variables, the state variables being mapped into the communication objects; and

providing interaction between the 3D UI and the appliance by a communication protocol integrated with the appliance.

8. The method of claim 7, wherein providing interaction between the 3D UI and the appliance by a communication protocol integrated with the appliance comprising:

when a first operation is applied to the 3D UI, updating the state variables of the appliance and in turn the communication objects of the appliance; and

activating a first action on the appliance in the physical environment via the communication protocol.

9. The method of claim 7, wherein providing interaction between the 3D UI and the appliance by a communication protocol integrated with the appliance comprising:

when a second action is applied to the appliance, updating the communication objects via the communication protocol and in turn the state variables of the appliance; and

presenting the second action on the corresponding 3D widget.

10. The method of claim 7, wherein abstracting the appliance to obtain an appliance functionality description comprises:

assigning default values and valid value ranges to the state variables of the appliance.

11. The method of claim 7, further comprising:

attaching event handlers to the operations of the state variables to deal with interactions with the corresponding widgets.

12. A method of generating a 3D user interface (UI) in a controller for an appliance in a physical environment, comprising:

reading state variables of the appliance from an appliance functionality description, which further includes operations of the state variables and relationship among the state variables;

searching a 3D widget template library to select a 3D widget template for each state variable as a function of its variable type; and

generating a widget for each state variable by using the selected matching 3D widget template.

13. The method of claim 12, wherein generating a widget for each state variable by using the selected matching 3D widget template comprises:

generating an interactive component for the widget, to which an interaction is applicable to operate the appliance, wherein the 3D widget template defines geometric model and behavior of the correspond state variable.

14. The method of claim 12, wherein generating a widget for each state variable by using the selected matching 3D widget template comprises:

generating a dock component and a text annotation for the widget, to form a background of the interactive component of the widget.

15. The method of claim 12, further comprising:

defining and adding a new 3D widget template into the 3D widget template library, if no 3D widget template is selected in the 3D widget template library.

16. The method of claim 12, wherein searching a 3D widget template library to select a 3D widget template for each state variable as a function of its variable type comprises:

comparing the type of each state variable to each 3D widget template within the 3D widget template library.

17. The method of claim 12, further comprising:

if the state variables have operations, attaching event handlers to the operations of the state variables to deal with interactions with the corresponding widgets.

18. A method of laying out a 3D user interface (UI) in a controller for an appliance in a physical environment, comprising:

receiving an appliance functionality description including state variables of the appliance, operations of the state variables, and relationship among the state variables;

searching a 3D widget layout template library to select a 3D widget layout template as a function of the type of each state variable of the appliance;

generating a 3D widget for each state variable of the appliance by a 3D widget modeling system; and

laying out the generated 3D widgets based on the selected 3D widget layout template.

19. The method of claim 18, wherein searching a 3D widget layout template library to select a 3D widget layout template as a function of the type of each state variable of the appliance comprises:

comparing the type of each state variable with each 3D widget layout template within the 3D widget layout template library.

20. The method of claim 18, further comprising:

if the state variables have operations, attaching event handlers to the operations of the state variables of the appliance to deal with interactions with the corresponding 3D widgets.