CROSS-PLATFORM DATA VISUALIZATIONS USING COMMON DESCRIPTIONS

• US 20140125672A1
• Filed: 11/02/2012
• Published: 05/08/2014
• Est. Priority Date: 11/02/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Winternitz, Fabian O., Unnithan, Aditya, Hill, David Paul, Davis, Jeffrey A.
• Granted Patent: US 9,075,618 B2
• Time in Patent Office: Days
• US Class Current: 345/440
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 8/38 for implementing user interfaces
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

The present invention extends to methods, systems, and computer program products for cross-platform data visualizations using common descriptions. Embodiments of the invention provide mechanisms for simplifying software development and enhanced code reliability. A user interface, along with interactive and dynamic characteristics, can be described (programmatically and/or declaratively) independently of any specific device platform. User interface qualities can be described using a generic scene graph structure with attached behaviors. The generic scene graph can then be used to drive the user interface on any number of different computing platforms using platform specific (e.g., rendering, gesture recognition, etc.) sub-systems. Platform specific sub-systems can vary based on technologies that are used by the platform specific sub-systems. For example, rendering sub-systems can vary based on the technology used to render graphical data, such as, for example, Open GL, XAML, Direct X, Quartz, etc.

Claims

1. At a computer system, the computer system including a user interface management system, the user-interface management system including a scene graph processing layer and a platform specific user interface sub-system, the scene graph processing layer aware of the data visualization processing techniques available to the platform specific user interface sub-system, a method for outputting a data visualization, the method comprising:

the scene graph processing layer accessing a generic scene graph from common code that is executable on a plurality of different technology platforms, the generic scene graph representing a user interface, the generic scene graph defining the data visualizations independent of any of a plurality of different platform specific user interface sub-systems, including the platform specific user interface sub-system, the generic scene graph containing scene graph metadata that describes characteristics of the data visualizations;

the scene graph processing layer interpreting the structure and data of at least portions of the generic scene graph into specific operations that are processed by a platform specific rendering sub-system, the scene graph processing layer makes decisions on how to process individual scene graph nodes or groups of scene graph nodes based on the characteristics of rendering subsystems a platform provides;

the scene graph processing layer sending the specific operations to the platform specific rendering sub-system for processing; and

the platform specific rendering sub-system outputting the data visualizations by processing the specific operations.

3. The method of claim 1, wherein the platform specific rendering sub-system is a platform specific communication sub-system, the platform specific communication sub-system connected to a platform specific rendering sub-system via network communication.

4. The method of claim 1, wherein accessing a generic scene graph comprises accessing a generic scene graph that includes a specified type of node;

and further comprising;

the scene graph processing layer inferring the scene graph metadata based on inclusion of the specified type of node in the generic scene graph prior to generating the specific operations.

5. The method of claim 1, wherein accessing a generic scene graph comprises accessing a generic scene graph that includes a specified number of nodes;

and further comprising;

the scene graph processing layer inferring the scene graph metadata based on the specified number of nodes prior to generating the specific operations.

6. The method of claim 1, wherein interpreting the structure and data of at least portions of the generic scene graph into specific operations comprises interpreting the structure and data of at least portions of the generic scene graph into specific operations for rendering the data visualizations using retained mode.

7. The method of claim 1, wherein interpreting the structure and data of at least portions of the generic scene graph into specific operations comprises interpreting the structure and data of at least portions of the generic scene graph into specific operations for rendering the data visualizations using immediate mode.

2. The method of claim 2, wherein the platform specific rendering sub-system uses one of:

Direct X®

, XAML, Open GL, Cocoa, or Quartz®

to render the platform specific graphical data.

8. At a computer system, the computer system including a user interface management system, the user-interface management system including a scene graph processing layer and a platform specific user interface sub-system, the scene graph processing layer aware of the gesture recognition processing techniques available to the platform specific user interface sub-system, a method for processing an input gesture from a data visualization, the method comprising:

the scene graph processing layer accessing a generic scene graph from common code that is executable on a plurality of different technology platforms, the generic scene graph representing a user interface, the generic scene graph defining behaviors for input gesture processing independent of any of a plurality of different platform specific user interface sub-systems, including the platform specific user interface sub-system, the generic scene graph containing scene graph metadata that describes characteristics of the behaviors;

the scene graph processing layer receiving input gesture data from the platform specific user interface sub-system, the input gesture data for an input gesture recognized by utilizing a gesture recognition technique available to the platform specific user interface sub-system;

the scene graph processing layer adjusting the input gesture data for consistency with the generic scene graph based on the utilized gesture recognition technique and in view of the scene graph metadata; and

the scene graph processing layer routing the adjusted input gesture data to an appropriate behavior in the generic scene graph.

9. The method of claim 8, wherein the platform specific user interface sub-system is a platform specific user input sub-system.

10. The method of claim 8, wherein receiving input gesture data comprises receiving input gesture data for a recognized mouse or keyboard input gesture.

11. The method of claim 8, wherein receiving input gesture data comprises receiving input gesture data for a recognized touch input gesture.

12. The method of claim 8, wherein receiving input gesture data comprises receiving input gesture data for a recognized voice or video input gesture.

13. A computer system, the computer system comprising:

one or more processors;

system memory;

one or more components of a specific technology platform;

one or more computer storage devices having stored thereon computer-executable instructions representing a scene graph processing layer, the scene graph processing layer aware of the data visualization processing techniques available to the specific technology platform, the scene graph processing layer aware of the gesture recognition processing techniques available to the specific technology platform, the scene graph processing layer configured to;

access a generic scene graph from common code that is executable on a plurality of different technology platforms, the generic scene graph representing a user interface, the generic scene graph;

defining one or more data visualizations independent of any of a plurality of different technology platforms, including the specified technology platform;

defining behaviors for input gesture processing independent of any of the plurality of different technology platforms, including the specified technology platform; and

indicating scene graph metadata that both describes characteristics of the one or more data visualizations and describes characteristics of the behaviors for input gesture processing;

output a data visualization by;

interpreting the structure and data of at least portions of the generic scene graph into specific operations that are processed by the platform specific user interface sub-system, making decisions on how to process individual scene graph nodes or groups of scene graph nodes based on the characteristics of rendering subsystems a platform provides; and

sending the specific operations to the specific technology platform for output by processing the specific operations; and

process an input gesture by;

receiving input gesture data from the specific technology platform, the input gesture data for an input gesture recognized by utilizing a gesture recognition technique available to the specific technology platform;

adjusting the input gesture data for consistency with the generic scene graph based on the utilized gesture recognition technique and in view of the scene graph metadata; and

routing the adjusted input gesture data to an appropriate behavior in the generic scene graph.

14. The system of claim 13, wherein the specific technology platform uses one of:

Direct X®

, XAML, Open GL, Cocoa, or Quartz®

to render the platform specific graphical data.

15. The system of claim 13, wherein the one or more components of the specific technology platform include a rendering sub-system.

16. The system of claim 13, wherein the one or more components of the specific technology platform include a communication sub-system, the communication sub-system connected to a rendering sub-system via network communication.

17. The system of claim 13, wherein interpreting the structure and data of at least portions of the generic scene graph into specific operations comprises interpreting the structure and data of at least portions of the generic scene graph into specific operations for rendering the data visualizations using retained mode.

18. The system of claim 13, wherein interpreting the structure and data of at least portions of the generic scene graph into specific operations comprises interpreting the structure and data of at least portions of the generic scene graph into specific operations for rendering the data visualizations using immediate mode.

19. The system of claim 13, wherein receiving input gesture data comprises receiving input gesture data for one of:

a recognized mouse input gesture, a recognized keyboard input gesture, a recognized voice input gesture, or a recognized video input gesture.

20. The system of claim 13, wherein receiving input gesture data comprises receiving input gesture data for a recognized touch gesture.