Dynamic user experience with semantic rich objects

• US 7,774,713 B2
• Filed: 06/28/2005
• Issued: 08/10/2010
• Est. Priority Date: 06/28/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Stojanovic, Alexander, Mital, Vijay
• Primary Examiner(s): Vu; Kieu
• Assistant Examiner(s): Shih; Haoshian
• Application Number:
  US11/168,913
• Publication Number:
  US 20060294509A1
• Time in Patent Office: 1,869 Days
• Field of Search: 715/769, 715/762, 715/763, 704/270, 707 3- 5, 717/105, 717/108, 717/109, 717/111
• US Class Current: 715/763
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Semantic rich objects for a dynamic user experience. One or more actions in the process are represented by semantic rich objects (SROs) for exposing functionality of the process. Each of the SROs includes metadata associated with an action and an execution environment therefor. In response to an interactive gesture from a user selecting at least one of the SROs to perform a desired action in the process, metadata is extracted from the selected SROs to determine whether the desired action is available in the execution environment of the process.

Claims

1. A computerized method of providing a dynamic user experience within an application program, said computerized method comprising:

graphically presenting one or more semantic rich objects (SROs) as icons to a user for exposing functionality of the application program, said SROs each having data with a type associated therewith, including metadata therein associated with one or more actions in the application program and including a storage for the data relating to an execution environment for executing the one or more actions, said application program operating externally with respect to the SROs;

identifying an interactive gesture, said identified interactive gesture including selection of at least two different types of the presented SROs in combination from the user via overlapping the icons of the two different types of the presented SROs, said each of the at least two different types of the presented SROs storing data in the storage;

evaluating the stored data from the at least two different types of the presented SROs in the interactive gesture based on a set of semantic rules;

extracting the metadata from the at least two different types of the presented SROs in response to the evaluating;

applying the set of semantic rules to the evaluated data and the extracted metadata;

in response to the applying, generating a semantic format executable by the one or more actions of the application program; and

providing at least a portion of the one or more actions of the application program in response to the extracted metadata associated therewith from each of the two different types of the presented SROs as a function of the received data and the generated semantic format.

2. The computerized method of claim 1 wherein each of the SROs further includes code for execution of the action, and wherein extracting the metadata comprises executing the included code to determine whether the desired action is available in the execution environment of the process in response to the received interactive gesture.

3. The computerized method of claim 1 further comprising providing, in response to the received interactive gesture, a set of available actions for the user to interact with the process based on the extracted metadata.

4. The computerized method of claim 3 wherein providing the set of available actions comprises at least one of the following:

displaying the set of available actions exposed through the SROs in a form-based user interface and generating a user interface for selectively displaying the set of available actions for the user to interact with the process, and wherein the process includes one or more of the following;

an application program, an application software, and a workflow processing application.

5. The computerized method of claim 1 further comprising performing the desired action based on the extracted metadata and the received interactive gesture.

6. The computerized method of claim 1 further comprising providing a plurality of SROs in response to a query from a user, and further comprising correlating the extracted metadata with one or more of the following:

a performance history of the execution of the process, prior user interactions with the execution of the process, and a process execution model.

7. The computerized method of claim 1 further comprising composing a set of available actions as a semantic sentence that includes actions represented by the two different types of SROs.

8. The computerized method of claim 1 wherein one or more computer storage media include computer-executable instructions for performing the computerized method recited in claim 1.

9. A system for actively generating behaviors for a user to interact within a process, said system comprising:

a memory for storing one or more semantic rich objects (SROs), each of said SROs including a type and metadata therein corresponding to a behavior and an execution environment therefor, each of said SROs also including data relating to execution of the behavior, said behavior and said execution environment operating externally with respect to the stored one or more SROs;

an interface for presenting the one or more SROs to the user, wherein the interface is further configured for receiving a selection of one or more SROs from the user, wherein the interface receives a user selection for associating one type of SRO with another type of SRO from the user;

a processor for executing computer-executable instructions for;

evaluating the data from the one type of SRO and the data from the other type of SRO in the received user selection based on a set of semantic rules;

in response to the evaluating, extracting the metadata from the one type of SRO and the metadata from the other type of SRO;

generating a set of available behaviors to the user in response to the extracting by selecting at least a portion of behaviors from the one type of SRO and at least a portion of behaviors from the other type of SRO according to the extracted metadata from the one type of SRO and the extracted metadata from the other type of SRO; and

wherein the interface provides the generated set of behaviors to the user for interacting with the data in the one type of SRO and the data in the other type of SRO from the user selection and the process in the execution environment.

10. The system of claim 9 further comprising a user interface having a user selection device for receiving the selection of the one or more SROs from the user.

11. The system of claim 9 wherein the processor is further configured to map the received selection from the interface to the stored metadata of each of the SROs.

12. The system of claim 11 further comprising a data structure mapping a set of speech acts to the stored metadata of each of the SROs.

13. The system of claim 9 wherein the processor is configured to compose a set of available actions as a semantic sentence that includes actions represented by the two different types of SROs.

14. The system of claim 9 wherein the processor is further configured to execute the stored data relating to execution of the behavior to generate the set of behaviors based on the stored metadata and the received selection from the interface.

15. The system of claim 9 wherein,

the interface is configured to provide one or more SROs in response to a query from the user, selectively provides the generated set of behaviors of the SROs based on the execution environment of the process, and receive a selection of one or more SROs from the user via a form-based user interface, and

the processor is further configured to correlate the stored metadata with one or more of the following:

a performance history of the execution of the process, the prior user interactions with the execution of the process, and the process execution model.

16. A computer storage medium having stored thereon a data structure for a semantic rich object, said computer storage medium comprising:

an object data field for storing data relating to the semantic rich object, said object data field further storing a type data relating to the stored data of the semantic rich object;

a first data field of the semantic rich object including metadata representing one or more actions associated with a process for operating on the data relating to the semantic object stored in the object data field;

a second data field of the semantic rich object including metadata representing an execution environment of the process for operating on the data relating to the semantic object stored in the object data field, said first data field being associated with said second data field to define a set of available actions to be performed in the execution environment of the process; and

a third data field of the semantic rich object including metadata representing a set of semantic rules, said semantic rules defining a semantic format executable by the one or more actions associated with the process based on the one or more interactive gestures associated with the metadata in the first data field and the metadata in the second data field for operating on the data relating to the semantic object stored in the object data field, wherein the semantic rich object is presented to a user graphically as an icon, wherein the set of semantic rules is applied when data in the object data field of a type of the semantic rich objects relates to data in the object data field of another type of the semantic rich objects according to the defined semantic format, and wherein at least a portion from the set of available actions based on the metadata in the first data field of the type of the semantic rich objects and the other type of the semantic rich objects are provided to the user.

17. The computer storage medium of claim 16 wherein metadata in the third data field includes metadata representing a set of interactive gestures and a set of speech acts, said set of interactive gestures and said set of speech acts being mapped to the metadata in the first data field and the metadata in the second data field.

18. The computer storage medium of claim 16 further comprising one or more additional data fields including metadata associated with one or more of the following:

a set of process ambient service definition, interactive gesture pins, process extensibility methods, a process object model, a performance history of the execution of the process, the prior user interactions with the execution of the process, execution of the behavior, and a process execution model.