MULTI-MODAL CONTENT PRESENTATION

• US 20070226635A1
• Filed: 03/24/2006
• Published: 09/27/2007
• Est. Priority Date: 03/24/2006
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Goebel, Steffen, Hartmann, Falk, Kadner, Kay, Pohl, Christoph
• Granted Patent: US 7,487,453 B2
• Time in Patent Office: Days
• US Class Current: 715/745
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents

Abstract

A method is provided that includes receiving a user input, the user input having been input in a user interface in one of multiple modalities. The method also includes accessing, in response to receiving the user input, a multi-modality content document including content information and presentation information, the presentation information supporting presentation of the content information in each of the multiple modalities. In addition, the method includes accessing, in response to receiving the user input, metadata for the user interface, the metadata indicating that the user interface provides a first modality and a second modality for interfacing with a user. First-modality instructions are generated based on the accessed multi-modality content document and the accessed metadata, the first-modality instructions providing instructions for presenting the content information on the user interface using the first modality. Second-modality instructions are generated based on the accessed multi-modality content document and the accessed metadata, the second-modality instructions providing instructions for presenting the content information on the user interface using the second modality.

Claims

1. A method comprising:

receiving a user input, the user input having been input in a user interface in one of multiple modalities;

accessing, in response to receiving the user input, a multi-modality content document including content information and presentation information, the presentation information supporting presentation of the content information in each of the multiple modalities;

accessing, in response to receiving the user input, metadata for the user interface, the metadata indicating that the user interface provides a first modality and a second modality for interfacing with a user;

generating first-modality instructions based on the accessed multi-modality content document and the accessed metadata, the first-modality instructions providing instructions for presenting the content information on the user interface using the first modality; and

generating second-modality instructions based on the accessed multi-modality content document and the accessed metadata, the second-modality instructions providing instructions for presenting the content information on the user interface using the second modality.

2. The method of claim 1 wherein:

the content information includes a first content portion and a second content portion,

the metadata includes a user interface attribute related to pagination, and

generating the first-modality instructions and the second-modality instructions comprises;

determining a particular first-modality pagination describing pagination between the first content portion and the second content portion during presentation on the user interface using the first modality;

determining a particular second-modality pagination describing pagination between the first content portion and the second content portion during presentation on the user interface using the second modality, wherein the particular first-modality pagination and the particular second-modality pagination are determined based on (i) desired order of presentation of the first content portion and the second content portion in the first modality on the user interface, (ii) desired order of presentation of the first content portion and the second content portion in the second modality on the user interface, and (iii) the user interface attribute related to pagination;

generating, based on the particular first-modality pagination, the first-modality instructions including instructions for (i) presenting the first content portion on the user interface during a first time interval, and (ii) presenting the second content portion on the user interface during a second time interval; and

generating, based on the particular second-modality pagination, the second-modality instructions including instructions for (i) presenting the first content portion on the user interface during the first time interval, and (ii) presenting the second content portion on the user interface during the second time interval.

3. The method of claim 2 wherein pagination is further based on other metadata.

4. The method of claim 2 wherein the user interface attribute includes a screen size indicator.

5. The method of claim 2 wherein generating the first-modality instructions and the second-modality instructions further comprises:

determining multiple first-modality paginations, including the particular first-modality pagination, describing pagination between the first content portion and the second content portion during presentation on the user interface using the first modality; and

determining multiple second-modality paginations, including the particular second-modality pagination, describing pagination between the first content portion and the second content portion during presentation on the user interface using the second modality, wherein each of the multiple first-modality paginations and second-modality paginations are determined based on (i) desired order of presentation of the first content portion and the second content portion in the first modality on the user interface, (ii) desired order of presentation of the first content portion and the second content portion in the second modality on the user interface, and (iii) the user interface attribute related to pagination;

forming a first pagination scenario, the forming including associating with a first of the multiple first-modality paginations a first of the multiple second-modality paginations;

determining for the first pagination scenario a first distance characterizing whether (i) the first content portion is configured to be presented on the user interface during a first common time interval using both the first modality and the second modality and (ii) the second content portion is configured to be presented on the user interface during a second common time interval using both the first modality and the second modality;

forming a second pagination scenario, the forming including associating with a second of the multiple first-modality paginations a second of the multiple second-modality paginations;

determining for the second pagination scenario a second distance characterizing whether (i) the first content portion is configured to be presented on the user interface during a first common time interval using both the first modality and the second modality and (ii) the second content portion is configured to be presented on the user interface during a second common time interval using both the first modality and the second modality;

comparing the first distance with the second distance; and

selecting a pagination scenario from the first pagination scenario and the second pagination scenario based on a result of the comparing.

6. The method of claim 5 wherein:

the content information is divided into multiple elements,

the first content portion includes at least a first of the multiple elements,

the second content portion includes at least a second of the multiple elements,

the first pagination scenario dictates when elements of the first content portion and elements of the second content portion are presented,

determining the first distance comprises determining, for the first pagination scenario, a number of element-inserts and a number of element-removals that are required in order that (i) the first content portion is configured to be presented on the user interface during a first common time interval using both the first modality and the second modality and (ii) the second content portion is configured to be presented on the user interface during a second common time interval using both the first modality and the second modality, and

the first distance is based on the number of element-inserts and the number of element-removals.

7. The method of claim 5 wherein selecting a pagination scenario from the first pagination scenario and the second pagination scenario based on a result of the comparing further includes selecting the pagination scenario based on the minimal pagination distance.

8. The method of claim 1 further comprising providing both the first-modality instructions and the second-modality instructions to the user interface.

9. The method of claim 1 wherein the presentation information of the multi-modality content document includes (i) a first layout container including presentation information for presenting content using the first modality and (ii) a second layout container including presentation information for presenting content using the second modality.

10. The method of claim 1 wherein the presentation information of the multi-modality content document includes a generic layout container including presentation information for presenting content using multiple modalities.

11. The method of claim 1 wherein generating the first-modality instructions comprises a browser generating the first-modality instructions for execution by a processor that controls the user interface.

12. The method of claim 1 wherein generating the first-modality instructions comprises generating a first-modality content document that includes the first-modality instructions.

13. The method of claim 12 wherein generating the first-modality content document comprises generating one or more of a Speech Recognition Grammar Specification (SRGS) document, a Hypertext Markup Language (HTML) document, a Voice Extensible Markup Language (VXML) document, or a Speech Synthesis Markup Language (SSML) document.

14. The method of claim 1 wherein the metadata further indicates one or more of (i) content types supported by the user interface or (ii) scalability of graphics on the user interface.

15. An apparatus comprising:

means for receiving a user input, the user input having been input in a user interface in one of multiple modalities;

means for accessing, in response to receiving the user input, a multi-modality content document including content information and presentation information, the presentation information supporting presentation of the content information in each of the multiple modalities;

means for accessing, in response to receiving the user input, metadata for the user interface, the metadata indicating that the user interface provides a first modality and a second modality for interfacing with a user;

means for generating first-modality instructions based on the accessed multi-modality content document and the accessed metadata, the first-modality instructions providing instructions for presenting the content information on the user interface using the first modality; and

means for generating second-modality instructions based on the accessed multi-modality content document and the accessed metadata, the second-modality instructions providing instructions for presenting the content information on the user interface using the second modality.

16. The apparatus of claim 15 wherein the means for generating the first-modality instructions is disposed local to the user interface.

17. An apparatus comprising a computer-readable medium that includes instructions for performing at least the following:

receiving a user input, the user input having been input in a user interface in one of multiple modalities;

accessing, in response to receiving the user input, a multi-modality content document including content information and presentation information, the presentation information supporting presentation of the content information in each of the multiple modalities;

accessing, in response to receiving the user input, metadata for the user interface, the metadata indicating that the user interface provides a first modality and a second modality for interfacing with a user;

generating first-modality instructions based on the accessed multi-modality content document and the accessed metadata, the first-modality instructions providing instructions for presenting the content information on the user interface using the first modality; and

generating second-modality instructions based on the accessed multi-modality content document and the accessed metadata, the second-modality instructions providing instructions for presenting the content information on the user interface using the second modality.

18. The apparatus of claim 17 wherein the apparatus comprises one or more of a portable storage device or a processing device.