Distributed interactive augmentation of display output

• US 8,495,492 B2
• Filed: 03/14/2007
• Issued: 07/23/2013
• Est. Priority Date: 03/14/2006
• Status: Active Grant

• Current Assignee: Technische Universitã¤T Darmstadt
• Original Assignee: Technische Universitã¤T Darmstadt
• Inventors: Mhlhuser, Eberhard Max, Trompler, Christoph, Turban, Georg
• Primary Examiner(s): STORK, KYLE R
• Application Number:
  US11/686,279
• Publication Number:
  US 20070220412A1
• Time in Patent Office: 2,323 Days
• Field of Search: 715/202, 715/203, 715/204, 715/230, 715/233, 715/718, 715/730, 715/731, 715/732, 715/751, 715/759, 715/200, 715/273
• US Class Current: 715/273
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F

Abstract

Systems, methods, and apparatus are provided for distributed augmentation. In one implementation, a system is provided. The system includes one or more presentation applications, each presentation application operable to provide a presentation output and a plurality of augmentation and presentation control (iAPC) modules operable to provide user augmentations of the presentation output. The system also includes one or more distributed augmentation and presentation management (DAPm) modules operable to separately manage augmentations from the presentation output and to provide an augmented presentation output and one or more display devices for presenting the augmented presentation output.

Claims

1. A system comprising:

one or more presentation applications, each presentation application operable to provide a respective presentation output, wherein the presentation output is configured to be displayed or projected;

a plurality of augmentation and presentation control (iAPC) modules operable to provide user augmentations of the presentation output, the user augmentations being associated with the presentation output without modifying the presentation output, wherein each iAPC includes a respective remote virtual frame buffer each virtual frame buffer hosting one of the presentation threads from the one or more presentation applications, wherein the iAPC provides an augmentation user interface to users having permission to augment the presentation output as a separate layer over a presented augmentation layer and the presentation output;

one or more distributed augmentation and presentation management (DAPm) modules operable to separately manage augmentations from the respective presentation output received from the plurality of iAPC modules, wherein the DAPm modules each include one or more virtual frame buffers such that the augmentations are separately maintained in one or more augmentation layers distinct from the presentation output received from the one or more presentation applications and to provide an augmented presentation output, where the augmented presentation output provides synchronous presentation of the augmentations and the presentation output; and

one or more display devices for presenting the augmented presentation output, wherein each of the one or more display devices is coupled to one or more iAPC modules, and wherein the iAPC modules receive the augmented presentation output from one or more DAPm modules.

2. The system of claim 1, further comprising:

one or more post-processing augmentation/presentation control (pAPC) modules operable to further process the augmented presentation output, including providing non-interactive augmentations.

3. The system of claim 2, further comprising:

one or more output devices operable to present the augmented presentation output received from the one or more pAPC modules.

4. The system of claim 2, wherein the one or more pAPC modules present the augmented presentation output as part of a presentation surface configured to present a number of tiles of a projection area as a single large area and wherein each tile displays different data.

5. The system of claim 1, wherein providing the augmented presentation output includes overlaying the presentation output with the augmentation layers.

6. The system of claim 1, where the DAPm initially provides the presentation output for presentation to one or more users;

receives augmentations to the presentation output from the iAPC; and

combines the presentation output with the received augmentations to provide the augmented presentation output.

7. The system of claim 1, wherein management of multiple presentation threads by the DAPM module includes creation or deletion of presentation streams and corresponding VFB'"'"'s as needed.

8. The system of claim 7, wherein each thread is associated with a VFB/DAPm pair and wherein the pairing can be dynamically changed.

9. The system of claim 1, wherein providing synchronous presentation includes synchronizing temporal aspects, location aspects, and semantic aspects of the augmentations and the presentation output.

10. The system of claim 1, wherein the DAPm modules manage synchronization between DAPm modules.

11. A method comprising:

receiving one or more presentation outputs, each presentation output of the one or more presentation outputs being associated with a particular presentation application and configured for display;

receiving a first augmentation to a first presentation output of the one or more of the presentation outputs, the first augmentation being associated with the first presentation output without modifying the first presentation output, wherein the first augmentation is received at an augmentation user interface presented to users having permission to augment the presentation output as a separate layer over a presented augmentation layer and the presentation output, and wherein each of the one or more presentation outputs is hosted by a respective remote virtual frame buffer;

generating, using one or more processors, a first augmented presentation output, the first augmented presentation output received from a first presentation application, the first augmented presentation output corresponding to a combination of the first augmentation and the first presentation output, where the combination provides synchronous presentation of the first augmentation and first presentation output while separately maintaining the first augmentation in a distinct augmentation layer from the first presentation output such that the first augmentation and first presentation output are independent streams; and

providing the augmented presentation output for display.

12. The method of claim 11, where the one or more presentation outputs comprise multiple threads.

13. The method of claim 11, where receiving the first augmentation to the first presentation output includes receiving an augmentation associated with particular content of the first presentation output.

14. The method of claim 13, where the first augmentation is an annotation to the first presentation output.

15. The method of claim 11, where generating the first augmented presentation output includes:

generating an augmentation layer corresponding to the received first augmentation;

synchronizing the augmentation layer to the first presentation output; and

overlaying the synchronized augmentation layer to the first presentation output.

16. The method of claim 15, where synchronizing the augmentation layer to the first presentation output includes synchronizing temporal aspects including a relationship between the first augmentation and a time of a corresponding portion of the presentation output.

17. The method of claim 11, further comprising:

receiving additional augmentations from one or more users.

18. The method of claim 17, where each user is associated with a particular augmentation layer.

19. The method of claim 17, where each presentation output is associated with a particular augmentation layer.

20. The method of claim 11, further comprising:

displaying an output corresponding to each presentation output as separate tiles.

21. The method of claim 11, further comprising:

displaying the presentation output without the first augmentation on a first display device associated with one or more users while displaying the augmented presentation output on a second display device.

22. The method of claim 11, further comprising:

generating a second augmentation; and

determining for each of a plurality of display devices whether to include the second augmentation in the augmented presentation output.

23. The method of claim 11, where the first augmentation expires a specified time after display in the augmented presentation output.

24. The method of claim 11, where the first augmentation is removed when a second augmentation is generated.