Method and apparatus for compositing various types of content

• US 8,656,311 B1
• Filed: 01/07/2007
• Issued: 02/18/2014
• Est. Priority Date: 01/07/2007
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Harper, John, Dyke, Kenneth C.
• Primary Examiner(s): Vital, Pierre
• Assistant Examiner(s): Raab, Christopher J
• Application Number:
  US11/620,720
• Time in Patent Office: 2,599 Days
• Field of Search: 707/3, 707/10, 707/100, 707/101, 707/102, 707/829, 715/201, 715/202, 715/854, 345/530, 345/531, 345/532, 345/533
• US Class Current: 715/854
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

At least some embodiments of the present disclosure include a method for compositing media and non-media content of user interface for display on a device. The method includes constructing a data structure having a hierarchy of layers associated with the user interface of the device. The method further includes determining whether each layer of the data structure is associated with media or non-media content. The data structure or layer tree is traversed in order to determine whether each of the layers of the data structure is associated with media or non-media content. The method further includes detaching a layer associated with media content from the data structure. The method further includes storing media content in a first memory location. The method further includes storing non-media content in a second memory location. The method further includes compositing the media and non-media content for display on the device.

Claims

1. A machine implemented method for compositing media and non-media content of a user interface for display on a device, comprising:

constructing a data structure having a hierarchy of layers for storing media and non-media content associated with the user interface to be displayed on a display of the device;

traversing layers of the hierarchy of the data structure to determine whether each layer of the data structure is associated with media or non-media content to distinguish between media and non-media content for scanning the media content and the non-media content to the display at different rates;

detaching any layer associated with media content from the data structure;

storing media content in a first memory location;

storing non-media content in a second memory location;

compositing the media and non-media content from the first and second memory locations; and

displaying the composited media and non-media content representing the user interface on the display of the device, wherein the media content includes time varying video content and the non-media content includes non-video content that does not require frequent updating.

2. The method as in claim 1, wherein compositing the media and non-media content comprises:

retrieving the media content from the first memory location;

retrieving the non-media content from the second memory location; and

scanning the media and non-media content directly to the display.

3. The method as in claim 2, wherein the media content is scanned to the display at a rate of substantially thirty frames per second.

4. The method as in claim 2, wherein the non-media content is scanned to the display at a rate of substantially one frame per second.

5. The method as in claim 1, wherein the media content comprises video content and time varying audio content.

6. The method as in claim 2, further comprising:

scaling the media content prior to sending the media content to the display.

7. The method as in claim 1, wherein the device is one of:

a data processing device, a portable device, a portable data processing device, a multi touch device, a multi touch portable device, a wireless device, and a cell phone.

23. The method as in claim 1, wherein the media content is scanned to the display at a first rate that is higher than a second rate of scanning the non-media content to the display, wherein the non-media content includes a user interface view for the time varying video content of the media content.

8. A non-transitory machine readable storage medium storing executable program instructions which when executed cause a data processing system to perform a method comprising:

constructing a data structure having a hierarchy of layers for storing media and non-media content associated with the user interface to be displayed on a display of the device;

traversing layers of the hierarchy of the data structure to determine whether each layer of the data structure is associated with media or non-media content to distinguish between media and non-media content for scanning the media content and the non-media content to the display at different rates;

detaching any layer associated with media content from the data structure;

storing media content in a first memory location;

storing non-media content in a second memory location;

compositing the media and non-media content from the first and second memory locations; and

displaying the composited media and non-media content representing the user interface on the display of the device, wherein the media content includes time varying video content and the non-media content includes non-video content that does not require frequent updating.

9. The medium as in claim 8, wherein compositing the media and non-media content comprises:

retrieving the media content from the first memory location;

retrieving the non-media content from the second memory location; and

scanning the media and non-media content directly to the display.

10. The medium as in claim 8, wherein the media content is scanned to the display at a rate of substantially thirty frames per second.

11. The medium as in claim 8, wherein the device is one of:

a data processing device, a portable device, a portable data processing device, a multi touch device, a multi touch portable device, a wireless device, and a cell phone.

12. The medium as in claim 8, wherein the media content comprises video content and time varying audio content.

13. The medium as in claim 8, further comprising:

scaling the media content prior to sending the media content to the display.

14. An apparatus, comprising:

means for constructing a data structure having a hierarchy of layers for storing media and non-media content associated with the user interface to be displayed on a display of the device;

means for traversing layers of the hierarchy of the data structure to determine whether each layer of the data structure is associated with media or non-media content to distinguish between media and non-media content for scanning the media content and the non-media content to the display at different rates;

means for detaching any layer associated with media content from the data structure;

means for storing media content in a first memory location;

means for storing non-media content in a second memory location;

means for compositing the media and non-media content from the first and second memory locations; and

means for displaying the composited media and non-media content representing the user interface on the display of the device, wherein means for displaying includes a processor and a memory having the first and second memory locations, wherein the media content includes time varying video content and the non-media content includes non-video content that does not require frequent updating.

15. The apparatus as in claim 14, wherein compositing the media and non-media content comprises:

means for retrieving the media content from the first memory location;

means for retrieving the non-media content from the second memory location; and

means for scanning the media and non-media content directly to the display.

16. The apparatus as in claim 15, wherein the media content is scanned to the display at a rate of substantially thirty frames per second.

17. The apparatus as in claim 15, further comprising:

means for scaling the media content prior to sending the media content to the display.

18. A device comprising:

an input panel which is configured to receive user input;

a display device integrated with the input panel, the display device configured to display media and non-media content at the same time;

a central processing unit coupled to the input panel, the processor being configured to execute one or more programs in order to construct a first data structure having a hierarchy of layers with each layer associated with media or non-media content, based on at least the user input,

a memory coupled to the processor, the memory being configured to store the media content in a first memory location and the non-media content in a second memory location, wherein the media content and the non-media content are to be composited to represent a user interface to be displayed on the display device, wherein the processor is further configured to traverse layers of the hierarchy of the data structure to determine whether each layer of the data structure is associated with media or non-media content to distinguish between media and non-media content for scanning the media content and the non-media content to the display device at different rates and to detach any layer associated with media content from the data structure, wherein the media content includes time varying video content and the non-media content includes non-video content that does not require frequent updating.

19. The device as in claim 18, wherein the processor is further configured to composite the media and non-media content for display on the display device.

20. The device as in claim 18 further comprising:

a graphics processing unit coupled to the memory, the graphics processing unit configured to receive instructions indicating the locations of the media and non-media content in the memory.

21. The device as in claim 20 wherein the graphics processing unit is further configured to composite the media and non-media content for display on the display device.

22. The device as in claim 18, wherein the device is one of:

a data processing device, a portable device, a portable data processing device, a multi touch device, a multi touch portable device, a wireless device, and a cell phone.