User interfaces

• US 8,713,460 B2
• Filed: 02/13/2006
• Issued: 04/29/2014
• Est. Priority Date: 02/12/2005
• Status: Active Grant

• Current Assignee: Siemens Industry Software Limited (Siemens AG)
• Original Assignee: Mentor Graphics Corporation (Siemens AG)
• Inventors: Kendall, Geoff, Rivers, Mark
• Primary Examiner(s): Chow, Doon
• Assistant Examiner(s): Kumar, Anil N
• Application Number:
  US11/884,182
• Publication Number:
  US 20090217187A1
• Time in Patent Office: 2,997 Days
• Field of Search: None
• US Class Current: 715/765
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image

Abstract

User interface (UI) techniques, and more particularly to graphical user interface (GUI) techniques providing 3-dimensional (3-D) renditions. A method of displaying one or more graphical objects, the method being carried out in an electronic device, the device having processing circuitry, memory and a display device, the method comprising: obtaining first image data, the first image data defining at least one two-dimensional graphical component; performing a transformation operation on the first image data to generate second image data, the second image data defining, for the or each graphical component, a modified form of the graphical component; using said second image data, displaying the modified form whereby the or each graphical component has the appearance of having a component of dimension perpendicular the plane of the display device.

Claims

1. A method of displaying one or more graphical objects, the method being carried out in an electronic device, the device having processing circuitry, memory and a display device, the method comprising:

obtaining first image data, the first image data defining at least one two-dimensional graphical component;

performing a transformation operation on the first image data to generate second image data, the second image data defining, for the or each graphical component, a modified form of the graphical component; and

using said second image data, displaying the modified form whereby the or each graphical component has the appearance of having a component of dimension perpendicular the plane of the display device;

wherein performing the transformation operation includes determining whether a first graphical component that is in modified form intersects with a concurrently displayed second graphical component that is in modified form and, if they are intersecting, determining whether to merge the first and second intersecting graphical components based on an area of the merged components;

wherein determining whether to merge the first and second intersecting graphical components comprises;

(a) determining if both of the first and second intersecting graphical components comprise dirty regions;

(b) if the determination in (a) is positive, then determining whether merging the concurrently displayed first and second intersecting graphical components would be more efficient than leaving them as concurrently displayed, intersecting graphical components; and

(c) if the determination in (b) is positive, then merging the first and second intersecting graphical components to form a merged region that comprises a geometrical union of the first and second intersecting graphical components.

6. The method of claim 1, wherein the transformation operation further includes:

initializing a region list from bounding boxes of all elements shown in a preceding video frame;

adding a region to the region list for the bounding box of each element contained in the new frame;

sorting the bounding boxes into order of distance from the plane of the display (z-order); and

discarding regions that are hidden behind others or are outside the periphery of the display.

7. The method of claim 1, wherein determining whether to merge the first and second graphical components further comprises:

(d) determining if one of the first and second graphical components comprises a dirty region and the other comprises a non-dirty region; and

(e) if the determination in (d) is positive, then marking the non-dirty region as dirty.

8. The method of claim 1, wherein determining whether to merge the first and second graphical components further comprises:

(d) determining whether the merged region would intersect a non-dirty region prior to merging the first and second graphical components; and

(e) if the determination in (d) is positive, then not merging the first and second graphical components.

9. The method of claim 1, wherein performing the transformation operation further comprises:

painting a 2D background into each dirty region; and

painting all dirty regions in Z-order.

10. The method of claim 1, wherein determining whether merging the first and second graphical components would be more efficient than not merging them comprises determining if a resulting merged region would have an area that is less than the product of a threshold factor multiplied by the sum of the areas of the first and second graphical components.

11. The method of claim 10, wherein the threshold factor is about 1.4.

12. The method of claim 1, wherein a given graphical component or region comprises a dirty region when the region'"'"'s constituent graphical elements change during the transformation and need to be repainted.

2. A method of displaying one or more moving graphical objects, the method being carried out in an electronic device, the device including processing circuitry, memory and a display device, the method comprising:

(a) obtaining first image data, the first image data defining at least one two-dimensional graphical component;

(b) for each of one or more instants during a predetermined time period,

(b1) performing a transformation operation on the first image data to generate second image data, the second image data defining, for the or each graphical component, a modified form of the graphical component;

(b2) using said second image data, displaying the modified form;

thereby at one more of said instants generating a rendition of the or each graphical component so as to appear to having a component of dimension perpendicular the plane of the display device, whereby the two-dimensional graphical component appears to move within a three-dimensional space; and

further including determining whether or not to merge the two-dimensional graphical component with another concurrently displayed, intersecting, two-dimensional graphical component based on whether or not an area of the merger is less than a product of a threshold factor multiplied by a sum of areas of the graphical components individually,

wherein determining whether to merge the two-dimensional graphical component with the another concurrently displayed, intersecting, two-dimensional graphical component comprises;

(c) determining if both of the two-dimensional intersecting graphical components comprise dirty regions;

(d) if the determination in (c) is positive, then determining whether merging the concurrently displayed two-dimensional intersecting graphical components would be more efficient than leaving them as concurrently displayed, intersecting graphical components; and

(e) if the determination in (d) is positive, then merging the two-dimensional intersecting graphical components to form a merged region that comprises a geometrical union of the first and second intersecting graphical components.

3. The method of claim 2, wherein steps (b1) and (b2) are carried out at a plurality of instants, thereby giving the appearance of gradual natural motion.

4. The method of claim 2, wherein the predetermined period is any suitable length to give satisfying appearance, including one of the following:

k hundredths of a second (where 1<

=k<

=10), m tenths of a second (where 1<

=m<

=10), or n seconds (where 1<

=n<

=10).

5. The method of claim 2, wherein, during the predetermined period, the transformation operation and display step is carried out p tens of times (where 1<

=p<

=10), q hundreds of times (where 1<

=q<

=10), or r thousands of times (where 1<

=r<

=10), so as to give an acceptable appearance of movement.

13. A computer-readable non-transitory storage having instructions thereon for executing a method of displaying one or more graphical objects, the method comprising:

obtaining first image data, the first image data defining at least one two-dimensional graphical component;

performing a transformation operation on the first image data to generate second image data, the second image data defining, for the or each graphical component, a modified form of the graphical component; and

using said second image data, displaying the modified form whereby the or each graphical component has the appearance of having a component of dimension perpendicular the plane of the display device;

wherein performing the transformation operation includes determining whether a first graphical component that is in modified form intersects with a concurrently displayed second graphical component that is in modified form and, if they are intersecting, determining whether to merge the first and second intersecting graphical components based on an area of the merged components;

wherein determining whether to merge the first and second intersecting graphical components comprises;

(a) determining if both of the first and second intersecting graphical components comprise dirty regions;

(b) if the determination in (a) is positive, then determining whether merging the concurrently displayed first and second intersecting graphical components would be more efficient than leaving them as concurrently displayed, intersecting graphical components; and

(c) if the determination in (b) is positive, then merging the first and second intersecting graphical components to form a merged region that comprises a geometrical union of the first and second intersecting graphical components.

14. The computer-readable non-transitory storage of claim 13, wherein the transformation operation further includes:

initializing a region list from bounding boxes of all elements shown in a preceding video frame;

adding a region to the region list for the bounding box of each element contained in the new frame;

sorting the bounding boxes into order of distance from the plane of the display (z-order); and

discarding regions that are hidden behind others or are outside the periphery of the display.

15. The computer-readable non-transitory storage of claim 13, wherein determining whether to merge the first and second graphical components further comprises:

(d) determining if one of the first and second graphical components comprises a dirty region and the other comprises a non-dirty region; and

(e) if the determination in (d) is positive, then marking the non-dirty region as dirty.

16. The computer-readable non-transitory storage of claim 13, wherein determining whether to merge the first and second graphical components further comprises:

(d) determining whether the merged region would intersect a non-dirty region prior to merging the first and second graphical components; and

(e) if the determination in (d) is positive, then not merging the first and second graphical components.

17. The computer-readable non-transitory storage of claim 13, wherein performing the transformation operation further comprises:

painting a 2D background into each dirty region; and

painting all dirty regions in Z-order.

18. The computer-readable non-transitory storage of claim 13, wherein determining whether merging the first and second graphical components would be more efficient than not merging them comprises determining if a resulting merged region would have an area that is less than the product of a threshold factor multiplied by the sum of the areas of the first and second graphical components.

19. The computer-readable non-transitory storage of claim 18, wherein the threshold factor is about 1.4.

20. The computer-readable non-transitory storage of claim 13, wherein a given graphical component or region comprises a dirty region when the region'"'"'s constituent graphical elements change during the transformation and need to be repainted.