Window display device with a three-dimensional orientation of windows

• US 6,577,330 B1
• Filed: 01/18/2000
• Issued: 06/10/2003
• Est. Priority Date: 08/12/1997
• Status: Expired due to Fees

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Tsuda, Kenjirou, Inagaki, Satoru
• Primary Examiner(s): Cabeca, John
• Assistant Examiner(s): Detwiler, Brian J
• Application Number:
  US09/463,198
• Time in Patent Office: 1,239 Days
• Field of Search: 345/781, 345/782, 345/848, 345/849, 345/850, 345/851, 345/852, 345/427
• US Class Current: 715/782
• CPC Class Codes: G06F 2203/04802 3D-info-object: information is displayed on the internal or external surface of a three dimensional manipulable object, e.g. on the faces of a cube that can be rotated by the user
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

A storage unit 5102 stores image data for a window, and a three-dimensional position calculating unit 5104 calculates a position for the window in a virtual three-dimensional space so that it appears to be at a position inclined in the depth direction. A perspective projection unit 5106 transforms image data so that a window positioned at the calculated coordinates is projected onto a screen in perspective, taking a point in front of the screen as a viewpoint. A frame memory unit 5107 holds image data for the window on which perspective projection has been performed, and a display unit 5108 displays the window on the screen.

Claims

1. A window display device displaying a plurality of windows on a screen, the window display device comprising:

a display data holding means for holding display data representing display content of the plurality of windows;

a position determining means for determining a position for each window in a virtual three-dimensional (3D) space, the positions being such that the windows are not parallel to the screen, wherein the position determining means determines 3D coordinates for each vertex of a window as the position for the window;

an input means for acquiring an indication from a user, wherein the position determining means determines positions according to indications acquired from the input means;

an window analyzing means for detecting, as a key part for each window, a part of the display content fulfilling certain conditions, detection being performed based on display data held by the display data holding means,

wherein the position determining means determines the position of windows so that the key parts detected by the window analyzing means are displayed near the front of the virtual 3D space;

a perspective projection means for transforming the display data to project the windows onto the screen in perspective based on the determined positions; and

a display means for displaying the transformed display data on the screen.

2. The window display device of claim 1, wherein the window analyzing means determines whether a title is included in the display data for each window and, if a title is included, detects a part of a window in which the title is displayed as the key part.

3. The window display device of claim 1, wherein the window analyzing means determines whether a text and an indication for displaying the text horizontally are included in the display data for each window, and if the text and the indication are included, detects a left side of a window when viewed from the front as the key part.

4. The window display device of claim 1, wherein the window analyzing means determines whether a text, and an indication for displaying the text vertically are included in the display data for each window, and if the text and the indication are included, detects a right side of a window when viewed from the front as the key part.

5. The window display device of claim 1, wherein the position determining means includes an automatic alignment unit for determining the position of a plurality of windows so that the windows are placed in a predetermined arrangement, where at least one window is not parallel with the screen.

6. The window display device of claim 5, wherein the position determining means further includes:

a means for determining a position according to an indication acquired from the input means, so that a window displayed in the alignment position is newly displayed facing the front; and

a means for determining a position according to an indication acquired from the input means, so that a window displayed facing the front is returned to the alignment position.

7. The window display device of claim 1, further comprising a window information image position changing means for editing display data stored in the display data holding means so that a window information image is displayed near the front of the virtual 3D space, the window information image being formed from a title bar and a menu bar, and editing being performed based on a position determined by the position determining means.

8. The window display device of claim 1, further comprising a window information image orientation changing means for controlling the position determining means so that, in a window where a window information image is displayed near the front of the virtual 3D space, the window information image faces the front, while the remaining part of the window is not parallel with the screen.

9. The window display device of claim 1, further comprising:

an unreadable area determining means for determining whether an unreadable area displaying characters smaller than a specified size has been generated from display data transformed by the perspective projection means;

a scroll button position determining means for determining a position for a scroll button when an unreadable area has been determined to exist, the position determined as if the unreadable area was not displayed in the window; and

a scroll bar attachment means for editing the display data so that a scroll bar image, including a scroll button arranged at the determined position, is attached to the window.

10. The window display device for claim 1, further comprising:

a layer separating means for separating the display data held in the display data holding means, the display data corresponding to a window with a layered construction having a base layer and at least one upper layer, and the display data for each upper layer, but not the base layer, being separated; and

a window layer generating means for generating display data for a window layer corresponding to each separated layer,

wherein the position determining means determines a position for each window layer parallel with and a specified distance from the base layer.

11. The window display device of claim 1, further comprising:

a keyword searching means for searching display data held in the display data holding means for parts that match a keyword acquired from the input means, and

a window layer generating means for generating display data for a new window layer in which the keyword is arranged at positions corresponding to the matching parts, and storing the generated display data in the display data holding means,

wherein the position determining means determines a position for the window layer parallel with and a specified distance from a display object window.

12. The window display device of claim 1, further comprising:

an inter-related window specifying means for specifying a group of inter-related windows from the plurality of displayed windows; and

a link information storage means for generating and storing link information used to line up the group of inter-related windows in a concertina arrangement with frames of neighboring windows touching,

wherein the position determining means determines positions of the group of inter-related windows based on the link information.

13. The window display device of claim 12, further comprising:

a link information changing means for changing the link information according to an indication acquired from the input means, so that the group of inter-related windows is lined up with part of the group excluded,

wherein the position determining means determines positions for the group of inter-related windows, apart from the excluded part of the group, based on the changed link information.

14. The window display device of claim 13, further comprising:

an inter-window editing means for editing display data held in the display data holding means according to an indication acquired from the input means, so that editing of display content can be performed between two windows in the group of inter-related windows.

15. A window displaying device displaying a plurality of windows on a screen, the window display device comprising:

a display data holding means for holding display data representing display content of the plurality of windows;

a position determining means for determining a position for each window in a virtual three-dimensional (3D) space, by determining 3D coordinates for each vertex of a window as the position for the window, the positions being such that the windows are not parallel to the screen;

an input means for acquiring an indication from a user, wherein the position determining means determines positions according to indications acquired from the input means;

a window information image orientation changing means for controlling the position determining means so that, in a window where a window information image is displayed near the front of the virtual 3D space, the window information image faces the front, while the remaining part of the window is not parallel with the screen;

a perspective projection means for transforming the display data to project the windows onto the screen in perspective based on the determined positions; and

a display means for displaying the transformed display data on the screen.

16. A window display device displaying a plurality of windows on a screen, the window display device comprising:

a display data holding means for holding display data representing display content of the plurality of windows;

a position determining means for determining a position for each window in a virtual three-dimensional (3D) space, by determining 3D coordinates for each vertex of a window as the position for the window, the positions being such that the windows are not parallel to the screen;

an input means for acquiring an indication from a user, wherein the position determining means determines positions according to indications acquired from the input means;

a layer separating means for separating display data held in the display data holding means, the display data corresponding to a window with a layered construction having a base layer and at least one upper layer, but not the base layer, being separated;

a window layer generating means for generating display data for a window layer corresponding to each separated layer, wherein the position determining means determines a position for each window layer parallel with and a specified distance from the base layer;

a perspective projection means for transforming the display data to project the windows onto the screen in perspective based on the determining positions; and

a display means for displaying the transformed display data on the screen.

17. A window display device displaying a plurality of windows on a screen, the window display device comprising:

a display data holding means for holding display data representing display content of the plurality of windows;

a position determining means for determining a position for each window in a virtual three-dimensional (3D) space, by determining 3D coordinates for each vertex of a window as the position for the window, the positions being such that the windows are not parallel to the screen;

an input means for acquiring an indication from a user, wherein the position determining means determines positions according to indications acquired from the input means

a keyword searching means for searching display data held in the display data holding means for parts that match a keyword acquired from the input means, and

a window layer generating means for generating display data for a new window layer in which the keyword is arranged at positions corresponding to the matching parts, and storing the generated display data in the display data holding means,

wherein the position determining means determines a position for the window layer parallel with and a specified distance from a display object window;

a perspective projection means for transforming the display data to project the windows onto the screen in perspective based on the determined positions; and

a display means for displaying the transformed display data on the screen.

18. A window display device displaying a plurality of windows on a screen, the window display device comprising:

a display data holding means for holding a display data representing display content of the plurality of windows;

a position determining means for determining a position for each window in a virtual three-dimension (3D) space, by determining 3D coordinates for each vertex of a window as the position for the window, the positions being such that the windows are not parallel to the screen;

an input means for acquiring an indication from a user, wherein the position determining means determines positions according to indications acquired from the input means

a perspective projection means for transforming the display data to project the windows onto the screen in perspective based on the determined positions; and

a display means for displaying the transformed display data on the screen, wherein the display data holding means includes;

a character data storage unit storing character data formed of character information displayed in the windows; and

a window data storage unit storing window data formed of information concerning non-character information displayed in the windows,

wherein the position placement determining means determines a position for each window and the corresponding characters according to an indication acquired from the input means, and

the perspective projection means includes;

a perspective transform unit generating two-dimensional data showing the vertices and window image data by performing a perspective projection transform on the window data, the perspective projection transform being performed based on window positions determined by the position determining means;

a character data transform unit generating character image data by transforming the character data, based on character positions determined by the position determining unit and two-dimensional data showing vertices obtained by the transform performed by the perspective transform unit; and

a combining unit combining image data generated by the perspective transform unit and image data generated by the character data transform unit,

the display means displaying image data combined by the combining unit, wherein the character data storage unit includes;

a character information storage unit storing information relating to characters displayed in the window, the information relating to characters displayed in the window, the information including at least character position information showing positions of the characters in virtual space, character identifying information specifying which characters should be displayed, and character size information; and

a character image storage unit storing character images,

wherein the window data storage unit stores graphics data, comprising window display content excluding characters, and window coordinate information showing a position of the window in the virtual 3D space,

the position determining means changes the window data and the character position information in response to an indication acquired from the input means;

the perspective transform unit performs a perspective projection transform on window data that has been changed by the position determining means,

the character data transform unit includes;

a character position information conversion unit converting character position information changed by the position determining means into screen position information;

a character image selecting unit selecting a character image from the character images stored in the character image storage unit, the character image corresponding to the character identifying information; and

a character image transform unit transforming the character image selected by the character image selecting unit in accordance with the result of the position changing process performed by the position determining means,

wherein the combining unit combines a character image transformed by the character image transform unit with image data generated by the perspective transform unit, so that the character image is placed at a position in the window shown by the screen position information produced by the character position information conversion unit, the window having been transformed onto the screen by the perspective transform unit,

wherein the character image storage unit stores character images on which a similarity transform has been performed for each of a plurality of representative positions for a window in the virtual 3D space, the similarity transform being related to the shape of the window produced at each of the plurality of positions as a result of a perspective projection transform;

the character image selecting unit selects a character image from the character image storage unit, the character image coinciding with character identifying information stored in the character information storage unit, and with window coordinate information stored in the window data storage unit; and

the character image transform unit finds a scale based on character position information stored in the character information storage unit, and changes the size of the character image selected by the character image selecting unit, the size of the character image being changed according to the scale.

19. The window display device of claim 18, further comprising a coordinate system converting means for changing a scale of a coordinate axis showing screen positions, the scale being changed according to window coordinate information stored in the window data storage unit,

wherein the perspective projection unit performs a perspective projection transform, converting the coordinate system to a coordinate system having a coordinate axis with the changed scale.

20. A window display device displaying a plurality of windows on a screen, the window display device comprising:

a display data holding means for holding a display data representing display content of the plurality of windows;

a position determining means for determining a position for each window in a virtual three-dimensional (3D) space, by determining 3D coordinates for each vertex of a window as the position for the window, the positions being such that the windows are not parallel to the screen;

an input means for acquiring an indication from a user, wherein the position determining means determines positions according to indications acquired from the input means

a perspective projection means for transforming the display data to project the windows onto the screen in perspective based on the determined positions; and

a display means for displaying the transformed display data on the screen, wherein the display data holding means includes;

a character data storage unit storing character data formed of character information displayed in the windows; and

a window data storage unit storing window data formed of information concerning non-character information displayed in the windows,

wherein the position placement determining means determines a position for each window and the corresponding characters according to an indication acquired from the input means, and

the perspective projection means includes;

a perspective transform unit generating two-dimensional data showing the vertices and window image data by performing a perspective projection transform on the window data, the perspective projection transform being performed based on window positions determined by the position determining means;

a character data transform unit generating character image data by transforming the character data, based on character positions determined by the position determining unit and two-dimensional data showing vertices obtained by the transform performed by the perspective transform unit; and

a combining unit combining image data generated by the perspective transform unit and image data generated by the character data transform unit,

the display means displaying image data combined by the combining unit,

wherein the character data storage unit includes;

a character information storage unit storing information relating to characters displayed in the window, the information relating to characters displayed in the window, the information including at least character position information showing positions of the characters in virtual space, character identifying information specifying which characters should be displayed, and character size information; and

a character image storage unit storing character images,

wherein the window data storage unit stores graphics data, comprising window display content excluding characters, and window coordinate information showing a position of the window in the virtual 3D space,

the position determining means changes the window data and the character position information in response to an indication acquired from the input means;

the perspective transform unit performs a perspective projection transform on window data that has been changed by the position determining means,

the character data transform unit includes;

a character position information conversion unit converting character position information changed by the position determining means into screen position information;

a character image selecting unit selecting a character image from the character images stored in the character image storage unit, the character image corresponding to the character identifying information; and

a character image transform unit transforming the character image selected by the character image selecting unit in accordance with the result of the position changing process performed by the position determining means,

wherein the combining unit combines a character image transformed by the character image transform unit with image data generated by the perspective transform unit, so that the character image is placed at a position in the window shown by the screen position information produced by the character position information conversion unit, the window having been transformed onto the screen by the perspective transform unit,

further comprising a character image readability judging means for (1) calculating a size for a transformed character image transformed by the character image transform unit from (a) size information stored in the character information storage unit, and (b) a scale calculated from character position information stored in the character information storage unit, (2) judging whether the transformed character image is readable by comparing the calculated size with a threshold value, and (3) informing the character image selecting unit of the judgment result,

wherein the character image storage unit further stores symbol images;

the character image selecting unit selects a specified symbol image from the character image storage unit to be displayed instead of a character image if the character image readability judging means judges that the character is unreadable;

the character image transform unit does not perform transform processing on the symbol image, and

the combining unit combines the symbol image with image data generated by the perspective transform unit so that the symbol image is placed at a position in a window shown by screen position information produced by the character position information conversion unit, the window having been transformed onto the screen by the perspective transform unit.

21. The window display device of claim 20, wherein the character image transform unit includes:

a shape transform unit transforming the shape of the character image using a similarity transform related to the shape of the window transformed by the perspective transform unit; and

a scaling unit finding a scale based on character position information stored in the character information storage unit, and changing the size of the character image according to the scale.

22. The window display device of claim 20, wherein the character image transform unit calculates a scale based on character position information stored in the character information storage unit, and changes the size of the character image according to the scale.

23. The window display information of claim 20, wherein the character image transform unit includes:

an aspect ratio conversion unit finding an aspect ratio based on character position information stored in the character information storage unit and changing the length to width ratio of the character image according to the aspect ratio; and

a scaling unit finding a scale based on character position information stored in the character information storage unit, and changing the size of the character image according to the scale.

24. A window display device displaying a plurality of windows on a screen, the window display device comprising:

a display data holding means for holding a display data representing display content of the plurality of windows;

a position determining means for determining a position for each window in a virtual three-dimensional (3D) space, by determining 3D coordinates for each vertex of a window as the position for the window, the positions being such that the windows are not parallel to the screen;

an input means for acquiring an indication from a user, wherein the position determining means determines positions according to indications acquired from the input means a perspective projection means for transforming the display data to project the windows onto the screen in perspective based on the determined positions; and

a display means for displaying the transformed display data on the screen, wherein the display data holding means includes;

a character data storage unit storing character data formed of character information displayed in the windows; and

a window data storage unit storing window data formed of information concerning non-character information displayed in the windows,

wherein the position placement determining means determines a position for each window and the corresponding characters according to an indication acquired from the input means, and

the perspective projection means includes;

a perspective transform unit generating two-dimensional data showing the vertices and window image data by performing a perspective projection transform on the window data, the perspective projection transform being performed based on window positions determined by the position determining means;

a character data transform unit generating character image data by transforming the character data, based on character positions determined by the position determining unit and two-dimensional data showing vertices obtained by the transform performed by the perspective transform unit; and

a combining unit combining image data generated by the perspective transform unit and image data generated by the character data transform unit,

the display means displaying image data combined by the combining unit,

wherein the character data storage unit includes;

a character information storage unit storing information relating to characters displayed in the window, the information relating to characters displayed in the window, the information including at least character position information showing positions of the characters in virtual space, character identifying information specifying which characters should be displayed, and character size information; and

a character image storage unit storing character images,

wherein the window data storage unit stores graphics data, comprising window display content excluding characters, and window coordinate information showing a position of the window in the virtual 3D space,

the position determining means changes the window data and the character position information in response to an indication acquired from the input means;

the perspective transform unit performs a perspective projection transform on window data that has been changed by the position determining means,

the character data transform unit includes;

a character position information conversion unit converting character position information changed by the position determining means into screen position information;

a character image selecting unit selecting a character image from the character images stored in the character image storage unit, the character image corresponding to the character identifying information; and

a character image transform unit transforming the character image selected by the character image selecting unit in accordance with the result of the position changing process performed by the position determining means,

wherein the combining unit combines a character image transformed by the character image transform unit with image data generated by the perspective transform unit, so that the character image is placed at a position in the window shown by the screen position information produced by the character position information conversion unit, the window having been transformed onto the screen by the perspective transform unit,

wherein the character information storage unit further stores font type information showing character font types;

the character image storage unit stores character images for each font type; and

the window display device further includes a character image readability judging means for (1) calculating a size for a transformed character image transformed by the character image transform unit from (a) size information stored in the character information storage unit, and (b) a scale calculated from character position information stored in the character information storage unit, (2) judging if the font type of the transformed character image is readable by comparing the calculated size with a threshold value, and (3) informing the character image selecting unit of the judgment result,

wherein, when the font type of a character is judged unreadable by the character image readability judging means, the character image selecting unit selects a character image with a specified font type, ignoring font type information stored in the character information storage unit.